Sign up

Using ddox with GtkD for documentation

Like some other GtkD users, I find using the documentation a bit frustrating so I thought I would take a stab at an alternative, namely ddox. If you are unfamiliar with ddox, it's used to provide the documentation for Vibe and Phobos and a link to the project can be found here https://github.com/rejectedsoftware/ddox. The one nice thing with ddox is that it provides a searchable interface so you can just type into a text box and it will show the entries that match rather then scroll up and down a long list like we do currently with candydocs. With a project like GtkD which has a plethora of classes I find this a big time saver.

This post is about getting ddox working with GtkD, note that while what I am posting here works it's not perfect, I'm posting in the hopes of getting some discussion going and maybe get to a point where we can improve the official GtkD documentation. On that note, here's my instructions.

Using ddox consists of two script files, the first script file generates the docs.json file required by ddox while the other starts the ddox server which serves the documentation. The script that generates the documentation is called makeddox.sh and goes in the root Gtkd directory, it's a peer to the existing makedocs.sh. The script creates a ddox directory (similar to the existing docs directory), the contents of the script file are as follows:

#!/bin/sh

mkdir -p ddox/temp

echo "MODULES =" > modules.ddoc ;grep -h -e "^module" src/* -r | sort -u | sed 's/;//' | sed 's/\r//' |  sed 's/module \(.*\)$/\t$(MODULE \1)/' >> modules.ddoc
grep -h -e "^module" srcgl/* -r | sort -u | sed 's/;//' | sed 's/\r//' |  sed 's/module \(.*\)$/\t$(MODULE \1)/' >> modules.ddoc
grep -h -e "^module" srcsv/* -r | sort -u | sed 's/;//' | sed 's/\r//' |  sed 's/module \(.*\)$/\t$(MODULE \1)/' >> modules.ddoc
grep -h -e "^module" srcvte/* -r | sort -u | sed 's/;//' | sed 's/\r//' |  sed 's/module \(.*\)$/\t$(MODULE \1)/' >> modules.ddoc

dmd -o- -D -X -Xfddox/docs.json -Ddddox/temp modules.ddoc docs/candy.ddoc \
	src/gtk/*  src/gtkc/* src/glib/* src/gio/* src/gdk/* \
	src/gobject/* src/gthread/* src/atk/* \
	src/pango/* src/cairo/* src/gdkpixbuf/* \
	srcgl/glgdk/*  srcgl/glgtk/*  srcgl/gtkglc/* \
	srcsv/gsv/*  srcsv/gsvc/* \
	srcvte/vtec/* srcvte/vte/* -op

# Delete all html files generated by D ddocs
rm -rf ddox/temp/*

# insert a fake comment for all modules so that ddox doesn't filter out the modules since 
# none of them have comments
echo Adding comment to modules
sed -i 's/"kind" : "module",/"kind" : "module", "comment" : " ",/g' ddox/docs.json

# Filter out everything except public members that are documented
dub run ddox -- filter ddox/docs.json --only-documented --min-protection Public

# Use dub to run ddox and generate offline ddox documentation
# This doesn't work, hangs on gdk.Windows
#dub run ddox -- generate-html docs.json ddox

This will create a ddox/docs.json file, now we are ready to run the ddox server. Before executing the next step, you need to copy the /public directory from ddox into the GtkD/ddox directory so that the ddox web server can access the css and js files it needs to serve the documentation. Once you've done that, use the following command to start the ddox server from the root GtkD directory:

dub run ddox -- serve-html ddox/docs.json --navigation-type=ModuleTree --web-file-dir=ddox/public

If everything went to plan you should see a nicely stylized site with the GtkD documentation. If you what you see instead is plan HTML with no styling you made a mistake in copying the ddox public web files, you can view source on a page and see where ddox is looking for the files.

Some issues and potential improvements as follows:

  • The ddox generation of off-line documentation hangs on gdk.Window so running the ddox server is required, need to look into this more

  • AFAIK GtkD candydocs only generates docs for things which have documentation comments, for example if you look at the source code for vte.Terminal there are lots of un-documented public members which don't show up in candydoc. ddox has a switch to do the same thing, only-documented, but since none of the modules are documented they get stripped out leaving you with an empty docs.json file. If the project wanted to switch to ddox at some point in the future it would be nice if the auto-generation could include a blank doc comment for modules similar to what it does for classes. Right now I'm handling this in the script with sed to insert a blank comment

  • GtkD has a style of a 1:1 match between module and class, this is a bit annoying with respect to browsing the ddox documentation since you have to click on the module first and then the class. For the search facility in ddox it's not an issue since you can directly click on the class but it would be nice to have a pass-through when there is only one documented item in the module.

  • I'm not a fan of the default ddox style of showing the class with only summary info and forcing the user to click on the member to see it's full definition. Need to spend some time figuring out if there is way to generate a single page for each class with all of the info.

Anyway, that's it for now, like I said I'm hoping this will kick off some discussion about GtkD documentation and we can get a bit of critical mass to improve the situation.

Re: Using ddox with GtkD for documentation

Here's a screenshot of the GtkD ddox generated documentation:

http://gexperts.com/img/ddox_gtk.png

Re: Using ddox with GtkD for documentation

ddox also hangs when you are using serve-html and try to access
gdk.Window.Window.

One problem seems to be the “ and ” characters in the comments, but
there are more issues it hangs on.

Re: Using ddox with GtkD for documentation

On Fri, 25 Dec 2015 11:29:03 +0100, Mike Wey wrote:

ddox also hangs when you are using serve-html and try to access
gdk.Window.Window.

One problem seems to be the “ and ” characters in the comments, but
there are more issues it hangs on.

I'll have a look at both issues and see if it's something simple to fix in their diet templates.

Also, as an FYI I noticed that the <template> reference in Widget causes the ddox documentation to not display in the browser however when I compile the docs in candydocs using makedocs.sh locally I see the same issue but the on-line docs look fine. Is it possible something changed in DMD that is causing this, I'm using 2.0.69 in Arch Linux.

Re: Using ddox with GtkD for documentation

On Fri, 25 Dec 2015 11:29:03 +0100, Mike Wey wrote:

ddox also hangs when you are using serve-html and try to access
gdk.Window.Window.

One problem seems to be the “ and ” characters in the comments, but
there are more issues it hangs on.

The problem appears to be unicode in general, ddox does a lot of character parsing of the comments in order to create links between various entities and I suspect that the multi-byte unicode characters in UTF-8 are screwing it up. I updated the script to convert unicode double and single quotes to their ASCII equivalents and then run the rest through iconv for everything else to generate an ASCII docs.json. This then works fine with ddox and it can successfully generate documentation for all classes. A quick look through the generated docs look fine to me from a readability point of view, I don't see any obvious problems but I suspect Mike you'd be in a better position to know where the problem areas are likely to be.

The updated script is below. Mike I'd be willing to put some additional effort into modifying the ddox diet templates that generate the pages to try to optimize them for GtkD but is using ddox something the project would be willing to consider? I don't want to invest a bunch of effort in this if it's a non-starter for GtkD.

#!/bin/sh

mkdir -p ddox/temp

echo "Generating JSON file for modules"

echo "MODULES =" > modules.ddoc ;grep -h -e "^module" src/* -r | sort -u | sed 's/;//' | sed 's/\r//' |  sed 's/module \(.*\)$/\t$(MODULE \1)/' >> modules.ddoc
grep -h -e "^module" srcgl/* -r | sort -u | sed 's/;//' | sed 's/\r//' |  sed 's/module \(.*\)$/\t$(MODULE \1)/' >> modules.ddoc
grep -h -e "^module" srcsv/* -r | sort -u | sed 's/;//' | sed 's/\r//' |  sed 's/module \(.*\)$/\t$(MODULE \1)/' >> modules.ddoc
grep -h -e "^module" srcvte/* -r | sort -u | sed 's/;//' | sed 's/\r//' |  sed 's/module \(.*\)$/\t$(MODULE \1)/' >> modules.ddoc

dmd -o- -D -X -Xfddox/docs.json -Ddddox/temp modules.ddoc docs/candy.ddoc \
	src/gtk/*  src/gtkc/* src/glib/* src/gio/* src/gdk/* \
	src/gobject/* src/gthread/* src/atk/* \
	src/pango/* src/cairo/* src/gdkpixbuf/* \
	srcgl/glgdk/*  srcgl/glgtk/*  srcgl/gtkglc/* \
	srcsv/gsv/*  srcsv/gsvc/* \
	srcvte/vtec/* srcvte/vte/* -op

# Delete all html files generated by D ddocs
rm -rf ddox/temp/*

#insert a fake comment for all modules so that ddox doesn't filter out the modules
echo "Adding comment to modules"
sed -i 's/"kind" : "module",/"kind" : "module", "comment" : " ",/g' ddox/docs.json

#Fix problem with unicode quotes by replacing them with ASCII quotes
echo Replacing unicode double and single quotes with ASCII equivalent
sed -i 's/“/\&quot;/g' ddox/docs.json
sed -i 's/”/\&quot;/g' ddox/docs.json
sed -i 's/’/\&#39;/g' ddox/docs.json

echo "Convert UTF-8 to ASCII for everything else"
mv ddox/docs.json ddox/docs_utf8.json
iconv -f utf8 -t ascii -c ddox/docs_utf8.json > ddox/docs.json

# Filter out everything except public members
dub run ddox -- filter ddox/docs.json --only-documented --min-protection Public

# Use dub to run ddox and generate offline ddox documentation
dub run ddox -- generate-html --navigation-type=moduleTree ddox/docs.json ddox

Re: Using ddox with GtkD for documentation

On 12/27/2015 06:29 PM, Gerald Nunn wrote:

On Fri, 25 Dec 2015 11:29:03 +0100, Mike Wey wrote:

ddox also hangs when you are using serve-html and try to access
gdk.Window.Window.

One problem seems to be the “ and ” characters in the comments, but
there are more issues it hangs on.

The problem appears to be unicode in general, ddox does a lot of character parsing of the comments in order to create links between various entities and I suspect that the multi-byte unicode characters in UTF-8 are screwing it up. I updated the script to convert unicode double and single quotes to their ASCII equivalents and then run the rest through iconv for everything else to generate an ASCII docs.json. This then works fine with ddox and it can successfully generate documentation for all classes. A quick look through the generated docs look fine to me from a readability point of view, I don't see any obvious problems but I suspect Mike you'd be in a better position to know where the problem areas are likely to be.

The updated script is below. Mike I'd be willing to put some additional effort into modifying the ddox diet templates that generate the pages to try to optimize them for GtkD but is using ddox something the project would be willing to consider? I don't want to invest a bunch of effort in this if it's a non-starter for GtkD.

I've added some sed commands to fix the documentation for gtk.Widget and
gtk.Builder. And added the script to the GtkD repository.

Re: Using ddox with GtkD for documentation

I've added some sed commands to fix the documentation for gtk.Widget and
gtk.Builder. And added the script to the GtkD repository.

Where in the repository is it as I'm not able to find it?

Re: Using ddox with GtkD for documentation

On 01/16/2016 03:57 PM, Gerald Nunn wrote:

I've added some sed commands to fix the documentation for gtk.Widget and
gtk.Builder. And added the script to the GtkD repository.

Where in the repository is it as I'm not able to find it?

Should be there now.