Documenting The Outpost2 Api

[TH300]

Recently, I started coding a new Outpost2 mission and while reading some API headers looking for useful functions, I realized that I did again forget how some functions work (the comments don't explain everything) and that it is quite a pain to get a good overview of everything.

I believe that people who are new to op2 coding have even more trouble. We have tutorials, but those don't help if you want to know how one particular function works.

My suggestion is that we create a html documentation of the whole Outpost2 API with Doxygen. Doxygen automatically creates html files from C++ source files (and special comments in those files). The created html document is clean and easy to navigate. Here is an example.

Now, I want to know if I can go ahead and commit doxygen comments to the existing api tree on svn. All that will be changed are comments. The api wouldn't be altered. The headers would grow quite a bit and doxygen configuration file(s) would be added.

Furthermore, I'll be happy if I get help, because its a lot and chances are that I won't find out how some functions work.

If we finish this, we can put the html documentation on the website. Imo this would be a great resource for beginners as well as experienced programmers.

[Hooman]

I'm having a bit of trouble forming an opinion on this. I believe you are correct that this needs to be addressed. I'm less certain that your proposal is the way to do it. I would like to see more comments in the code. I'm hesitant to suggest commenting to that extent though, as it can drown the code in comments, making it hard to check header files as a quick reference once you're already familiar with the functionality. But part of that is I've never used a system like Doxygen before. I have no sense of what using such a system is really like in practice. I've heard such systems work well with code folding, but I still use MSVC 6 for Outpost 2 code, which doesn't have code folding. That would make it harder to ignore the bulk of documentation.

I'm also not entirely sure this addresses the overview part that you've brought up. I understand how this addresses documenting individual functions. I'm not sure how these systems are used to document a group of similar functions, such as all trigger creation functions, which all share certain commonalities. Or similarly, how to document a group of related member functions for a class, separate from the entire class.

Lastly, I believe one of the strengths of these documentation systems is that putting the documentation in with the code makes it easier to keep both code and documentation in sync. But due to the no-source-code nature of Outpost 2, I don't see the code in the header files ever changing, so the problem of keeping things in sync doesn't seem to exist here.


I would like better documentation. Both of individual functions, and overview of how to use sets of functions, or general concerns for certain features. I would love to put documentation on the website somehow. I also wouldn't mind a small intro to C++ that's targeted at what you'll need to know for level programming. I'm not completely sold on using Doxygen to generate that documentation. I'm also not exactly against the idea either. I'm not sure I'll have a sense of whether or not it's a good idea until someone actually tries it. I guess I just have a strange sense that another method or another tool might be more appropriate. But then, I may be overlooking some of Doxygen's features.


I'd be curious to know what other people think about the documentation. In particular, would having lots of documentation in the header files themselves be good, or distracting. I think having more and better documentation is something most people will want. I'm not so sure if people want it all in the header files. It might be nice to document a small file from the API to compare what it looks like with and without the extra documentation. From what I've seen of other projects, the amount of comments for the documentation often dwarfs the actual code itself by an order of magnitude.


What should go in the documentation? Perhaps:
 - function signature
 - description of all inputs
 - description of return value
 - error conditions
 - behavior or performance notes
 - example usage
 - multiple examples of usage?
 - links to related functions or classes
Including it all could be useful, but would take up quite a bit of space.
 

[TH300]

I understand your concern of huge comment blocks making the header files unreadable. I thought, that would no longer be necessary once we have a doxygen documentation, but well... There is the option to put the documentation (source) outside the C++ headers. The headers wouldn't be touched at all. It would be a bit more work, because all class and function definitions would have to be copied. In addition would we still have to put the documentation source files somewhere near the C++ headers (but possibly in a different directory).

Also, if you like to have everything integrated in MSVC++, doxygen can be used to create CHM help files. PDF is also a possible output format.

Let me explain the overview part, as I see it: Right now I see a function definition in a header which takes an object of a class as parameter which is defined in another file. I have to search for the other file (sometimes it doesn't have the name of the class that I'm looking for) and find the class definition in it. - In my eyes thats far too complicated. - In a doxygen documentation I simply click on the name of the class that I want to inspect (wherever it occurs) and I'm there.

As for tutorials and examples which are not part of the op2 api, but related to it: That is possible. We can create pages which describe no particular class and no particular function. But in my opinion tutorials are a bit off the scope of an API documentation (that doesn't mean I'm against putting them on the website).

What should go in the documentation? Perhaps:
- function signature
- description of all inputs
- description of return value
- error conditions
- behavior or performance notes
- example usage
- multiple examples of usage?
- links to related functions or classes

Yea, thats sort of what I have in mind. And, as I said, there is no necessity to put it in the header files.

[Hooman]

I'm thinking I might want to explore this idea as a branch. I suppose one of the benefits of using Doxygen is that it seems to be able to parse C++ code. That's a nice feature to have for ensuring correctness of at least that part of the documentation.

Have you checked for any alternative tools? I looked briefly, but didn't see anything too promising looking. Mostly just similar tools with support for other languages. Some of them might also be usable with C++ code, but probably couldn't parse it. Seemed like you'd have to manually mark up the structure and relationships for them to appear in the documentation with the other tools.


I've also heard mention of an SVN plugin to automatically generate/update Doxygen documentation whenever a commit occurs. That might be a nice automation tool to have.
 

[TH300]

As for alternative tools, I found Natural Docs, which I personally don't like as much, because I'm getting confused by its output (its hierarchy seems to be file oriented, whereas I'd prefer a class oriented hierarchy). And it doesn't seem to automatically link as much as Doxygen does.

Anyways, many popular C/C++ libraries use Doxygen. It has proven to work. What can we do wrong if we use it?

[Hooman]

Ahh yes, I came across the Natural Docs option the other day. I kind of dismissed it thought after I noticed the "language support" page. It lists C/C++ under the "Basic Language Support" section, rather than the "Full Language Support" section. After reading the difference between full and basic language support I figured this wouldn't be as useful of a tool. It seems to require manually marking certain things for them to appear in the documentation.
 

[Hooman]

I have renewed interest in this idea. There does need to be more documentation, and a better overview. The header files probably are a good place for it. I think function level documentation, and general overviews of larger topics would be quite welcome.

Tutorials I think should be kept separate, but mostly because I don't know how they'd fit in with the header files documentation.

[Leviathan]

Hi

Documenting Outpost 2 in general would be great, both playing the game and creating content for it. We made a good start in the past with the wiki, there are some good forum threads and also comments in the SDK/on the SVN etc.

We badly need documentation so that new users can get into programing for Outpost 2.

I feel that the documentation should be seperate to the source code and maintained via a web based system. CHM/PDF is nice, and so is auto documentation creation but its not necessary. The most importiant thing is that we actually go about having a up to date, clean and orgnized documentation.

With a wiki system we can have one category for API pages and another for tutorials.

Anyone else interested in helping?

[Hooman]

I've been suffering through bad and non-existent internet connections these last few weeks. It's making the CHM or PDF option sound nice actually. There's probably a way to write the content once, and have it compiled into various formats. A wiki sounds more modern. Do you know of any tools to work with multiple formats of documentation at the same time?

[leeor_net]

I think TH300's proposal of using Doxygen is not only valid but exactly the tool that we should use. Doxygen can generate HTML pages but it can also generate a lot of other formats as well. There are a few mediawiki extensions that seem to do the job but they're not maintained.

It shouldn't be hard to modify Doxygen to output wiki formatted text... the problem is in converting it from text to pages. MediaWiki would be an issue as everything is stored in a SQL database.

DokuWiki, on the other hand, uses flat text files. If we modified Doxygen to output text files using DokuWiki markup, those text files could then just be uploaded to the wiki server and bam, done deal. Could even automate it and have the server run Doxygen whenever a new commit is made to the OP2 API svn directories and automagically rewrite the files for us.

Moving on to TH300 and Hooman's concerns of large comment blocks in the headers, there shouldn't be. If you're going to do a lot of explaining of the behind the scenes and how to do things, that's where a Wiki comes in handy.

Comment blocks using Doxygen styled formatting shouldn't clutter things up too much unless we're creating functions that have too many parameters. I haven't looked at the API source at all so I can't make a judgement, but I'm strongly in favor of updating the code comments.


EDIT:

Preliminary lookup of having Doxygen output DokuWiki or MediaWiki compatible code was not promising and converting HTML to a wiki format was also not very promising.

But really when I think about it, I see the documentation generated by Doxygen as the 'Manual' which only explains how to use a function, class, structure, etc. vs a Wiki which goes into the details of the why's and when's. I still vote for a doxygen generated documentation which can still be automatically generated and copied to any location, really, and made available any time there are SVN commits to the Outpost2 code.

[Hooman]

Yes, there seems to be two separate needs for documentation. One is the reference style for a single function, which I think goes well in a header file, which documentation can be auto generated from. The other is the higher level overview and using sets of functions together to accomplish specific tasks. The later seems too verbose to put into header files, and I think it makes more sense as a wiki. Auto generating wiki content from source seems to miss the point of a wiki.

I do like the auto-build idea, of auto compiling new documentation upon check in. We need more stuff like that.

The current server SVN setup seems a bit lacking to me. We have a central repository, but no real backup. The full data is not available while offline. There is limited ability to add server side hooks. I think only Leviathan has access to the server? I think I'm finally giving in to the Git fundamentalists. And the auto build and test system evangelists too. I suppose there is always git-svn though.

[leeor_net]

Sounds like we're on the same page. The Doxygen comments do actually work well in a header and can be used to auto-generate an HTML manual of the code but can also output LaTeX and RTF files. I've never seen how these look but if they're anywhere near as good as the HTML they'll make a great download and even can be converted to PDF rather easily.

The Wiki is your support page where you go for the how's, why's and 'theory' behind things.

[Hooman]

Does anyone have any experience with tools to convert a wiki into a book format? That could be useful for generating high level overview documentation with examples from any contributed wiki content. I was wondering how articles are ordered when placed in a linear book format. I assume some kind of table of contents needs to be generated first to guide the order.

I see MediaWiki has a PDF Export plugin which is geared towards printing individual articles, and manually grouping articles together when printing. The extension page also links to a PDF Book plugin. I was originally looking for something that could run as a sort of commit hook to generate new output whenever there was an update. It looks like this plugin generates a book on demand, which I think works just as well. Either way, it's a one click download of up to date data. I'd want it to include a generation date though, in case multiple book versions end up floating around on a hard drive. A generation date would also be useful in the event the wiki data was ever lost, and someone wanted to restore the data from downloaded books.

[leeor_net]

I have looked into these sorts of tools and both major platforms have plugins with a PDF Export option. I think we've settled on DokuWiki so we'll need to install the Export to PDF for that.

Though I wonder if it would instead make sense to have it export to another format, may RTF? That way we can edit it into a a book format?

The Doxygen generated pages can also be offered as a download for offline browsing. I do it all the time with NAS2D when I don't have access to the internet. I just load up my offline Doxygen manual and go from there so that's an option as well.


EDIT:

DokuWiki has an Export to ODT plugin which would probably make more sense for this than an RTF export.

[Hooman]

I just took a look at the DokuWiki page. It looked decent, and I saw a DW2PDF extension that sounded at least a little promising.

I don't quite understand the RTF comment. What do you mean by edit into a book format? I would hope an export option generates a book format, without need for further editing. I wouldn't want to re-edit into a book format every time someone makes a minor update to the wiki.

[leeor_net]

RTF as in Rich Text Format... but if an export can be done directly into PDF and the results are fine, I'm game. My problem is that I'm always really, really super picky about the way written materials look and flow. It's the perfectionist in me -- it gets in the way sometimes.

I'm also typing without really thinking about it... so some things may come out as incoherent or flat out wrong, etc.