Calling for help in documentation

Posted by Ingo Schommer on 2 June 2010

The criticism we hear most often from the developer community is that, with the exception of our books, SilverStripe needs to improve its developer documentation. And it's a fair point.  

We made some decisions regarding our documentation a while back that we thought would work at the time - a best guess as to how to proceed. But these turned out to limit us in ways we simply didn't foresee.

First, the wiki format used on doc.silverstripe.org since we've opensourced in 2007 didn't scale well. Now, we chose a wiki because this way we could have everyone contribute without putting any barriers in the way. The problem was that it wasn't used that much. Sure, anyone could press the "edit" button, but in practice, few developers did - and information quickly became outdated. 

Complicating that was that there were months-long gaps between when we implemented new features into the test builds and when we finally released them. Because the Wiki was always "live," we couldn't change the documentation for features we didn't know would make it into release until close to, or after, the release. So you're likely to forget the code later on, or not be able to do it at the same level of detail as you would have been able to when the code was fresh. 

Then, our API documentation at api.silverstripe.org was poorly presented, not targeted at actual releases, and often severely outdated. In order to enable developers to effectively look up information like method parameters or class usage, we need to do a lot better than this. As a starting point, we've completely rewritten and "decluttered" the phpDocumentor-based templates (see announcement and discussion).

Now, we do not have a master plan on how to improve the situation, but here are some some high-level ideas that we want to put out for community feedback.

First, we could start a documentation team that can review existing material and keep editorial oversight on new content. This will be an important milestone in raising the bar for our documentation quality, and we're looking for community members to have a significant stake in this.

Second, we'd like to move some of the documentation closer to the code, and into version control. There needs to be a separate set of documentation for upcoming releases of SilverStripe than there is for the current version, at the time when features are implemented, not six months afterwards. This means converting a great amount of text from the current DokuWiki format into our syntax of choice, Markdown.

Third, the doc.silverstripe.org wiki should become more of a "conceptual glue" for highlevel concepts like "forms" or "themes", and point people towards the detailed documentation inside the API. 

Fourth, alongside the move of documentation parts to version control, a clear separation of "official" documentation from user contributed recipes should help to keep information relevant and manageable. To be clear: This doesn't change the fact that anybody can contribute to "official" docs, but hopefully we can establish a more proactive editorial process through a documentation team to ensure new content lands in the right place.

Fifth, we're considering a unified search interface across both doc.silverstripe.org and api.silverstripe.org can help to avoid duplicated and hidden information. This will be based on the OpenSearch specification, which will allow you to integrate a search right in your browser search bar for quick access.

Sixth, and finally, we want to document our way of writing documentation (meta documentation, anyone?!) - guidelines what needs to go where, in which style and format.

Now, documentation is a time and labour intensive process, and to be honest, we're quite a small core team thats already swamped with other responsibilities. We've had a massive push in getting the first and second SilverStripe book out, but also need to stay committed to having up-to-date and freely available documentation on top of that.

We hope to get a discussion going to find out where we need to focus and if we can make it a priority in the overall community to improve SilverStripe documentation. And to do that, we're hoping you can provide suggestions in the comments section of this post.

What annoys you the most about SilverStripe documentation? Where do you go to get information about developing in SilverStripe? And would you be willing to contribute to documentation efforts?

Post your comment

Note: Comments are moderated and won't show until they are approved

Comments

  • > How much of the books content is put into the documentation? From my point of view it would be great if all of it was, but obviously there are profitability concerns there.

    Any direct content transfer from the book to the online documentation is unlikely, as we don't hold the digital publishing rights ourselves unfortunately. That has nothing to do with profitability on our end, i"d be more than happy to creative-commons license the whole thing :)

    Posted by Ingo, 4 years ago

  • I generally spend time bouncing between forums and docs, plus looking through the code.

    I think a recipes area would facilitate most of queries in the forum, and I like the fact they are in a separate area to most of the docs.

    Often the API docs are non-existent so it would be great to get that back up and running.

    I agree that docs should be linked to a version, that has confused me a couple of times, however appreciate that it is probably more work.

    I'm not very experienced at documenting so am unable to offer much at this stage, but may be interested to contribute in the future once you've settled on a structure etc.

    How much of the books content is put into the documentation? From my point of view it would be great if all of it was, but obviously there are profitability concerns there.

    Posted by John Milmine, 4 years ago

  • Hey Jim, I think documentation on github aligns very well with our other source control plans. We originally planned to bundle docs alongside modules like sapphire in a new docs/ folder (thats the assumption of Will's sapphiredocs module). The goal was that every SilverStripe project would be able to generate its own documentation (incl. all installed modules). I'll have to talk to Sam how docs on git fits in the overall picture of making SilverStripe available on git, perhaps it makes more sense to have the documentation in a separate repo.

    The diem approach looks really interesting - would you be able to share your Markdown conversions of doc.ss.org and the proof of concept? Did you use pandoc for HTML->Markdown, or some converter for DokuWiki format?

    Posted by Ingo, 4 years ago

  • I am glad to see there is thought towards restructuring the documentation!

    My suggestions:
    After having used symfony ( http://symfony-project.org ) and Doctrine ( http://doctrine-project.org ), I have grown to love the way they handle documentation.

    It has similar concept to django's documentation and takes advantage of version control ( http://github.com/symfony/symfony1-docs ). The documentation is written in Markdown and each "branch" of the version control can represent a different project version and/or language that the documentation is written in.

    Using github promotes contribution since anyone can easily fork the documentation and make changes on their own and then can be merged back with the official branch.

    The next hard part is actually displaying the markdown documentation in a relevant user interface. Since symfony's documentation website itself is not open source, I have looked around for a way to easily display the markdown documentation that is kept in SVN/github/wherever.

    Enter diem-project.org. Diem project uses the exact same concepts to display their documentation (using Markdown, github) as seen here: http://diem-project.org/diem-5-1 .
    The great thing about diem project, is that the diem-project.org website is open source ( http://diem-project.org/diem-5-1/doc/en/howtos/open-source-projects ). After looking at the code, the web application synchronizes the documentation from github and handles the presentation of the code based on version (either 5.0 or 5.1) and by culture (English is the default, notice the "en" in the URL). The sidebar menu and the contextual links are key for making this documentation a great user experience.

    Notice though that they have a few different "sections" for the documentation: a "howto", "reference", and "tutorial", which covers several different styles of documentation that I believe SilverStripe is trying to achieve (for example "recipes" vs "getting started". Also note that diem-project has a concept of "snippets" ( http://diem-project.org/community/snippets ), which allows users to submit their own snippets of code (could be inspiration for community-added documentation or any kind of user-submitted markdown).

    As you can tell, I've been thinking about this for a while. A few months ago, I actually took portions of the SilverStripe dokuwiki and converted the html to markdown, split it up into relevant sections, and plugged it into a local installation of diem as a proof of concept and it worked very well. I would be more than willing to share the proof of concept publicly if there is enough interest for this idea.


    As for opensearch, I would suggest using Solr. I have noticed some SilverStripe related solr activity here: http://github.com/nyeholt/silverstripe-solr . Not really familiar with Sphinx, but I know that Solr is excellent with searching document based records (as opposed to normalized relational database records), which is the format that better matches the Markdown documentation and phpdoc API reference files.

    Posted by Jim Muir, 4 years ago

  • Great! It's good to see you guys moving in the right direction.

    Comments:

    1) Even one man in charge of docs and writing up some basic tuts on the Silverstripe will probably make a great impact.

    2) As noted in all other posts the search feature is awful. I'm pretty sure that you can get a quick and dirty fix for the search to make just simple relevant queries in the beginning.

    3) Building the docs on top of the API is also a good idea. Starting out with only the simple functions and classes.

    4) There are actually a lot of good explanations in the docs system that might be a tad outdated but still relevant. The only problem with those is that they are very hard to reach unless you really dig in.

    5) A good example of documentation sites are those of the most popular systems out there:
    wordpress.org
    jquery.com
    php.net (a bit less but also good)

    Keep up the good work!

    Viva la SilverStripe

    Posted by Yotam, 4 years ago

  • By the way, we've already got open.ss.org set up to trac documentation requests and patches: http://open.silverstripe.org/query?status=inprogress&status=new&status=replyneeded&status=reviewed&component=Documentation&order=priority&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component

    The existing tickets will need some review, they've collected a lot of dust on the shelves there...

    To file new tickets that you think are reasonably well described and actionable, use http://open.silverstripe.org/newticket?component=Documentation

    Posted by Ingo, 4 years ago

  • Hendrik: "The search feature used in all silverstripe areas (forum, doc, modules...) is very bad. I always got thousands of useless results, not sorted in any reasonable order. "

    There's only so much we can do to fix this - one thing I'd love to see is sphinxsearch.com for ss.org. In terms of DokuWiki (doc.ss.org) and Trac (open.ss.org) searches, our hands are tied unfortunately.

    Mo: " always assumed that the api docs were auto generated from http://svn.silverstripe.com/ repo. If that is not the case, could they be?"

    Yes they are, but the cronjob was broken for a loong long time. You can see the generation date at the bottom of each page on api.silverstripe.org

    Duncan: "Finally, from my point of view I suggest somebody puts together a basic how to for the more well used parts of the CMS."

    Yeah, the actual structure will be a challenge. I've started a wiki page to at least collect what could be improved/merged/purged: http://doc.silverstripe.org/tmp:documentation-restructuring - feel free to add to this.

    Pingu: "I think that initially, monitoring the forums is a pretty good way to make the existing documentation miles better, as this is where we developers turn to when we can't figure something out."

    We've actually tried this a while ago through Google Highly Open Participation Contest (trawl the forums to find documentation requests) - I think everybody in the community can to be more proactive about this (see wiki page mentioned above), its hard for us to keep tabs on frequency of questions etc.

    Would any of you be willing to invest some time and go through disqus comments on doc.ss.org?
    I think they point to a lot of deficiencies already, most of which can be fixed straight in the wiki (code examples not working, unclear descriptions, omissions). Any larger restructuring ideas could go on the mentioned wiki page. There's three ways in which you can trawl through disqus comments:
    1. http://doc.silverstripe.org/start?do=index - just look through every page
    2. Subscribe to the RSS feed: http://silverstripe-doc.disqus.com/c/13398/comments.rss
    3. I can provide a Disqus XML export with all comments (800KB)
    So, Hendrik, Duncan, Mo, Pingu, Keith - anybody keen to start?

    Posted by Ingo, 4 years ago

  • Hi Ingo,
    great to read this blog entry. Since I started using Silverstripe in 2009 there have always been moments when I cursed the lack in documentation.
    I can point out some things I just got into my mind:

    One very demotivating point are general questions in the silverstripe forum that have not been answered a long time by anybody. That makes a very strange impression about a project like silverstripe.

    The search feature used in all silverstripe areas (forum, doc, modules...) is very bad. I always got thousands of useless results, not sorted in any reasonable order.

    It is necessary to tell the reader of a documentation, for which version(s) the given explanations are valid.

    If the goal is to move parts of the documentation closer to the code, it would be wise to put some parts inside the code files, thats where its mostly needed.
    Description and instructions for usage for all (or at least the public) functions and classes.

    So much for now, greetings
    Hendrik

    Posted by Hendrik Schaper, 4 years ago

  • A step in the right direction. I imagine the poor documentation is the number one reason why new users abandon SS after deciding to adopt it.
    Im not a strong developer and nearly abandoned it but after reading the first book and much persistence with the online docs and forum I managed to learn just enough to get to grips with the basics.
    Google is your friend currently. If you search for "silverstripe setQuality" for example you get targeted help from all sources. If you try to find help via the SS website on that subject you will slowly go insane!
    Finally, from my point of view I suggest somebody puts together a basic how to for the more well used parts of the CMS. Eg. Making a page type with custom fields/variables. Creating a dataObject and displaying the results. The current tutorials and examples given are very high level and don't help users that are just cutting their teeth with object oriented PHP.
    I believe doing this would help increase the user base and take up of SS and hopefully boost interest and support for what is an excellent framework.

    Posted by Duncan Kendrick, 4 years ago

  • I still like how django's docs work, although I would like more of them if I am fair:

    http://docs.djangoproject.com/en/1.2/

    The main thing is that each version has its own docs portal, which keeps the docs separate and helps keep things a bit tidier. For example:

    http://docs.djangoproject.com/en/1.2/ - Version 1.2
    http://docs.djangoproject.com/en/1.1/ - Version 1.1
    http://docs.djangoproject.com/en/1.0/ - Version 1.0

    Also, it would be nicer if the docs used a wider template. It can get quite hard to read some of the current pages, especially if they have diagrams or large code blocks.

    I also think it would be nice if the "recipes" section of the Wiki was separated out at a higher level of the navigation. This seems to be be a popular section to post content to, and so possibly should be independent of the main 'docs' section.

    I always assumed that the api docs were auto generated from http://svn.silverstripe.com/ repo. If that is not the case, could they be? It seems most of the documentation you would need for what classes and methods do are there, and I generally use inline documentation most of the time these days (and I think a lot of other developers do too).

    Just me thoughts anyway, hope they are of some use.

    Mo

    Posted by Mo, 4 years ago

RSS feed for comments on this page | RSS feed for all comments

Want to know more about the company that brought you SilverStripe? Then check out SilverStripe.com

Comments on this website? Please give feedback.