One month ago, Senior Project Manager Julian Meadow (aka J), along with colleagues Paul Clarke and Ryan O'Hara, usability tested SilverStripe 3.0 beta 2 with a number of clients around Wellington, to better understand if the user interface is intuitive or if there are areas that could be improved. I sat down with J to hear more about the long and winding road that is usability testing.
After more staff within SilverStripe became involved with the development of SilverStripe 3.0 and started playing with the early beta releases, we realised there were a number of usability issues that needed to be addressed.
I’d encourage anyone building something online that has some form of complexity or workflow to do some usability testing, even if it’s just showing it to a couple of other colleagues and getting their feedback.
We tested the main sections of the back-end interface, the main parts a web administrator would use on a daily basis. We did this by preparing a list of typical tasks, including adding a new user, adding and organising files, creating, editing and publishing a new page, rearranging and deleting pages.
We decided to only test with users that had previous experience in using SilverStripe, and to test at their desk where possible. Two of us from SilverStripe attended each test session, one to run through the tasks with the participant, the other to observe and take notes.
After the testing sessions were completed we (Paul, Ryan and myself, i.e. two designers and a PM with usability experience) reviewed all the results, agreed and grouped our findings and presented these to key staff working on the development of SilverStripe 3.0. During this presentation we made a point of not discussing solutions and instead focussed on agreeing on the main findings i.e. agreeing on areas that needed to be addressed because participants found them confusing or hard to use.
We (again Paul, Ryan and myself, plus Sean for technical input) then worked on developing a number of recommendations. Efforts were focussed on suggestions of how to fix the usability issues raised. We presented these and together agreed on the resolutions and order in which they should be fixed i.e. which ones to fix prior to a full stable release (in beta 3) versus which ones to fix later in version 3.1. Prioritisation was based on a number of factors, including technical complexity and user impact.
Initially we tested with four individuals. We were prepared to test with more participants if needed (typically up to six participants is adequate), but found after testing with four we had collected enough feedback to work with.
The results followed the 80/20 rule, 80% of the issues were common and reported (reinforced) by all participants, the other 20% of the issues were scattered and more specific.
We were surprised how consistently all the test participants tried to use the search user fields to create a new user. A number of factors combined to confuse the participants; bad button labelling and positioning, confusing search boxes and icons. As a result we’re going to rename the “Add New” button to “Add User” and move it from the far right to the left of the screen (following the original designs), plus we’re going to tidy up the user search interface.
Also, none of the participants could log out. One user spent two minutes looking at the interface and finally had to give up. Noone understood that the power icon was the logout button.
Participants were confused by the filters pull-out, most of the time it’s there, but when editing a page it gets replaced by the site-tree.
The biggest finding for us was that all participants felt they weren’t getting enough feedback after completing an action, to confirm it had been done. Examples include participants uploading the same file multiple times because they didn’t get confirmation each time they completed the task and participants were unsure if their draft page had been saved.
On the positive side, all participants liked the new and improved look of the SilverStripe 3 back-end interface.
We’re looking at a number of ways to address the issues we found. Most changes are small cosmetic things, like changing button labels, using less ambiguous and more familiar icons, and changing positioning so it’s more intuitive with the task. We’re revamping the actions buttons at the bottom of the page editing area, improving labels and tips on the site-tree, improving options to drag’n-drop and multi-select, and a bunch of other stuff to make the user interface more user friendly.
We’re presently resolving the high priority issues (due in the beta 3 release) and will then work on resolving the other outstanding issues, but will most likely hold-off introducing these until release version 3.1
Yes, we’re planning to usability test beta 3 after it’s been released, to check the issues we’ve resolved have improved the SilverStripe 3.0 interface and to explore some of the ideas we’re proposing to develop for later releases.