Paper prototyping: the next iteration(s)

As explained in our previous post, we changed concept from Rankr to Featr.
To test the interface with actual users, we created a paper prototype. Such a prototype allows to get fast feedback and incorporate changes rapidly, because all you have to do is draw a new design. We'll now discuss our first iteration and evaluation.

Our original main screen looked is presented below for easy comparison.

Of course, we also included some additional menus etc. The workflow for our application is described in our previous post.

1. Method

   The very purpose of a paper prototype seems to give explicit feedback on the user interface using the prototype as a means of efficient and concrete communication. The user can pinpoint very specific issues. Performing tasks with the mock-up shows what is perceived as difficult and where the user's mental model at first diverges from the one the the developer has in mind.
   
   For that reason, we chose to evaluate by means of user interviews, asking people to perform several tasks with our model. Initially, we briefly explained the concept of our application and the purpose of the prototype. We encouraged the user to share any comments and thought they might have while working with the application.
   In most cases, two interviewers where present when conducting an interview. One took on the role of 'computer': he simulated the behaviour of the program when the user performed an action. The other took notes of what the user did and said, after a task had been given.
   Such an interview mostly took about half an hour to complete. At the end of each session, we asked the users to give a general qualitative assessment and remarks about some weaknesses that were especially prominent.
   
   As said before, this method was the best match for our needs. It came closest to a real user experience because it involves the user directly. A questionnaire is useful to give a general assessment, but seems (generally spoken) incapable of directly pointing out conceptual or interface-related problems since it is a closed approach. In this early development stage, it's precisely the open, direct, flexible and concrete approach offered by (structured) interviews we were after.
   



2. Carrying out the evaluation

   Test users
   We were lucky to have test users who experience the needs we want to address with this web application namely, information overload and how to deal with it. Professor Duval was so kind to forward our call for testers to his project people. We eventually got 4 responses. Three of these people were subjected to an interview. Additionally, one master student took part as well.
   
   To clarify, they mostly don't experience the need as extreme as presented in class.
   As one of our test persons told earlier in class, although his needs are higher than the average home user, his communication stream is far smaller than that of e.g. professor Duval's. This could be a reason why speed or efficiency was hardly ever mentioned. Most user remarks concerned effectiveness, clarity and consistency.
   
   Set-up and tasks
   All tests took place in the hall of the computer science department of the KU Leuven. The user was sitting at a table with the prototype in front of him. We sat at the other side of the table, taking note and simulating the program.
   The fact we use a (paper) prototype is in itself a barrier to perceive the situation as real, a perception that might even be stronger because of the presence of interviewers. Yet, the environment (with lots of students around) and the working atmosphere might contribute to a sense of 'being at work' for the test users (which in fact they were, albeit not in their office). Furthermore, since they were familiar with the process of paper prototyping, this was not such a big issue. issue.
   
   We used following tasks:
   
   • File some messages for tomorrow at work
   • File a message for Monday next week
   • File a message for Monday in two weeks
   • Move a message to an archive
   • Check messages to be checked at work today
   • Remove 2 messages
   • Open a message
   • Move a message  to tomorrow at work
   • Prioritize a message
   • Check only facebook messages
   • Search messages with a specific word
   • select all messages
   • Log out
     



3. Analysis and results

   The written comments we gathered were analysed after the tests had been carried out. This too, was done qualitatively, since remarks were qualitative as well. The discussion below highlights some of the problems we encountered and the solutions we incorporated in our next model. These include extra features, conceptual changes and some graphical additions .
   
   Obscurity and inconsistency of the 'New' and 'Processed' mode
   All people had troubles with the two modes concept. No one really grasped the concept of the 'New' and 'Processed' tab at first glance. Therefore, we changed the name of the latter to 'To Do', since this really describes what it is about. As a related issue, people tried to check messages afterwards without changing mode. This was an inconsistency in our design: the side bar behaves differently in 'New' and 'Processed/ToDo' mode. We corrected this - if no messages are selected, the bar behaves as a simple browser, redirecting the user to the the ToDo mode if necessary. Otherwise, it moves the messages, remaining in the same mode (indicated by the tab at the top).
   
   Reusing the week days asks for another ordering in the pane
   Filing a message posed problems. Next week Monday was actually the Monday on top of the week days, (reusing the day that had already passed this week for next week) while Monday in two weeks should be filed at 'Week+'. The participants used 'Week+' for both. Guided by their remarks, we changed the ordering of the week days to put the current one at the top. We also changed 'Week+' to 'Later'.
   
   The need for support functionality
   When people were unable to perform an action, they would like to find some more information. Therefore, we included a help function, which was not a feature in the original version.
   
   The unclarity of the search scope
   The searching posed problems. People didn't know the scope. We clarified this by allowing search directly from the bar to have local scope, whereas the appearance of an advanced search bar allows users to specify the scope (and other options) themselves - a system similar to many existing applications.
   
   Opening and closing a message: what happens next?
   Opening a messages happens by double-clicking on it. All users figured this out easily. They new from earlier tasks that a single click only selected a message. So far so good, but what happens if you close the message without processing it? At first we just left it in the overview as it was before, which was confusing to users. We changed this to a classical mailbox approach: messages change colour (become lighter) once read.
   
   Ordering and grouping messages
   Originally, all messages were sorted chronologically. Some users expressed the need to use other criteria (e.g. would like to process all messages from a specific sender first). We included several orderings to deal with that. They also wanted to group messages in their task lists. We included this functionality using tags.
   
   Difficulties with tagging and prioritizing
   Tagging or prioritizing was not an easy task. Originally, we had the idea of dragging a message to the top to prioritize it (in ToDo mode, priority was the ordering criterion). We facilitated the concept and integrated it with the tags with a bar at the bottom of the screen. It emerges only when messages are selected and allows for three priority levels and tagging.
   
   Icons to clarify tasks
   Although Trash and Archive were quite clear to everyone, we included icons as well. This also serves to remind people of the desktop nature of the application (allowing for dragging, box select, double clicking etc...)
   



4. Conclusion and plans

   As you notice, a lot of changes were incorporated based on user remarks. Yet, one question remains. We got almost no remarks about efficiency of our approach. People focused on whether something was easy or possible, not whether it was fast. However, this really is the issue with information overload, so we'll have to ask explicitly in the future. Also, we don't know yet whether our changes are successful (though some of them were already included in the latest test). Up to the next iteration...
   
   The next iteration will be similar to this one, using the paper prototype. We will conduct (hopefully) more user interviews with the same tasks, prompting the test users for comments and focusing a little more on the efficiency of the design.
   If we succeed in conducting more interviews, the analysis could be done more quantitatively to assess the importance of a user opinion. How many users thought this as well? This is the main difference with the first iteration, in which we tested the concept and changed from a crude to a more workable version.
   After our second analysis, we will judge whether the prototype is stable enough to start a second phase: the digital prototype.
   

To conclude, eventually, our interface looks like this now