This blog post contains the second iteration of our paper prototype. For this iteration we evaluated the changes we discussed in the first iteration to see whether they’ve caused the desired effect.
1. Method
As in the first iteration, we did some user tests with the paper prototyping method. The purpose this time was to test whether the changes after the first iteration leaded to the desired improvements.
We combined the paper prototyping with a little survey we conducted after each user interview. That way we hoped to get an honest opinion of the test users regarding the usefulness of the application and some design choices we made, whereas only paper prototyping doesn’t really investigates this. The survey contained some general questions concerning the user-friendliness and usefulness of the application as well as some design choices. We added the design choice questions to check whether some design decisions we had some doubts about where indeed right.
The usefulness and user-friendliness questions came from the CSUQ questionnaire.We choose this questionnaire because it is not particularly long and really focuses on comfort of use and efficiency of the interface. We've dropped some questions, for example those on error handling, because they didn't really apply to our prototype and we didn’t want to browbeat our test users, as we also wanted to ask some questions concerning design choices. The complete survey can be found in our previous blog post.
2. Test persons
We had some problems finding test persons. We did a call for help on Twitter but didn’t receive any response, even though professor Duval was kind enough to re-tweet it to all of his followers. Finally we were glad found 3 master students (2 master in computer science and 1 master literature) and 1 recently graduated ICT’er who wanted to help us. All of them are in there early twenties and have different sources of information.
3. Set up
Most tests where executed at the computer science department with one of us acting as computer and one of us as interviewer. The computers job was to make the paper prototype react as if it was implemented on a computer. The interviewers job was to delegate the tasks to the test user and write down what the test user says en does. The user tasks where the same as for the first iteration, as we wanted to see whether the changes after that iteration have the desired impact. The tasks:
- File some messages for tomorrow at work
- File a message for Monday next week
- File a message for Monday in two weeks
- Move a message to an archive folder
- Check messages to be checked at work today
- Remove 2 messages
- Open a message
- Move a message to tomorrow at work
- Prioritize a message
- Check only facebook messages
- Search messages with a specific word
- Select all messages
- Log out
Our paper prototype we conducted the usability tests with for this iteration looked like this:
After the user interview the interviewer conducted the survey described above.
4. Analysis methodology and Results
As a first part of our analysis, we looked back at the problems the users experienced in the previous iteration and the impact of the changes we made. Secondly we introduce some new changes related to the newly discovered issues. Finally we have a look at the answers on our questionnaire.
4.1 Retrospection on the previous iteration
We will now go through the effect of the changes we incorporated in our design as response to remarks of the users in the previous iteration.
Obscurity and inconsistency of the 'New' and 'Processed' mode
By changing the behaviour of the left side panel as to make it more consistent in the two modes, the problem was solved. Now, when a user clicked on one of the labels when a message was selected, the message was moved, as before. In contrast, when nothing was selected, the user was forwarded to the ‘ToDo’ mode automatically, where the panel was used in a similar way. No-one experienced problems anymore. However, no-one really noticed the change of mode either. This rightly ask the question whether these ‘modes’ are necessary at all. We’ll come back to this in the next part where we discuss changes for the next iteration.
Reusing the week days asks for another ordering in the pane
Changing the ‘Week+’ label to ‘Later’ was effective. Everyone immediately understood that this label referred to messages that they intended to read in more than a week from now.
Reordering of the week days turned out to be more clear as well. Previously, the fact that weekdays could be reused was somewhat unclear since they were listed in the classical week order. Now, the days were listed in the order they would occur in the coming week. One user wanted to use the ‘Later’ label initially when we asked to file a message for next week. However, when only briefly looking at the interface, she understood days could be reused.
The need for support functionality
This problem didn’t come up again. No one really asked for support since all functionality appeared clear. Yet, we do believe it is always useful to have a central place to find information. We intend to explicitly ask the users about this in the next iteration.
The unclarity of the search scope
When we asked users to search for a specific message, they understood they could use the advanced search. Yet, the terminology for choosing the scope of the search (a dropdown list with the label ‘Where’) was unclear.
Difficulties with tagging and prioritizing
The prioritizing of messages were solved by showing a pop-up bar at the bottom of the screen when a message was selected. Users were given the option to prioritize a message using three priority levels. They can also tag one message for messages for grouping purposes.
Ordering and grouping messages
The systems we introduced for sorting and grouping were grasped easily by all users. The concept of tagging a message as a way to group is familiar to them. The sort functionality’s location on the screen was apparent.
Icons to clarify tasks
We use icons on several places in our application. As a means to indicate the source of the message this was greatly appreciated by all test users. On the other hand although they liked the ‘Trash’, ‘Archive’ and ‘Spam’ icons they had doubts to the usefulness of the calender icon for week days.
4.2 Analysis of new problems and possible solutions
‘New’ and ‘ToDo’ tabs are irrelevantAs we mentioned in our discussion of the solution for the left side panel users hardly ever noticed the mode transition which renders tabs useless. Therefore we decided to drop them. We will introduce a ‘New’ label in the left side bar. Because the modes are no longer used the number of messages filed for certain day will be indicated all the time instead of only in ‘ToDo’ mode.
Insufficiency of time categories
When explicitly asked about it users expressed the need for more categories. Before if user files a message for later it would reappear in the ‘New’ stream one week later. We will expand this in the next iteration. ‘Next Week’ and ‘Next Month’ allow a message to be reintroduced next week or next month respectively. In this new view ‘Later’ refers to anything later than one month. Using this system reduces the users’ workload since they don’t have to postpone their messages again and again every week.
Sorting by priority
In our previous prototype we assumed messages in ‘ToDo’ mode would always be sorted by priority. Because it wasn’t always clear we decided to explicitate this by adding it as an option in the ‘sort by’ section.
Missed messages
When a user fails to read message on the day she/he intended these messages will be moved to a special ‘Missed’ category. It will be coloured red when any messages have been missed. As with the other categories the number of messages will be shown.
Source selection
To clarify the source of messages for filtering we will replace the names by icons. This stems from remarks we got and the effectiveness of the same icons on the messages. Several users experienced difficulties when asked to filter by source. Instead of deselecting all sources to be filtered away (by using ‘clear all’) they tried to achieve it by simply clicking the requested source. We are not really sure about good alternatives. One possibility is to achieve the intended effect by double clicking the source. Of course it has the advantage of directness but is rather uncommon in web applications. The alternative is leaving the functionality unaltered which requires the user to do perform two actions (clicking ‘clear all’ and the source). Since the former solution also allows the second strategy we will include it as an additional feature.
Changes to the search functionality
As mentioned earlier the term ‘where’ wasn’t associated with the scope of the search. We will replace it by ‘Search in’.
Another problem we encountered was the inability to erase the search filter. Thus we will add ‘clear search’ button.
Where am I?
When reading messages it can be difficult to remember which category you’re currently in. A solution to this is the highlighting the current category’s label and adding a small subtitle to indicate the specific section (eg. ‘At Work’).
Reorganizing the labels
As apparent from the changes previously described a lot of labels will be added. This asks for restructuring. As a result we intend to use the bottom of the screen for labels as well. The left bar will contain the ‘New’ and ‘Missed’ labels and all the days of the week. The bottom bar will host ‘Next Week’, ‘Next Month’, ‘Later’, ’Trash’, ’Spam’ and ‘Archive’. For now all labels will have equal size. In the next iteration users will be asked their opinions about which ones will be used more often and should be larger.
4.3 User questions
As described above, we also conducted a survey to get additional information. Especially the positive and negative points the test users summed up and their opinions on design decisions we made were very usefull. The questionnaire about usefulness and user-friendliness was less helpful because for that kind of questionnaire you actually need more test users.
The negative points the test users summed up where handled in the previous section. The positive point most users gave was that it was handy to handle all your information streams in one application.
The results of the usefulness/user-friendliness questions can be found in the graph below. The response to these questions was mainly positive. Overall test users had the impression the application could be helpfull and was more or less user-friendly.
The design choice questions gave the following insights. The priority bar which appears on the bottom of the screen when a message is selected was a good choice for most of the test users. One guy did suggest we might want to let the bar appear near the message which is selected. This will be considered, but it may cause problems when more than 1 message is selected. ToDo wasn’t a good name for most of the test users, but this problem is handled by making that tab obsolete. The number and size of the messages was good, although one test user remarked that you might want to make this dependent on the resolution of the screen. We will think about this. The icons did offer an added value so we will definitely keep using them. The time-categories where clear for the test users, although some suggested to add a ‘next week’ and ‘next month’-category. The fact that the application behaves like a desktop (drag-and-drop) was clear for the test users.
5.Conclusion
As is clear from the evaluation, several issues remain unsolved, leading to as many changes in our prototype. Yet, we will continue our evaluation iterations as planned and proceed to the implementation of the digital prototype.
This is based on several observations. First of all, the main concept and layout are clear to the user. Although several aspects of the interface are still under construction, we also start to come upon issues related to the look of the actual interface, such as colour. Secondly, if we look at the questionnaires (although we have only few), we can see the responses are rather positive. This indicates a level of satisfaction, the fact that the desired functionality and effectiveness are possible given the interface we have now. To finetune the design, we will proceed digitally from now on.
Yet, some issues were raised in this post. We will certainly ask the user the following questions in the next iteration:
- The need for ‘help’ functionality
- The method for source selection
- The new positioning of the category labels
- Are some labels used more often, should they be bigger?
We updated our paper prototype one last time to have a clear sight on what we wanted to build in our first digital iteration. The result is shown below.
No comments:
Post a Comment