Digital prototype iteration 2 (validation)

Introduction

This post describes the validation of the second iteration of our digital prototype. After the first iteration of the prototype, we caught some minor problems and suggested some solutions. In this iteration, we simply wanted to test whether those changes had the desired effect.

1. Method and set-up

The method and set-up are the same as the previous iteration, only we left the questionnaire behind, since the goal was testing whether the small changes had the desired effect. We executed some user interviews in the hall of CW, and let them do exactly the same tasks is in the previous iteration.

To conduct the interviews, we had our test subject sitting behind a computer in the hall of the department, carrying out tasks as instructed by a team member. A second team member took notes of the process, watching the screen and asking some general questions afterwards.
Our prototype is shown in the picture below.

2. Test subjects

As with the previous test, we used engineering students as test subjects, because they were the easiest to find and we are only testing the functionality in this iteration, not the efficiency. For the tests of our implementation, we hope to find a more varied test panel.

After two test subjects, the same conclusions came up twice, so we decided not to test any further.

3. Analysis and results

Firstly, we'll look back on the changes and problems that came out of the previous iteration. Secondly, we'll discuss new problems that arose in this iteration.

3.1 Review on the changes from previous iteration

Changing priority names
Naming the priorities high, normal, low instead of 1,2,3 completely solved the problems of test users not knowing which was the highest priority. None of the users had problems when we asked them to label a message with the highest priority.

Recovery from error

When the test users were asked to undo a delete operation (bring a message they sent to trash back), all the users still went looking in the trash to retrieve the message. When we asked them if they considered using the ‘undo’-button, they indicated they would have used it if they had seen it. To solve this problem we will increase the size of the undo-image en type the words ‘UNDO’ next to it.

Shortcuts

We included in our digital prototype the functionality that you can start a search operation by pressing enter after you entered the search term. All test user indeed used the enter button to start their search, so this is definitely an improvement.

As for the delete button, Axure didn’t give the possibility to implement a removal of messages on delete. Since none of the test users tried to use the delete button to remove any of the messages, this functionality wasn’t really missed. Of course it wouldn’t hurt adding this functionality, but it’s not indispensable.

Read messages

We tested whether it was clear to the user that read messages change colour by showing them a list of messages and asking which of them had already been read. It was clear to all test users that the least bright messages were the one that had already been read.

Advanced search revisited

The advanced search is now not automatically shown once you search for a term, in this was an improvement. The uncertainty of whether or the test users should press search again disappeared.

The advanced search panel itself now looked like in the figure below. It was a lot clearer for the test users how it worked now. The fact that the source selection is now done on the right bar where it always happens, wasn’t clear for all test users, but once we explained it, they could see why it was logical to place it there. For the rest everything in the advanced search panel was clear for the test users in this iteration. The advanced search panel is shown below.

Where am I?

When we asked people after a certain task where they were in the application, they were able to answer us right away. When we asked they how they knew where they were, they told us they used the breadcrumbs on the top bar. One test user did remark that the breadcrumbs may be a bit larger. We actually agreed so we decided to make it a bit larger so people will definitely know where in the application they are.

4.2 New problems and solutions

The unclearness of the next week, next month and later tabs

We came across one new problem in the user test. It wasn’t clear for the users what time period we meant with the next week, next month and later tabs. We already figured out ourselves that this could create problems for the users, so we specifically asked them what time period they thought the different tabs represented. They indicated that these time periods were indeed confusing. We asked them whether it would be more clear when we would name the tabs “>1week”, “>2weeks” and “>month”. They told that this would be more clear, but this is one of the we would check in a following iteration, if we had the time.

4. Conclusion

This second iteration with our digital prototype seems to prove that, aside from some very small remarks, we're on the right track for the functionality of our application. Of course the efficiency still needs to be tested with our implementation. This will be done in the following week. The problems of the previous iteration have largely been solved. For the one newly discovered problem we suggested a solution to the test users and they seemed to agree with it. Of course, to be absolutely sure we’d have to do a third iteration of the digital prototype, but we prefer testing our implementation to check the efficiency of our application.