First Iteration Test Findings

After completing our user testing of the first prototype the next step was to evaluate these findings to identify where improvements could be made. This involved reviewing the observational videos in detail, collating our individual notes and coming to an agreement as to where improvements could be best made.

Observational Testing Summary

As part of the class for Robert Griffin we were asked to summarise two of the user observational videos into one video no longer than 15 minutes. The video below is  a summary of User #4 and User #5 observational interviews with my own observational notes included.

Key Findings

What worked

Feature Description User Need/Goal
General flow Most users had only minor issues with completing tasks and were able to progress to the order completion page with ease Site Architecture. Homepage layout refinement. Responsiveness
Ticket selection Ticket selection was great improved from he original website with little if any issues Improved booking process
Accordion The accordion worked across mobile and desktop Improved booking process
Final payment  People found it useful and easy to use Information Feedback

Table:1 Successful features

There was a marked improvement in ticket, seat and food selection. The task of selecting a time was made simpler by presenting only times for the selected day first and all possible times as buttons; this was an improvement on the current site which presented all days and times in table format.

Ticket selection was made clearer through segregating. Users appeared to be able to clearly identify and book tickets while also performing the task of adding food to their bookings with ease.

The accordion option which allowed participants see all stages of the booking selection process on the one screen, created a sense of trust as participants could easily move back and forward through their preferred options without the need to reload pages.

The final payment screen also seemed to be very successful as all of the test participants can be heard reviewing their order, the lack of which in the current site was something which was of great annoyance. Again this added to the sense of trust when using the prototype.

The videos show that users were able to perform the core tasks of booking tickets and food in a reasonable length of time, on average around five minutes, with a considerably lower problem and error rate than original website. Users progressed through each stage of the booking process with very little confusion, and were able to complete the various elements of the scenario with ease.

What could be improved

Feature Description  User Need/Goal
Filter Most if not all users did not see the filter Site Architecture
Which ticket is right. Users failed to see this  could have been greatly improved through being more prominent Information Feedback
Swipe Gesture The swipe up gesture on mobile to access the booking summary conflicted with the swipe up Apple OS gesture Improved booking process
Accordion Arrows on accordion that clear as to what they indicating Information Feedback
Account login Not all users recognised this icon Information Feedback
Seat selection Pre-selecting seats confused some users Information Feedback

Table:2 Features for improvement

To begin with, there were issues did occur was on the homepage of the site. This was mainly down to us not being able to enable all elements of the prototype. Users expected certain areas of the prototype to respond to interactions, but due to time constraints, it was not possible to enable all of these options. This created a very narrow path for participants to follow, although most did seem to follow it automatically. Once users tried to interact with non-active elements it caused a large amount of confusion, as can be seen with User 04 in the video.

Some features were completely unnoticed by the participants such as the search function and the movie filter feature on the “Whats on..” page.  For a number of participants, the lack of informational feedback from the prototype was an issue, with the issue scoring the highest amongst of all of those recorded. In some cases such as the information on tickets, users failed to see it, in other instances we failed to provide enough feedback to users such as when seats were pre-selected.

On mobile, the biggest issue was the swipe up gesture for the booking summary. This interaction was to close to the swipe up on Apple OS which launches the control panel.

For a very small amount of users the account login icon had no meaning and they failed to complete the task associated with it.


Categorisation matrix  Prototype Results


Issue 1.1 1.2 1.3 1.4 1.5 1.6 1.7 2.1 2.2 3.1 3.2
a 3 3 1 2 1 4
b 1 1 2 1 3
c 1 2 1 1 1 1
d 1 2 1 1
e 1
f 1 1
g 1 2 4 1 2 3
h 1 1 1 1 2
Totals 7 4 0 6 5 5 2 4 3 5 15

Table:3 Categorisation matrix prototype results

Interface problems 
1.1 Visual evidence of confusion about the interface
1.2 Verbal evidence of confusion about the interface
1.3 Visual evidence of confusion about the interface through misleading information
1.4 Visual or Verbal evidence of confusion about the interface due to user misunderstanding functionality
1.5 Visual evidence of user selecting invalid option due to misleading or poor navigational information
1.6 Visual or Verbal evidence of expected functionality missing
1.7 Visual or Verbal evidence of user selecting option unintentionally
Informational Problems
2.1 Verbal evidence of confusion about the information displayed
2.2 Verbal evidence of user being blocked from progressing through missing or hidden information
Interaction Problems
3.1 Verbal evidence of user being confused due to negative or invalid website feedback
3.2 Visual evidence of user being confused due to poor website feedback

Table:4 Categorisation matrix schema

Categorisation matrix comparison

Although not the absolute truth due the limited time we had to carry out the overall project the comparing the totals of the two categorisation matrices does show  marked drop in the number of issues.


Original Site Totals 21 15 5 7 8 3 3 23 8 7 3
Prototype Totals 7 4 0 6 5 5 2 4 3 5 15

Table:5 matrix comparison results

Leave a Reply

Your email address will not be published.

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑