The task-based observational research was undertaken by all members of the team, with the goal of identifying pain points for users when using the Movies at Dundrum website. These were categorised and converted into experience maps so as to identify user goals. Each member of the team carried out at least two separate observations
Preparation
To begin with, we created a set of test scenarios which we felt would best reflect the tasks that most users undertook on a cinema website, based on the data the number one task was clearly finding and booking tickets, we felt it would be useful to include a task of purchasing food, for two reasons. One being the current “Movies @ Dundrum” website already provided this functionality and two, the quantitative survey data showed that users would be likely to buy food online if the option was available
We created a simple usability test plan with two scenarios this can be viewed here.
The Actual Tests
Each of us tested two to four users. In all cases screen capture, audio and notes where used as well as video in some cases. The videos for my own personal tests covered four users using a mixture of laptops and tablet devices and are as follows
User #1
| Subject: | User #1 |
| Age: | 64 |
| Subject IT literacy: | Poor |
| Date: | 02/12/2017 |
| Time: | 17:26 |
| Interviewer: | Gavin Stokes |
| Space: | Seated at a desk |
| Device: | Laptop, Screen resolution 1600 x 1024 |
| Task: | Book two tickets for a showing of “Murder on the Orient Express” for the 9th of September |
| Recording devices: | Video, audio, written note |
| Full Transcript: | Google Doc |
| Notes: | -User finds home confusing and scrolls a lot -Filter button causes confusion as the content is reloading below it is there in association -Quite a high level of confusion -Page reloads cause confusion -Interactive items cause issue as high-lite is same colour as page -Mistake breadcrumb nav for wizard/next step -Cant locate where ticket selection is -Ticket selection is confusing as it has 0-10 with no instruction as to what it means -Seat selection proceed button is confusing, is labelled as select seats which user sees as instruction |
Table:1 User #1 interview notes
User #1 had many issues with the overall site. Some, but not all of the issues are discussed here, for a detailed summary of the issues see the categorisation matrix at the bottom of the page.
Page reloads caused a large amount of confusion, in one case, in particular, the user could not make the connection between the filter functionality on the home page, and the reloading of the movies below the filter. This was probably due to a number of reasons, the filter functionality highlighted in (fig 2) in red, is not clearly separated from he the image carousel above it, which causes the user to assume the filter affects this instead. This poor mapping of controls as pointed out by Cooper et al. (2014) can cause the user to have to stop and think increasing cognitive load and the level of confusion. The second possibility is that the filter results, actually load below the filter option and off screen after a page load.
Another issue was the poor labelling of controls and the failure to supply information to the user as to their function. For User #1 who has little to no experience of what might be considered established norms the following two features caused a large amount of confusion.
The breadcrumb in (fig 3) was mistakenly taken to be a form of navigation which could progress the user to the next stage, there is no information explaining what this does, such as a “You are here” indicator.
The user can be seen at time stamp 8:11 trying to use this when the content on the page becomes confusing due to a large advert on the page which hides the next required steps off-screen. She eventually selects the login to account button in the hope of finding a payment option.
There was a secondary observation here from the testers perspective. I took my eyes of the user to make note of the confusion, upon returning my attention to the user, I found they had somehow landed on the account login page, always have two testers if possible.
The ‘Choose These Seats’ button in fig 4 caused confusion as the user felt it was asking her to perform the task of choosing seats which she had just performed, time stamp 15:29. Upon trying to the move to the next step in the process the user drew a blank as to what to do next. This led her to try the breadcrumb function, and various other options before having to be promoted. A secondary issue with the seating selection was that certain seating arrangements appeared of screen with a large amount of white space above them, essentially causing them to not appear, time stamp 14:08.
User #2
| Subject: | User #2 |
| Age: | 68 |
| Subject IT literacy: | Beginner |
| Date: | 02/12/2017 |
| Time: | 08:26 |
| Interviewer: | Gavin Stokes |
| Space: | Seated at kitchen table |
| Device: | Laptop, Screen resolution 1600 x 1024 |
| Task: | Book two tickets for a showing of “Murder on the Orient Express” for the 9th of September |
| Recording devices: | Video, audio, written note |
| Full Transcript: | Google Doc |
| Notes: | -Immediately tries to locate a find or search option but cant see the one on the home page as he has landed on an interior page. -Back image (curtain image) has a book now button which is only an image -Cant locate search option on homepage -Mezz title causes confusion as he cant understand what it is |
Table:2 User #2 interview notes
Key Issues
Entering the site via google and not landing on the homepage causes some confusion as the user is looking for a search option, which is only available no the homepage. There is book tickets now button which is actually part of a background and performs no function. After clicking this (time stamp 1:04) the user then feels its necessary to go back to google to access the homepage.
On returning to the site the user misses the filter feature on the home page and scrolls down the site to find the desired movie and selects the wrong date from the predefined options. This then loads the movie page for that date whereupon there is no option to change the date (time stamp 2:46). The user proceeds through the booking process in the hope there is an option to change the date later. The user was then prompted to start over. After starting over a second time there is an option to select the date which he then finds.
The booking process from here on goes relatively smoothly with a few questions around types of tickets.
(For detailed summary of the issues see the categorisation matrix )
User #3
| Subject: | User #3 |
| Age: | 39 |
| Subject IT literacy: | Average |
| Date: | 02/12/2017 |
| Time: | 11:48 |
| Interviewer: | Gavin Stokes |
| Space: | Seated at kitchen table |
| Device: | iPad mini |
| Task: | Book four kids tickets and one adult tickets. Add food for each child. For any movie and time you want |
| Recording devices: | Video, audio, written note |
| Full Transcript: | No transcript available due to time constraints |
| Notes: | -User uses the title of the movie on the homepage to access times -Large food advert is seen as clickable but inst. |
Table:3 User #3 interview notes
Key Issues
The first issue User #3 encounters is the advert that appears on the ticket page (fig 3). The user understands this to be clickable, upon realising it is not she scrolls down and proceeds with the process. The ticket options presented are confusing for the user and she is unsure which option to select. The naming conventions are not descriptive and only add to the confusion “Large ticket and combo meal”, “Medium ticket and combo meal” (time stamp 3:10). There appears to be no kids options. The User is eventually forced to select two medium options but is not happy with the selection and returns to the ticket options.
On the tickets screen the advert shows a kids deal option but it is is not available under the list of options for tickets and food (time stamp 5:45).
The user returns to the homepage and tries a few different options to see if she can find a booking option for four kids tickets and and 1 adult with food but cant find anything. She restarts the booking process again and after selecting a different date she is presented with the advertised meal deal. It appears that the kids deals are not available for some reason when booking on a Friday.
(For detailed summary of the issues see the categorisation matrix )
User #4
| Subject: | User #4 |
| Age: | 40 |
| Subject IT literacy: | Average |
| Date: | 02/12/2017 |
| Time: | 2:16 (Video was cut short due to battery) |
| Interviewer: | Gavin Stokes |
| Space: | Seated at kitchen table |
| Device: | iPad mini |
| Task: | Book four kids tickets and one adult tickets. Add food for each child. For any movie and time you want |
| Recording devices: | Video, audio, written note |
| Full Transcript: | No transcript available due to time constraints |
| Notes: | -User uses the title of the movie on the homepage to access times -Large food advert is seen as clickable but inst. |
Table:4 User #4 interview notes
Key Issues
The user had no real issues with completing the task of booking tickets. He did fail to complete the task of booking tickets with food. This was due to confusion over the advertisement on the tickets page. The user had assumed that the family deal included food, and was not aware of this even after completing the booking.
The lack of payment review screen makes if difficult it not impossible to see what has booked as only a final total is presented. If the user had of add this review option they would probably have not completed the booking.
(For detailed summary of the issues see the categorisation matrix )
User testing Categorisation matrix
A total of eight participants took part in the observational study of the ‘Movies @ Dundrum’. From these a total of 10 tests where carried performing a series of tasks mainly centred around the booking process, with or without the the addition of food.
Upon completion of the video observational work, transcripts where created for some of the videos but due to time contrarians this was not possible for all. We then proceeded to categorise the data from these interviews using a categorisation scheme. As suggested by Rogers, Preece and Sharp (2015), we allowed the categorisation scheme to be suggested by the data. Once we had established the scheme using one video we then proceeded to see if it was valid by applying it to the other video observations in the study and allowing other members of the group apply it independently.
Although not 100% accurate and even though it does not identify exact issues, it did give us an excellent starting point in identifying and cataloguing issues with ‘Movies @ Dundrum’. And when later combined with a user experience map helped us in identifying key pain points and goals.
| Participant | 1.1 | 1.2 | 1.3 | 1.4 | 1.5 | 1.6 | 1.7 | 2.1 | 2.2 | 3.1 | 3.2 |
| A | 3 | 4 | 0 | 1 | 2 | 2 | 0 | 2 | 1 | 0 | 0 |
| B | 7 | 2 | 2 | 5 | 2 | 0 | 1 | 2 | 5 | 3 | 1 |
| C1 | 2 | 4 | 3 | 1 | 2 | 0 | 1 | 6 | 1 | 1 | 1 |
| C2 | 2 | 2 | |||||||||
| D1 | 2 | 7 | |||||||||
| D2 | 1 | 1 | 1 | 1 | 3 | ||||||
| E | 3 | 2 | 1 | 3 | 1 | 1 | 1 | ||||
| E2 | 1 | 1 | |||||||||
| F | 1 | 1 | |||||||||
| Totals | 21 | 15 | 5 | 7 | 8 | 3 | 3 | 23 | 8 | 7 | 3 |
Table:5 Categorisation matrix results
| — | Interface problems |
| 1.1 | Visual evidence of confusion about the interface |
| 1.2 | Verbal evidence of confusion about the interface |
| 1.3 | Visual evidence of confusion about the interface through misleading information |
| 1.4 | Visual or Verbal evidence of confusion about the interface due to user misunderstanding functionality |
| 1.5 | Visual evidence of user selecting invalid option due to misleading or poor navigational information |
| 1.6 | Visual or Verbal evidence of expected functionality missing |
| 1.7 | Visual or Verbal evidence of user selecting option unintentionally |
| — | Informational Problems |
| 2.1 | Verbal evidence of confusion about the information displayed |
| 2.2 | Verbal evidence of user being blocked from progressing through missing or hidden information |
| — | Interaction Problems |
| 3.1 | Verbal evidence of user being confused due to negative or invalid website feedback |
| 3.2 | Visual evidence of user being confused due to poor website feedback |
Table:6 Categorisation matrix schema
Additional Management Feedback
One of the reasons we chose Dundrum cinema was that we had direct access to management which allowed us a ask management a series of qualitative questions (Table 7). Although not exactly informative as regards what changes might be beneficial from a management perspective it did encourage us to consider the business while making or improvements. Some of the answers to the questions also supported not only our decision to focus on a responsive website which would function across all devices, but also some customers desire to have a customer account.
| Qs: Relating to ticket sales, do your customers normally book tickets through your website, app or in person at the cinema? |
As:
|
| Qs: What are the top things you want people to know about or do on your website or app? |
| As: Get info book films |
| Qs: Are you aware of any feedback or issues people have when using your website or app, and if so, could you elaborate? |
| As: Website outdated and not mobile friendly As booking takes place on a different site, Sorting Analysis can be difficult |
| Qs: If you could improve 3 things on the website or app, what would you change? |
| As: Complete revamp, Web site: modernise allow log into account, App nothing shouting at me |
| Qs: We noticed you have a link to listen to 98FM on your site. Do you have a relationship with 98 FM and if so what is the nature of that relationship? |
| As:Yes Debatable |
| Qs: You mentioned that you’d like to have a log in for customers – Can I ask why you want this? I am assuming it is to (a) store credit card details to make booking easier and (b) perhaps allow them to set preferences for what they’d like to see, but would like to clarify |
| As: Yes to store credit card, loyalty points and history |
| Q: Have you undertaken any analysis to identify a difference in customers booking different types of screenings, such as live events and regular movies, if so so was there a difference in the customer type? We are trying to understand the different potential target markets that Movies @ Dundrum has. |
| As: Yes we have many customer profiles on F/B. We also have separate email databases for 1. Movies 2. Live events a. Opera b. Ballet c. Exhibition d. Theatre |
Table:7 Management questions and responses
Leave a Reply