Qualitative Research

The task-based observational research was undertaken by all members of the team, with the goal of identifying pain points for users when using the Movies at Dundrum website. These were categorised and converted into experience maps so as to identify user goals. Each member of the team carried out at least two separate observations

Preparation

To begin with, we created a set of test scenarios which we felt would best reflect the tasks that most users undertook on a cinema website, based on the data the number one task was clearly finding and booking tickets, we felt it would be useful to include a task of purchasing food, for two reasons. One being the current “Movies @ Dundrum” website already provided this functionality and two, the quantitative survey data showed that users would be likely to buy food online if the option was available

Fig 1: Would consider buying food online

We created a simple usability test plan with two scenarios this can be viewed here.

The Actual Tests

Each of us tested two to four users. In all cases screen capture, audio and notes where used as well as video in some cases. The videos for my own personal tests covered four users using a mixture of laptops and tablet devices and are as follows

User #1

Subject: User #1
Age: 64
Subject IT literacy: Poor
Date: 02/12/2017
Time: 17:26
Interviewer: Gavin Stokes
Space: Seated at a desk
Device: Laptop, Screen resolution 1600 x 1024
Task: Book two tickets for a showing of “Murder on the Orient Express” for the 9th of September
Recording devices: Video, audio, written note
Full Transcript: Google Doc
Notes: -User finds home confusing and scrolls a lot
-Filter button causes confusion as the content is reloading below it is   there in association
-Quite a high level of confusion
-Page reloads cause confusion
-Interactive items cause issue as high-lite is same colour as page
-Mistake breadcrumb nav for wizard/next step
-Cant locate where ticket selection is
-Ticket selection is confusing as it has 0-10 with no instruction as to what    it means
-Seat selection proceed button is confusing, is labelled as select seats       which user sees as instruction

Table:1 User #1 interview notes

User #1 had many issues with the overall site. Some, but not all of the issues are discussed here, for a detailed summary of the issues see the categorisation matrix at the bottom of the page.

Page reloads caused a large amount of confusion, in one case, in particular, the user could not make the connection between the filter functionality on the home page, and the reloading of the movies below the filter. This was probably due to a number of reasons, the filter functionality highlighted in (fig 2) in red, is not clearly separated from he the image carousel above it, which causes the user to assume the filter affects this instead. This poor mapping of controls as pointed out by Cooper et al. (2014) can cause the user to have to stop and think increasing cognitive load and the level of confusion. The second possibility is that the filter results, actually load below the filter option and off screen after a page load.

Fig 2: Movie Filter

Another issue was the poor labelling of controls and the failure to supply information to the user as to their function. For User #1 who has little to no experience of what might be considered established norms the following two features caused a large amount of confusion.

The breadcrumb in (fig 3) was mistakenly taken to be a form of navigation which could progress the user to the next stage, there is no information explaining what this does, such as a “You are here” indicator.

The user can be seen at time stamp 8:11 trying to use this when the content on the page becomes confusing due to a large advert on the page which hides the next required steps off-screen. She eventually selects the login to account button in the hope of finding a payment option.
There was a secondary observation here from the testers perspective. I took my eyes of the user to make note of the confusion, upon returning my attention to the user, I found they had somehow landed on the account login page, always have two testers if possible.

Fig 3: Breadcrumb

The ‘Choose These Seats’ button in fig 4 caused confusion as the user felt it was asking her to perform the task of choosing seats which she had just performed, time stamp 15:29. Upon trying to the move to the next step in the process the user drew a blank as to what to do next. This led her to try the breadcrumb function, and various other options before having to be promoted. A secondary issue with the seating selection was that certain seating arrangements appeared of screen with a large amount of white space above them, essentially causing them to not appear, time stamp 14:08.

Fig 4: Chosee seats

User #2

Subject: User #2
Age: 68
Subject IT literacy: Beginner
Date: 02/12/2017
Time: 08:26
Interviewer: Gavin Stokes
Space: Seated at kitchen table
Device: Laptop, Screen resolution 1600 x 1024
Task: Book two tickets for a showing of “Murder on the Orient Express” for the 9th of September
Recording devices: Video, audio, written note
Full Transcript: Google Doc
Notes: -Immediately tries to locate a find or search option but cant see the one   on the home page as he has landed on an interior page.
-Back image (curtain image) has a book now button which is only an     image
-Cant locate search option on homepage
-Mezz title causes confusion as he cant understand what it is

Table:2 User #2 interview notes

Key Issues

Entering the site via google and not landing on the homepage causes some confusion as the user is looking for a search option, which is only available no the homepage. There is book tickets now button which is actually part of a background and performs no function. After clicking this  (time stamp 1:04) the user then feels its necessary to go back to google to access the homepage.

On returning to the site the user misses the filter feature on the home page and scrolls down the site to find the desired movie and selects the wrong date from the predefined options. This then loads the movie page for that date whereupon there is no option to change the date (time stamp 2:46). The user proceeds through the booking process in the hope there is an option to change the date later. The user was then prompted to start over. After starting over a second time there is an option to select the date which he then finds.

The booking process from here on goes relatively smoothly with a few questions around types of tickets.

(For detailed summary of the issues see the categorisation matrix )

 

User #3

Subject: User #3
Age: 39
Subject IT literacy: Average
Date: 02/12/2017
Time: 11:48
Interviewer: Gavin Stokes
Space: Seated at kitchen table
Device: iPad mini
Task: Book four kids tickets and one adult tickets. Add food for each child. For any movie and time you want
Recording devices: Video, audio, written note
Full Transcript: No transcript available due to time constraints
Notes: -User uses the title of the movie on the homepage to access times
-Large food advert is seen as clickable but inst.

Table:3 User #3 interview notes

Key Issues

The first issue User #3 encounters is the advert that appears on the ticket page (fig 3). The user understands this to be clickable, upon realising it is not she scrolls down and proceeds with the process. The ticket options presented are confusing for the user and she is unsure which option to select. The naming conventions are not descriptive and only add to the confusion “Large ticket and combo meal”, “Medium ticket and combo meal” (time stamp 3:10). There appears to be no kids options. The User is eventually forced to select two medium options but is not happy with the selection and returns to the ticket options.
On the tickets screen the advert shows a kids deal option but it is is not available under the list of options for tickets and food (time stamp 5:45).

Fig 3: Meal deal advertisment

The user returns to the homepage and tries a few different options to see if she can find a booking option for four kids tickets and and 1 adult with food but cant find anything. She restarts the booking process again and after selecting a different date she is presented with the advertised meal deal. It appears that the kids deals are not available for some reason when booking on a Friday.

(For detailed summary of the issues see the categorisation matrix )

User #4

Subject: User #4
Age: 40
Subject IT literacy: Average
Date: 02/12/2017
Time: 2:16 (Video was cut short due to battery)
Interviewer: Gavin Stokes
Space: Seated at kitchen table
Device: iPad mini
Task: Book four kids tickets and one adult tickets. Add food for each child. For any movie and time you want
Recording devices: Video, audio, written note
Full Transcript: No transcript available due to time constraints
Notes: -User uses the title of the movie on the homepage to access times
-Large food advert is seen as clickable but inst.

Table:4 User #4 interview notes

Key Issues

The user had no real issues  with completing the task of booking tickets. He did fail to  complete the task of booking tickets with food. This was due to confusion over the advertisement on the tickets page. The user had assumed that the family deal included food, and was not aware of this even after completing the booking.

The lack of payment review screen makes if difficult it not impossible to see what has booked as only a final total is presented. If the user had of add this review option they would probably have not completed the booking.

(For detailed summary of the issues see the categorisation matrix )

User testing Categorisation matrix

A total of eight participants took part in the observational study of the ‘Movies @ Dundrum’. From these a total of 10 tests where carried performing a series of tasks mainly centred around the booking process, with or without the the addition of food.

Upon completion of the video observational work, transcripts where created for some of the videos but due to time contrarians this was not possible for all. We then proceeded to categorise the data from these interviews using a categorisation scheme. As suggested by Rogers, Preece and Sharp (2015), we allowed the categorisation scheme to be suggested by the data. Once we had established the scheme using one video we then proceeded to see if it was valid by applying it to the other video observations in the study and allowing other members of the group apply it independently.

Although not 100% accurate and even though it does not identify exact issues, it did give us an excellent starting point in  identifying and cataloguing issues with ‘Movies @ Dundrum’. And when later combined with a user experience map helped us in identifying key pain points and goals.

Participant 1.1 1.2 1.3 1.4 1.5 1.6 1.7 2.1 2.2 3.1 3.2
A 3 4 0 1 2 2 0 2 1 0 0
B 7 2 2 5 2 0 1 2 5 3 1
C1 2 4 3 1 2 0 1 6 1 1 1
C2 2 2
D1 2 7
D2 1 1 1 1 3
E 3 2 1 3 1 1 1
E2 1 1
F 1 1
Totals 21 15 5 7 8 3 3 23 8 7 3

Table:5 Categorisation matrix results

Interface problems 
1.1 Visual evidence of confusion about the interface
1.2 Verbal evidence of confusion about the interface
1.3 Visual evidence of confusion about the interface through misleading information
1.4 Visual or Verbal evidence of confusion about the interface due to user misunderstanding functionality
1.5 Visual evidence of user selecting invalid option due to misleading or poor navigational information
1.6 Visual or Verbal evidence of expected functionality missing
1.7 Visual or Verbal evidence of user selecting option unintentionally
Informational Problems
2.1 Verbal evidence of confusion about the information displayed
2.2 Verbal evidence of user being blocked from progressing through missing or hidden information
Interaction Problems
3.1 Verbal evidence of user being confused due to negative or invalid website feedback
3.2 Visual evidence of user being confused due to poor website feedback

Table:6 Categorisation matrix schema

Additional Management Feedback

One of the reasons we chose Dundrum cinema was that we had direct access to management which allowed us a ask management a series of qualitative questions (Table 7). Although not exactly informative as regards what changes might be beneficial from a management perspective it did encourage us to consider the business while making or improvements. Some of the answers to the questions also supported not only our decision to focus on a responsive website which would function across all devices, but also some customers desire to have a customer account.

Qs: Relating to ticket sales, do your customers normally book tickets through your website, app or in person at the cinema?
As:

  • Approx. 50% web
  • 10% app
  • Assumption: 40% in cinema sales
 Qs: What are the top things you want people to know about or do on your website or app?
 As: Get info book films
 Qs: Are you aware of any feedback or issues people have when using your website or app, and if so, could you elaborate? 
As: Website outdated and not mobile friendly As booking takes place on a different site,
Sorting Analysis can be difficult
Qs: If you could improve 3 things on the website or app, what would you change?
As: Complete revamp, Web site: modernise allow log into account, App nothing shouting at me
Qs: We noticed you have a link to listen to 98FM on your site. Do you have a relationship with 98 FM and if so what is the nature of that relationship?
As:Yes Debatable
Qs: You mentioned that you’d like to have a log in for customers – Can I ask why you want this? I am assuming it is to (a) store credit card details to make booking easier and (b) perhaps allow them to set preferences for what they’d like to see, but would like to clarify
As: Yes to store credit card, loyalty points and history
Q: Have you undertaken any analysis to identify a difference in customers booking different types of screenings, such as live events and regular movies, if so so was there a difference in the customer type? We are trying to understand the different potential target markets that Movies @ Dundrum has.
As: Yes we have many customer profiles on F/B. We also have separate email databases for 1. Movies 2. Live events a. Opera b. Ballet c. Exhibition d. Theatre

Table:7 Management questions and responses

Leave a Reply

Your email address will not be published.

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑