Requirements
- Need to put together a detailed user testing plan, to make the testing progress more straightforward once it comes to setting it up.
- Need to find out information about the best details to include in this user testing plan, to be as detailed and efficient as possible.
- Need to know what's crucial to include and whether some elements may not apply depending on the testing process.
Research
- Carried out research into what to include in a user testing plan and how detailed some sections should be, found 2 detailed webpages that were very insightful and provided a lot of unique information.
- Webpage #1, from playbookUX, detailed in steps what to consider, and included the importance of planning the objective of the testing, creating observation and task lists, and defining what to test and how to do so.
- 'Generally, you can collect two types of data – qualitative and quantitative. Qualitative data focuses on user behavior and attitude, while quantitative data concentrates on numbers and metrics.' - Step 3, Define what you'll test and how you'll test it, Lines 1-2.
- This webpage also included information in each step on how to make the plan efficient and understandable for other people, like being transparent when outlining the objectives, or narrowing a test group down to just people from the target market.
- Webpage #2, from usability.gov, outlined some more concise elements of a user testing plan, like scope and metrics (Quantitative data and/or questions given to user testers during their session).
- 'Include the questions you are going to ask the participants... overall ease, satisfaction and likelihood to use/recommend questions when the sessions is completed.' - Metrics, Lines 1 + 3-4.
- This webpage also went into detail about how to identify different test metrics, like successful task completion, critical errors or subjective measures.
Research sources:
Elevate your UX strategy: 9 tips for crafting an effective usability test plan (2023) PlaybookUX. Available at: https://www.playbookux.com/elevate-your-ux-strategy-9-tips-for-crafting-an-effective-usability-test-plan/ (Accessed: February 3, 2023).
Affairs, A.S.for P. (2013) Planning a usability test, Usability.gov. Department of Health and Human Services. Available at: https://www.usability.gov/how-to-and-tools/methods/planning-usability-testing.html (Accessed: February 3, 2023).
Implementation
- Used elements of both webpages found through research, and identified elements like scope, metrics, critical errors, participants and subjective measures.
- Scope is to test our minimum viable product that will be presented at the end of the pre-production release - the aim is to test before this presentation and be able to present some user feedback to the panel.
- Purpose of the testing will center around garnering opinions on mechanics like anomalies, narrative presentation, and to also see how users play through a game with optional tasks to complete (ie. will they complete these tasks?). The goal of the testing is to have testers play through the first day of our game and experience the mechanics and story elements.
- Participants will be recruited online for now (hoping to present the game to other students and get realtime feedback too) - the game will be available via Reddit and a survey will be provided for testers to fill out after they play through the game.
- I believe some elements, like observations, would be difficult to obtain with the current testing method of allowing people online to play through the game without the participants willingly recording themselves playing the game, and not all may consent to this, and so this element can be overlooked and hopefully implemented in future with in-person participants.
- Metrics will be qualitative and the survey created reflects that - the questions are opinion-based. We will also be looking to see if users are satisfied with the gameplay and want to see more, and if the game was easy to use and play through.
A screenshot depicting one of the questions in our survey and how it will appear to user testers
Analysis
- Overall, I think I was able to use implement both research sources equally when writing out my research plan.
- I would like to, in future, try to run more tests with user testers being present in real time, as it would be easier to obtain information on some elements like observations.
- It was really interesting looking over my two research sources, as both brought up points that I would have assumed to have been very basic, but having forgotten them since my last UX design role, even these initial tips are crucial and help to form a descriptive and efficient framework for a user test plan.
- I now know that, in our team's case, obtaining observation information like physical reactions and realtime problems would be difficult to obtain and so can be left out of the user test plan for now. However, both resources make it clear how important it is to include as many elements to a user test plan as possible and to try not to leave any out, as this provides the most detail when it comes to carrying out the user testing.