Doc Ketchum’s Murder Circus
Case Study
Summary
The production team and I collaborated to identify the highest-priority research questions, focusing on usability and key attitudinal variables on clarity, usability, character narratives, and artistic style. I conducted a remote, semi-directed, unmoderated playtest (usability tests with surveys) to gather feedback. The test provided critical feedback that led to immediate design revisions, refined our research strategy, and improved our recruitment approach for future tests, ultimately enhancing the game's user experience and testing processes.
Situation
While the production team was enthusiastic about the project, our initial research questions were broad and ambiguous. With usability and various attitudinal variables to consider and limited by time and budget, we had only one opportunity to conduct user testing to make significant improvements.
Tasks
My primary challenge was to select high-priority research questions while combining usability and key attitudinal variables. This required a strategic approach to ensure comprehensive testing and drive the game's development.
Actions
The production team and I prioritized research questions based on urgency and potential impact, narrowing our focus to usability and key attitudinal variables critical for the mission's success.
I developed a comprehensive strategy to extract maximum insights from remote, unmoderated testing sessions, utilize an incomplete game build, and incorporate lorem-ipsum placeholders where the content was unavailable.
The game designer, producer, and I identified the required participant criteria. Due to the study's unmoderated, remote, attitudinal, and short-notice nature, I aimed for 30 participants from one assumed consumer segment (we did not have access to market research).
I curated a series of interpolated surveys to assess usability and attitudes surrounding character narratives, artistic style, accessibility, navigation, and clarity regarding the new release's special mode (designed to be separate from the main game).
Results
Due to the short notice, steep requirements, and narrow selection criteria, we could only recruit 9 participants. Regardless, the usability portion of the test yielded invaluable results that significantly improved subsequent builds. The feedback from these sessions confirmed the need for character narrative adjustments, validated lighting choices, and clarified the in-game store's utility. Users understood the new release's separation from the main game, alleviating our initial concern. This feedback was the cornerstone of our design improvements.
A screenshot of a slide from the playtest results presentation depicts subjective clarity across multiple variables.
Outcomes
Design Changes: Based on the testing insights, the product team promptly implemented the recommended improvements, which led to a better-defined and more engaging user experience.
Research Strategy: Reflecting on the process, I recognized the value of more stringent prioritization and transparent communication. I planned to narrow the focus to only the most critical usability issues, potentially excluding attitudinal questions to streamline the testing process.
Experiential Learning: We fine-tuned our participant recruitment strategy and planning timeline for future tests, aiming for more inclusive selection criteria and scheduled sessions well in advance. This approach improved participation rates and turnaround times while broadening our participant pool beyond the initially targeted consumer segment.
These adaptations addressed immediate usability issues and set a new precedent for approaching game testing under tight constraints. They ensured that we continued to make data-driven decisions that enhanced our players' experiences.