User Testing for new features in the Eviction Menu

Background

This eviction menu's purpose is to enable the player to choose the traitor (correct) in order to reach the game's ending based on the information/clues provided by the game's plot.

In Sprint 2, our team attempted to add detailed information/clues to the eviction menu by adding character profile photographs to each NPC/eviction card and an information box that can be opened by clicking on each card.

Thus, this user testing tested and assessed the UI design and fundamental interaction flow of the eviction menu, especially for the info window, to obtain feedback on aesthetic and functional effects in the current design, as well as expected new features in the next phase. The feedback about aesthetics and function obtained from this user testing is beneficial and conducive for development in the next sprint.

Introduction

This user test aims to gauge users' (respondents') satisfaction and opinion on the present Detailed Eviction Card and Information/clues Window. These tests concentrate primarily on the appearance and operation of these new features added to the eviction menu. (Note: The data from user testing might also be synchronously given back to developers for development and functional optimization.)

Test Plan

Users (survey participants): people from other Teams in the studio, other students with an interest in the game, and potential users.

1. Task:

• Enter the interface of the Eviction Menu, browse and hover the different components

(Purpose: To determine whether the player can realize and find that an individual NPC/eviction card has additional functionality.)

• Click on one NPC/eviction card

(Purpose: To determine whether the interaction flow is clear and understood, then gather feedback on the popup effect used to develop the clues window.)

• Browse the information/clues window interface after clicking one NPC/character card

(Purpose: To determine the degree of satisfaction with the interface design of the information clue window, including color palette and layout.)

• Exit this information clue box and return to the main interface of the eviction menu

(Purpose: To determine whether the player understands how to exit the information/clues window.)

2. Get feedback: Responses were gathered through the questionnaire form. Besides, online interviews were conducted to complement data collection and analysis on outlier data deemed crucial or low-scoring.

3. Display testing after development: Different devices will have different screen sizes and resolutions. Testing whether the display effect of each operation is typical to determine that the interface works with most devices.

Process

15 participants were chosen for the testing process, and they came from a variety of groups, including:

• People from other teams in the same studio
• Team members' classmates and acquaintances

1.Test tasks: Respondents (Users) are required to perform the four mentioned tasks outlined in the Test plan; the process and results will be documented. (10 of them were evaluated online using Zoom screen sharing. The remaining 5 users were evaluated in person.)

2.Questionnaire: All responders (users) will be asked to fill out a brief questionnaire regarding their experience with the mentioned tasks once they have finished them.

3.Online interviews: After gathering and evaluating all questionnaire forms, we supplemented our survey results with online interviews and selected three respondents who expressed dissatisfaction, and another 2 expressed high satisfaction.

---

Questionnaire

The distributed questionnaire to all 15 respondents that participated in the test:

The user testing questionnaire(google form)

The user testing questionnaire(Tencent questionnaire)

Online Testing and Interviews

Due to the constraints of the situation, the majority of responders performed the tasks online (via zoom). And the intention of the interview was to get more particular responses to the specific question.

Important Questions

Questionnaire

Analysis and Summary

Questionnaire analysis

Q1Q3 primarily investigates the satisfaction with the interaction process of triggering this information/clues window. The majority of people regarded that the NPC/eviction cards with the info/clue box feature aligned their expectations (86.6%), which means that our design with the interaction effect(slight enlarging the card size and changing a brighter color) could let players realize that clicking on NPC cards to access more information to a large extent, indicating that there is a minimal likelihood that significant optimization is required (didn't add too many element styles to make the interaction look complicated). Similar to previous surveys, this feedback also led us to discover that some additional factors may influence player satisfaction with interactions, which will be addressed in the analysis of interviews.

Q2 reflects the degree of satisfaction with the UI design of the layout and color palette.
The number of neutral and unsatisfied respondents(who selected 1, 2, and 3) reached 53.4%, slightly exceeding the number of people who were relatively satisfied with the current design. This evidence adequately demonstrates some potential design problems in the interface of this info/clues window, which cannot be ignored and need to be further addressed in online interviews.

Q4 is a vote on several potential future new features. "failure/successful results" and "times of failure" are almost evenly matched, occupying the overwhelming majority of respondents, 86.6 percent, which will be considered for implementation in the next sprint.

Interview analysis

In addition to the questionnaire, an interview was used to glean further specific details for the data findings.

Regarding the color scheme, the following feedback was received:

1. Several dissatisfied respondents regarded the light blue and semitransparent background color of the info/clues window as inappropriate, which could result in a bad game experience. This is because the info/clues window appears above eviction cards with NPC information, which might cause issues with the text display of the clue window (e.g., text overlapping; poor typography of components), and some users find the translucent window to blend in the main eviction menu interface too much with the backdrop (it looks messy).

The following feedback is about the design concept:

2. Some disappointed responders have design requirements. They believe we should employ a higher aesthetic design concept to convey the ocean's mystique and retro, as well as use some particular pattern to get the feature of this info/clues window ( display the collected details).

• The color scheme is not aligned with users' expectations, and the background color with high transparency will make the window not clear enough.
• The decorative elements of the current info window are possibly inconsistent with the previous design.
• Some respondents are expected to be more in tune with the game theme (e.g., reduce the feeling of cuteness and brightness while appropriately increasing the sense of retro and mysterious ocean).

In terms of the interaction, the following feedback was obtained:

3. A small number of participants felt a little confused when they exited the information/clues window. It may be because they do not realize that just clicking any area (whether inside or outside the info window) could leave this interface. And there are NPC cards with information under the information/clue window, which makes them a little overwhelmed. However, this way of exiting the interface is very common to be used, so that it may be due to this participant's lack of experience in human-computer interaction.

Summary

During this sprint, the majority of functions' design and interaction procedures are as intended, but many tiny difficulties must be optimized. Actually, the above issues have been optimized and improved before being shipped to developers in this sprint:

1. The designer has thoroughly fine-tuned the aesthetic and color palette (The light and semitransparent background color is replaced with a darker solid (transparency: 90%) blue to make players browse the content more efficiently, but also have a little bit of water sense to fit the ocean theme.

2. Moreover, the designer modified the border of the window to a more aesthetically pleasing vintage scroll to better match the game's theme and the information window's feature. Also, some decorative elements (like the envelope and dolphins) are improved into a more vintage style to align with the game theme.

3. After user testing, other teams defined and updated several character profiles and related information. Our designers complement and refine the layout of the information on NPC/character cards.

Future Testing/Evaluation

The next sprint's user testing will be based on the information gleaned from this one's tests and feedback. We've planned the future testing and evaluation that may be needed to ensure that the team's efforts are well-aligned with what needs to be accomplished in the next sprint:

• Add the function of displaying selection failure/successful results
• Add feedback on incorrect selections (e.g., the limited number of incorrect choices in the game and the number of times choices have been wrong)
• If possible, add an item clue bar, using item icons or the combination of text and pictures to show info/clues (currently only text clues)

(Note: The specific functionalities that may be accomplished must also be established based on the task's progression and studio collaboration.)

1. Communicate with relevant Teams that may cooperate to Share thoughts and plans with each other.
2. Consider the implementation's viability and the integration's complexity via brainstorming and dialogue.
3. Test if the integrated player eviction menu can meet users' expectations at this time (e.g., Whether the error prompt can be triggered normally / whether the user feels that the interaction process can be made clearer with the help of the error prompt / whether the feedback of the selection result can be effectively presented etc.).
4. Discuss with the relevant teams and assess whether or not changes are required.
5. Invite studio members and other students (friends or future users) to evaluate the design, interaction flow, and functioning.