<My Role>
Research lead (team of 3)

<Background and Goals>
As part of ideating a new augmented reality (AR) workbook series for at ARpedia (Woongjin Thinkbig), I was the project lead, where I oversaw the whole product lifecycle from initial research, content design, prototyping, testing, software development, system management, and launch.

This usability test was conducted on a grade 2 English Language Arts workbook prototype, with the goal of identifying problems and areas of improvement.

<Research Questions>
(1) Can users (grade 2 students) use the workbook independently without adult help?
(2) Can users successfully complete 1 workbook within 15 minutes?
(3) What app features are working well and what's not?
(4) Are there under-served user needs?
(5) Is it engaging and fun?

<Methodology>
(1) Usability test (15 min session/ea.);
(2) Post-test user survey (w/ System Usability Scale, SUS);
(3) Post-test teacher interviews (30 min session/ea.)

*The usability test was conducted in a school setting, where students tested the prototype and teachers observed. Teacher in-depth interviews were also administered to receive additional insights, given the nature of the educational product.

- Participants (usability study): 20 grade 2 students from a private school in Massachusetts without prior exposure to ARpedia.

- Participants (interview): 3 teachers (1 tech teacher, 2 homeroom teachers)

<Key Learnings>
(1) [RQ #1-2] The content is age/level-appropriate and can be completed in 15 minutes, although it was confusing to use for some users. Some students spent more time figuring out how to use it and less on the activities, which eventually prevented them from completing the workbook within the given time.
→ AR technology is still quite new for many users, and therefore the initial product on-boarding process needs to be more simple, concise and intuitive.
→ Need to provide more guidance on initial product on-boarding (app content + help functions).

(2) [RQ #3] The train markers sometimes did not work (technical bug) on page 2-3.
→ Need to improve the recognition rate of marker images.

(3) [RQ#4-5] The product has too many pieces (cards, markers, reading device etc.) that can overwhelm users and decrease motivation/engagement, especially if they are struggling to understand how to use the product in the first place. Unfamiliar technology and product features can creates unnecessary cognitive load, which negatively impacts learning motivation (e.g. some students were frustrated and reported that they did not want to try the product again).
→ Should simplify materials and decrease number of supplemental learning items.

(4) [RQ #5] The content was engaging and educational. Students especially enjoyed the gamified contents on the fishing pages. Rate of engagement and desire to repeat the experience was positively correlated to rate of completion.
→ May have higher engagement and completion rates if it was easier to use.
→ Need to provide more guidance on initial product on-boarding (app content + help functions).

*For additional findings and learnings, please contact lim.jsy@gmail.com

<Recommendations>
(1) Provide more guidance on initial product on-boarding, either as app content, help features or via instructional video materials.
(2) Simplify materials and decrease the number of supplemental learning items.
(3) Improve recognition rate of marker images.

<Impact>
(1) An initial product on-boarding game has been created for users to learn how to use all parts of the product (app, workbook, coloring markers and action markers).
(2) Pop-up help bubbles were added in the app to assist first-time users with product on-boarding.
(3) An additional "How to set up" on-boarding informative mini-video was created to explain how to use the product. In a classroom setting, teachers can use this video to introduce the product to students.
(4) Revised and decreased the number of supplemental learning items for each workbook.
(5) Improved marker recognition rates with image correction and design changes.

<Reflection>
Augmented reality (AR) makes learning content more engaging and fun for learners. However, AR technology is relatively new to most of the general audience, and therefore more guidance is needed for new users to understand how to use the technology in the product. Diverse ways of refining the product design to be more intuitive is needed. For this particular project, once the "on-boarding process" is improved, another usability study should be conducted as a follow-up to ensure that the experience is more intuitive for users.