Case Study ‣ UX Research, Delivery Phase
Optimizing the Assignment Builder for Back to School Season
Background
Ahead of the Back to School season, Newsela added new features to grant teachers more control and flexibility when creating assignments. The enhanced assignment builder includes options to lock reading levels, set start dates, and assign specific activities, among other customizations.
These improvements were initially rolled out to 20% of users through an Optimizely experiment, enabling the Product team to gather insights and refine the features iteratively before a full-scale release.
We hypothesized that paid teachers in the experiment group would assign at higher rates than the control; and have a higher conversion rate of initiating to completing the assignment.
The control group (80% of users) saw the current assignment builder
The experiment group (20% of users) saw the new assignment builder
Research Objective
We wanted to understand how teachers in the experiment group felt about the new features as they were gradually released over a span of 2 months (2-3 features released biweekly). We wanted to know what features were working well and what could be better when creating assignments on Newsela.
My Role
I led the qualitative component of this study, conducting interviews with teacher users. I also facilitated weekly check-ins to share insights with my counterparts (two Product Managers and a Product Analyst) that were conducting quantitative analyses on the experiment and control groups at this time.
Artifacts
Bi-Weekly 1-pager
This 1-page summary, released bi-weekly, kept stakeholders and collaborators engaged throughout the process. It summarized what feature(s) was tested, details of the participants, and average ratings of the feature's usability and meaningfulness. It also included a summary table highlighting what was working well, what still needed improvement, and recommended action items.
Final Summary of Findings
The final summary of findings included a study overview, a roll-up summary of all interviews, detailed findings for each feature, UX research recommendations, and clips and links to full interview videos.
1-pager template
Key Learnings
Findings from the interviews revealed that teachers highly valued the new assignment builder. For example:
Select activities: This feature helps teachers facilitate student actions and assess comprehension.
Lock reading level: This feature enables teachers to ensure text accessibility and rigor.
Additionally, we learned that most teachers use smaller screens like Chromebooks, which leads to issues like excessive scrolling and hiding key functions at certain breakpoints. With these insights, the Product team optimized the assignment editor for smaller screens.
Impact
The learnings from this study provided the validation needed for a full-scale release for Back to School season. As hypothesized, teachers are assigning content more frequently than before, with a higher conversion rate from initiating to completing assignments.
This study also demonstrated how mixing methods can create more robust and meaningful research, allowing the team to proactively address potential issues to enhance the user experience.