Initial usability interviews were conducted with 13 teachers at Kansas City Virtual Academy and University Academy to understand teachers’ first impressions with Pango’s lesson planning platform. The central research question for these interviews was “Do teachers understand how to use Pango and see the value in it during their first encounter with the platform?”
Research Team
Dr. Trisha Callella
Dr. Erin Huebert
Schools Team
Andrea Cook
Karnissa Caldwell
Jorge Holgiun
Graphic Design
Alexander Sheppard
Usability Strengths
The resources are easy to find and it Is easy to search for and filter resources.
"I didn’t know there were going to be all these resources. So that’s nice!"
“I don’t have to dig and dig to find what I need.”
“I loved the graphics that went along with what it was teaching."
The graphics were visually appealing - especially the calming, cool colors.
“I like the graphics, the lessons, and the simple wording.”
Integration with Google Classroom and Quizlet resources.
“Oh good - they have Quizlet!”
“We do everything in Google Classroom, so the integration will be nice.”
Class template is helpful for creating common lesson plans.
“This class template is awesome, especially for new teachers. This framework is helpful.”
Usability Barriers
Lesson Creation
The vast majority of teachers struggled with having to create a lesson before seeing the platform and its function and features. For most - particularly elementary teachers that teach a number of different content areas - they didn't want to make a commitment to one class or lesson at the beginning as they worried it would limit what they'd see once they entered the platform. The majority of teachers said they didn't understand why they were selecting a lesson as they hadn't yet had an opportunity to see what they would actually do once inside of Pango.
TEACHER QUOTES
"Asking me at log in was too specific. Commitment to class, grade level, etc. Almost made me turn around and stop using it. Don’t know what I’m doing this for yet."
“if I pick one, am I only going to be limited to one. Wish there was a general ed. I don’t know what to pick here.” Picks other. “If it only gave me limiting choices I’d probably leave the website right now.”
“I don't want to start with resources or creating a lesson. I want to start with just labeling everything - goals, standards, etc. - and then plug in resources. It starts too small.”
“I don’t know what to title my lesson. I don’t know what to pick for unit/topic. Hoping I can change this later.”
“You don’t get to view the resources until you choose a specific lesson, and I don't want to start that narrow.”
"...must select one grade level but middle school and high school teachers often teach more than one."
Setting up the timetable
Setting up the timetable comes too early in the process as, similarly to above, teachers have yet to see or understand what they are creating. All teachers just clicked on a period to be able to advance past it. For elementary teachers, the periods didn't work for them as they have shorter time blocks and more of them throughout the day. Most teachers said later that they valued the timetable feature, but that it would be better to set that up later.
TEACHER QUOTES
"Change the periods to time specific, customize to times or subject areas."
"Timetable is confusing. I don't know what I'm setting up yet or what will happen after I do."
"Calendar screen is overwhelming."
"It gave me the whole month, I wish it would do one week at a time. Oh, here’s a week."
"I have group from 9-9:30, wish it was times rather than periods... set up like high school. Am I going to be able to change this? Can I delete this?"
Navigating the Standards Section
None of the teachers could find the MO standards until pointed out to them. One or two happened upon them by accident but couldn't find them again. They also didn't understand to select MO standards first, and that it then loaded into the template for them to choose the specific standard within it.
The main problem is that by default, English National Standards is there and you have to click "Customize Standards" to find it. Even then, Missouri Standards doesn't all appear because they see an ellipsis which means they don't notice it.
Lastly, "Pango Standards" is more prominent and nobody knows that that means. The main suggestion is to be asked during profile set up which standards they use, so that when they go to select a standard, the default option is their own and not the English National Standards. From there, it would be very easy.
TEACHER QUOTES
"They look different from the MO standards and I was wondering where to find MO standards."
"I was confused by “Pango standards.”
"I don't know what any of these standards are or these acronyms."
"I didn't see Missouri Learning Standards because it was blocked out. I just saw "dot, dot, dot" and stopped reading."
Lesson Planning Usability
While the structure of the lesson plan template itself was intuitive and had all the relevant parts for teachers, there were a couple usability features that many teachers struggled with or found frustrating. Because the resources bar kept popping up, they could not see the whole lesson plan template and it confused or overwhelmed them. Additionally, with the timetable at the top, for many teachers, there was too much happening on the screen and they didn't know where to start.
TEACHER QUOTES
"When this all popped up at once it kind of gave me a heart attack."
"At first it was overwhelming and I felt lost. But then it keeps me interested. Periods hung me up and huge calendar that came up at once." (calendar at top)
"It all came at me at once, the calendar and resources all popped up at once. It was overwhelming and didn’t know what to do first."
"I would like to build my calendar first then look at resources, then decide where I want to put my resources in. I don’t know how to get back."
Expectations vs. Reality
Relevant Resources for Upper Grades
Middle and high school teachers struggled with finding relevant resources at first, but wanted to play around more. They would get frustrated when they couldn’t find an activity/game for their age and began to think it was geared more toward elementary, which didn't align with their hopes/expectations.
TEACHER QUOTES
"Not a lot for my specific subject area."
"No activities above grade six."
"Geared more toward elementary than high school."
"I would jump ship now. I don’t have anything for my age group, then I would have to do things on my own. Probably same goes for my colleagues."
"In high school, we have a hard time coming up with activities, so that’s a tool I’m looking for. There’s nothing for high school on here."
"I didn't see any high school links on the website (before entering), so I probably would have not signed up because it looks to be for elementary."
"If there is no science stuff I would not be interested in going further. I struggled to find relevant science stuff for high school quickly. Disappointed about the resources"
Aligning Resources with Learning Standards
Most teachers expected to be able to find a resource and align it with their learning standards. While they were pleased to see the standard alignment feature, all struggled with the usability of it, particularly in finding their preferred learning standard.
TEACHER QUOTES
"I expect to be able to convert the resources to MO standards."
Pre-Made Lesson Plans
About half the teachers expected there to be pre-made lesson plans. All teachers saw the utility of the lesson planning template, but many teachers wanted to see a whole lesson plan already created that they could download or share. To the extent that such an option already exists, teachers couldn't find it.
TEACHER QUOTES
"I am looking for pre-made lessons that align to our standards that first year teachers could use."
Filtering Resources
All teachers expected to be able to filter and search for resources and the “Resources” feature was the most prominent and easy-to-use for all the teachers. About half the teachers didn't have expectations beyond this, which meant that they were pleasantly surprised to see the lesson planning feature but also confused/overwhelmed when first prompted to set up a lesson, as they were expecting to first see resources. For some, the perceived complexity and/or surprise with the lesson-planning feature turned them off. They felt it would take too much time to learn Pango. For them, their emphasis was simply on finding good resources quickly.
TEACHER QUOTES
"I did not expect the format to be what I saw as far as setting up class periods right off."
"I thought you could look through resources before setting up or have the option to look for resources."
"Not enough time in my day to play with it to learn how to use it."
"I would not use if I only had my plan-time to use. I would need to find relevant content quite quickly because I don’t have time to figure it out."
Wonderings
- The teachers are wondering if and how to push the content out to their students.
- Concern: How much does it cost?
- Teachers noticed the standards and appreciate them but wonder if each state will have standards listed in the future. They also wonder if there is a way to make their own standards the default standard from the beginning.
- Some middle school and high school teachers are wondering how to save and share.
- Teachers had questions around the international framing and language used (i.e. KS2).
Perceived Value
One Stop-Shop for Resources and Lesson Planning
“This could be a one-stop shop!”
“I like that I am able to keep all of my lesson plans in one place. (Currently I am using Excel to track lessons.)”
More Efficient for Lesson Planning
“I believe there will be less stress for teachers due to the time they can save with narrowing resources.”
Great for Collaboration
“This could be one planning template rather than all of our plans looking different.”
“Sharing with team is a great feature.”
Helpful for sharing with substitute teachers/assistants
“This would be great for substitute teachers. Or, we teachers often have to fill in for each other, so this would make that easier.”
Accessible on multiple devices
“I like that it’s able to be on my phone”