thesisbeans

Live testing results: week 1

Last week has been a whirlwind of activity. Most of it was spent trying to get testers onboard with varying levels of success, gathering their reactions, and interviewing those who actually made it to the cooking stage.

There were both successes and failures. First, the successes:

  • People responded very positively to the playful visual design, including several fully grown men with luxuriant beards. I win!
  • People largely understood the core concept and expressed enthusiasm for it.
  • For the most part, people had little trouble completing key tasks when asked to: sending an invite, viewing one, responding to one, navigating to a cooking room, etc.
  • The code I wrote didn’t completely splatter everywhere as soon as someone tried using it.

But there were some failures, too:

  • The biggest failing was that, despite expressing enthusiasm towards the concept (“I want to try this!”), few people actually got as far as sending an invitation, and fewer still made it to the cooking stage. As I soon realized, it’s a lot to expect, but there may also be some flaws in the userflow. I’ll write more about this in the next post, as well as make some guesses as to what needs to be done to fix it.
  • One or two people had some confusion over whether the cooking happens concurrently, so I need to be a little more explicit about the “real time” aspect.
  • OpenTok sound quality was terrible for the first live cooking session done by independent testers (But to the testers’ credit, they figured out a way around it by just calling each other on the phone while cooking).

There were a host of small bugs too. Those are to be expected, and are relatively easy to fix.

Overall, actually seeing Hotpot live in the real world has been completely eye-opening. As I alluded to in my last post, you just can’t argue with a live prototype. It shows you how things really are, regardless of what you steadfastly believed in design fiction land. The real challenge now is how to interpret and implement changes by the thesis deadline, which is fast approaching. That will be the ultimate game of prioritization.