ARIS is a user-friendly, open-source platform for creating and playing mobile games, tours and interactive stories. Using GPS and QR Codes, ARIS players experience a hybrid world of virtual interactive characters, items, and media placed in physical space.
I had been taking a class on research and evaluation class during graduate school when my instructor had asked me some questions about quantitative and qualitative data. One of the questions she asked went something like “If you were asking your participants to go through a software program to see what they are doing, would that be qualitative or quantitative?” I replied something to the extent of it being mostly qualitative, but I would say it would partly be qualitative if instead of just observing, you had the participants speak out loud what they were thinking. The next thing I knew, I was asked to help do a user test for a program she is doing a research project with called Aris.
Aris, developed out of the University of Wisconsin Madison, is a mobile development platform. They use Aris to teach children in school how to do programming by using simple techniques. At Utah State University, a research group that I am closely associated with runs a mobile game workshop where about 10 children with ages ranging from 8-14 get together and make their own mobile app games similar to Pokemon Go. There is no coding required, however, they can create items, scenes, and quests together using Aris, and on an iOS device they can play their own games, as well as share them with others. Hopefully by teaching these children Aris, they will have a deeper interest in STEM fields and be less intimidated by its complexity. During the user test, I had been asked to test the factory section, which is both new and difficult to use.
Before going in to do the user tests, I read the chapter in “Don’t make me think” on how to run a user test to refresh my memory. I had also read many other blog posts on user testing so I was familiar with what they were, and how they were facilitated, but I had never done a formal user test before. I found the script that Steve Krug uses in his “Rocket Surgery Made Easy” work, and remodeled it to fit the children that I would be doing the user tests with.
I followed Don Norman’s suggestion on getting 5 participants, and made sure that each of them knew that they weren’t going to be tested. I really wanted to emphasize this point to them because I remember when I was their age I always got nervous during a test and wouldn’t perform as well. For this session, I had 3 specific tasks related to the factories in Aris. They are:
I was pleased with the experience range of the participants. I had one person who didn’t know anything about Aris, and another who was familiar and very confident in answering all of my questions and completed each task quickly.
With this range of experience, I was able to find 3 things that I thought would be helpful for Aris to have a better user experience. The first one was being able to create a factory in a fixed location. I wish I could see into the developer's mind to find out why they made the decisions that they made when it came to this task, but I only am able to assume. The way this currently works is to set factories at a location, you put in the latitude and longitude into a text field. The issue is that you can’t drop a pin on a map, but you would need to go to a new window and somehow find the latitude and longitude. I assume that writing code to drop a pin on a map is more difficult than having the latitude and longitude fields, but on the user's end, it would make it much easier and faster, thus making it more used.
The second suggestion that I found by these user tests is clearing up the relation between items and factories. When asked to make a factory of turtles, the user would create a factory and name it turtles. Later most realized that you need an item of “Turtles” for the factory, so there would be a factory named “Turtles” and an item named “Turtles.” I believe that the developers intended the name of the factories to be called “Turtles Factory” to avoid confusion. I would make a simple suggestion to have the input field say “Factory Name.” This would also help the others who didn’t understand the relationship between items and factories to realize that calling the factory by the name of the item does not mean that the factory will produce the name.
The third issue I found through these user tests is that there were so many checkbox fields available that the students would check and uncheck some boxes quickly. When I asked why they selected or unselected those boxes they would reply “I actually don’t know what that does. This is just how I have always done it.” That was a surprising answer to me, but it almost appears that they didn’t know and didn’t care because they didn’t see a difference. My suggestion is to have an “advanced” button that would expand the additional options that would need to be changed. Another thing that could help would be an icon in the corner that the students could click and it would take them to that specific point in the documentation explaining exactly what the options are.
I had a lot of fun doing a real user test. Just like they say everywhere that I have referenced, user tests are pretty easy to do, and they don’t require a lot of experience. In fact, even though this was my very first user testing session that I facilitated, I was able to find plenty of issues that may not have been known before. After doing this, I would like to actually participate in another user testing session for another company to see how they do these sessions differently. That way, the next time I am asked to help a research group with this type of evaluation, I will have more ideas to help them improve their product.