Evaluation of StreetLearn prototype took place
On 14.11.11. a half day evaluation with 6 students and one tutor of the faculty of cultural science took place.
We gave a short introductory presentation to the participants and then asked them to play the game. During the game we gave some support to participants and could answer questions. After the game session, we asked participants to fill out a questionnaire and we additionally collected some feedback in a wrap-up discussion.
Four of the students where female, two where male. Any two of the participants where in age groups 36-50, 52-65, above 65. Their previous experience with technology enhanced learning, computer games and serious games was rather low (avg. 2.5 on a scale from 1 to 6, where 6 is “I fully agree”).
The rated the use of serious games for education as helpful (avg. 5/6) and the idea behind streetlearn as interesting (avg. 5/6). However, some of them claimed, that learners should concentrate on learning rather than gaming (avg. 3.3/6)
Participants were asked to rate the user interface of the StreetLearn prototype (on scales of 1 to 5, where 5 is “I fully agree”). They rated it to be appealing (avg. 3.5/5), partly easy to use (2.5/5), and useful for the game (3.5/5), only partly they think, the UI is obstructing (2.5/5), too complex (2/5), or confusing (2/5).
Learning contentThe learning content about the amsterdam grachtengordel was rated as interesting (4.2/5), relevant (4/5), understandable (4.2/5), nicely presented (3.8/5), comprehensive (3.5/5). Only few participants rated the content as too complex (2.2/5).
The game story in the amsterdam drug milieu was perceived to be well connected to the learning content (4/5). However, opinions on the usefulness of the game story where not clear: the story was rated only by half of the participants as interesting (2.8/5), motivating (2.7/5), helpful (2.7/5). It was also by half of the participants rated as confusing (2.8/5), distracting (2.8/5), or too simple (2.7/5).
We received further input from the participants about different aspects of the prototype. See below an excerpt of the comments received.
Overall- you don’t get bored, time flies- fun, enjoy, kept me occupied- information is helpful- good mixture of text, movie, game elements- connection of past/present elements is good (e.g. today’s function of westerkerk as gay symbol)
- it is sometimes unclear what to do next
- sometimes items are a bit hidden
- relation grachtengordel / drug story weak
- more information- more test questions
- more own content (no wikipedia etc.)
- the police story is a bit distracting from the learning content. Maybe a historical figure/scene would link content and story better.
- Too busy with game, too distracted from learning
Overall we are satisfied with the outcome of the evaluation. Participants where highly interested and motivated. We received very positive feedback for concept and idea behind StreetLearn and some critical remarks about its current prototypical status. The feedback given is very helpful for further improvements to the system.