First Prototype Trial

I did a pre-trial test study today, where five users (university staff and students) were asked to try the prototype and answer the questionnaire that will be used in the proper study. As expected the Povey version of the display had some issues that affected the way it was used. The initial prototype (finding points) got a mixed response, with some positive feedback and some less so. I will conduct a more detailed look at the feedback over the next few days, but I wanted to get down some initial observations while they are still fresh.

• In discussion after the participant had tried the display, it became evident that there focus was more on testing the technology, rather than looking at and learning about the painting, whilst this is to be expected (the novelty factor) it poses a slight problem for the study. I think that a partial solution is to be careful about the wording about what is expected of them (letting them know that 1) the display is designed to help them understand the painting 2) they should use the display for as long as they want 3) they will be asked questions about their experience of the display and, 4) that they will be asked questions about how well they understand the painting after using the display.
• One user expressed that they were less impressed with the display, as they were expecting something more advanced (a more visually stimulating experience with more of a technology wow factor, like the advanced depictions in films), However they did mention that they are more into there gadgets, and less interested in the content side of the display. I think part of the issue is that I have specifically chosen quite a simple aesthetic as the study is more focused on the impact of different underlying structures (through comparison), than the visual style. They also mentioned that they wanted to see the painting and content in more of a merged experience on the screen, with more interactivity with the displayed content.
•  I think most of the users were unsure how to use the display at first, I think that a better on device introductions (with visual material - illustration, photo or video) may work better, especially for the less technology competent user. 
• As expected most of the visitors complained of their arms getting tired as they needed to hold the tablet in such a high position in order to get it to track. 
• Some users seemed uncertain about where to stand and the distance they should be from the painting. Whilst some moved around to find the best position others seemed less willing to do so. The problem was that with the Povey trial version some points required you to be close and some further away, however the users seemed to find a mid position and only looked at the points that tracked from that distance. In the proper version it will be better to have all points track from about the same distance, and tell the users the optimum distance at which they should hold the device (e.g. 1 meter) from the artefact.
• Some users said that they gave up as they were not sure how many content points there were to find, and as such were not sure if they had finished.
• Glare was a bit of an issue with some of the tracking points, where the painting was by the bright light source of the window. This resulted in the user pointing the device to where a tracking point is and getting no response. 
• I found the observation scale quite hard to use as it is quite subjective (it is impossible to tell if the user is enjoying the display). I think most of the category's are OK, and a bit more practice at using the scale will be useful. It is also essential to have the 'cant tell' option and use it where no observation judgement can be identified.
• For the proper study I need to be more consistent in how the questions are asked. In the pre-trial I took the initial standpoint of holding the questionnaire and reading out the question but allowing the participant to scan the available options (multiple choice) and say which one they wanted. One participant expressed a preference for being given the questionnaire and filling it in on there own. This may have the benefit that they will be more willing to put down their true opinion, by there is more possibility that they will not answer all the questions.

Study Confirmation

Good news, after some waiting permission to carry out my PhD study at the intended site has been granted (the site I wanted as it has a very good wall  mural, which would be perfect for augmented interpretation trials). Hopefully things will progress after I get back from Vietnam. Of key importance is getting the display transferred over to the mural, this will involve:

• Choosing the information points (and taking a reference photo for each point). I intend to ask the staff / volunteers which points they think are really interesting as they will probably know the mural well.
• Writing the content for each point.
• Modifying the android app with the new content (hopefully this should not be to hard).

In terms of conducting the trail days, I was thinking of testing the different versions of the display on different Saturdays (in order to try and capture a similar study population). There is a bit of a time limit at the moment as the property closes for the winter in early November so I need to get the initial two trials in before then. I was also intending to conduct a small survey before the prototype testing, to gauge the site visitors expressed interest in this form of interpretation.

Over the next few days I was also intending to do the small pretrial prototype test of the version of the display I built for the Povey mural. This will be useful as a pretest for the proper study (ironing out any issues with the questionnaires and other study tools). However as I have identified that the mural is not really suitable for the type of display I am testing, the pretrial will be a chance to identify if the users also find this to be the case, and explore what use issues they encountered. As the is not at a site with passing visitors (like the intended study site), I will probably have to ask a small number of students to be test subjects (a.k.a willing volunteers). I was thinking that about three to five subjects should suffice (which is similar to other pretest studies I have read about).

Interesting Conference Title

In line with the previous post, I have had the thought that my paper for the conference should be something along the lines of:

What's the problem with Edward Poveys Hall of Illusions? - initial explorations with augmented interpretation.

I think this might offer an interesting paper, and will tie into writing the chapter.

The First Post

So the time has come for my first blog post (long overdue), the following blog entries relate to my developing  PhD work which is concerned with using new media (specifically augmented reality) as an interpretation tool for the heritage sector. In an attempt to finally get organised and make some progress with my writing, I have  decided to keep a blog of my developing production work, research and general musings.

I had a meeting about my work yesterday which raised some interesting points that I have had to ponder over. Firstly, I have probably focused to much on building the prototype (and making sure that it is all working ), an have possibly neglected recording the process. The point was raised that I have already done enough work for a chapter which needs to be written up. This would be:

• Chapter 1: Creating and testing the initial prototype (which for initial convenience was built around the Edward Povey Hall of Illusions mural at Bangor University).
• Chapter 2: Choosing the proper heritage site and trailing the basic agented display.
• Chapter 3: Trailing and comparing the alternative treasure hunt style display.

I think the problem was that I never really intended to use the Povey painting as I always wanted to work at a dedicated heritage site (e.g. a national trust site) as this has the benefit of a stream of visitors (the willing study population) to try the display. However, approaching a site was put back due to teaching responsibility and it seemed like a good idea to make sure that I could get the display working (by testing it on the Povey painting) before approaching the intended site. Whilst I have got the display working with the Povey painting, it turns out that the painting is not quite suitable for the type of augmented display I am putting together (the main issue is the scale of the painting, and the distance from it that you need to stand). Because of this, it was my intention to move on to transferring the painting over to the selected heritage site, however it was pointed out that I am missing an opportunity in not turning the Povey pre-trial into a chapter in its own right, and exploring in detail why the painting was not a suitable subject for the augmented interpretation (the term I have come to use to explain what I am doing). I think that this would be a really good approach. With this in mind I think that I need to do a few things.

• Write the questions to test the test subjects understanding in relation to the augmented painting.
• Put together a short usability issue questionnaire (to identify the main user issues with the Povey version of the display).
• Run the pre trial with three test subjects (probably willing students).

I need to try and do this as soon as possible, as I have a conference coming up and it would be good to use the pre-trial as the focus.