This blog post is about diary studies and how to go about conducting them.

What are diary studies?

Diary studies, otherwise known as User Research Diaries or “Cultural probes” were pioneered for use in design research by William “Bill” Gaver, Professor at Goldsmiths London. Interestingly he doesn’t analyse diary content, nor does he create scenarios or personas from them instead using them as a base from which to validate other data. He does not create personas, preferring instead to revisit the raw data.

What are diary studies good for?

Recruit carefully for diary studies

• Allow a little longer than usual for recruitment. It’s more important to recruit well than to recruit on time.
• Pay participants more. More effort is required of them.
• Screen participants yourself. Trust your first impression. If you don’t think they will be suitable, don’t recruit them.
• Note the anticipated schedule of activities of the participant in the screening process i.e. get an idea of what activities they plan to do over the period of your study just in case they are lying  and saying yes just for the money!
• It is harder to find participants who are willing to go to the effort of filling out a diary every day – so take the time getting the right people.
• Brief the participant about what effort is involved

• Their information is confidential
• Their identity will not be exposed
• They are free to withdraw from the study at any time

Brief participants about what is involved

DO

• Instruct participants how to use the diaries
• Remind participants what EFFORT is involved, the level of effort expected of them
• Give participants a checklist of the materials they need to return
• Provide written instructions – they will read it, and it saves you explaining at length
• If you can its better to have all participants start on the same day. Makes your job easier later when you have to track their diaries down.

Prepare diary materials thoughtfully

When preparing diary packs, prepare a diary for each week of the study, camera (optional), instructions, reply paid envelopes.
You need to make a few decisions upfront — do you want to use structured or unstructured diaries? Unstructured diaries give the participant a blank slate. Structured diaries set some questions or areas for response. I would recommend structured diaries. They give people a sense of what you want, otherwise it can be too hard for them to start. Consider including a survey, it gets people used to filling something in, as well as gathering useful information.
DO

• Label the diaries so you can tell who they came from when you receive them
• Offer participants a choice between a paper diary OR digital diary.

DONT

• Identify the participants

Choosing between paper and digital diaries

Paper diaries are good because they’re low fi, low fuss

Paper diaries are bad because:

• Participants tend to contribute longer answers
• Easy set up – I’ve used Posterous. A colleague has even given someone a Word document to email back. Posterous only requires the participant to send an email to submit their diary entry.

Using posterous:

• It’s not suitable for less technical participants
• It’s harder to set up a structured diary
• Because its unstructured, there can be less detail as to the “why” behind the activities.

Conducting photo diaries and providing cameras

• Photos bring the participant to life, and clients love them
• Great at showcasing the absurd (e.g. paperwork involved in financial processes)
• BUT people don’t know what to photograph. Let them know what you are looking for
• Don’t expect too many photos
• Why don’t people take photos? – people feel like spies, scared they could get in trouble, its an unnatural behaviour.

What’s worked

• Getting people to post back a memory card

What hasn’t worked

• Getting participants to email photos. Uploading is a chore!

Expect difficulty getting materials back

In 2005-06 Australia post delivered 94.9% of letters on time or early. Despite this many* diary participants in studies I have been involved with so far did not post their diaries back on time. Expect most diaries to come 2 days – 1.5 weeks later than they should. Expect that when you phone your participants they will say “it’s in the post”. Participants will lie about getting their homework done, and sending it to you.
DO

• Pay on receipt! But make a good faith payment before the participant starts, on receipt of the diary materials. No diary, no payment.
• Use reply paid EXPRESS envelopes for paper diaries. They arrive quicker, they’re traceable and they are a sign that the diaries need to be returned pronto!

Touch base with participants throughout the study

Debrief entries with participants. If conducting a long study, encourage them to participate more. Keep a log file to track volume of entries, who is contributing, who is not.

BUT remember that people have day jobs. its really hard to get in touch with participants during the day. You kind of get the sense that they may be avoiding you!
Ask them when its best to contact them when you are recruiting them. Be prepared that half of the effort in the project will be getting hold of participants.

DO

• Ask the best time to call
• Interfere as little as possible
• Offer gentle reminders

DON’T

• Spam your participants. Remember this is meant to be unobtrusive research.

Set some creative tasks

Tasks are fun and people are surprisingly creative with their responses. Tasks are great for seeing what the participant thinks about the process they are undertaking and great for eliciting their attitude, their emotions,  and how they see themselves in the process. The results provide powerful metaphors for the whole experience.
Example tasks you can set:

• If this process was an animal what would it be?
• What celebrity best represents you in the process Quote: “I feel like Jennifer Anniston. I’m on an emotional rollercoaster.”
• What object best describes the main person you are dealing with: “My xxxxx is like a stream locomotive. I know they will get there … eventually!”

DO

• Set tasks throughout the study, not all at the end

DON’T

• Expect everyone to get it, or to comply

End the study with an interview

Use diary studies to GROUND the participant in their actions during the interview or contextual inquiry.
DON’T

• Expect to yield insights from the diaries alone.

DO

• Prepare your interview questions from the diaries as you receive them
• Pick up the last diary at the interview
• Arrange the final time for the interview upfront
• Ensure the interview is conducted in the most appropriate environment, you still want to see the participant “in situ”

Thank you!

This blog post was taken from a presentation I gave at Different’s Friday Talks. Friday talks are our little learning and sharing fest.

All drawings were done by my lovely boyfriend Colin Stokes. Diary war stories came courtesy of my experiences and those of Matthew Ballesteros, Jason Crane and Vicki Lane at Different.

Recommended reading on diary studies

• http://www.sleepsurvey.net.au/the-sleep-survey/about/
• http://www.infodesign.com.au/usabilityresources/culturalprobes
• http://www.gurtle.com/ppov/2009/12/06/a-summary-of-user-research-methods#cultural-probes
• http://www.usit.com.au/2010/08/13/using-posterous-as-an-online-cultural-probe-user-research-diary/

The original presentation has also been published on Slide share.

Correction

* Thanks again to Vicki who pointed out to me that many of her participants did comply and did send diaries back on time. I had previously stated that all participants were late. Maybe I have just been unlucky.

Guardian infographic on public expenditure. "For 670 billion pounds, there must be a space for design there" -- Lauren Tan

Earlier this week Lauren Tan presented at Service Design Drinks on her university research paper. In it she looked at 2007 DOTT (Design of the Times) design projects in the public and social space.

“This PhD programme aims to identify and understand how design methodology is used in the public and social sector and the contributions it can make to the broader context of sustainable development.”
– Lauren Tan.  Reference: http://northumbria.academia.edu/LaurenTan

Tan researched the methodology of a number of agencies and within the context of service design found the role of the designer to be that of a:

• creator
• researcher
• provocateur
• facilitator
• social entrepreneur
• capability builder
• strategist

Lauren’s talk gave everyone the opportunity to compare their work with what is being done overseas. Lauren ran us through 2 case studies.

Case study 1. Alzheimer 100

The agency: Thinkpublic

Design role: Designer as co-creator

The problem: The difficulty faced by those recently diagnosed with Alzheimer’s and their support network of learning about the services available to them.

The challenge: Foreseeing the future of dementia services.

The approach: Co-design workshops with service provider stakeholders, researchers, carers and sufferers. These workshops were preceded by an extensive research phase that defined the themes of the design project.

The deliverable: Recommendations for a dementia adviser concierge service. A poster for carers to navigate the service network.

The outcome: Two years after the conclusion of the project the UK government published a strategy document. One recommendation, for a dementia advisor to facilitate easy access to care, support and advice following diagnosis, directly spoke to the research project i.e. the project was written into government policy. The Dementia Adviser Service is now being rolled out in the UK by the Alzheimer’s Society.

Case study 2. Low Carb lane

The agency: Live|Work

Design role: Designer as provocateur

The problem: Reduce carbon footprints, while not compromising heating needs, and while tackling fuel poverty.

The challenge: By 2016 all homes to have a net zero carbon foot print with all carbon output offset by household activities.

The approach: Design research with residents of Castle Terrace.

The deliverable: A financial product concept articulated through a scenario. Called “Saverbox” it is an interest free energy loan for energy saving measures. Repayments are based on actual energy savings.

The outcome: Several years after the project a government agency released a similar product  for small business.

Ultimately what this talk spoke to was the growing scope of design. User experience design and strategy is slowly colonising problems belonging to other fields. It can do so effectively with the inclusion of the user in defining the problem and sometimes even helping to design the solution and through the use of design tools to communicate its outcomes.

Related links

• A write up of the night is also available on the Service Designing Australia blog, the hosts of the night.
• Lauren Tan speaks more on research methodologies.

Senova's challenge: can we design our industry together?

Melis Senova began her presentation with this premise: if you interpret every choice as a design decision, you can look at your life as a designed experience. And concluded: if we all design our lives, can we design our industry?

Her presentation was a challenge to apply the principles and characteristics of user experience  design – collaboration, transparency, sharing, iterative, open, designerly, innovative, empathetic, cross disciplinary – to design our experience of working in our industry.

Melis Senova: attributes of user experience

Melis Senova: Business as usual

Zafer Bilda opened the night with a service design case study that followed a familiar working approach (1) define the business objective (2) survey the landscape with a comparative analysis (3) conduct observational research (4) create customer and staff personas (5) study service tools in use to design a new experience.

Perhaps we can take this working model and apply it to Senova’s challenge. Is a bar camp in order?

Service Design Drinks 6 was a full house.  Service Design Drinks 7 will be on 23 November and will be reviewing highlights from the upcoming Service Design conference in Berlin.

Related links

http://servicedesigning.com.au/2010/09/21/photos-from-sydney-service-design-drinks-thinks-6/

The talks couldn’t have been more different at this weeks service design drinks. Stephen Cox, Customer Experience Manager at Westpac opened the night with a presentation on touch-point workshops. Janna DeVylder from Meld Studios invited the audience to ponder whether the service designer has a role to play in the execution of projects. The first was a talk around design education, the second a discussion about design implementation.

Stephen Cox’s workshop is taken from At-One and a more detailed description of it is available at service-innovation.org. Participants are asked to design an experience for a persona across a journey using a range of touch-points cards. To see if the freshly designed experience stacks up, participants then consider if it works with a different persona; and if the experience is still coherent if touch-points are removed.

Stephen’s team is responsible for shifting Westpac culture to be more customer centric. But it seems his team could also affect  the corporate culture itself. The sessions involve participants from different areas of the bank that may otherwise not get the chance to work together, let alone meet one another. The workshop sessions have participants design their products and services for touch-points they may not have traditionally considered to be in their remit, and to consider their output as existing within a broad range of experiences.

The next discussion asked whether the service designer should get their hands dirty in implementation. Janna DeVylder asked “How do you design for the implementation of service?“ It’s an interesting question that goes to the heart of a utilitarians view on user experience design itself—that its easy to conjure up the perfect solution when you don’t have to be responsible for rolling it out.

At first the audience seemed to be in some sort of consensus that a service designer should play a role in implementation and all that is needed for this to happen is a suitable metaphor to communicate what they are doing in the trenches, be that director or composer. As the discussion unfolded other considerations arose:

• Is it just a matter of passing on appropriate tools to get the client to the next stage?
• How do you work towards change management?
• How do you make prototypes for service design?
• How do you test concepts?

Having worked in implementing web projects I advocated that producers are needed—people that are responsible for the detail but who understand the vision. Some people thought that I was still riffing on the director metaphor, but I wasn’t. By the end of the discussion everyone agreed concept and implementation are two very different types of engagements.

The first presentation was about building design thinking in a corporate culture. The second was about the designer playing a role end to end. Advocates need to be built within organisations. Someone else is ultimately responsible for the products, services and experiences that we may be designing. They are the ones who need to take ownership of the ideas as they are the best placed in seeing them through. I don’t think that concepts don’t get implemented because a designer isn’t there to realise their vision. Concepts don’t get implemented when they don’t have people championing their cause.

If you have not been, Service Design Drinks is a casual meet-up, where guest speakers present and take questions from the audience. It’s held regularly at the Trinity Bar in Surry Hills. The fourth event was held on 18 May and was attended by 30 or so user experience designers, including a team from Different.

The last event was a panel discussion moderated by Damian Kernahan, from Proto Partners. The panellists were:

• Opher Yom Tov, formally of Ideo, now working with BT
• Rod Farmer, Co-founder and Director of Research, Mobile Experience
• Susan Wolfe, Managing Director, Optimal Experience
• Faruk Avdi, from the NSW Department of Education and Training

The opening question put to the panel was “What is service design?”

• Opher Yom Tov: Weaving together discreet experiences in an ongoing relationship.
• Rod Farmer: Meaningful value at the point of interaction.
• Susan Wolfe: Thinking about the broader context. Experience design is service design.
• Faruk Avdi: The ecosystem a product exists; experience design is a subset of service design.

Much of the early discussion veered around product design, defined as designing for the experience and the utility that it brings, and not focussing exclusively on the product itself.

A conversation followed about the importance of leadership and change management in service design projects. Several challenges were discussed:

• How do you model relationships?
• If the bar is always moving, how do you exceed the standard that has been set?
• Companies do not measure people’s performance on delivering a whole service. What KPIs should be set to help put service initiatives into operation?

The final question put to the panel was, who’s doing it well? This was best answered by Ant from Different who said that the best service experiences are being had at the ma and pop stores. Everyone agreed; the challenge is getting big organisations behaving like small business. (Interestingly, Mark Pollard made the exact same analogy at a Social Media Club event when asked about the challenges businesses face in participating effectively in social media).

It was a lively discussion, where the panel conceded that service design is a fairly new term. It was interesting to see that the industry is still forming its opinions as to what service design really is, and how it differs to experience design and also interesting to see how the field of user experience is expanding and defining itself.

An edited version of this post first appeared on the blog Different UX

Postscript June 17, 2010

Audio of the event has been published on the Service Design Drinks blog:
http://servicedesigning.com.au/2010/06/17/service-design-thinks-drinks-4/

* Warning: this film review contains spoilers *

Ryan Bingham (played by George Clooney) is in the business of firing people. He travels the US first class, arrives at his destination to do what his clients are too scared to do themselves.

I couldn’t help but think of work as I watched this film. Not because people are being fired! But because the film is a great illustration of service design, and the pitfalls of an imperfect software implementation.

The opening of the film follows Ryan as he fast tracks airport check in with his first class frequent flyer VIP credentials. He receives personalised greetings and preferential treatment at every turn. Any interaction with the airport, airline, rental car service, hotel, is geared toward his status. Ryan is aware of the service he’s getting, he’s earned it,  it’s “the perfect experience”.

This is until a young go-getter, Natalie Keener, comes along and in an efficiency drive to cut costs and optimise the contract-for-fire business designs a system to take Ryan off the road.

The video conferencing set up is accompanied by a model that maps out the script for call centre operators to fire someone. The workflow includes what to do when the firee at the other end goes postal.

Can technology and a perfect workflow replace Ryan, and worse — stop his frequent flyer miles?

There are some great examples here for every experience  architect, and even a case study in why user centred design should be part of any software project.

Rating: 4 stars from me. 5 stars for EAs.

Related Links

Film review from a frequent flyer perspective. Hilarious.
http://www.examiner.com/examiner/x-2184-Frequent-Flyer-Examiner~y2009m12d4-Up-In-The-Air-with-George-Clooney-movie-review-by-a-frequent-flyer

Up in the Air Tweets. Great visualisation
http://www.upintheairtweets.com/

Up in the Air Official Website
http://www.theupintheairmovie.com