Tag: diary studies

Categories
Research Service design

The Dos and Don’ts of Diary Studies

What are diary studies?

Diary studies, otherwise known as User Research Diaries or “Cultural probes” were pioneered for use in design research by William “Bill” Gaver, Professor at Goldsmiths London. Interestingly he doesn’t analyse diary content, nor does he create scenarios or personas from them instead using them as a base from which to validate other data. He does not create personas, preferring instead to revisit the raw data.

Diary studies are used in longitudinal research — looking at people over a longer period of time than a typical Contextual Inquiry or interview can allow; and researching people when you could not otherwise be there with them.

What are diary studies good for?

  • Great for understanding the activities undertaken by participants, what they actually do.
  • Opportunity to witness subtle behaviour as participant can note activities they may not otherwise recall in a typical research interview, or contextual inquiry
  • Better than just a contextual inquiry for understanding the amount of separate actions and activities undertaken
  • Great for data on customer tasks and finding out the detail in these sequences
  • Great for witnessing all of the channels/touch points a customer encounters — so its perfect for service design, and the gathering of data for customer journeys
Consider a mix of research techniques, priming participants with diaries before your interview or contextual inquiry

Recruit carefully for diary studies

  • Allow a little longer than usual for recruitment. It’s more important to recruit well than to recruit on time.
  • Pay participants more. More effort is required of them.
  • Screen participants yourself. Trust your first impression. If you don’t think they will be suitable, don’t recruit them.
  • Note the anticipated schedule of activities of the participant in the screening process i.e. get an idea of what activities they plan to do over the period of your study just in case they are lying  and saying yes just for the money!
  • It is harder to find participants who are willing to go to the effort of filling out a diary every day – so take the time getting the right people.
  • Brief the participant about what effort is involved
And of course remember to tell participants:
  • Their information is confidential
  • Their identity will not be exposed
  • They are free to withdraw from the study at any time

Brief participants about what is involved

DO

  • Instruct participants how to use the diaries
  • Remind participants what EFFORT is involved, the level of effort expected of them
  • Give participants a checklist of the materials they need to return
  • Provide written instructions – they will read it, and it saves you explaining at length
  • If you can its better to have all participants start on the same day. Makes your job easier later when you have to track their diaries down.
DON’T
  • Don’t be leading about what content you are after of them, written, photographic or otherwise.

Prepare diary materials thoughtfully


When preparing diary packs, prepare a diary for each week of the study, camera (optional), instructions, reply paid envelopes.
You need to make a few decisions upfront — do you want to use structured or unstructured diaries? Unstructured diaries give the participant a blank slate. Structured diaries set some questions or areas for response. I would recommend structured diaries. They give people a sense of what you want, otherwise it can be too hard for them to start. Consider including a survey, it gets people used to filling something in, as well as gathering useful information.

DO

  • Label the diaries so you can tell who they came from when you receive them
  • Offer participants a choice between a paper diary OR digital diary.

DON’T

  • Identify the participants

Choosing between paper and digital diaries

Paper diaries are good because they’re low fi, low fuss. Paper diaries are bad because:

  • You ideally want people to carry them around and record things on the spot, but you can’t really tell if they do or not.
  • They have to go to the effort of posting it back to you
Digital diaries are good because:
  • Participants tend to contribute longer answers
  • Easy set up – I’ve used microblogging service Posterous in the past (sadly no longer around). A colleague has even given someone a Word document to email back.

Using microblogging and digital diary services:

  • Email submission is as good as instant
  • Easy for participants to include photos
  • Easy to track participants
  • Easy for participants to include you in on relevant emails and send you relevant web pages
  • You can use comments to ask questions
Digital diaries are bad because:
  • It’s not suitable for less technical participants
  • It’s harder to set up a structured diary
  • Because its unstructured, there can be less detail as to the “why” behind the activities.

Conducting photo diaries and providing cameras

  • Photos bring the participant to life, and clients love them
  • Great at showcasing the absurd (e.g. paperwork involved in financial processes)
  • BUT people don’t know what to photograph. Let them know what you are looking for
  • Don’t expect too many photos
  • Why don’t people take photos? – people feel like spies, scared they could get in trouble, its an unnatural behaviour.

What’s worked

  • Getting people to post back a memory card

What hasn’t worked

  • Getting participants to email photos. Uploading is a chore!

Expect difficulty getting materials back

In 2005-06 Australia post delivered 94.9% of letters on time or early. Despite this many* diary participants in studies I have been involved with so far did not post their diaries back on time. Expect most diaries to come 2 days – 1.5 weeks later than they should. Expect that when you phone your participants they will say “it’s in the post”. Participants will lie about getting their homework done, and sending it to you.

DO

  • Pay on receipt! But make a good faith payment before the participant starts, on receipt of the diary materials. No diary, no payment.
  • Use reply paid EXPRESS envelopes for paper diaries. They arrive quicker, they’re traceable and they are a sign that the diaries need to be returned pronto!

Touch base with participants throughout the study

Debrief entries with participants. If conducting a long study, encourage them to participate more. Keep a log file to track volume of entries, who is contributing, who is not.

BUT remember that people have day jobs. its really hard to get in touch with participants during the day. You kind of get the sense that they may be avoiding you!
Ask them when its best to contact them when you are recruiting them. Be prepared that half of the effort in the project will be getting hold of participants.

DO

  • Ask the best time to call
  • Interfere as little as possible
  • Offer gentle reminders

DON’T

  • Spam your participants. Remember this is meant to be unobtrusive research.

Set some creative tasks


Tasks are fun and people are surprisingly creative with their responses. Tasks are great for seeing what the participant thinks about the process they are undertaking and great for eliciting their attitude, their emotions,  and how they see themselves in the process. The results provide powerful metaphors for the whole experience.
Example tasks you can set:

  • If this process was an animal what would it be?
  • What celebrity best represents you in the process Quote: “I feel like Jennifer Anniston. I’m on an emotional rollercoaster.”
  • What object best describes the main person you are dealing with: “My xxxxx is like a stream locomotive. I know they will get there … eventually!”

DO

  • Set tasks throughout the study, not all at the end

DON’T

  • Expect everyone to get it, or to comply

End the study with an interview


Use diary studies to GROUND the participant in their actions during the interview or contextual inquiry.

DON’T

  • Expect to yield insights from the diaries alone.

DO

  • Prepare your interview questions from the diaries as you receive them
  • Pick up the last diary at the interview
  • Arrange the final time for the interview upfront
  • Ensure the interview is conducted in the most appropriate environment, you still want to see the participant “in situ”

Thank you!

This blog post was taken from a presentation I gave at Different’s Friday Talks. Friday talks are our little learning and sharing fest.

All drawings were done by my lovely boyfriend Colin Stokes. Diary war stories came courtesy of my experiences and those of Matthew Ballesteros, Jason Crane and Vicki Lane at Different.

Recommended reading on diary studies

The original presentation has also been published on Slide share.

Correction and updates

* Thanks again to Vicki who pointed out to me that many of her participants did comply and did send diaries back on time. I had previously stated that all participants were late. Maybe I have just been unlucky.

Earlier versions of this article wrote about using micro blogging tool Posterous for digital diary studies. Sadly, what was the best most stable microblogging service was acquired and disappeared soon after. Since then digital diary study software has emerged. Try it out or see if the likes of Tumblr works for you. The tool with the least steps and most minimal learning curve will always be best.

Categories
Research

Mappiness: the ultimate diary studies project

Thanks to Bec for introducing me to Mappiness and this video. Should be interesting for anyone who has run a diary studies project.