At the top of this post is an image of the State Library of NSW. Because strangely, using Google Gemini’s Deep Research mode reminded me of researching there in years past. When I was a kid I would go to the State’s Mitchell library, start my research, get out a book, read a section, go to the footnotes for the references, find that new book and follow that trail of reference breadcrumbs. And its kind of the same with Google Gemini’s deep research mode.

Context

A big project that represented a fundamental change in the product, with a set deadline rocketed to the front of our roadmaps. With little time and with questions that needed months to unpack, and usability lab resources we didn’t have I took a turn with Google Gemini .

Briefing 

It was valuable to kick start the AI research with the same process as any other research with the team. The key product and usability questions were workshoped with the team and this formed part of the brief for Google Gemini in Deep Research mode. This team work prior was better in this circumstance than jumping straight to a prompt so we could get answers to very specific questions.

Lesson learned

• Treat google gemini like a research brief and not a google search.

Anatomy of the prompt

I used this format for the prompt.

I also added the prompt I used in the google gemini report as an appendix item and made it clear how the report was generated. 

Lessons learned

• Provide transparency and traceability to how the report was generated
• Preserve the prompt so others might learn from it or improve it

Follow citations for insight

With some wise skeptics in the team I spot checked the citations to validate them. While the citations weren’t made up there were a lot of that were not relevant at all, bar a keyword. These were mostly product sales pages not published research or information pages.

The report was good but what was very good was reading the original sources for key questions. More nuance and specificity was available reading the original articles. 

Also worth noting the range of sources – informed by the prompt – that ranged from social forums (e.g. reddit) to industry sources, and published research. The published research seemed to be exclusively from research gate. The industry sources were good and there were articles about deeper customer research that I found after checking these original sources. 

What google gemini seems best at is finding really good sources that may not have come up in a google search. It feels like a search supercharge. Although so is Perplexity so perhaps I could have started there.

The findings that are generated are more aggregate information retrieval than insight so while they were comprehensive they were still a bit ‘greige’. 

Reading the report it generated in full plus reading original sources enabled me to provide actual insight.

I did edit the exported google doc to provide specific answers from the follow on reading I did. Adding my additions in italics or as comments felt like providing transparency to what was machine generated and what had a human hand. 

Lesson learned

• Go to the source to check its relevance
• Follow the rabbit to find more specific sources – don’t shortcut follow on reading of quality detailed content.
• Use it as a search supercharge (RIP Google search)
• Treat the report like a live document – any reader should highlight or strike through or even delete what is good and delete what is bad
• You’ll get data and findings not insight. 

Comparing to Google Notebook ML

For an experiment I manually added most of the sources (citations), plus a couple of others that I came across, and some company originated documents to a Google ML notebook. 

I asked the Google notebook the same questions from the research brief. The answers were very similar to the google gemini report, a little more specific and a bit less generic. Plus the inline reference preview on hover is handy. 

Unfortunately there is no URL bulk upload so this took ages. Bulk upload of URLs can be done in Claude but not Google Gemini yet. Let’s hope this is added soon – or better yet that Google enables a Gemini to Notebook workflow.

One of the uploaded sources was an internal research report. The Google ML answers made no distinction between hypothetical recommendations and real research or industry best practice. Another good reminder that this is just word salad and you need to reserve some judgement and check its outputs.

Lessons learned

• Exclude recommendations, ideas, and similar source material from Google Notebook ML

Overall was it valuable?

Yes – the google gemini report found relevant industry and user research at speed which helped the team understand the design patters/product features in market.

The output also found great research we wouldn’t be able to replicate from a budget or time perspective. 

There are still outstanding questions we have that we can do primary and focussed research on but it provided a leg up that validated hypothesis and pushed assumptions over the line to speak with increased confidence to design choices.

The inputs need to be treated similarly to any research brief – that is, with thoughtful consultation and effort – even if that effort is just a prompt. The outputs too need to be read and scrutinised to go beyond the generic and get the best results.