How do participants really feel about AI interviews? We conducted a research battle royale to find out.

AI-moderated research promises interview-quality insight at survey speed and scale, but most of the conversation focuses on the researcher’s side. What happens when you ask the people answering the questions?

To find out how respondents actually feel about being interviewed by AI, our in-house research team conducted what they deemed a “research battle royale” of more than 1,000 interviews across three methods: an unmoderated survey, AI-moderated interviews, and moderated interviews.

I’ll take you through their findings below.

How we structured the battle royale

For our team’s study, participants were randomly assigned to three study types: 

• A static Typeform survey (N=501)
• An AI-moderated study using our internal Research Flow product (N=501) 
• A human-moderated Zoom interview (N=51, a benchmark capped by how many interviews a team can run)

The topic was an important one for us: the buying journey for online subscriptions. 

All three methods used the same 10 core questions with open-ended follow-ups, and every study invited participants to share their thoughts and experiences through recorded video responses.

In the survey, the follow-up never changed: when using Research Flow, it adapted to participants' responses, and in moderated interviews, our research team used their best judgment, built on years of user research experience.

Our research aimed to answer a few key questions:

• Who emerged victorious in the battle royale overall in terms of quality of insights, as measured by the time spent on the study, response length, and richness of those responses?
• Which method held respondents’ attention from start to finish?
• Which method was most enjoyable?
• Did respondents feel like they were able to express their true thoughts and opinions?

A research battle royale for the ages: Our findings

So which battle royale method ruled them all? By and large it was the AI-moderated interview format. Here’s what our analysis uncovered:

1. Participants spent more than double the time on AI interviews

Research Flow participants spent 2x more time completing the study compared to static surveys and about half the time compared to a live, moderator-led interview.

Click to expand

2. AI interviews lead to longer responses

Research Flow participants used 4.7x more words per open-ended question than static surveys and about the same number of words than when completing human-moderated interviews.

Click to expand

3. Responses in AI interviews also had greater depth

Research Flow participants were 2.5x more likely to provide contextual details when answering compared to static surveys and 1.3x more likely compared to human-moderated interviews.

Click to expand

Which method held the most attention?

Participants in our AI-moderated interviews via Research Flow reported paying significantly more attention to the study compared to other modalities.

In response to our research question, “Did the study hold your attention from start to finish?”, 61% of the participants in the Research Flow-enabled study said yes, the highest engagement among the three methods. 

Click to expand

What about the delight factor?

Our next question, “Overall, how much did you enjoy completing this study?” yielded similar findings, with 53% of Research Flow participants reporting more enjoyment from the study compared to other modalities.

Click to expand

Were responses more honest and candid?

When we asked, “Did you feel like you were able to express your true thoughts and opinions?” we saw a similar response: Research Flow participants reported being significantly more open and honest compared to other modalities (71%).

Click to expand

How do participants feel about AI interviews? Final thoughts.

So, how do research participants really feel about AI-moderated interviews? Overwhelmingly positive, according to our team’s research. Here’s a quick rundown:

• The participants were more deeply engaged, gave richer answers to questions, and found the experience more enjoyable than the two other methods
• Research Flow answers were 2.5x more likely than surveys to include context (e.g., past experiences, expectations, relationships, emotions), and 1.3x more likely than interviews
• Surprisingly, people were not less shy in talking to the robots: 71% of the study participants said they could fully express their true thoughts

As important, a reality check: our analysis showed that moderated interviews and unmoderated surveys still have their place in research based on participants’ feedback. One major takeaway from our analysis was that quality goes beyond an either/or quant versus qual proposition—a mixed methods approach can be equally beneficial to gleaning the strongest insights. It could then come down to a question of resourcing (i.e., time and scale), to determine the best approach for your organization.