In the book "Originals - How Non-Conformists Move the World" by Adam Grant, there's a passage worth your attention when working on that "big next idea" where you feel like you need confirmation from the market through research first.
Adam Grant analyzed why original ideas in different industries have been predicted by both test audiences and managers to fail, but instead became massive successes: The Harry Potter books, Seinfeld TV show, Star Wars film, Xbox etc.
"[...] neither test audiences nor managers are ideal judges of creative ideas. They’re too prone to false negatives; they focus too much on reasons to reject an idea and stick too closely to existing prototypes. And we’ve seen that creators struggle as well, because they’re too positive about their own ideas. But there is one group of forecasters that does come close to attaining mastery: fellow creators evaluating one another’s ideas."
All of the above 'guaranteed fails' survived and became successes because test audiences and managers have been ignored by the original creators. Instead, their gut feeling was fed with energy by other fellow creators:
"In Berg’s study of circus acts, the most accurate predictors of whether a video would get liked, shared, and funded were peers evaluating one another. When artists assessed one another’s performances, they were about twice as accurate as managers and test audiences in predicting how often the videos would be shared. Compared to creators, managers and test audiences were 56 percent and 55 percent more prone to major false negatives, undervaluing a strong, novel performance by five ranks or more in the set of ten they viewed."
Take-away: For original and creative ideas, get comfortable relying more on peer feedback. With this insight, you can more clearly understand potential frustrations when conducting market research with test audiences, when it comes to imagination, rather than validation. Simply put: Accept each other's evaluation even more to become more accurate predictors of what's to come. David Ogilvy summarized it pretty well:
The trouble with market research is that people don't think what they feel, they don't say what they think, and they don't do what they say.
Surprisingly, many companies are reluctant to accept this truth. Others don't need to be reminded of this truth. Instead, they trust fellow creators.
Therein lies your competitive advantage.