Context
At the heart of the Glimpse Project is our methodological claim that we can elicit the texture of spiritual-yearning and religious-belief by every person, by asking a single question.
Perhaps a safer claim to make is: this method is the “least imperfect” method for eliciting this data, and doing it in a way that allows us to process the data using advanced quantitative methods.
This post explains the parameters of the One Perfect Question we’re aiming to develop, describes some of our beta-testing of one of the candidate-questions, and talks about where we’ve used a similar “one perfect question” method in other research.
The parameters of the One Perfect Question
The first part of the study seeks to develop a “one perfect question” that can help to reveal the nature of Spiritual Yearning.
We believe that this instrument will deliver a breakthrough in researchers’ and practitioners’ ability to explore Spiritual Yearning and religious-belief. This question could catalyse public engagement and everyday private conversations about spirituality and belief.
We have not arrived at the right question yet - although we have trialled one candidate question with promising results, and we have emerging-hypotheses around some of the alternate candidate questions (see phase 1 of the Approach section). Here are the four pre-requisite attributes that this “one perfect question” must have:
Open: so that the answers it elicits are given without bias or preconception
Single: so its answers can be interpreted and mapped by Machine Learning to reveal new layers of meaning
Universal: so that it can be asked to anybody and everybody, irrespective of their level of religious adherence or cultural/religious context
Simple and memorable: So it can be used for public-engagement and everyday conversation.
Early Beta Testing of One Candidate Question
One version of the “magic question” we’ve devised (in beta-testing) for this Glimpse project is:
The “magic question” we’ve devised (in beta-testing) for this Glimpse project is:
This article gives some context and background about this new research-method.
Background to this Research Method
The Magic Key Question method is used in consumer and behavioural research to elicit intuitive, high-veracity responses in a format that enables quantitative analysis. It’s useful in eliciting a simple answer from a complex set of unformed beliefs about a topic or experience; and to obtain answers in a structure that allows for advanced data-analysis.
We at Neighbourly Lab use this method in our ethnographic-research - to try to “dig deeper” into people’s intuitive or unconscious feelings. And similar questions are often used by qualitative researchers, often couched as “thought experiments”.
Examples of similar magic-key questions in other contexts:
Perhaps the most widely known example of the deprivation thought-experiment is from the BBC Radio Programme “Desert Island Discs”. It’s been running since the 1940’s and asks celebrities “If you were stranded on a desert-island and could take only one luxury-item with you, what would it be?” The answer to that question (arguably) elicits some new information about the celebrity-guest. On an edition I heard recently, Ed Sheeran asked for a lifetime supply of tomato-ketchup.
The first major application of this method in corporate-research was asking participants to give a numerical score, and was popularised by the “Net Promoter Score” method developed by Fred Reichfeld in 2004-6*. For example, if somebody has just returned from a holiday to the “Star Hotel”, their travel-agent might email them to ask the NPS question “On a scale of 0 to 10, how likely are you to recommend the Star to a friend or colleague?”.
Now, with the advances in Natural Language Processing and Machine-Learning powered methods of data-clustering and analysis, researchers are able to elicit and process unstructured verbal responses, as well as structured numerical ones. Our goal is to design a perfect question or set of questions that have the same high validity as methods such as the Net Promoter Score, but enables respondents to answer using their own words. The central property of the question is that it will allow respondents to give the research team information that is both accurate and otherwise hard to articulate.
The Key Success Factors for Framing the Magic-Key Question in the best way
To frame the “One Magic Key” question in the best-possible way, it requires adherence to these guidelines:
A: The question must be designed for maximum clarity and answerability.
The question should be written in plain language. That is, language that is simple, direct, and concise, avoiding jargon or technical terms that would not be easily understood by most people.
The question should be written in neutral language, avoiding words that suggest a judgement or opinion, either explicitly or implicitly. The test we always apply is “Does the question avoid assuming anything about the answerer's views?”
In applying these tests, we follow guidelines from experts in electoral referendum design. For example here from the UK Electoral Commission.
B: The “one Magic Key” question must be positioned within the Questionnaire in a way that minimises framing bias.
In the run-up to the Magic-key question, we’ll introduce the context in a very general neutral way, to signal to the answerer that the question is about what they think, but to avoid any preconceptions about religion, spirituality etc, and also to seek to avoid any directiveness about the level of committed position (eg avoid words like “belief”) etc.
We’ll ask a simple permission-question to prepare the respondent for the “one magic question” such as, “The next question asks about what you think about things, is that ok for us to ask that question?”
The questionnaire will ask respondents several other questions about their or their family’s religious affiliation, ethnicity, frequency of thinking about spirituality etc: to avoid these questions affecting how respondents answer the “magic-key question” we will ask these questions after they’ve answered the “magic-key” question.
C: the question should be phrased in a way that makes it memorable.
The memorability of the one magic question allows it to have onward use and become part of the public-discourse.
Our experience in testing these question about spiritual conceptions has showed that respondents are interested in, and can remember, their response to the question. Often the answerer hears themselves expressing something for the first time, and perhaps even surprising themselves with the answer. Researchers using this method frequently hear the respondent using language that indicates that they are themselves articulating the answering in a fresh way, eg “I suppose it’s the canals that makes Batley Batley, isn't it”** or “I’d have to say 7 or 8”. When asked in a group setting, people began to explore their own answers and others, prompting conversations in which respondents felt that they had learned something about themselves and their companions. Respondents to surveys that use this method are therefore likely to share that answer with other people, ask the question to other people to hear other answers, and be curious to find out the overall results of the surveys. Importantly, it’s the accessibility and memorability of the singular question that facilitates this type of onward engagement, in contrast to traditional long-form surveys with closed questions that don’t have this same potential.
*Reichheld, Fred. The ultimate question 2.0 (revised and expanded edition): How net promoter companies thrive in a customer-driven world. Harvard Business Review Press, 2011.
*This example is taken from a recent piece of research completed by Neighbourly Lab for the UK Government which sought to understand the degree of “pride in place” amongst the residents of Batley (a town in Yorkshire). We asked respondents “What makes Batley Batley?” as our one magic question to unlock what people most associate Batley with and their attitudes towards the town.
Comments