Delphi method - a way to make better business decisions through collective wisdom
Making business decisions is challenging. Especially when the stakes are high and the available information is incomplete or ambiguous. In such situations, we often rely on the opinions of experts. But what to do when different experts have different opinions? How to choose the best solution when everyone looks at the problem from a different perspective?
What is the Delphi method?
The Delphic method is a systematic approach to decision-making that uses the collective wisdom of experts. It was developed by the RAND Corporation in the 1940s, initially for forecasting the development of military technology. The name refers to the ancient oracle at Delphi. Although, unlike mysterious predictions, the method is based on a rigorous scientific methodology.
Experts in a Delphic study do not have to be only people with scientific degrees. Life experience is also important here. The more different perspectives on the same phenomenon, the better. At the same time, all people invited to the study must be competent to answer the question.
The result of a Delphi study is always a number. This allows us to descend from the level of vague generalities (maybe, a lot, little, rarely, often, important, unimportant) to the level of specifics (23%, 1,290 patients, €77,258, priority 1, priority 2, priority 3) and also to quantify the level of expert consensus (expressed in standard deviation). This makes the Delphi method widely used in fields such as forecasting, strategic planning, risk assessment, and wherever we want to make decisions based on reliable data about some phenomenon, need, or future, and it is difficult to obtain it through other studies, such as laboratory experiments.
How it works. - Four pillars of the Delphi method's effectiveness
In a nutshell, a Delphi study involves an iterative and anonymous process of sharing information among a group of experts. Alternating rounds of numerical estimation and justification of one's position. Once a predetermined level of consensus has been reached, or after the stipulated round or rounds have been held, the final results come from the last round of estimation, and a record of the arguments can be used to explain them.
Delphi research can take many forms and variations, so it is worth looking at four basic features of the study that are absolutely essential:
Anonymity
Experts are not supposed to know who presented what factual information such as individual estimates and arguments. The purpose of anonymity is to maintain as much objectivity as possible. Anonymity encourages independent thinking, freely expressing one's thoughts and changing one's mind without “losing face.” It allows us to focus solely on the subject matter of the issue, and not on who is talking about it, whether they are liked by us and what social respectability they have. It also eliminates a number of negative effects of groupthink such as group pressure, shyness, harboring unpopular opinions, or following the opinion of those perceived as a leader or authority.
Iterability
The Delphi study assumes that each expert can improve his position. Therefore, the collection of information is conducted in rounds. Typically, two or three rounds are sufficient. Processes that are too long can lead to forced consensus - a situation in which experts change their answers mainly to end the survey already.
Statistical analysis
Both the survey result and the consensus measure should be objective statistical results. There are two elements that need to be calculated each time after the estimation phase. The mean - the main result of the phase, and the standard deviation - how far from the mean the typical expert's answer is. The smaller the measure of the standard deviation, the greater the consensus. This way of counting results allows us to maintain objective and transparent measurements.
Feedback
The Delphi survey assumes that experts revise their positions by seeing the responses of other panelists and comparing them with their own. This applies to both the aggregate numerical result of the round and the qualitative analysis. An important part of the qualitative process is revealing the reasons for giving one result and not another so that the whole group can decide based on the knowledge they have gathered together. A key tool here is argumentation, which aims to bring about consensus through dialogue.
How does Swarmcheck support each of the pillars?
We have built a system that allows you to conduct a Delphi survey in real-time. This means that instead of a long and cumbersome process of sending surveys, collecting responses, and compiling results, everything happens automatically in the system. As a result, results can be obtained even in a single meeting. There is also an option to set a time per round and work asynchronously, which may be needed for experts who cannot gather at the same time.
Swarmcheck APP takes the utmost care of the pillars of the method's effectiveness:
The numerical estimation process and the anonymous creation of a common argument map that unifies the discussion of the entire group for a given question throughout the process, without the need for the study moderator to develop conclusions.
The moderator's built-in tools for managing rounds allow for a smooth transition through the phases of the survey. The moderator sees the numerical results in real time and reveals them to the participants with a single click. In the discussion phase, the system tags new arguments on the same map, so everyone can see what the new information is and how it relates to previously gathered knowledge.
The system collects individual results and automatically, in real time, conducts statistical analysis for all rounds. The system also has additional analytical functions to analyze response ranges, as well as quantitative and graphical analysis of discussions.
Clear feedback is automatically generated for all participants. Summarizing the results is easy not only for the users but for the person ordering the survey. Unlike many other discussion methods in a Delphi study, the use of group argumentation mapping allows for verbal justification of where the final numerical results came from. With these features, preparing a report or scientific publication is fabulously easy.
Experience Delphi method
A computer-assisted Delphi study is a fast and proven method for gathering reliable information. By properly selecting a group and sticking to the four pillars of the method, you too can conduct your own Delphi study.
If you are wondering how to do it, write to us and we will give you a free consultation.
Bibliography
Khodyakov, D., Grant, S., Kroger, J., & Bauman, M. (2023). RAND methodological guidance for conducting and critically appraising Delphi panels. RAND. https://www.rand.org/pubs/tools/TLA3082-1.html
