Delphi Method for Expert Risk Assessment

Delphi Method for Risk Assessment

What is the Delphi Method?

The Delphi method is a structured communication technique for gathering and refining expert opinions through iterative rounds of anonymous feedback.

When to Use Delphi

• Limited historical data
• Complex, novel risks
• Need for consensus
• Expert judgment required
• Emerging risk assessment

Core Characteristics

1. Anonymity: Experts respond independently
2. Iteration: Multiple rounds of feedback
3. Controlled Feedback: Aggregated responses shared
4. Statistical Analysis: Results analyzed quantitatively

Traditional Delphi Process

Round 1: Initial Survey

• Open-ended questions
• Experts provide estimates independently
• No interaction between participants

Round 2: Feedback and Revision

• Share anonymized summary statistics
• Participants review group responses
• Revise estimates if desired
• Explain reasoning for outlier positions

Round 3+ : Convergence

• Continue iterations until:
  • Consensus reached
  • Stable disagreement understood
  • Predetermined rounds completed
• Typically 3-4 rounds sufficient

Risk Assessment Applications

Likelihood Estimation

• Expert estimates of event probability
• Calibrate using reference classes
• Aggregate for consensus view

Impact Assessment

• Qualitative and quantitative estimates
• Multiple impact dimensions
• Range estimation (min, most likely, max)

Emerging Risk Identification

• Identify potential future risks
• Assess importance and timing
• Prioritize monitoring

Scenario Probability

• Assign probabilities to scenarios
• Validate scenario plausibility
• Inform stress testing

Facilitation Guidelines

Expert Selection

• Diverse expertise
• Relevant experience
• Independent thinkers
• 7-15 participants ideal

Question Design

• Clear, unambiguous wording
• Avoid leading questions
• Include calibration questions
• Use consistent scales

Feedback Presentation

• Summary statistics (median, IQR)
• Anonymized rationale for outliers
• Visual displays (box plots, histograms)
• Highlight areas of consensus/divergence

Convergence Criteria

• Interquartile range stability
• Kendall's coefficient of concordance
• Predetermined number of rounds
• Practical time constraints

Variations

• Real-time Delphi: Online, continuous
• Policy Delphi: Focus on options, not consensus
• Mini-Delphi: Single meeting, rapid iteration
• Decision Delphi: Paired with decision analysis

Limitations

• Time-intensive
• Expert availability
• Groupthink possibility (despite anonymity)
• Not suitable for well-quantified risks