By Kent E. Frese, Ph.D. — Industrial-Organizational Psychologist and Founder, TeamLMI
The High Cost of "Going With Your Gut"
Hiring decisions are among the most consequential choices any organization makes. A single mis-hire at the managerial level can cost an organization between one and three times the position's annual salary when accounting for recruitment costs, lost productivity, onboarding expenses, and the ripple effects on team morale (Society for Human Resource Management, 2022). Yet despite these stakes, many organizations still rely on unstructured interviews — free-flowing conversations where each candidate is asked different questions, evaluated on different criteria, and judged largely by the interviewer's intuition.
The research is unambiguous: unstructured interviews are one of the least effective selection methods available. In a landmark meta-analysis, Schmidt and Hunter (1998) found that unstructured interviews have a validity coefficient of just .38 for predicting job performance, while structured interviews achieve a validity of .51 — a substantial improvement that translates directly into better hires, lower turnover, and stronger organizational performance. More recent meta-analytic work by Sackett and colleagues (2022) has reaffirmed these findings, placing structured interviews among the top-tier selection tools alongside cognitive ability tests and work sample assessments.
So why do unstructured interviews persist? The answer lies in a collection of well-documented cognitive biases. Interviewers tend to form rapid first impressions within the first few minutes — sometimes seconds — and then spend the remainder of the conversation seeking confirmation (Barrick, Swider, and Stewart, 2010). They are swayed by similarity-to-self bias, halo effects, and contrast effects that have nothing to do with the candidate's ability to perform the job. Structured interviews are specifically designed to neutralize these biases, replacing subjective "gut feelings" with systematic, evidence-based evaluation.
What Makes an Interview "Structured"?
A structured interview is not simply a list of prepared questions. It is a comprehensive selection methodology built on three core principles: standardization, job relevance, and systematic scoring. When all three elements are present, the interview becomes a reliable measurement instrument rather than a casual conversation.
Standardization
Every candidate for a given role is asked the same questions, in the same order, under comparable conditions. This ensures that differences in interview ratings reflect genuine differences in candidate quality — not differences in the questions they happened to be asked. Standardization also means that interviewers follow a consistent protocol for probing and follow-up, reducing the influence of interviewer-specific tendencies.
Job Relevance
Every question in a structured interview is directly linked to a competency, behavior, or knowledge domain identified through a job analysis. This ensures that the interview measures what actually matters for success in the role. Two types of questions dominate structured interviews:
- Behavioral questions ask candidates to describe specific past experiences that demonstrate a target competency. The underlying premise — rooted in behavioral consistency theory (Wernimont and Campbell, 1968) — is that past behavior is the best predictor of future behavior.
- Situational questions present candidates with hypothetical job-relevant scenarios and ask how they would respond. Developed from goal-setting theory by Latham, Saari, Pursell, and Campion (1980), these questions assess judgment, problem-solving, and decision-making in context.
Both question types significantly outperform the vague, open-ended prompts typical of unstructured interviews ("Tell me about yourself," "What's your greatest weakness?"). Research by Huffcutt, Conway, Roth, and Stone (2001) demonstrates that behavioral and situational questions tap into distinct but complementary dimensions of candidate capability, and using both types together provides the most complete picture.
Systematic Scoring
Perhaps the most critical — and most frequently overlooked — element of structure is the use of anchored rating scales. For each question, interviewers reference a pre-developed rubric that defines what constitutes a poor, adequate, good, and excellent response. This transforms evaluation from a subjective impression ("I liked that answer") into a criterion-referenced judgment ("That response demonstrated three of the four behavioral indicators for this competency"). Campion, Palmer, and Campion (1997) identified anchored rating scales as one of the single most important components of interview structure for increasing predictive validity.
A Step-by-Step Guide to Building a Structured Interview
Creating an effective structured interview guide requires deliberate effort up front, but the payoff is substantial. The following framework provides a practical, research-informed process that any organization can implement.
Step 1: Conduct a Job Analysis
Before writing a single question, identify the critical competencies, knowledge areas, and behaviors required for success in the role. This can be accomplished through a combination of subject matter expert interviews, review of existing job descriptions, observation of high performers, and analysis of critical incidents. The goal is to produce a prioritized list of five to eight core competencies — enough to capture the essential requirements of the role without making the interview unwieldy.
For example, a job analysis for a project manager might identify the following competencies: strategic planning, stakeholder communication, conflict resolution, results orientation, and team leadership.
Step 2: Develop Competency-Based Questions
Write two to three questions per competency, mixing behavioral and situational formats. Behavioral questions should use the "STAR" prompt structure — asking candidates to describe a specific Situation, the Task they faced, the Action they took, and the Result they achieved. Situational questions should present realistic, job-relevant dilemmas that require candidates to articulate their reasoning and approach.
Keep the language clear and free of jargon. Avoid leading questions or questions that telegraph the desired answer. Each question should be open-ended enough to allow candidates to demonstrate the depth and breadth of their experience.
Step 3: Build Anchored Rating Scales
For each question, develop a scoring rubric on a five-point scale with behavioral anchors. A "1" should describe a response that is vague, irrelevant, or demonstrates a lack of the target competency. A "3" should describe an adequate response that meets basic expectations. A "5" should describe a response that provides specific, compelling evidence of the competency at a high level. Include example responses at each anchor point to calibrate interviewers and reduce rater drift.
Step 4: Train Your Interviewers
Even the best-designed interview guide will underperform if interviewers are not trained in how to use it. Training should cover the rationale for structured interviewing, how to ask questions without leading the candidate, how to take effective notes, how to use the rating scales, and how to avoid common rating errors such as halo effect, leniency bias, and central tendency. Research by Levashina, Hartwell, Morgeson, and Campion (2014) shows that interviewer training is one of the most cost-effective interventions for improving hiring quality.
Step 5: Pilot, Evaluate, and Refine
Before full deployment, pilot the interview guide with a small group of candidates or internal role-players. Gather feedback from interviewers on question clarity, scoring ease, and time management. Evaluate inter-rater reliability — the degree to which different interviewers arrive at similar scores for the same candidate. Use these data to revise questions, sharpen anchors, and improve the overall process. A structured interview is a living document that should be updated as roles evolve and new performance data become available.
Behavioral Interview Questions Mapped to Competencies
The following examples illustrate how behavioral and situational questions can be linked to specific competencies. These are designed for a mid-level leadership role but can be adapted for any position by adjusting the competency framework to match the job analysis.
Leadership and Influence
- Behavioral: "Describe a time when you needed to gain buy-in from a team or stakeholder group that was resistant to a change you were proposing. What was the situation, what actions did you take, and what was the outcome?"
- Situational: "Imagine you have been asked to lead a cross-functional initiative, but two department heads have publicly expressed skepticism about the project's value. How would you approach building their support?"
Problem-Solving and Decision-Making
- Behavioral: "Tell me about a complex problem you faced at work where the right course of action was not immediately clear. Walk me through how you analyzed the situation, what options you considered, and what decision you ultimately made."
- Situational: "You discover that a product your team launched three months ago has a quality issue affecting approximately 10% of customers. Fixing it will require pulling resources from a major upcoming initiative. How would you approach this decision?"
Communication and Collaboration
- Behavioral: "Give me an example of a time when you had to communicate a difficult or unpopular message to a team or colleague. How did you approach it, and how did the other party respond?"
- Situational: "You are leading a project team and notice that two team members have a persistent interpersonal conflict that is affecting the group's productivity and morale. What steps would you take to address it?"
Results Orientation and Accountability
- Behavioral: "Describe a situation where you were responsible for achieving a challenging goal or target. What obstacles did you encounter, and what specific steps did you take to stay on track?"
- Situational: "Midway through a critical project, you realize that your team is unlikely to meet the original deadline without compromising quality. How would you handle this situation with your team and your stakeholders?"
Adaptability and Learning Agility
- Behavioral: "Tell me about a time when you were asked to take on a responsibility or role that was significantly outside your comfort zone or area of expertise. How did you approach it?"
- Situational: "Your organization is implementing a new technology platform that will fundamentally change how your team operates. Many team members are apprehensive. How would you manage both your own transition and your team's adoption of the new system?"
Notice that each question is open-ended, competency-specific, and designed to elicit concrete evidence of behavior rather than abstract self-assessments. When paired with anchored rating scales, these questions give interviewers a reliable framework for distinguishing candidates who talk about competencies from those who demonstrate them.
Beyond the Interview: Building a Complete Selection System
Structured interviews are powerful, but they are most effective as part of a multi-method selection system. Research consistently demonstrates that combining assessments that measure different constructs — such as cognitive ability, personality, and behavioral tendencies — produces higher overall predictive validity than any single method alone (Schmidt and Hunter, 1998). This is because each method captures a different facet of candidate capability, and combining them reduces the measurement error inherent in any one tool.
For example, pairing a structured interview with a validated behavioral assessment such as DISC — which measures observable behavioral tendencies including dominance, influence, steadiness, and conscientiousness — provides insight into how a candidate is likely to interact with colleagues, manage stress, and approach tasks. Adding a personality inventory like the ELLSI, which assesses deeper trait-level characteristics, helps predict longer-term cultural fit and leadership potential. Cognitive ability testing rounds out the picture by assessing the capacity for learning, reasoning, and problem-solving that underlies performance in complex roles.
The key is that each component of the selection system should be validated against the specific competencies and performance outcomes that matter for the role. Science-based hiring processes integrate these tools into a coherent framework where each assessment contributes unique, non-redundant information to the hiring decision.
The goal of a selection system is not to find the "perfect" candidate — it is to systematically increase the probability of making a good hire while reducing the probability of making a bad one.
Organizations that adopt structured, multi-method selection processes routinely see measurable improvements in quality of hire, time to productivity, employee retention, and even team performance. These are not marginal gains. Research by Schmidt, Oh, and Shaffer (2016) estimates that the economic value of moving from low-validity to high-validity selection methods can reach hundreds of thousands of dollars per hire for professional and managerial roles.
Making the Shift: From Intuition to Evidence
Transitioning from unstructured to structured interviewing requires more than a new set of questions. It requires a shift in organizational mindset — from viewing interviews as opportunities to "get to know" candidates to treating them as measurement tools that must be designed, calibrated, and continuously improved. This shift can meet resistance from hiring managers who trust their instincts and are skeptical of standardized processes.
The most effective way to overcome this resistance is through education and results. When hiring managers understand the evidence behind structured methods — and when they see the tangible impact on hiring outcomes — adoption increases rapidly. Providing interviewer training, sharing validation data, and involving hiring managers in the interview design process all contribute to buy-in and sustained implementation.
It is also worth noting that structured interviews are not only more valid — they are also more legally defensible. Because every candidate is evaluated against the same job-relevant criteria using the same standardized process, structured interviews are far less susceptible to legal challenges based on disparate treatment or adverse impact (Williamson, Campion, Malos, Roehling, and Campion, 1997). In an era of increasing scrutiny around hiring practices and DEI commitments, this is a significant practical advantage.
Organizations committed to developing strong leaders understand that leadership begins with selection. The competencies assessed during hiring — communication, strategic thinking, adaptability, accountability — are the same competencies that define effective leadership throughout an individual's career. Building a rigorous, structured hiring process is not just an HR initiative; it is a strategic investment in the organization's future leadership pipeline.
Ready to upgrade your hiring process? TeamLMI helps organizations design and implement science-based selection systems — including structured interview guides, behavioral assessments, and validated selection criteria — that dramatically reduce mis-hires and improve quality of hire. Learn more about TeamLMI's Recruiting & Hiring services, or contact TeamLMI today to start a conversation about building a stronger, more evidence-based approach to talent selection.