What Does "Going With Your Gut" Cost?

Hiring decisions are among the most consequential choices any organization makes. Even a routine hire is expensive. SHRM benchmarking puts the average cost per hire near $4,700 (Navarra, 2022), and a mis-hire adds lost productivity, onboarding time, and strain on the team on top of that. Yet despite these stakes, many organizations still rely on unstructured interviews — free-flowing conversations where each candidate is asked different questions, evaluated on different criteria, and judged largely by the interviewer's intuition.

The research is clear: unstructured interviews are one of the least effective selection methods available. In a landmark meta-analysis, Schmidt and Hunter (1998) estimated the validity of unstructured interviews at .38 for predicting job performance and structured interviews at .51. Sackett and colleagues (2022) revised those estimates downward, but structured interviews still ranked first among common selection methods (.42), far ahead of unstructured interviews (.19).

So why do unstructured interviews persist? The answer lies in a collection of well-documented cognitive biases. Interviewers form impressions quickly, and impressions formed in the first few minutes of rapport-building predict their later ratings and job offers (Barrick, Swider, & Stewart, 2010). They are swayed by similarity-to-self bias, halo effects, and contrast effects that have nothing to do with the candidate's ability to perform the job. Structured interviews are specifically designed to neutralize these biases, replacing subjective "gut feelings" with systematic, evidence-based evaluation.

What Makes an Interview "Structured"?

A structured interview is not simply a list of prepared questions. It is a comprehensive selection methodology built on three core principles: standardization, job relevance, and systematic scoring. When all three elements are present, the interview becomes a reliable measurement instrument rather than a casual conversation.

Standardization

Every candidate for a given role is asked the same questions, in the same order, under comparable conditions. This ensures that differences in interview ratings reflect genuine differences in candidate quality — not differences in the questions they happened to be asked. Standardization also means that interviewers follow a consistent protocol for probing and follow-up, reducing the influence of interviewer-specific tendencies.

Job Relevance

Every question in a structured interview is directly linked to a competency, behavior, or knowledge domain identified through a job analysis. This ensures that the interview measures what actually matters for success in the role. Two types of questions dominate structured interviews:

  • Behavioral questions ask candidates to describe specific past experiences that demonstrate a target competency. The underlying premise — rooted in behavioral consistency theory (Wernimont & Campbell, 1968) — is that past behavior is the best predictor of future behavior.
  • Situational questions present candidates with hypothetical job-relevant scenarios and ask how they would respond. Developed from goal-setting theory by Latham, Saari, Pursell, and Campion (1980), these questions assess judgment, problem-solving, and decision-making in context.

Both question types significantly outperform the vague, open-ended prompts typical of unstructured interviews ("Tell me about yourself," "What's your greatest weakness?"). Huffcutt, Weekley, Wiesner, DeGroot, and Jones (2001) found that both question types predicted performance, with behavioral questions holding up better for higher-level roles, which is why many structured interviews use both.

Systematic Scoring

Perhaps the most critical — and most frequently overlooked — element of structure is the use of anchored rating scales. For each question, interviewers reference a pre-developed rubric that defines what constitutes a poor, adequate, good, and excellent response. This transforms evaluation from a subjective impression ("I liked that answer") into a criterion-referenced judgment ("That response demonstrated three of the four behavioral indicators for this competency"). Campion, Palmer, and Campion (1997) identified anchored rating scales as one of the single most important components of interview structure for increasing predictive validity.

How Do You Build a Structured Interview?

Creating an effective structured interview guide requires deliberate effort up front, but the payoff is substantial. The following framework provides a practical, research-informed process that any organization can implement.

Step 1: Conduct a Job Analysis

Before writing a single question, identify the critical competencies, knowledge areas, and behaviors required for success in the role. This can be accomplished through a combination of subject matter expert interviews, review of existing job descriptions, observation of high performers, and analysis of critical incidents. The goal is to produce a prioritized list of five to eight core competencies — enough to capture the essential requirements of the role without making the interview unwieldy.

For example, a job analysis for a project manager might identify the following competencies: strategic planning, stakeholder communication, conflict resolution, results orientation, and team leadership.

Step 2: Develop Competency-Based Questions

Write two to three questions per competency, mixing behavioral and situational formats. Behavioral questions should use the "STAR" prompt structure — asking candidates to describe a specific Situation, the Task they faced, the Action they took, and the Result they achieved. Situational questions should present realistic, job-relevant dilemmas that require candidates to articulate their reasoning and approach.

Keep the language clear and free of jargon. Avoid leading questions or questions that telegraph the desired answer. Each question should be open-ended enough to allow candidates to demonstrate the depth and breadth of their experience.

Step 3: Build Anchored Rating Scales

For each question, develop a scoring rubric on a five-point scale with behavioral anchors. A "1" should describe a response that is vague, irrelevant, or demonstrates a lack of the target competency. A "3" should describe an adequate response that meets basic expectations. A "5" should describe a response that provides specific, compelling evidence of the competency at a high level. Include example responses at each anchor point to calibrate interviewers and reduce scoring drift.

Step 4: Train Your Interviewers

Even the best-designed interview guide will underperform if interviewers are not trained in how to use it. Training should cover the rationale for structured interviewing, how to ask questions without leading the candidate, how to take effective notes, how to use the rating scales, and how to avoid common rating errors such as halo effect, leniency bias, and central tendency. Levashina, Hartwell, Morgeson, and Campion (2014) identify interviewer training as a core element of a well-structured interview.

Step 5: Pilot, Evaluate, and Refine

Before full deployment, pilot the interview guide with a small group of candidates or internal role-players. Gather feedback from interviewers on question clarity, scoring ease, and time management. Evaluate inter-rater reliability — the degree to which different interviewers arrive at similar scores for the same candidate. Use these data to revise questions, sharpen anchors, and improve the overall process. A structured interview is a living document that should be updated as roles evolve and new performance data become available.

What Do Good Interview Questions Look Like?

The following examples illustrate how behavioral and situational questions can be linked to specific competencies. These are designed for a mid-level leadership role but can be adapted for any position by adjusting the competency framework to match the job analysis.

Leadership and Influence

  • Behavioral: "Describe a time when you needed to gain buy-in from a team or stakeholder group that was resistant to a change you were proposing. What was the situation, what actions did you take, and what was the outcome?"
  • Situational: "Imagine you have been asked to lead a cross-functional initiative, but two department heads have publicly expressed skepticism about the project's value. How would you approach building their support?"

Problem-Solving and Decision-Making

  • Behavioral: "Tell me about a complex problem you faced at work where the right course of action was not immediately clear. Walk me through how you analyzed the situation, what options you considered, and what decision you ultimately made."
  • Situational: "You discover that a product your team launched three months ago has a quality issue affecting approximately 10% of customers. Fixing it will require pulling resources from a major upcoming initiative. How would you approach this decision?"

Communication and Collaboration

  • Behavioral: "Give me an example of a time when you had to communicate a difficult or unpopular message to a team or colleague. How did you approach it, and how did the other party respond?"
  • Situational: "You are leading a project team and notice that two team members have a persistent interpersonal conflict that is affecting the group's productivity and morale. What steps would you take to address it?"

Results Orientation and Accountability

  • Behavioral: "Describe a situation where you were responsible for achieving a challenging goal or target. What obstacles did you encounter, and what specific steps did you take to stay on track?"
  • Situational: "Midway through a critical project, you realize that your team is unlikely to meet the original deadline without compromising quality. How would you handle this situation with your team and your stakeholders?"

Adaptability and Learning Agility

  • Behavioral: "Tell me about a time when you were asked to take on a responsibility or role that was significantly outside your comfort zone or area of expertise. How did you approach it?"
  • Situational: "Your organization is implementing a new technology platform that will fundamentally change how your team operates. Many team members are apprehensive. How would you manage both your own transition and your team's adoption of the new system?"

Notice that each question is open-ended, competency-specific, and designed to elicit concrete evidence of behavior rather than abstract self-assessments. When paired with anchored rating scales, these questions give interviewers a reliable framework for distinguishing candidates who talk about competencies from those who demonstrate them.

What Else Belongs in a Selection System?

Structured interviews are powerful, but they are most effective as part of a multi-method selection system. Research shows that combining methods that measure different things, such as cognitive ability, personality, and a structured interview, predicts performance better than any single method alone (Schmidt & Hunter, 1998). This is because each method captures a different facet of candidate capability, and combining them reduces the measurement error inherent in any one tool.

For example, pairing a structured interview with a personality assessment built for selection, such as the ELLSIx Hiring Assessment, provides insight into how a candidate is likely to interact with colleagues, manage stress, and approach tasks. Its forced-choice format makes the results hard for a motivated candidate to shape, and it measures six trait factors, including Honesty & Humility. Structured reference checks through RefCheck add the view of people who have actually worked with the candidate. Cognitive ability testing, such as the Reasoning Assessment, rounds out the picture by assessing the capacity for learning, reasoning, and problem-solving that underlies performance in complex roles.

The key is that each component of the selection system should be validated against the specific competencies and performance outcomes that matter for the role. Science-based hiring processes integrate these tools into a coherent framework where each assessment contributes unique, non-redundant information to the hiring decision.

Organizations that adopt structured, multi-method selection can expect better hires and fewer early exits. Because strong performers produce far more value than average ones, moving to more valid methods can repay its cost many times over for every year a hire stays (Schmidt, Oh, & Shaffer, 2016).

How Do You Make the Shift from Intuition to Evidence?

Transitioning from unstructured to structured interviewing requires more than a new set of questions. It requires a shift in organizational mindset — from viewing interviews as opportunities to "get to know" candidates to treating them as measurement tools that must be designed, calibrated, and continuously improved. This shift can meet resistance from hiring managers who trust their instincts and are skeptical of standardized processes.

The most effective way to overcome this resistance is through education and results. When hiring managers understand the evidence behind structured methods — and when they see the tangible impact on hiring outcomes — adoption increases rapidly. Providing interviewer training, sharing validation data, and involving hiring managers in the interview design process all contribute to buy-in and sustained implementation.

It is also worth noting that structured interviews are not only more valid — they are also more legally defensible. Because every candidate is evaluated against the same job-relevant criteria using the same standardized process, structured interviews are far less susceptible to legal challenges based on disparate treatment or adverse impact (Williamson, Campion, Malos, Roehling, & Campion, 1997). In an era of increasing scrutiny around hiring practices and DEI commitments, this is a significant practical advantage.

Organizations committed to developing strong leaders understand that leadership begins with selection. The competencies assessed during hiring — communication, strategic thinking, adaptability, accountability — are the same competencies that define effective leadership throughout an individual's career. Building a rigorous, structured hiring process is not just an HR initiative; it is a strategic investment in the organization's future leadership pipeline.

From Practice

This composite draws on several client engagements. A senior living operator of about 220 employees had lost most of its director team within a few years. Each departure meant another search, and each search ended the same way: the candidate who interviewed best got the job.

When we reviewed the process, the problem was plain. The interviews were unstructured conversations, and the people who did well in them were polished presenters. Nobody was asking how candidates had actually led a team through a staffing shortage, handled a struggling supervisor, or carried out a difficult change.

The organization rebuilt its director interviews around a short list of competencies from a job analysis, with behavioral and situational questions and an anchored rating scale for each. Interviewers were trained and scored independently before comparing notes. The next directors it hired were less polished in the room but stronger on the job, and the revolving door slowed.

Ready to upgrade your hiring process? TeamLMI helps organizations design and implement science-based selection systems, including structured interview guides, hiring assessments, and job-related selection criteria, that reduce the odds of a costly mis-hire. Learn more about TeamLMI's Recruiting & Hiring services, or contact TeamLMI today to start a conversation about building a stronger, more evidence-based approach to talent selection. For a closer look at what a bad hire really costs, see The Hidden Cost of Bad Hires.

Sources

Research cited in this article, with links to the published work where available.

  • Barrick, M. R., Swider, B. W., & Stewart, G. L. (2010). Initial evaluations in the interview: Relationships with subsequent interviewer evaluations and employment offers. Journal of Applied Psychology, 95(6), 1163–1172. https://doi.org/10.1037/a0019918
  • Campion, M. A., Palmer, D. K., & Campion, J. E. (1997). A review of structure in the selection interview. Personnel Psychology, 50(3), 655–702. https://doi.org/10.1111/j.1744-6570.1997.tb00709.x
  • Huffcutt, A. I., Weekley, J. A., Wiesner, W. H., DeGroot, T. G., & Jones, C. (2001). Comparison of situational and behavior description interview questions for higher-level positions. Personnel Psychology, 54(3), 619–644. https://doi.org/10.1111/j.1744-6570.2001.tb00225.x
  • Latham, G. P., Saari, L. M., Pursell, E. D., & Campion, M. A. (1980). The situational interview. Journal of Applied Psychology, 65(4), 422–427. https://doi.org/10.1037/0021-9010.65.4.422
  • Levashina, J., Hartwell, C. J., Morgeson, F. P., & Campion, M. A. (2014). The structured employment interview: Narrative and quantitative review of the research literature. Personnel Psychology, 67(1), 241–293. https://doi.org/10.1111/peps.12052
  • Navarra, K. (2022, April 11). The real costs of recruitment. SHRM. https://www.shrm.org/topics-tools/news/talent-acquisition/real-costs-recruitment
  • Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040–2068. https://doi.org/10.1037/apl0000994
  • Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274. https://doi.org/10.1037/0033-2909.124.2.262
  • Schmidt, F. L., Oh, I.-S., & Shaffer, J. A. (2016). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 100 years of research findings [Working paper]. Fox School of Business, Temple University.
  • Wernimont, P. F., & Campbell, J. P. (1968). Signs, samples, and criteria. Journal of Applied Psychology, 52(5), 372–376. https://doi.org/10.1037/h0026244
  • Williamson, L. G., Campion, J. E., Malos, S. B., Roehling, M. V., & Campion, M. A. (1997). Employment interview on trial: Linking interview structure with litigation outcomes. Journal of Applied Psychology, 82(6), 900–912. https://doi.org/10.1037/0021-9010.82.6.900

About the Author

Kent E. Frese, Ph.D. is the Managing Partner of TeamLMI and an Industrial-Organizational Psychologist with over 20 years of experience. He works primarily with small and mid-sized businesses — from manufacturing and technology firms to professional services and family-owned companies — on leadership development, talent strategy, and long-term succession planning. Dr. Frese is a member of SIOP (Society for Industrial-Organizational Psychology) and has guided hundreds of leaders and organizations through assessment-driven development and transition.