What the Evidence Supports, and Where it Stops

8 hours ago

3 min read

Part 4 of 4 in the series Training new supervisors with Logic Puzzles for Managers, Volume 1.

Parts 1 to 3 made the case that supervisors drive engagement, productivity and turnover, that practice beats a lecture, and that a logic puzzle with a structured debrief gives new supervisors that practice. A fair question follows: has anyone measured lecture against practice-based training and traced the difference all the way to productivity or turnover? I looked for that study and didn't find one that does all of it in a single design. What the literature offers is a chain of well-measured links, and each link is strong.

Study	What it compared or measured	Finding
Freeman et al. 2014¹	Lecture versus active learning, 225 studies	Exam scores about 6 percent higher with active learning; lecture students 1.5 times as likely to fail
Burke et al. 2006²	Least versus most engaging safety training, 95 studies, 20,991 participants	Most engaging methods about three times as effective for knowledge and skill; larger reductions in accidents and injuries
Arthur et al. 2003³	Organizational training methods, 162 studies	Effect sizes of about 0.60 to 0.63 overall; lecture more effective than its reputation
Lacerenza et al. 2017⁴	Leadership training design, 335 samples	Transfer effect of 0.82 and results effect of 0.72; strongest with practice, feedback and spaced sessions
Tannenbaum and Cerasoli 2013⁵	Debriefs versus no debrief, 46 samples	Performance about 25 percent higher; strongest when facilitated, structured and aimed at the team
Klein et al. 2009⁶	Team building interventions	Moderate positive effect, strongest on trust, cohesion and coordination
Saks and Belcourt 2006⁷	Application of training over time	Estimated application fell from 62 percent to 34 percent over one year
Gallup Q12 2024⁸	Top versus bottom quartile engagement	Productivity 14 to 18 percent higher; turnover 21 to 51 percent lower
Lazear, Shaw and Stanton 2015⁹	Bottom versus top 10 percent supervisors	Output gain larger than adding a tenth worker to a nine-member team
Hoffman and Tadelis 2021¹⁰	10th versus 90th percentile people managers	Employees about 60 percent less likely to leave

Read together, the links support a reasonable conclusion. Practice-based, engaging training produces more learning and more transfer than passive instruction, and in safety training it produces better operating outcomes. Leadership training built on practice, feedback and spacing improves organizational results, and structured, facilitated debriefs add about a quarter to performance on their own. Better supervisors raise engagement, output and retention. What the literature doesn't give you is a single percentage that says "replace your lecture with puzzles and turnover falls by X." Anyone who quotes that number to you is inventing it, and I'd rather give you the measures to produce your own.

If you're planning a new supervisor program for the coming year, I'd welcome 20 minutes to help you set up the baseline and the measures before the first session, so your results answer the question the research leaves open. Email mpalm@convergentresults.com with "Cohort" in the subject line. If you run Volume 1 with a cohort, I'd like to hear what you measure, too.

Notes

1. Scott Freeman et al., "Active Learning Increases Student Performance in Science, Engineering, and Mathematics," Proceedings of the National Academy of Sciences 111, no. 23 (2014): 8410 to 15, https://doi.org/10.1073/pnas.1319030111.

2. Michael J. Burke et al., "Relative Effectiveness of Worker Safety and Health Training Methods," American Journal of Public Health 96, no. 2 (2006): 315 to 24.

3. Winfred Arthur Jr. et al., "Effectiveness of Training in Organizations: A Meta-Analysis of Design and Evaluation Features," Journal of Applied Psychology 88, no. 2 (2003): 234 to 45.

4. Christina N. Lacerenza et al., "Leadership Training Design, Delivery, and Implementation: A Meta-Analysis," Journal of Applied Psychology 102, no. 12 (2017): 1686 to 1718, https://doi.org/10.1037/apl0000241.

5. Scott I. Tannenbaum and Christopher P. Cerasoli, "Do Team and Individual Debriefs Enhance Performance? A Meta-Analysis," Human Factors 55, no. 1 (2013): 231 to 45, https://doi.org/10.1177/0018720812448394.

6. Cameron Klein et al., "Does Team Building Work?," Small Group Research 40, no. 2 (2009): 181 to 222, https://doi.org/10.1177/1046496408328821.

7. Alan M. Saks and Monica Belcourt, "An Investigation of Training Activities and Transfer of Training in Organizations," Human Resource Management 45, no. 4 (2006): 629 to 48, https://doi.org/10.1002/hrm.20135.

8. Gallup, The Relationship Between Engagement at Work and Organizational Outcomes: 2024 Q12 Meta-Analysis, 11th ed. (Washington, DC: Gallup, 2024), https://www.gallup.com/workplace/321725/gallup-q12-meta-analysis-report.aspx.

9. Edward P. Lazear, Kathryn L. Shaw, and Christopher T. Stanton, "The Value of Bosses," Journal of Labor Economics 33, no. 4 (2015): 823 to 61.

10. Mitchell Hoffman and Steven Tadelis, "People Management Skills, Employee Attrition, and Manager Rewards: An Empirical Analysis," Journal of Political Economy 129, no. 1 (2021): 243 to 85.