Bookmark and ShareShare
Friday, April 19, 2013

No, Low Performers Are Not More Engaged Than High Performers

By Jim Harter, Gallup Chief Scientist for Workplace and Wellbeing

A new report by the consulting firm Leadership IQ claims that in 42% of organizations, poor performers are actually more likely to be engaged at work than are high performers. This finding appears to contradict almost two decades’ worth of management research.

Like any claims that end up in popular media, we need to take a close look at the evidence to know what the data are really saying, and compare that research to other research completed on the topic. New research should be continuous and challenge existing wisdom. Past findings should be debated. But, we should always examine the quality of the basis for new findings in the context of past findings.

Gallup in 1997 completed its first meta-analysis -- or study of many studies -- on employee engagement using data from 1,135 business units. The central question we wanted to answer was whether workplace conditions correlate with business outcomes such as profit, productivity, customer perceptions of service, and employee retention. We just completed our eighth iteration of this same analysis, which now includes 49,928 business units across 34 countries, and analyzes even more outcomes, such as quality (defects), safety (accidents), absenteeism, and shrinkage (theft). Various iterations of this meta-analysis have been published in peer-reviewed, top tier academic journals and books, including a longitudinal causal analysis.

The central, and consistent, finding has been that employee engagement -- which Gallup defines as 12 specific workplace elements -- predicts objective performance outcomes within organizations and this relationship holds true across very different types of organizations. Meta-analysis enables us to test whether the correlations are consistent or different across organizations. The data are very clear: if engagement is measured appropriately, it predicts business outcomes across organizations.

Now, this doesn’t mean that highly productive employees are never disengaged or that low-performing employees are never engaged. But, what our study does tell us is that when business units have more engaged employees their probability of success improves substantially. In fact, those with high engagement nearly double the odds of success compared with those with low engagement.

So, working on management elements, things like clarifying expectations, giving people an opportunity to do what they do best, giving employees developmental opportunities, and holding people accountable for quality work increase the odds that business units within your organization will be successful.

Here at Gallup, we take all new research on employee engagement seriously and have reviewed the Leadership IQ study closely. Based on the available material from the company -- we contacted Leadership IQ for their methodological details, but did not receive a response -- we see six potential questions to consider.

  1. What is the quality of the performance measurement Leadership IQ used to gauge the performance of individual employees? The Leadership IQ study based its findings on performance appraisal instead of on objective performance data, such as productivity, profit, turnover, customer engagement, safety, absenteeism, quality, and shrinkage. Gallup research has long-proven the clear connection between its Q12 employee engagement metric and these substantive outcomes.
  2. Why does Leadership IQ present their findings for only one organization? It is unclear why they would do this when they claim to have studied thousands of organizations. Gallup includes all available studies in its meta-analysis, which now includes more than 49,000 business or work units and 1.4 million employees in those units. 
  3. What percentage of the population does each organization in the study represent -- and how large are the samples in each organization? Sampling error and sampling bias can greatly distort conclusions when a near census isn’t obtained from the organizations studied. Gallup attains an average 85% response rate in its employee engagement studies. 
  4. Does the study control for other factors that can influence engagement and performance or do they distort the results? The Leadership IQ study says there is a relationship between being a low performer and being engaged. But, there are likely other factors that could be influencing engagement and performance, such as employee tenure, position, level in the organization, etc. 
  5. Over what time periods did the study measure engagement and performance? Gallup collects a great deal of longitudinal data and includes predictive validity estimates in its analyses (that is, measuring engagement in one year and outcomes in the next year). These studies allow for stronger conclusions of causality than studies where engagement and performance are collected concurrently.
  6. What is the quality of the engagement measure and what validation evidence is available? The use of the term “engagement” has skyrocketed in the last decade. Nearly anyone using an employee survey refers to their survey as “employee engagement” -- regardless of the quality or content of questions being asked. Gallup, since the early 1990s, has validated its unique Q12 employee engagement metric by examining the workplace elements that consistently predict various performance outcomes, including profitability, productivity, employee retention, and customer perceptions of service.
The Leadership IQ finding that in 42% of organizations there is an inverse relationship between engagement and performance likely means that their measurement of engagement, performance, or both is less than optimum. In addition, it is possible that any one of the possibilities outlined above is distorting the results they are obtaining.

While the Leadership IQ study may seem novel or intriguingly counterintuitive, it, unfortunately, provides little substance to back up its claims.

The bottom line, though, is that employee engagement, in the way Gallup defines it, is one of the most important concepts for any business to master. Further research -- whether from Gallup or another company -- on employee engagement is vitally important in helping more companies integrate it and use it to their advantage. The important thing to remember is to look closely at the quality of the research in any new paper before judging the value of its findings.

6 comments:

davidbowles said...
April 23, 2013 at 2:17 AM  

Jim thank you for crushing...in the nicest possible way...this "research" from Leadership IQ. It is amazing to watch how so many mainstream publications (WSJ, FT, etc.) have picked this up. Perhaps, with the popularity of employee engagement, as you point out above, there is a need to vent an opposing viewpoint just to try and create some balance. Also too many "experts" are now out there with little real experience in the key elements which are essential for this field, including even the basics of research methodology and organizational psychology. This in turn gives rise to blogs and even articles supporting Leadership IQ's position.

I am with you, that creating the great work environment in which people choose to engage must be the core part of any successful organization's...and country's...mission in an increasingly competitive world.

DB
co-author, The High Engagement Work Culture: Balancing ME and WE (Macmillan, 2012)

Anonymous said...
April 23, 2013 at 8:45 AM  

It's pretty obvious to anyone who looks at the two studies that Gallup and Leadership IQ are looking at two very different things. Gallup's poll does not differentiate between high and low performig employees; instead they look at organizational performance. Leadership IQ looked at individual performance and engagement. Comparing the two sets of data is like comparing oranges and potatoes. I suspect that organizations that perform well and have highly engaged employees fall into the 58% of companies Leadership IQ identified as having engaged high performers. Instead of bashing a study that on the surface seems to disagree with your findings, why doesn't Gallup seek to understand what it REALLY takes to be engaged at work (hint--you don't have to have a best friend there).

Sharon said...
April 23, 2013 at 8:52 AM  

Jim: We've been discussing this on the Employee Engagement Ning and would love to hear more about your perspective:

http://employeeengagement.ning.com/forum/topics/what-are-your-thoughts-on-the-hbr-article-your-least-engaged

davidbowles said...
April 23, 2013 at 9:44 AM  

Anonymous: Jim Harter says that Leadership IQ refused to share their methodology, so we cannot check as to whether they made mistakes. In the ongoing discussion about this "research" which Sharon refers to above (on the Employee Engagement Network) we have discussed in depth many methodological issues with it.

Your mention of "having a best friend at work" shows that you basically do not understand how questionnaires are created and how they work. Gallup can speak for itself, but as far as this question goes it is greatly misunderstood: just like in medical science, a symptom can point to a medical condition while having nothing to do with causing it or having the potential to make it go away. Having a friend at work has merely been determined (after a vast amount of research) to be a SIGN that the individual is likely to be high on the engagement scale, not a CAUSE of that.

DB
co-author, The High Engagement Work Culture: Balancing ME and WE (Macmillan, 2012)

Anonymous said...
May 1, 2013 at 10:44 PM  

David, I do understand how questionnaires are created and how they work. Ideally they are based on theory and have some form of construct validity. The Q12 may have criterion validity but what does it really measure? I am not so sure that it is "engagement."

Anonymous said...
November 30, 2013 at 6:47 AM  

The Leadership IQ report is in my view the worst form of PR tosh. That the media have taken it up demonstrates than on a slow day, any news will do. However, the PR has no doubt worked for them. In fact, a bit like the PR around the Q12. Now there's no denying that Gallup have done their research well (as opposed to the Leadership IQ), but I think the PR machine supporting the Q12 has over the years massively distorted it's focus and value. Despite the hype, in reality the Q12 entirely fails to measure engagement, it just measures some of the antecedents of engagement. But by being limited for packaging purposes to 12 items, the Q12 then fails to measure many very well known drivers of engagement. Now I have no doubt that in their work with clients Gallup is at pains to extend the measurement tool in a sensitive way beyond these 12 questions, however the PR machine unhelpfully suggests that all you need is 12. Why 12? Because it is a 'magic number', found to be repeated through the ages in mythology, religion, calendar systems and so-on. So if you're going to market something, stick with a formula for the packaging that has worked for millennia.... Having met one or two very intelligent current and ex-gallup employees, I do know that there is embarrassment at the way the Q12 sometimes gets presented. I'm not entirely sure this blog will have eased that pain....

Post a Comment

Comments are moderated by Gallup and may not appear on this blog until they have been reviewed and deemed appropriate for posting.

Copyright © 2010 Gallup, Inc. All rights reserved. | Terms of Use | Privacy Statement