Our Head of Reporting and Analytics adds his thoughts on the nuts and bolts of engagement.
Greta Roberts, who I have a lot of time for, yesterday published an article called Employee Engagement? It’s Just A Meaningless, “Feel Good” Business Metric. Looking at our data and analysis we disagree.
Greta describes Engagement as a middle measure. We would probably use the term ‘leading indicator’. I would argue both are emotional terms, but both are potentially accurate.
The article mentions 6 reasons, each of which I think are worth addressing.
Employee engagement isn’t the goal. Business performance is the goal.
I suspect a good parallel is that customer satisfaction isn’t a goal, sales is. Greta mentions that most businesses don’t and can’t link performance to engagement results. My view is that (a) I suspect more businesses could do this if they wanted – just ask their survey provider and (b) this argument is more a criticism of how engagement is measured and analysed than it is of the usefulness of measuring engagement.
Justifying any kind of program based on someone else’s research is a less than rigorous business practice.
OK, there isn’t much to disagree about this. Much of the published research has (a) been mostly correlations, written as to provide an impression of causation (b) written by consultants with a tool to sell. We’ve done a big literature review on this as part of a presentation to an industry group.
OK, so one could level this claim on us, however we’d be more than willing to quantify the results with clients. I think it’s necessary to justify the ongoing use.
Are these studies useless? For the analyst they’re not. I would argue one of the ways of using them is as hypotheses to test. Testing others’ findings in your own context is a valuable activity for analysts. In some ways, it’s one of the advantages of using a firm with deep domain expertise, like Greta’s firm or our own – we often know where to concentrate analysis resources.
I propose that you should be starting an engagement project with the objective to link engagement to performance. Design the intervention on that basis. Run an experiment but certainly capture the data in a way to enable you to do the analysis.
Rigorous analytics often show little or no correlation between high engagement and an increase in business performance or a decrease in turnover.
This conflicts with what we’re seeing from our data and analysis.
Now, let me be clear, there is muddied water between the relationship between engagement and business performance. The key issue is that whilst we see engagement linked to performance we also see employee populations of higher performing companies more engaged.
There are a few ways of doing this analysis to disentangle the results. First, you need to capture engagement at an individual basis. In truth most survey firms do this. Most surveys are confidential rather than anonymous. Whilst we don’t report or analyse groups below a certain number we do have the ability to do analysis using linked data. It’s pretty easy to tune ML algorithms to do this and also reduces the chance of overfitting.
A simple test to see if your survey is really anonymous – if you’re survey isn’t asking employees where they work they’re almost certainly linking the perception data to demographic data later because you’ll need to link to function etc for reporting purposes. There is no reason you couldn’t be linking individual performance data, for example from a CRM system this way.
Second, as engagement data is captured more regularly it’s becoming more valuable to analyse as a time series. The additional frequency is helpful identifying the order of events – i.e. which comes first, the engagement or the performance? We could use such patterns to infer causality. In fact when a presenter from IBM was questioned on how they had identified causality to engagement during a presentation I chaired at People Analytics 2015 it was via time-series data.
One of the reports we provide for clients with Workometry is the rates of which employees are shifting between engaged and disengaged states. We do this by looking at patterns of engagement at an individual level over time.
An earlier one of my posts used systems dynamics to explain why this is so important. I argued that it isn’t the amount of engagement that is important, but the rate that the business was disengaging people. Either way, you need to be measuring engagement to get this.
Engagement is a middle measure.
Lots and lots of KPIs are so called middle measures. The number of sales prospects doesn’t pay the bills; as I mentioned, customer satisfaction isn’t the same as sales. Few managers would question if you recommended measuring these.
We measure and target these measures because we see a link between the measures and what we really care about. Of course if there is no link then there is no point but (a) make sure that you’ve done the analysis with individually linked data (b) look at the rate of switching or other change over time metrics.
Engagement scores are not actionable.
There are two sides to this. Greta mentions that data isn’t linked which as I’ve argued is unlikely to be true. With this data it’s quite possible to identify events that precede a change in engagement at an individual level. Second we can ask employees about such events, and if we do this close to the event we increase the chance of getting accurate data. Workometry provides the option of triggered feedback requests by key events & integrates the data from this with regular pulse surveys.
Customer research suggests that such events are a big driver of customer engagement. Our research shows the same. The data clearly shows when a ‘shock’ event has occurred and where engagement at an individual level decreases over time. We can also see how and when people regain engagement (at a much lower frequency than those who become disengaged).
It’s a vanity metric for the company.
Can it be? Yes, of course. Nobody should measure engagement and accept the links as gospel. In fact, I would argue that there is little point measuring it if you accept it without being critical about it’s application in your firm – in these instances it would arguably be as useful to implement engagement-raising measures without measuring the outcomes. It might surprise you that some HR departments do initiatives without implementing measurement systems at the same time. It’s not even just limited to engagement!
Just because sometimes measurement is poor doesn’t necessarily imply that the measure should be abandoned or rubbished. If there is strong anecdotal evidence that a relationship often exists I think the smart approach is to try and replicate it with your own data, measured in the most effective manner possible.
From an analysis perspective what we’ve found useful as regards engagement are three things:
1) To link engagement with other measures at the individual level, measure it frequently and treat it as time-series data 2) To create derivatives of variables when doing feature-engineering pre-analysis 3) To use systems dynamics to understand how engagement interacts with other ‘harder’ variables. This simulation-based approach will often provide better forecasts and explanations than simple machine-learning approaches.
This post originally appeared on LinkedIn