{Grant, 2012. ROI is a poor measure of coaching success: towards a more holistic approach using a well-being and engagement framework. Coaching: An International Journal of Theory, Research, and Practice.}
Introduction
What is the full range of positive outcomes possible through coaching?
The wellbeing and engagement framework (WBEF) is a comprehensive and meaningful approach to evaluating the coaching impact.
Coaches benefit from validating their coaching services and demonstrating that they deliver value for money.
Organizations are attuned to the amount they spend on coaching and the justification of such expenditure in terms of financial return.
However, the evaluation of impact through a financial ROI is an unreliable and insufficient measure of coaching outcomes.
Evaluation is not just about financial payback. A humanistic framework is presented with the potential to provide a taxonomy from which to more comprehensively evaluate organization coaching interventions and some quantitative measures that might be effective in its operationalization.
What is ROI?
Kirkpatrick (1977) introduced a four-level taxonomy for evaluating training programs:
Level one is about evaluating trainees’ reactions – how much a participant likes or enjoys a training program, focusing on participants’ feelings rather than assessing any learning.
Level two is about evaluating learning – measuring the knowledge acquired, skills improved or developed, or changes in attitudes.
Level three is about assessing behavior change – the extent to which learnings acquired are subsequently manifested in the workplace through observable behavioral change.
Level four focuses on results that occur due to the training program. These could include increased sales, reduced costs, or any business-related metric.
All four levels should be evaluated to understand the full impact of a specific training program.
Phillips (1997) adds an explicit ROI step as level five.
Calculating ROI for coaching
The standard formula for calculating ROI involves
{[(estimated coaching benefits – costs of coaching) / costs of coaching] * 100%}
Why is ROI an unreliable and insufficient outcome measure?
A key factor in determining an ROI figure is the degree to which revenue can be attributed to the actual work of the client and the extent to which the coaching has enhanced such work.
It may not always be possible to draw a reliable direct causal line between coaching and financial return. The estimated financial benefits often represent highly subjective and contextually bound variables.
Furthermore, where the client’s work involves managing or directing others to attain organizational goals, the causal chain between coaching and eventual financial outcomes becomes even more tenuous. It is often difficult to delineate specific causal relationships between a coaching intervention and improvements in organizational metrics. In addition, ROI calculations ignore the impact of other variables, such as market context and team input.
The primary variable influencing financial impact is the client’s opportunity to shape revenue stream outcomes.
The way data are collected, reported, and interpreted may be biased due to vested interests in emphasizing commercial success and thus be primarily about marketing rather than measurement.
What might be a more effective means of evaluating organizational coaching success?
Poorly targeted coaching interventions that myopically focus on ensuring financial returns may inadvertently increase job-related stress as the client struggles to achieve unrealistic or inappropriate goals.
Well-targeted workplace coaching has the potential to deliver a wide range of positive outcomes, including increased workplace engagement, decreased stress, depression, and anxiety, increased resilience and wellbeing, as well as goal attainment. All of these variables can be satisfactorily assessed using pre-existing, validated measures.
Two important variables for coaching in organizations are wellbeing and workplace engagement.
Wellbeing is more than the absence of mental illness. It involves several facets of psychological wellbeing, including self-acceptance, purpose in life, positive relations with others, environmental mastery, and autonomy.
Workplace or employee engagement, the positive opposite of job burnout, can be understood as a state of high energy, strong involvement, and a strong sense of commitment to the performance of work functions.
The wellbeing and engagement framework (WBEF)
The WBEF has two dimensions: a wellbeing dimension (high-low) and a workplace engagement dimension (high-low), qualitatively represented as a 2×2 matrix or in a four-quadrant view.
Area of flourishing
Individuals in the area of flourishing (upper right area; UR) experience elevated wellbeing and high levels of engagement. They would be highly involved with and absorbed in their work, have a well-developed sense of work-related meaning and purpose, and enjoy positive relations with work colleagues. This area may represent the ideal or target state in a workplace coaching context.
Area of acquiescence
Individuals in the area of acquiescent (upper left area; UL) are happy but disengaged or have good levels of wellbeing but low levels of workplace engagement. These are physically and emotionally present employees who are not actively engaged with the organization’s goals or day-to-day work. It can be expected that these individuals will eventually drift into a state of languishing.
Area of languishing
Individuals who are languishing have relatively low levels of wellbeing, but not high levels of depression, anxiety, or stress and also have only moderate levels of workplace engagement. Their working lives lack vigor, energy, and resilience, usually associated with high workplace engagement and personal flourishing.
Area of distressed but functional
Individuals in the area of distressed but functional (lower right area; LR) have comparatively high levels of workplace engagement and may be highly functional in terms of workplace performance, but may also be mildly depressed, have high levels of anxiety, or be chronically stressed.
Clients here will likely present issues in line with the coaching context, such as delegation, motivation, time management, staff retention, or interpersonal communication difficulties or conflicts.
The mental health issues may warrant a clinical approach and require counseling support.
Area of burnout and psychopathology: the disengaged and distressed
Individuals in this area (lower left area; LL) may have relatively high levels of mental illness, and would have very low levels of workplace engagement, and may well be experiencing major symptoms of job burnout, high levels of cynicism, and physical and mental exhaustion.
Individuals in this quadrant may be more suited for counseling or a therapeutic intervention than coaching.
Using the WBEF
The WBEF builds on and extends the core facets of Kirkpatrick’s (1977) taxonomy. Within an organizational framework of wellbeing and engagement, we may evaluate the results that occur due to the coaching program at the four levels of emotions and reactions, learning, behavioral change, and the results of wellbeing and engagement or moving individuals into the area of flourishing.
The WBEF can help determine the focus of the coaching intervention; that is, if the coaching should be primarily aimed at relieving stress, increasing wellbeing, or enhancing engagement through the pursuit of goals that are meaningful and poignant for the individual.
As a diagnostic tool, the WBEF could help determine whether the individuals would benefit from counseling or coaching.
Measurements relevant to the WBEF
The wellbeing dimension could be operationalized with pre-existing quantitative measures.
The Positive Affect and Negative Affect Scale (PANAS) would indicate high positive affect and low negative affect for the upper section of the WBEF and low positive affect and high negative affect for the lower sections of the WBEF.
Other measures to be used for the wellbeing dimension include the Depression, Anxiety, and Stress Scales for the lower section of this dimension and the Warwick-Edinburgh Mental Wellbeing Scale for the upper section.
The engagement dimension could be measured with the Gallup Q12 workplace engagement survey or the Utrecht Work Engagement Scale.
An individual’s scores on these measures would locate them within a specific quadrant in the WBEF. By taking repeated measures over time, it may be possible to track changes and thus calculate the impact of a coaching intervention on wellbeing and engagement.
Key points
People tend to pay attention to what is measured, and such measurement tends to change behavior. The primary metrics used to assess coaching interventions can significantly impact the coaching agenda.
Organizations and workplaces are social and psychological contexts in which people live, work, and relate. The real potential of coaching is to create a broad range of positive humanistic outcomes in people’s development, growth, and wellbeing, in addition to the financial outcomes associated with engagement and achievement.