Thursday, 12 November 2009
Methodological and Measurement challenges in coaching research
How might you respond when you see a research article that claims that:'75% of coached respondents reported that thier relationships with others in their team had improved by 50%. At first sight this looks like a 'good result' for coaching, and adds further data to just how great coaching is. But does it?
Here are a number of questions we should ask before accepting this study at face value.
1 What was the quality of the inputs (e.g time, quality of the coaches)
2 What exactly was improved?
3 Did the respondents merely move from being 'Very ineffective' to just 'Ineffective'?
4 Who reported the changes - were these self -reports, with all the inherent biases of this approach.
One of the challenges faced by coaching researchers is defining what we actually mean by coaching - the precise nature of the intervetion. We also need proof that coaches actually do what they say they do or think they do. Furthermore, one of the most common research designs that we use, pre-post is inherently weak as a design; most of the measures that we use are not sensitive to detecting change. The impact of coaching may even work in a reverse direction (even though the intervention is effective!) For example, take a manager who is highly unreflective and rates herself quite highly on a trait; the effect of coaching might be to improve her reflection, with the result that she now rates herself lower on the trait because she has more self-understanding!
A better design than pre-post is the use of repeat measures both pre and post. Hence: AAABBB
In this design a series of repeat measures on the same person are taken (AAA); then repeat post measures are taken on this person (BBB). This allows for some stability of measurement.
Wednesday, 8 July 2009
In a seminal paper, Goode (1960) outlines a range of features that an occupation acquires for it to become a profession, including:
• The profession determines its own standards of education and training
• Professional practice is often legally recognised by some form of license to practice
• Licensing and admission boards are manned by members of the profession
• Most legislation concerned with the profession is shaped by members of the profession themselves
• The practitioner is relatively free of lay evaluation and control
• The norms and practices enforced by the profession are more stringent than legal controls
• Members strongly identify with the profession and are affiliated to it
• Members see themselves as ‘tied’ to the profession and have no plans to leave it.
So, is coaching an occupation or profession? A glance at the web sites of some of the coaching associations suggests a claim for the latter. Certainly, there have been extensive moves over recent years, for example, to ensure that most of the associations have a written code of ethics that members are required to adhere to. There is also extensive education and training, including the creation of standards for coaching programmes.
Problems, however, remain. It would be difficult to argue that within coaching there is a'body of knowledge' that is widely accepted. Indeed, it could be argued that there are at least two strongly contested bodies of knowledge between psychological/therapeutic approaches to coaching and the more business focused approaches - to name but two.
At Surrey, we are currently completing the analysis of a major online survey (with 267 responses) that sought the views of managers (as beneficiaries of coaching) as to what they were looking for in a coach. Personal factors such as the coach's ability to listen, provide empathy and constructive support came out highly positively correlated with decisions to select a coach. However, criteria such as the possession of qualifications, including coaching qualifications, were negatively correlated with the decision to select a coach. So, far from coaching qualifications being seen as factor in choosing a coach, they tend to put people off! This finding is not good news for the coaching associations who see qualifications and credentialing as one of the cornerstones of the push towards professionalisation of the industry.
There is, however, an alternative perspective. Rather than placing the emphasis on formal qualifications, professionalisation could be nurtured through other forms of personal development. Mentoring by more experienced coaches would be just one possibility.
More details on the results of our survey will be provided in later reports on this site.
Do you have a view on whether coaching is, or should be a profession? If so, do post a comment.
Wednesday, 24 June 2009
- What value can research bring to the profession of coaching?
- What should we be researching?
In the discussion, it was claimed that, at least in Small and Medium Sized enterprises, there is no push to evidence that coaching works. The corporate market, however, is different, with a keen desire to measure coaching's Return on Investment. However, organistions are trying to do this, using the kinds of traditional methods they apply to face-to-face training programmes. What we really need is research that measures some of the long-term benefits of coaching.
I couldn't agree more.
For more information about the London Coaching Group see:
Monday, 15 June 2009
This is a thought-provoking article, on an intriguing subject – the potential links between intuition and coaching. As human beings, we all use intuition to varying degrees. The same, argues Mavor, can be said about our role as coaches. She acknowledges, however, that apart from a trickle of studies, very little research has been undertaken on this subject.
But what do we mean by intuition? There is, as yet, no universally accepted definition. Mavor presents a number of alternative perspectives. Dane and Pratt (2007: 40) for example, regard intuitions as ‘affectively charged judgements that arise through rapid, nonconscious, and holistic associations’. Hence, intuition contains features such as: ‘gut feelings or gut instincts’; speed – they arise rapidly; nonconscious information processing; and holistic associations including patterns, structures or schemas held in long-term memory.
The Mavor study used semi-structured interviews with 14 experienced executive coaches (8 males and 6 females) with an average of 14.5 years experience as a coach. The coaches were asked to report retrospectively on intuitive experiences in either one-to-one or group coaching. A series of 15 broad questions, elicited from the literature on intuition, were posed, each interview lasting approximately two hours.
The findings suggest that intuition is, indeed, very much present in coaching conversations. One coach, for example, talked about ‘out of the blue’ experience. The intuition ‘came from nowhere’. But it cannot be deliberately ‘called up’. Looking for it makes it difficult to find. The key seems to be being open and maintaining a ‘soft focus’, allowing intuition to give you messages and clues.
Intuition is more likely to be accessed if the coach has self-belief and self-confidence in what they do. But it is also essential that the coach gets themselves in the right physical, mental and emotional state to help them access and apply their intuition. This includes the coach’s:
- Attention to their own well-being
- The preparation they undertake before the coaching session
- The rituals or routines they use before the session to get into ‘the zone’
- Their ability to stay present and focused during the session
Preparation for a session depends on the individual coach. Some would read through the notes from previous sessions; others would look through coaching models or frameworks. The key, however, was letting go of analytical thoughts, of getting ‘grounded’ and quieting the mind. It meant being congruent, receptive, fresh, attentive and calm. This helps to develop the vital ingredient of rapport which allows the coaching conversation to access deeper levels of communication and beliefs, attitudes, emotions and feelings. Yet it also means having a level of detachment and objectiveness in accessing and applying intuition, and to present an observation as an offering as opposed to a profound truth. As one coach said, it means being “willing to put it out there and willing to get it wrong”. This is not a celebration of ignorance. As one coach commented, “you have to know your stuff”. Hence, intuition is mainly used by more experienced coaches. This is because they operate at an unconscious competence level. Experience enables coaches to chunk information so that they can store and retrieve it easily (Hayashi 2001).
It would be wrong to read too much into what is, at best, a small scale study. However, the findings here on intuition in coaching, seem largely consistent with much of the general literature on intuition. The study raises some important themes that are certainly worthy of further exploration.
Dane, E., and Pratt, M. (2007) Exploring intuition and its role in managerial decision-making, Academy of Management Review, 32, 33-54.
Hayashi, A.M. (2001) When to trust your gut. Harvard Business Review at Large, Feb, 59-65.
Saturday, 6 June 2009
Setting up a mentoring practice in your organisation
Mentoring practice overview (map)
Recruiting mentors. Who should be a mentor – professional profile.
Matching mentors and mentees
The stages of mentoring
Setting direction. Planning and conducting the first meeting; planning and agreeing time; confidentiality; boundaries; expectations.
Moving on – what happens next?
Giving and receiving feedback
Measuring progress – formal or informal?
What can go wrong?
Raising causes for concern
A Peer Learning event: Mentoring in VET - European perspectives, will be held (venue to be announed) on 21 and 22 October, 2009. If you would like further details, please contact me at D.E.Gray@surrey.ac.uk
Friday, 22 May 2009
Please contact me at the following address if you would like a full copy of the article:
Monday, 11 May 2009
What is clear to me is that coaching is a highly skilled, multi-faceted service which requires a very broad range of competencies and experience. These include business experience (in a variety of organisations) and knowledge of how organisations function (organisational behaviour), as well as at least a working knowledge of human psychology. Then there are a host of other compentencies such as empathy, listening skills, honesty, integrity etc etc. Finally, most good coaches understand, and are able to apply, one or more coaching methodology - indeed, good coaches are usually able to work with a flexible range of methodologies, depending on the needs of the coachee. Can all of this be learnt on a two-day training programme? I think not.
So, to conclude. We should welcome the fact that organisations are waking up to the power of coaching and that some of them are putting managers through 'The manager as coach' programmes. But we should be clear about the strengths and limitations of such programmes. It is not surprising that many organisations find that they have to use a blend of both internal and external coaches.
(1) Coaching at the sharp end: developing and supporting the line manager as coach, available at:
Friday, 8 May 2009
Coaching services (executive coaching, leadership coaching, life coaching, to name but three) have been growing exponentially over the last 10 years. Many (especially large) public and private organisations have been spending considerable amounts of money hiring external coaches or training their own internal coaches. The question is: does coaching work?
This blog hopes to encourage debate on this important issue.