Assessment in Surgical Training - Evolution or Revolution?
created 01 December 2016
How do we measure competence? Modern surgical training has become competence-based, yet the assessment of competence is an inexact science.
The acquisition of any skill requires description, demonstration and performance under varying levels of supervision until the individual is considered competent. In the ISCP we now have a comprehensive syllabus for all surgical specialties, a resource to record evidence of training progression and a process to assess knowledge, clinical and technical skills as well as professionalism and judgement. Such a resource was not available to my generation, who were brought up in a system with much more time to get practical experience but with varying assessment and feedback.
In my training you knew if you were making progress if you were allowed to do the next case. This had its limitations, however, and I well recall being heavily criticised in morbidity meetings for complications with procedures - which was not constructive criticism. This experience did develop reflection on my practice, not least to try to avoid such criticism.
Getting the balance right is one of the issues we face with a competence based approach. It requires a greater input from trainers, as the work place based assessments (WBAs) require regular feedback and are time-consuming. Meanwhile, both the EWTR and service pressures have reduced the time available. As a result WBAs have generated a “tick box” culture, which by definition has an adverse effect on the quality of training.
The ISCP Evaluation highlighted the limitations of existing WBA practices and many respondents, although in favour of WBAs, found the practicalities restrictive. The ISCP started with a modest number of recommended WBAs. This has expanded to approximately one per week. Advocates of larger numbers argue that more regular WBAs will enhance trainer – trainee interaction. The counter argument is the adverse effect on quality because of the emphasis on quantity removing the focus from actual assessments.
We need to return to a more flexible, informal approach to workplace assessment. The trainer – trainee interaction should be a daily occurrence, but this does not mean that every event needs to be formally recorded. I accept the old apprenticeship model was limited. However, learning did occur - partly through instruction but also by observation and discussion. We do not have the luxury of time for such observation now, but we can use the apprentice model with close supervision.
The introduction of the Supervised Learning Event (SLE), often compared to a driving lesson, has much to commend it. It provides an assessment on a day to day basis of several skills, covering performance, feedback and discussing what needs to be improved and performed at the next encounter. This process can be applied to any of the current WBA tools and could be recorded in a log book or diary of activity. We have to be careful about adding another acronym to an already confusing list, but I do believe that, with this approach, we can create a good and productive experience for both trainer and trainee and encourage both feedback and reflection. We all reflect on our practices as a matter of routine – it is part of being a professional and striving for improvement. This approach would complement the more structured recording of WBAs, which should occur at least once a week
The most difficult feature of assessment is, I believe, writing constructive criticism, including both positive and negative comments. We are overall better at negative comments than we are at positive reinforcement - not only confirming satisfactory performance but also praising excellence. We need to ensure both are used equally, as the tick in the satisfactory box alone is worthless. As part of the ISCP revision we intend to add descriptors of all aspects of performance for each WBA to encourage consistent and reproducible commentary and feedback. We also plan to ensure that due emphasis is given to the complexity of a procedure or encounter in order to address the tension between competence and experience. Competence in a procedure, for example, does not mean ability to cope with every eventuality; this can only come with experience.
WBAs have become a core feature of surgical training over the last few years. They are evolving and, although we need to have some fixed processes to test learning, we need to ensure what we do on a daily basis is worthwhile and encourages and demonstrates progression. This will promote a high quality experience for both the trainee and trainer and also result in a high quality product of surgical training.
Bill Allum, ISCP Surgical Director
Lewis Ashman
(+admin +former tickbox admin)