Telling the PD Travel Story Part 4

Chapter 6 - Impact  

And so, we arrive at the end of our story and the promised land of improved student outcomes. The logic pathway from PD inception to improved outcomes is not at all straightforward and is so multi-causal that drawing a direct line between the two may be impossible. There are however some intermediary steps that may be easier to measure, and in themselves may be a useful for PD leaders to reflect on. 

In 1976 Donald Kirkpatrick produced a taxonomy of data types that could be used to evaluate training programmes in organisations as well as other interventions such as coaching and mentoring. Despite its age, the model remains a benchmark tool for evaluation and has inspired several adaptations including Kaufman’s Model of Learning Evaluation (1994) and Guskey’s (2000) work on evaluating professional development in schools. The four levels of Kirkpatrick’s model are as follows:  

Level 1 – Reaction: How do the participants feel about the training activity/programme they attended? To what extent are they satisfied customers?   

Level 2 – Learning: To what extent have the participants learned the information and skills?  

Level 3 – Behaviour: To what extent has the participant’s job behaviour changed because of attending the training activity/programme?  

Level 4 – Results: To what extent have results been affected by the training activity/programme?  

Integrating similar levels of evaluation depth into PD QA processes may help PD leaders reach a deeper understanding other the effectiveness and impact of their PD activities / programmes. 

So what? What next?  

So how can the concept of a PD travel story add value? Well, if we want to draw links between PD and student outcomes then we need to acknowledge that the quality of PD is key to the strength of that link. I guess my hope is that reflecting on and narrating the story of an PD activity may lead to improvements in its design, enactment and impact. My plan is to formalise the conversation that drove the creation of this post into something that I can use more systematically with my PD leaders. Below is a link to a document that I am going to trial at various stages during the PD life cycle to guide the reflection and decision making of my PD leads. Feel free to give it a try yourselves and let me know how you get on! 

References:  

• Blume B, Ford KJ, Surface E et al. (2019) A dynamic model of training transfer. Human Resource Management Review 29(2): 270–283.  

• Burke L and Hutchins H (2007) Training transfer: An integrative literature review. Human Resource Development Review 6: 263–296.  

• Department for Education (DfE) (2016) Standard for teachers’ professional development. Available at: www.gov.uk/government/publications/standard-for-teachers-professional-development (accessed 6 Oct 2023).  

• De Ruiter, R., & van der Klink, M. (2003). Personality characteristics and training transfer: A review. Journal of Workplace Learning, 15(3), 101-110.  

• Grohmann, A. K., Ehrhart, M. G., & Hofmann, D. (2014). Personality traits and training transfer: A meta-analysis. Journal of Personality and Social Psychology, 107(6), 917-938.  

• Grossman R and Salas E (2011) The transfer of training: What really matters. International Journal of Training and Development 15(2): 103–120.  

• Guskey, (2000) Gauge impact with five levels of data - https://tguskey.com/wp-content/uploads/Professional-Learning-1-Gauge-Impact-with-Five-Levels-of-Data.pdf  

• Heifetz, Ronald A., and Marty Linsky.(2017) Leadership on the Line: Staying Alive through the Dangers of Change, 2. Harvard Business Review Press. 

• Knowles, M. S. (1970). The modern practice of adult education: Andragogy versus pedagogy. New York, NY: Association Press.  

• Kaufman, R. and Keller, J. (1994) Levels of evaluation: beyond Kirkpatrick’. Human Resource Development Quarterly, Vol 54(4).  

• Kirkpatrick, D.L (1976) Evaluation of training. In R.L Craig (ed.) Training and development handbook. A guide to Huan Resource Development. New York: McGraw-Hill pp.181-91.  

• Knowles, M. S. (1980). The modern practice of adult education: From andragogy to pedagogy. New York, NY: Association Press.  

• McCrea, P. (2023). Developing expert teachers. Routledge.  

• Simpson J. A., Weiner E. S. C. 1989. The Oxford English Dictionary (2nd ed., Vol. 20). Oxford, United Kingdom: Clarendon  

• Wexley K and Latham G (1981) Developing and Training Human Resources in Organizations. Upper Saddle River, NJ: Prentice Hall.