Sensational Kids Program Evaluation Write-Up:

September 2021 to July 2022

 

Sensational Kids has continued using the Canadian Occupational Performance Measure (COPM; Law et al., 2000) for formal program evaluation. The COPM is a semi-structured interview tool for setting goals with clients that are meaningful to their daily lives; it has been found to be a preferred, valid, sensitive, and feasible method for ongoing outcome measurement in multidisciplinary pediatric therapy practice (Komar et al., 2016; Cusick et al., 2006, 2007; Komar et al., 2016; Verhoef et al., 2013; Wressle et al., 2003). Clients (in this case primarily caregivers/parents) rate their progress with each goal in terms of child performance and satisfaction with performance. This process was started in late September of 2021, and since that time 93 clients have been introduced to the program evaluation process. Second ratings are taken approximately 12 visits or three months after the first ratings are collected. Two clients have started the third round of ratings.

As part of the program evaluation process, across clients, services, and locations, 256 client-centered goals have been initiated, suggesting the variety of needs being addressed at Sensational Kids.

Since introducing the COPM as a program evaluation tool, 27 clients have completed a follow-up rating. Of those clients with follow-up ratings, 85% are reporting clinically meaningful changes in child performance related to individualized goals. A change score (the average difference in ratings across time) of 2 points is considered to reflect clinically meaningful difference – that is, change that would be noticeable and impactful on daily life for a client (Cusick et al., 2007; Law, 2021). At Sensational Kids, the average change score for client ratings of child performance is 3.53 points, more than a full point higher than would be required to observe meaningful difference in daily life. These perceived gains are further supported by therapist reports of client changes over the course of treatment, which also show a meaningful difference at an average of 2.94 points. Client satisfaction with child performance has also increased over the course of intervention at Sensational Kids, at an average of 3.46 points per client.

 

From this data, it can be seen that both client perception of performance on meaningful goals and their satisfaction in daily life have improved significantly over the course of intervention. Therefore, despite being in use for less than a year, the COPM has already begun to document numerically the profound positive changes that therapists and clients know to be true about intervention at Sensational Kids.

 

Sensational Kids Program Evaluation Brief:

September 2021 to July 2022

Sensational Kids began using the Canadian Occupational Performance Measure (COPM) in late September of 2021, and since that time 93 clients have been introduced to the program evaluation process. As part of the program evaluation process, across clients, services, and locations, 256 client-centered goals have been initiated, suggesting the variety of needs being addressed at Sensational Kids.

Since introducing the COPM as a program evaluation tool, 27 clients have completed a follow-up rating. Of those clients with follow-up ratings, 85% are reporting clinically meaningful changes in child performance related to individualized goals. A change score (the average difference in ratings across time) of 2 points is considered to reflect clinically meaningful difference; at Sensational Kids, the average change score for client ratings of child performance is 3.53 points. These perceived gains are further supported by therapist reports of client changes over the course of treatment, which show a similar average of 2.94 points. Client satisfaction with child performance has also increased over the course of intervention, at an average of 3.46 points per client.

• Sensational Kids Program Analysis – July 18, 2022
• Total number of clients involved in program evaluation: 93
• Total number of goals created for first cycle of program evaluation: 256
• Clients with follow-up data as of July 18, 2022: 27 (up from 11 in April 2022)
• Number of clients reporting clinically meaningful change in child performance: 23 (85%)
• Number of therapists reporting clinically meaningful change in child performance: 18 (75%)
• Average change score for client ratings of child performance: 3.53
• Average change score for client ratings of satisfaction: 3.46
• Average change score for therapist rating of child performance: 3.31
• Average change score for therapist satisfaction: 3.31

 

*Note: as of 2021, a change of 2+ points is considered clinically meaningful for performance ratings (Cusick et al., 2007; Law, 2021).

Sensational Kids is proud to have achieved a rating of 3.53, which is 1.5 points higher than what is considered to be clinically meaningful change of 2 points.

 

References

• Cusick, A., Lannin, N. A., & Lowe, K. (2007). Adapting the Canadian Occupational
  Performance Measure for use in a paediatric clinical trial. Disability and Rehabilitation,
  29(10), 761–766. https://doi.org/10.1080/09638280600929201
• Cusick, A., McIntyre, S., Novak, I., Lannin, N., & Lowe, K. (2006). A comparison of Goal
  Attainment Scaling and the Canadian Occupational Performance Measure for paediatric
  rehabilitation research. Pediatric Rehabilitation, 9(2), 149–157.
  https://doi.org/10.1080/13638490500235581
• Komar, A., Ashley, K., Hanna, K., Lavallee, J., Woodhouse, J., Bernstein, J., Andres, M., &
  Reed, N. (2016). Retrospective analysis of an ongoing group-based modified constraint induced movement therapy program for children with acquired brain injury. Physical &
  Occupational Therapy In Pediatrics, 36(2), 186–203.
  https://doi.org/10.3109/01942638.2015.1076557
• Law, M. C. (2021). Frequently asked questions: COPM. Retrieved July 30, 2021, from
  https://www.thecopm.ca/faq/
• Law, M. C., Baptiste, S., Carswell, A., McColl, M., Polatajko, H., & Pollock, N. (2000).
  Canadian occupational performance measure (3rd ed.). CAOT Publications ACE.
• Verhoef, J. A., Roebroeck, M. E., van Schaardenburgh, N., Floothuis, M. C., & Miedema, H. S.
  (2014). Improved occupational performance of young adults with a physical disability after
  a vocational rehabilitation intervention. Journal of Occupational Rehabilitation, 24(1), 42-
  51. https://doi.org/10.1007/s10926-013-9446-9
• Wressle, E., Lindstrand, J., Neher, M., Marcusson, J., & Henriksson, C. (2003). The Canadian
  Occupational Performance Measure as an outcome measure and team tool in a day treatment programme. Disability & Rehabilitation, 25(10), 497-506