O4305: Development of Standardized Patient Scripts and an Assessment Tool to Evaluate the Trauma-informed Care Communication Skills of Pediatric Trainees

Background: Pediatricians recognize the need for training in childhood trauma and resiliency. A review of the medical education literature reveals a handful of institutions delivering their own trauma-informed care (TIC) curricula. Most evaluation methods focused on Kirkpatrick level 1 (reaction) and level 2 (learning). At level 3 (application of learning), most studies varied in their measures. Our pediatric residency program adapted the AAP’s “Pediatric Approach to Trauma, Treatment, and Resilience” (PATTeR) course to implement TIC training. Following the PATTeR training, interns demonstrated significant changes in pre-post self-assessed knowledge and confidence scores. To evaluate higher-level learner outcomes, we developed standardized patient (SP) cases and an assessment tool to measure trainee communication skills in providing trauma-informed care in SP encounters.

Methods: We developed 2 SP scripts with our institution’s SP program: (1) an adolescent with somatic symptoms (headache) with recent history of trauma; (2) the parent of a toddler with developmental delay concerns following parental separation and mental health concerns in the home. For the assessment tool, we adapted the previously published Gap-Kalamazoo Communication Skills Assessment Form (GKCSAF).
We invited a TIC expert panel comprised of medical educators, pediatricians, and mental health experts from academic centers, community clinics, the AAP, and the National Child Traumatic Stress Network to review the GKCSAF content, SP scripts, and TIC guidelines, and used their feedback to create the GKCSAF adapted for Trauma-Informed Care (GKCSAF-TIC). The GKSCAF-TIC employs a 5-point Likert scale and incorporates the following TIC domains: relationship building, engagement, inquiry about adversity and trauma, understanding perspectives, response to distress, collaboration, empowerment, and closure.
The learners are first-year pediatric interns enrolled in their Developmental and Behavioral Pediatrics (DBP) rotation. At the beginning of the rotation (pre-intervention), each intern participates in a 15-minute SP encounter, and then receives the PATTeR training. At the end, each intern participates in a second 15-minute SP encounter (post-intervention). Both SP sessions are video-taped and scored by 2 trained faculty reviewers using the GKCSAF-TIC.

Results: The education program was piloted with 15 pediatric trainees. Content and response process for the assessment tool were supported by the development and rater training process. Cronbach’s alpha (internal consistency) ranged from 0.94-0.95 across the 2 faculty raters, and intra-class correlations (inter-rater reliability) ranged between 0.82 (pre-data) and 0.67 (post-data).

Conclusion: The GKCSAF-TIC demonstrates preliminary validity evidence as an assessment tool for TIC communication skills across the domains of content and response process. Initial internal structure is supported by relatively high internal consistency statistics and moderate to high inter-rater reliability, but further refinement is needed. The educational community can consider the standardized patient scripts and the TIC communication skills assessment tool for their TIC trainings.