O2055: Standardizing Training and Assessment: Creation of Pediatric Neurocritical Care Entrustable Professional Activities for Pediatric Critical Care Medicine Fellows

Background: Graduate medical education (GME) is increasingly adopting competency-based medical education (CBME) principles to standardize trainee assessment and ensure proficiency. Assessment frameworks have evolved from Competencies (e.g., Patient care) and Milestones (e.g., Performs fundamental physical examination) to specialty-board-sponsored entrustable professional activities (EPAs). EPAs integrate Competencies and Milestones into discrete tasks which physicians should be proficient at (e.g., pediatric critical care medicine (PCCM) EPAs; Figure 1). The current PCCM EPAs are broad and supplemental EPAs may aid trainees and training programs with curricula and competency-based decisions about trainee advancement. In particular, pediatric neurocritical care (PNCC) is a sub-field of PCCM wherein diverse institutional settings, variable practice patterns, and a wide array of pathology results in a variable trainee experience; without specific criteria for benchmarking, PCCM fellows have varying levels of proficiency in PNCC at the end of training. To address this gap, we developed a set of validated PNCC EPAs for PCCM fellows to 1) standardize PCCM fellow competency assessment in PNCC and 2) develop a prototype for creating other EPAs as medical education transitions to an outcomes-based CBME training model.

Methods: Local and national interprofessional groups simultaneously created comprehensive lists of essential PCCM PNCC-related activities guided by ABP testing content guidelines and a practice analysis of PCCM knowledge and skills. These activities formed a preliminary set of PCCM PNCC EPAs. Modified Delphi methodology was used to demonstrate content validity and craft acceptable wording of the EPAs. During iterative Delphi rounds using online surveys, experts used a 5-point scale to rate “how essential” and “how clear” each EPA was. The comprehensiveness of the EPA set was rated; opportunity was provided for free text comments and/or identification of additional content. Between rounds, data was analyzed and comments were incorporated into the drafted EPAs prior to survey redistribution. Content validity was estimated using the index of content validity (ICV; proportion of items receiving Likert scale rating of 4 or 5). A priori, EPAs with ICV ≥0.8 were interpreted as sufficient validity for inclusion, ≥0.6-0.79 were revised and revoted on, and < 0.6 were removed. Consensus was reached when each EPA’s essential rating and the EPA set comprehensiveness were ≥0.8. The Delphi panel then met virtually to group and finalize the EPAs (work ongoing).

Results: Experts from PCCM, PNCC, and GME were recruited from a variety of institutional settings. Response rate was 82% (31/38) for Delphi round one and 94% (29/31) for round two. Consensus was achieved with 13 EPAs (Figure 2) with 93% agreement on EPA set comprehensiveness.

Conclusion: Using modified Delphi methodology, we developed a set of PNCC EPAs with content validity for PCCM fellows. Conducted over a virtual platform, this process is readily reproducible for the development of other supplemental EPAs in PCCM and beyond.