Abstract

OS1-134

Quality and impact of survey research in pediatric anesthesiology

¹Ahmed S, ¹Syed F, ¹Miller R, ²Khawaja A, ¹Tumin D, ¹Tobias J
¹Nationwide Children's Hospital, Columbus, OH, United states; ²Joan C. Edwards School of Medicine Marshall University, Huntington, West virgi, United states

Introduction: Online survey research has rapidly emerged as a tool for studying clinical, educational, and professional issues in anesthesiology. Yet, validity of survey data may be threatened by low response rates and missing data. In this study, we examined the prevailing quality and impact of survey research in pediatric anesthesia professional societies. We hypothesized that high response rates would signal high validity of survey findings and contribute to greater impact within and beyond the academic literature.

Methods: We conducted a systematic search of the MEDLINE database for English-language original research published in 2010-2016 and involving online surveys of pediatric anesthesia professional societies in the United States, Canada, Great Britain, Australia and New Zealand. Articles were independently coded by 3 investigators to determine the response rate and use of reminders. Impact metrics included citation counts (from Google Scholar) and Altmetric scores (accounting for news and social media coverage), and were compared to measures of survey quality using Spearman rank correlations (rho). Interrater agreement was assessed using the intraclass correlation coefficient (ICC).

Results: Eleven articles were included, totaling 5,247 respondents (median sample size = 510). Interrater agreement was strong, with ICC of 0.99, 0.82, 0.99, and 1.00 for response rates, number of reminders, number of citations, and Altmetric scores. Ratings were averaged for further analysis. Response rates ranged from 24% to 52%, with a median of 36% (Table). Among the 11 studies, 5 mentioned a low response rate as a possible limitation to their conclusions. There were 5 surveys with no reminders, 2 surveys with 1 reminder, and 4 surveys with 2 reminders. Survey response rates were not correlated with the number of citations (range: 1, 70; rho = -0.1; p=0.723) or Altmetric scores (range: 1, 5; rho = 0.0; p=0.988). The number of reminders was not correlated with the response rate (rho = -0.1, p=0.827), number of citations (rho = -0.3, p=0.324), or Altmetric scores (rho = 0.2, p=0.535).

Discussion: Response rates are generally low in published surveys of pediatric anesthesia professional societies. Although use of reminders are commonly recommended to improve response rates, it was not correlated with survey response rates in this study. Response rates did not appear to influence articles’ impact, measured either by citation count or Altmetric scores. This suggests that even surveys with low response rates may generate valid and useful knowledge for pediatric anesthesiologists, although low response rates and small sample sizes can limit precision in survey estimates.

References:

1. Tait AR, Voepel-Lewis T. Survey research: it's just a few questions, right? Paediatr Anaesth 2015;25:656-62.
2. Dillman DA, Smyth JD, Christian LM. Internet, phone, mail, and mixed-mode surveys: the tailored design method. Hoboken, NJ: Wiley.