Development of a Telemedicine Screening Program During the COVID-19 Pandemic

Introduction

Recent technological improvements have driven the rapid evolution of telemedicine use by health care professionals.¹ The World Health Organization declared COVID-19 a pandemic on March 11, 2020, and telemedicine use has dramatically accelerated because of concerns for patient and health care provider safety.² The Centers for Medicare and Medicaid Services encouraged telemedicine use through expanded coverage, services, and payments for telemedicine. For patients with COVID-19, telemedicine can help with remote assessment and care provision. For those not infected with COVID-19, especially high-risk populations, telemedicine can provide convenient access to routine care without risking exposure within a medical facility.^3,4 Another major application of telemedicine is coordination of COVID-19 testing.⁵

Pediatric patients have their own unique challenges in using telemedicine.⁶ Although previous pediatric studies showed successful telemedicine use in multiple specialties, including critical care medicine, outpatient care, neonatology, neurology, and pulmonology,^7–11 the most recent data show that pediatric telemedicine use remains low.

Global emergencies represent a relatively unknown area of telemedicine use. Studies during the H1N1 pandemic found significant variability in adherence to screening and treatment guidelines.^12,13 Despite reports of telemedicine use during the H1N1 and COVID-19 pandemics, it has not been well studied.

Given concerns for increased transmission, lack of abundant testing resources, and the large geographical range of patients served by Augusta University Medical Center (AUMC), a telemedicine platform with frequently updated screening guidelines was implemented for COVID-19 testing in March 2020. As we learned more about COVID-19, guidelines often changed, raising questions about screeners’ adherence to them.

This study primarily aimed to understand adherence to telemedicine COVID-19 screening guidelines.

Methods

Population Identification

The study population included all health care providers and patients participating in an encounter in the Augusta University (AU) Health Express Care virtual care program (American Well Platform, Boston, MA) from March 22 to May 21, 2020.

All encounters were intended for COVID-19 screening, free for residents of Georgia and some parts of South Carolina, and available 24 h per day, 7 days per week. The virtual care program could be accessed by the person and health care professional through phone, laptop, computer, or tablet. If the telemedicine encounter was disconnected, the provider could call the patient back on their phone number and complete the visit through audio-only. Participants without access to virtual care program were able to call hot line and arrange for screening by physician through a landline phone service. Encounters that used landline for screening were not included for analysis in our study. After May 21, 2020, telemedicine screening was discontinued as more widespread testing became available throughout Georgia.

All providers were health care professionals employed by AUMC, with degrees including (allopathic) medical doctor, doctor of osteopathic medicine, bachelor of medicine/bachelor of surgery, doctor of medicine in dentistry, physician assistant, nurse practitioner, registered nurse, and more. Degrees were grouped for analysis. Provider specialties were also grouped for analysis, and included emergency medicine, internal medicine, family medicine, pediatrics, surgery, dermatology, neurology, psychiatry, and more. Some providers performed screening during clinic time, whereas others performed screening after business hours and were compensated hourly.

Telemedicine Screening

Before an encounter, patients were asked to type answers to various preset questions such as their contact information and health history. Within an encounter, the patient and provider could see and hear each other, and the provider screened the patient for COVID-19 based on screening guidelines. If patients required COVID-19 testing or quarantine, providers were instructed to use International Classification of Diseases (ICD)-10 code Z20.828—Contact with and (suspected) exposure to other viral communicable diseases. Otherwise, provider used ICD-10 code Z13.9—Encounter for screening, unspecified.

Screening Guidelines

Screening guidelines, formatted as annotated flowcharts, were developed by AUMC based on information from the Centers for Disease Control and Prevention (CDC) and the Georgia Department of Public Health. The latest guidelines were made viewable in the AU Health Express Care virtual care application to be viewed during the telemedicine visit and e-mailed to providers as they were updated. Twenty-three versions of the guidelines were produced by July 23, 2020, but some were never implemented because of the rapidity of changes, and only seven guideline versions were used within this study’s timeframe.

Provider Training

Providers were trained in the telemedicine screening process throughout the study period. In March and April 2020, training consisted of either an in-person or 1-h virtual training session. It later became a video that walked providers through the steps of a telemedicine screening visit.

Data Collection

Encounter-level data were exported from the AU American Well platform as CCR XML files. Using the programming language R, files were parsed to extract discrete variables, including patient-entered responses and provider-entered documentation. Data were converted to numbers to blind analyses. Positive and negative screens were determined by encounters’ diagnoses. The logic of the most current guideline version at the time of each encounter was evaluated to determine whether the patient should have had a positive or negative screen. The guideline version immediately prior was also evaluated.

Low-frequency specialties were combined in logical grouping to improve analytical power (e.g., cardiothoracic surgery was added to surgery). Providers without a match in the AUMC provider database were excluded (e.g., test providers, providers with incomplete names). Income data per U.S. zip code were obtained from the Internal Revenue Service.¹⁴

Data Analysis

Guideline adherence was defined as the provider’s diagnosis (positive or negative screen) aligning with the programmed guideline logic’s decision based on documented findings in the encounter note.

To explore correlations between screening guideline adherence, provider characteristics, and encounter timing, likelihood ratio tests were performed among the adult, pediatric, and cumulative populations. Likelihood ratio tests were also performed to investigate correlations between provider specialty and the proportion of pediatric patient encounters.

To evaluate the incomes of the study’s patient populations’ residential zip codes, the weighted average total income was calculated for all U.S. zip codes. Weighted average total incomes were compared between the U.S. population and the study’s patient population using a two-sample t test.

Statistical analysis was performed using R. All statistical tests were two-sided using a 0.05 significance level.

Results

There were 17,805 telemedicine calls during the study period. We removed four duplicate encounters. We excluded 2,385 encounters occurring before screening guideline version 17, since we lacked complete information on versions before version 17. We then excluded 179 encounters with providers with incomplete information and test providers. In our guideline adherence analyses, 208 encounters had a diagnosis other than those denoting a positive or negative screen. We could not evaluate whether strict adherence to guidelines would lead to a positive or negative screen for 1,429 encounters because of some allowance in the guidelines for clinical judgment. Finally, there were 13,600 encounters in our guideline adherence analyses.

Overall adherence to screening guidelines in the study population was 70% (of 13,600 encounters). Adherence in the adult population was 71% (of 13,201 encounters) and 57% (of 399 encounters) in the pediatric population. When providers did not follow guidelines, 72% decided upon a positive screen (therefore test and/or quarantine). Guidelines themselves determined that only 52% of encounters should have a positive screen.

Providers’ specialty (department) was significantly correlated with guideline adherence (p = 0.002). Psychiatry, neurology, and ophthalmology providers were the most adherent, with 83%, 80%, and 78%, respectively. Dermatology, emergency medicine, college of nursing, and internal medicine providers were the least adherent, with 61%, 62%, 64%, and 67%, respectively (Fig. 1). The radiation oncology department showed high levels of adherence to guidelines at 85%, although this department was represented by one provider with only 28 total screening encounters.

Departments at AUMC using telemedicine before implementation of COVID-19 telemedicine screening were psychiatry and neurology. Adherence to guideline in adult population presented in Figure 2.

Provider specialty had a significant effect on adherence in the pediatric population. College of nursing, surgery, radiology, and internal medicine providers were the most adherent, with 78%, 73%, 73%, and 63%, respectively. Neurology, obstetrics/gynecology, and pediatric providers were the least adherent, with 28%, 33%, and 41%, respectively (Fig. 3). Radiation oncology, anesthesiology, and psychiatry had the smallest number of pediatric screening encounters with 1, 2, and 6, respectively.

Years of training in residency and fellowship (PGY-1 through PGY-7) had no significant effects on adherence, although PGY-4 were the most adherent (78%).

Provider position had no significant effects on adherence. Advanced practice providers (RN/NP), residents, faculty, and fellows followed guidelines, with 73%, 70%, 69%, and 62%, respectively.

Numbers of screening encounters for each department and provider are presented in the Supplementary Tables S1 and S2.

Evaluation of the distribution of calls showed that between 6 am and 10 am calls were minimal, with increasing numbers of calls between 2 pm and 11 pm.

The weighted average total income of all U.S. tax returns filed to the Internal Revenue Service in 2017 was $74,941 compared with $60,637 in this study’s patient population (p < 0.001). The weighted average total income per U.S. zip code ranged from $7,840 to $2,212,640.

Discussion

We believe that this study provides useful information regarding telemedicine use for COVID-19 screening. Although, our data reflect provider behavior during the pandemic’s initial phase, the findings might be useful during the entire pandemic period. We were interested in adherence to screening guidelines for COVID-19 testing.

Telemedicine has several key strengths that can enhance an emergency response when global emergencies present. Telemedicine enables remote triaging for care, provides rapidly accessible information through technologies, helps patients navigate the health system, and access routine care during a pandemic. However, for telemedicine to be effective during the current pandemic and future global emergencies, we must ensure that telehealth is appropriately integrated into our health care system. Evaluation of providers’ guideline adherence can help with collecting baseline data that can be useful for future pandemic preparedness efforts and address potential areas for improved adherence. To our knowledge, no other studies to date have evaluated the adherence to COVID-19 testing guidelines during the pandemic.

There were 17,805 screening encounters during the telemedicine screening period, with 13,600 included in the final analysis. Pediatric patients were 2.9% of the screened population. Overall adherence to screening guidelines in the study population was 70%. Adherence in the adult population was 71% and in the pediatric population 57%, indicating that there remains room for improvement regarding providers’ adherence to recommended screening strategies. Previous research during the H1N1 pandemic in emergency departments showed <50% adherence to CDC guidelines regarding antiviral prescriptions (underprescribing) and 88% (undertesting) adherence regarding H1N1 testing.¹²

Health care providers in our study tended to override guidelines and recommend COVID-19 testing and quarantine, especially in pediatric population. This is possibly because of parental concern, providers’ level of comfort with children, and uncertainty about children’s outcome if affected by COVID-19. During the initial screening period, guidelines lacked specific recommendations for pediatric screening. The referral for testing at that time possibly was based on recommendations for adult patients and providers’ level of comfort and discretion. In addition, infants and children may not articulate symptoms, leading to increased nonspecific testing. Research on adherence to real-time changes in H1N1 guidelines that also included the pediatric population showed that nearly one-third of patients were tested for influenza despite lacking influenza-like symptoms.¹³

We noted that, in general, specialties with more telemedicine experience had higher adherence to screening guidelines (psychiatry and neurology). At AUMC, psychiatry implemented telemedicine early in the pandemic for routine visits, and neurology has used telemedicine for years in the form of Telestroke.

Our study did not show statistically significant differences in adherence based on provider degree. However, there were significant differences in adherence based on specialty, with college of nursing, surgery, and radiology being the most adherent. Pediatric specialty providers showed only 41% adherence. This may have occurred because pediatric providers were more concerned about children and their prognosis early in the disease when little was known about pediatric effects of COVID-19. The lack of statistically significant differences between provider position and experience with this particular method of screening when comparing adherence supports the idea that early or initial training is more important in increasing adherence than experience itself.

There was no feedback to providers to help learn or adapt; rapid feedback and reminders could improve guideline adherence. Standardizing early training for providers could also improve guideline adherence and allow wider implementation of telemedicine for other purposes. Future studies could include training data and help explain relationships between training, experience, and performance in telemedicine screening. Callers were generally located in lower income zip codes. Our efforts to advertise the service on the news, billboards, and social media apparently successfully captured lower socioeconomic status populations.

Limitations

This study had additional limitations not previously discussed. Possible COVID-19 was diagnosed based on limited knowledge and heightened awareness. Agreement between a provider’s decision and programmed guideline logic does not necessarily mean the provider used the guideline to inform his/her decision. The testing algorithm was updated frequently and we do not know if providers were aware of these updates when they did their screening, if they did not check their e-mail, or look at the latest update in the American Well platform. We did not follow providers during screenings or perform any quality improvements to improve the adherence rate.

We did not have health care providers state that they were aware of the latest guideline before their telemedicine shift, but this may have improved their knowledge and adherence to the current guidelines. Since it was available in the American Well platform, we could have made providers review the guidelines every time they logged in to the system to keep them updated. Furthermore, many providers personalized their documentation, making our data more error-prone. In addition, they sometimes put alternative diagnoses other than the two screening ICD-10 codes.

Without results of the testing and patients’ further course, we do not know how adherence to guidelines affected patient outcomes, or if introduction of COVID-19 screening through telemedicine affected patient flow in the emergency departments or primary care offices. The screening process was supported by a grant and we are not aware of the cost-effectiveness of this program. We do not know what percentage of participants used a landline phone service as they were not included in our analysis. Another limitation of our study is lack of a qualitative component. We do not know why such a high percentage of providers chose to override the guidelines. A questionnaire of a focus group could be used in future to understand reasons to override the guidelines. Last, our screening was performed by a single academic medical center and may not be representative of the nation.

Conclusions

Overall, this study provides proof of concept of a free telehealth screening platform during an ongoing pandemic. Our screening experience was effective and different specialties participated. Our patient population lived in lower than average income zip codes, suggesting that our free telemedicine screening program successfully reached populations with higher financial barriers to health care. Early training and a posteriori knowledge of telemedicine was likely key to screening guideline adherence.

Authors’ Contributions

All authors provided substantial contributions to the design, drafting, and final approval of this study and agree to be accountable for all aspects of this research.

Disclosure Statement

No competing financial interests exist.

Funding Information

No funding was received for this article.

Supplementary Material

Supplementary Table S1

Supplementary Table S2

References

• 1. Office of Health Policy, Office of the Assistant Secretary for Planning and Evaluation. Report to Congress: E-Health and Telemedicine. Washington, DC: U.S. Department of Health and Human Services, 2016. Google Scholar
• 2. Wijesooriya NR, Mishra V, Brand PLP, Rubin BK. COVID-19 and telehealth, education, and research adaptations. Paediatr Respir Rev 2020;35:38–42. Medline, Google Scholar
• 3. Dhaliwal JK, Hall TD, LaRue JL, et al. Expansion of telehealth in primary care during the COVID-19 pandemic: Benefits and barriers. J Am Assoc Nurse Pract 2021. [Epub ahead of print]; DOI: 10.1097/JXX.0000000000000626. Crossref, Google Scholar
• 4. Smith AC, Thomas E, Snoswell CL, et al. Telehealth for global emergencies: Implications for coronavirus disease 2019 (COVID-19). J Telemed Telecare 2020;26:309–313. Crossref, Medline, Google Scholar
• 5. Hollander JE, Carr BG. Virtually perfect? Telemedicine for Covid-19. N Engl J Med 2020;382:1679–1681. Crossref, Medline, Google Scholar
• 6. Burke BLJr., Hall RW, Section on telehealth care. Telemedicine: Pediatric applications. Pediatrics 2015;136:e293–e308. Crossref, Medline, Google Scholar
• 7. Marcin JP, Nesbitt TS, Kallas HJ, et al. Use of telemedicine to provide pediatric critical care inpatient consultations to underserved rural Northern California. J Pediatr 2004;144:375–380. Crossref, Medline, Google Scholar
• 8. McConnochie KM, Tan J, Wood NE, et al. Acute illness utilization patterns before and after telemedicine in childcare for inner-city children: A cohort study. Telemed J E Health 2007;13:381–390. Link, Google Scholar
• 9. Hall-Barrow J, Hall RW, Burke BLJr. Telemedicine and neonatal regionalization of care—Ensuring that the right baby gets to the right nursery. Pediatr Ann 2009;38:557–561. Crossref, Medline, Google Scholar
• 10. Qubty W, Patniyot I, Gelfand A. Telemedicine in a pediatric headache clinic: A prospective survey. Neurology 2018;90:e1702–e1705. Crossref, Medline, Google Scholar
• 11. Portnoy JM, Waller M, De Lurgio S, Dinakar C. Telemedicine is as effective as in-person visits for patients with asthma. Ann Allergy Asthma Immunol 2016;117:241–245. Crossref, Medline, Google Scholar
• 12. Hsieh YH, Kelen GD, Dugas AF, et al. Emergency physicians’ adherence to Center for Disease Control and Prevention guidance during the 2009 influenza A H1N1 Pandemic. West J Emerg Med 2013;14:191–199. Crossref, Medline, Google Scholar
• 13. Vijayan V, Jing J, Zangwill KM. Evaluation of diagnostic and therapeutic approaches for suspected influenza A(H1N1)pdm09 infection, 2009–2010. Emerg Infect Dis 2012;18:1414–1421. Crossref, Medline, Google Scholar
• 14. Internal Revenue Service. SOI Tax Stats—Individual Income Tax Statistics—2017 ZIP Code Data (SOI). Ogden, UT: Internal Revenue Service, 2017. Google Scholar