The diagnostic accuracy of saliva testing for SARS‐CoV‐2: A systematic review and meta‐analysis – Atieh – – Oral Diseases

2.1 Types of studies

Prospective and retrospective human studies that collected paired samples and compared saliva samples with OPSs and/or NPSs for the detection of SARS-CoV-2 were included in the analysis. Case reports, animal studies, letters to journal editors, studies that were not formally peer-reviewed, reports on viral shedding following the first episode of infection or those that did not report sufficient information were excluded. No language restrictions were imposed.

2.2 Types of participants

Adult individuals who were 18 years of age or older and were tested for COVID-19.

2.3 Types of diagnostic tests

Saliva sample (index test) and OPSs and/or NPSs (reference standard).

2.4 Outcome measures

2.5 Search strategy

The search strategy recommended by Faggion and co-workers (Faggion et al., 2013) was used to identify studies related to the diagnostic accuracy of saliva in detecting COVID-19. The following electronic databases were searched for published and unpublished trials up to November 12, 2020: MEDLINE, EMBASE, The Cochrane Central Register of Controlled Trials (CENTRAL), MetaRegister, ClinicalTrials.gov, and the system for information on Grey literature in Europe (http://www.opengrey.eu) (Table A1). The search was performed independently and in duplicate by two authors (M.A. and N.A.). The reference lists of all potentially eligible papers were examined for additional studies. The last 10 months of relevant journals (Clinical Infectious Diseases, Clinical Microbiology and Infection, Journal of Clinical Microbiology, Journal of Clinical Periodontology, Journal of Clinical Virology, Journal of Infectious Diseases, New England Journal of Medicine) were hand-searched to identify any eligible papers.

2.6 Selection of studies

Two authors (M.A. and N.A.) independently screened the retrieved citations in duplicate to identify human studies that were appropriate for inclusion. The initial screening was based on the title, abstract, and keywords. After discarding non-relevant studies, the full-texts of the remaining studies were examined against a standardized eligibility form. Any disagreements between the two authors were resolved by consulting a third author (M.G). When a duplicate publication (i.e., multiple publications of the same study) was identified, the one with more relevant information was selected. The studies that did not meet the inclusion criteria were excluded and the reasons for exclusion were reported.

2.7 Data collection

Two authors (M.A. and N.A.) independently used a standardized data extraction form to collect the following information from eligible studies: (1) Study characteristics: title, authors’ names, contact details, study location, language of publication, year of publication, published or unpublished data, source of study funding, and study design; (2) Participants: demographic characteristics, inclusion/exclusion criteria, number of participants, number of dropouts, and reasons for exclusion; (3) Interventions: number of participants tested for COVID-19 using saliva samples; (4) Comparison: number of participants tested for COVID-19 using OPSs and/or NPSs; and (5) Outcomes: True-positive, false-positive, false-negative, and true-negative values. Additional information was also obtained such as saliva storage, method and timing of collection and processing. All recorded data were verified by the two authors (M.A. and N.A.). Any disagreements were resolved by discussion or by seeking opinion of a third author (M.G.).

2.8 Quality assessment

The revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool (Whiting et al., 2011) was used to assess the quality of selected studies. The risk of bias was examined in four domains: participant, selection, index test, reference standard, and flow/timing. Applicability was evaluated in the first three domains (participant selection, index test, and reference standard). Two review authors (M.A. and N.A.) graded the quality of studies as low, high, or unclear based on specific criteria of the QUADAS-2 tool. Any disagreements were resolved by discussion or by seeking opinion of a third author (M.G.).

2.9 Statistical analysis and data synthesis

The reported true positives/negatives and false positives/negatives were transferred to 2 × 2 contingency table to calculate sensitivity and specificity as well as positive likelihood ratio (LR+), negative likelihood ratio (LR−), and diagnostic odds ratio (DOR). In order to avoid computational issues, a 0.5 was added to each cell that contained a 0 value in the 2 × 2 table (Dinnes et al., 2005). The discriminating ability of a diagnostic test was more reliably measured using LRs as they are less dependent on the prevalence rate. A LR+ of more than 10 and a LR− of less than 0.1 indicated a satisfactory discriminating diagnostic performance (Jaeschke et al., 1994). Random effects meta-analytic models were used to pool sensitivity, specificity, LR estimates with 95% confidence intervals (CIs). The heterogeneity between studies was evaluated visually using forest plots and statistically using Cochran Q chi-square test and I² statistic. A p-value of <0.10 and I² value of >50 indicated a substantial heterogeneity (Higgins et al., 2003). The potential causes of heterogeneity among studies were explained by using a meta-regression model.

The SROC curve was used to graphically present the interaction between sensitivity and specificity. The overall diagnostic ability of saliva was quantified using the area under the curve (AUC). AUC ranges of 0.5–0.7, 0.7–0.9, and 0.9–0.99 indicate poor, moderate, and very good accuracy, respectively. A perfect accuracy is shown by an AUC of 1.0 (Akobeng, 2007). The DOR is the ratio of the odds of positive test results in participants with COVID-19 compared with the odds of positive test results in those without COVID-19. DOR ranges from 0 to infinity with greater values suggesting greater accuracy. The potential for publication bias was assessed using the funnel plot which is created from standard error and estimated effect size (log DOR). Statistical analysis was performed using the midas package (Deeks et al., 2005; Glas et al., 2003) in Stata/MP (version 14; StataCorp, LLC), and methodological quality was assessed using Revman 5.4 (version 5.4; The Nordic Cochrane Centre, The Cochrane Collaboration).

3.1 Characteristics of study settings

The initial search of the databases identified 49 studies (Figure A1). The titles and abstracts were assessed independently and in duplicate by two review authors (M.A. and N.A.). As a result, the full-texts of 20 studies (Aita et al., 2020; Altawalah et al., 2020; Berenger et al., 2020; Binder et al., 2020; Chen et al., 2020; Guclu et al., 2020; Hanson et al., 2020; Iwasaki et al., 2020; Jamal et al., 2020; Kim et al., 2020; Landry et al., 2020; Moreno-Contreras et al., 2020; Pasomsub et al., 2019; Procop et al., 2020; Rao et al., 2020; Sakanashi et al., 2021; Senok et al., 2020; Vaz et al., 2020; Williams et al., 2020; Wyllie et al., 2020) were retrieved for detailed assessment. A total of 4 studies (Berenger et al., 2020; Iwasaki et al., 2020; Kim et al., 2020; Wyllie et al., 2020) were excluded, and 16 studies (Aita et al., 2020; Altawalah et al., 2020; Binder et al., 2020; Chen et al., 2020; Guclu et al., 2020; Hanson et al., 2020; Jamal et al., 2020; Landry et al., 2020; Moreno-Contreras et al., 2020; Pasomsub et al., 2019; Procop et al., 2020; Rao et al., 2020; Sakanashi et al., 2021; Senok et al., 2020; Vaz et al., 2020; Williams et al., 2020) were included in the present review (Table 1). The hand searching did not identify any additional studies. All studies were published in English and all together they included 2928 paired samples.

Of the 16 included studies, eight Studies (Altawalah et al., 2020; Binder et al., 2020; Chen et al., 2020; Hanson et al., 2020; Jamal et al., 2020; Moreno-Contreras et al., 2020; Pasomsub et al., 2019; Rao et al., 2020) were funded or supported by university or research institutes, while five studies (Aita et al., 2020; Guclu et al., 2020; Procop et al., 2020; Vaz et al., 2020; Williams et al., 2020) did not provide any information on funding. Three studies (Landry et al., 2020; Sakanashi et al., 2021; Senok et al., 2020) did not receive any funding, and their sampling was part of routine laboratory investigations.

3.2 Characteristics of participants

All participants were aged ≥18 years old. Five studies (Hanson et al., 2020; Landry et al., 2020; Pasomsub et al., 2019; Procop et al., 2020; Senok et al., 2020; Williams et al., 2020) collected samples from outpatients, with or without symptoms suggestive of COVID-19, attending test centers or screening clinics, while one study (Rao et al., 2020) collected samples from individuals staying in quarantine centers. Three studies (Altawalah et al., 2020; Binder et al., 2020; Guclu et al., 2020) included in-patients with confirmed or suspected COVID-19. Three studies (Aita et al., 2020; Chen et al., 2020; Jamal et al., 2020) included only confirmed COVID-19 in-patients with fever, dyspnea, pneumonia, anosmia, or gastrointestinal symptoms. In one of these studies (Jamal et al., 2020), 77% of the participants had at least one comorbidity. Three studies (Moreno-Contreras et al., 2020; Sakanashi et al., 2021; Vaz et al., 2020) collected samples from both in- and out-patients.

3.3 Characteristics of index test and reference standard

The NPS was collected in the standard way of passing the swab through the nostril and up to the posterior nasopharynx and then removing the swab while rotating. NPS was considered the reference standard in all included studies but two (Guclu et al., 2020; Moreno-Contreras et al., 2020), which included both OPS and NPS. In those two studies, the swab was passed into the posterior oropharynx prior to inserting it into one nostril. NPS were collected by trained healthcare workers in all studies except for one (Hanson et al., 2020), where patients were instructed to self-collect under the supervision of healthcare workers. Differences in the commercial kits and laboratory protocols to detect SARS-CoV-2 were noticed (Table 1). Fourteen studies (Aita et al., 2020; Altawalah et al., 2020; Binder et al., 2020; Chen et al., 2020; Guclu et al., 2020; Jamal et al., 2020; Landry et al., 2020; Moreno-Contreras et al., 2020; Pasomsub et al., 2019; Rao et al., 2020; Sakanashi et al., 2021; Senok et al., 2020; Vaz et al., 2020; Williams et al., 2020) used RT-PCR, while two studies used both RT-PCR and transcription mediated amplification (TMA) (Hanson et al., 2020; Procop et al., 2020).

With regard to saliva sampling, three studies (Altawalah et al., 2020; Chen et al., 2020; Procop et al., 2020) collected saliva by asking patients to “cough up,” while other studies collected saliva by passive drooling into a sterile container (Binder et al., 2020; Sakanashi et al., 2021) or spitting (Guclu et al., 2020; Jamal et al., 2020; Moreno-Contreras et al., 2020; Pasomsub et al., 2019; Rao et al., 2020; Senok et al., 2020; Vaz et al., 2020; Williams et al., 2020). Two studies (Hanson et al., 2020; Landry et al., 2020) used both drooling and spitting to collect saliva specimen. Only one study (Aita et al., 2020) used chewing on an absorbent material to encourage salivation.

3.4 Methodological quality

Two QUADAS-2 domains, the index test and reference standard, were associated with unclear concerns in all studies, as it was not clear whether the interpretation of the saliva test results was influenced by the knowledge of the outcome of OPS and/or NPS. For the domain of patient selection, three studies (Aita et al., 2020; Chen et al., 2020; Jamal et al., 2020) enrolled only patients with confirmed COVID-19 and therefore were judged to be at high risk of bias and concerns regarding applicability. The domain of flow and timing was associated with a low risk of bias in all studies except for five (Altawalah et al., 2020; Binder et al., 2020; Hanson et al., 2020; Moreno-Contreras et al., 2020; Williams et al., 2020), where some samples were not included in the analysis (Figure A2).

Only one study (Senok et al., 2020) reported a priori-power analysis to calculate the required sample size to examine the diagnostic accuracy of the saliva test.

3.5 Results of meta-analyses

All of the 16 studies were included in the analysis. The pooled sensitivity and specificity for saliva test were relatively high at 0.88 (95% CI 0.82–0.92) and 0.92 (95% CI 0.75–0.98), respectively (Figure 1). Consequently, the pooled LR+ and LR− were 11.6 (95% CI 3.2–42.5) and 0.13 (95% CI 0.09–0.20), respectively, indicating adequate diagnostic information. The DOR was calculated at 87 (95% CI 19–395) which indicates good diagnostic value. However, wide 95% CI indicates significant heterogeneity among the included studies. The AUC was calculated as 0.92 (95% CI 0.90–0.94; Figure 2) suggesting very good performance of the saliva tests in detecting SARS-CoV-2. The differences in clinical utility between saliva and OPS/NPS for diagnosis of COVID-19 were evaluated using Fagan plot analysis. In terms of detecting SARS-CoV-2, the probability of COVID-19 increased from 20% to 74% when the saliva test was positive and decreased to 3% when the results were negative (Figure 3). A funnel graph analysis showed no evidence of publication bias (Figure A3).

Six studies (Aita et al., 2020; Altawalah et al., 2020; Binder et al., 2020; Chen et al., 2020; Guclu et al., 2020; Jamal et al., 2020) examined the use of saliva tests among in-patients with confirmed or suspected diagnosis of COVID-19 without including healthy individuals or out-patients. Moreover, half of the studies (Aita et al., 2020; Binder et al., 2020; Chen et al., 2020; Guclu et al., 2020; Jamal et al., 2020; Moreno-Contreras et al., 2020; Sakanashi et al., 2021; Williams et al., 2020) had a sample size less than 100, and three studies (Altawalah et al., 2020; Chen et al., 2020; Procop et al., 2020) collected coughed-out saliva without using any measure to stimulate saliva. The meta-regression analysis was used to assess these potential sources of heterogeneity. Studies were divided into groups as follows: sample size (≥100 vs. <100 patients), characteristics of patients (in-patients vs. in-/out-patients), and method of saliva collection (coughed out vs. other methods). Higher pooled sensitivity was observed when the study included a sample size of ≥100 in-and out-patients compared to sample size of <100 in-patients only (p < 0.05). The method of saliva collection did not significantly alter the performance of saliva test (Table 2, Figure A4).

4.1 Agreement and disagreements with other systematic reviews

The question of whether saliva is a reliable sample for detecting SARS-CoV-2 has been addressed in other systematic reviews (Czumbel et al., 2020; Fakheran et al., 2020; Fernandes et al., 2020; Torretta et al., 2020). Common limitations across all these reviews were the limited number of included studies, the small sample sizes within studies and the lack of stringent selection criteria allowing the inclusion of non-peer-reviewed studies. In addition, while the reliability of saliva as a diagnostic specimen was cited in the previous reviews, the conclusions were less robust due to their acknowledged serious limitations. By contrast, the findings of the present review were based on a comprehensive search strategy and meta-analyses of 16 peer-reviewed studies with each having a control group of the standard NPS/OPS sample as an a priori criterion for inclusion in the review.

4.2 Strengths and weaknesses

Despite the limited number of included studies, this up-to-date systematic review gives an evidence-based appraisal on the performance of the saliva test as an alternative diagnostic tool to the standard reference NPS/OPS for the detection of the SARS-CoV-2. It comes at a very appropriate time considering the staggering number of infected cases across the globe and the ongoing need for a simple and effective screening and diagnostic tool. The findings from the meta-analyses in this review support the use of saliva in detecting SARS-CoV-2. We should, however, bear in mind that detection of a virus by RT-PCR does not indicate infectivity. Although viral cultures are not feasible as a general screening test, they are still required to confirm whether the virus is in fact infectious. In this context, there remains a need to validate the RT-PCR results, whether from NPS or saliva, against a viral culture before recommending NPS or saliva as the “gold” standard in the detection of SARS-CoV-2. Another element of relevance within this context is the need to identify the source of SARS-CoV-2 in saliva. Potential sources such as draining debris from nasopharyngeal epithelium, gingival crevicular fluid, secretions from infected salivary glands, and oral mucosal endothelial cells have all been proposed but remain inconclusive (Liu et al., 2011; Silva-Boghossian et al., 2013; To et al., 2020; Xu et al., 2020).

Nevertheless, the use of saliva for detecting other RNA viruses, including Zika and Ebola viruses, is well-documented (Gorchakov et al., 2019; Khurshid et al., 2019; Niedrig et al., 2018), and in our findings, saliva demonstrated good diagnostic accuracy with considerable similarities to the results obtained with the standard OPS/NPS.

The use of saliva will allow for self-collection of specimens and specimen collection in outpatient and community clinics. These possibilities will help reduce the overall cost of testing, including healthcare worker time and personal protective equipment (PPE) requirement, and reducing the healthcare workers’ risk of infection. In addition, the effectiveness of self-collected saliva was shown to be in moderate agreement with trained healthcare worker-collected NPS samples for detecting SARS-CoV-2 (Ku et al., 2021). On the other hand, the ability of the patient to understand the safe sampling instructions and the ability to collect sufficient quantity of saliva could be challenging (Torretta et al., 2020). The risks of disease spread may not be completely eliminated with the use of saliva sample as spitting or coughing is required to collect the saliva specimens. This in itself could provide a route for aerosol transmission (Sullivan et al., 2020), and the need for a standardized safe method for the saliva sample collection, therefore, remains an essential requirement.

Further research looking specifically at different confounding factors such as the method and timing of sample collection, the transport medium, storage, timing of RNA isolation, and detection is needed prior to mass application of the saliva sample test as a standard method for the detection of the SARS-CoV-2.