Comparison of Two Molecular Diagnostic Tests for COVID-19: Abbott RealTime SARS-CoV-2 and Allplex™ 2019-nCoV, in the Epidemic Context in Senegal

Background: The persistence of the rapid spread of the COVID-19 pandemic is linked to the appearance of several variants of SARS-CoV2 with an impact on biological diagnosis, treatment and vaccination. The United States Food and Drug Administration (FDA) has granted several SARS-CoV-2 detection tests Emergency Use Authorization (EUA) for diagnosis and better epidemiological surveillance. Thus, multiple RT-PCR tests have been developed and brought to market in order to meet the urgent need for the diagnosis of COVID-19. However, comparative data between these tests in clinical laboratories are scarcely available to assess their performance. Objective: To compare two molecular methods for detecting SARS-CoV-2: the RT-PCR, Allplex &#8482; 2019-nCoV tests on CFX96 Bio-Rad and the Abbott m2000sp/rt RealTime SARS-CoV-2. Materials and Methods: Nasopharyngeal and oropharyngeal swabs were taken from patients to diagnose SARS-CoV-2 infection. For each sample, we searched for the virus with two different RT-PCR tests: 1) first on Abbott m2000 SARS-CoV-2 targeting the N and RdRp genes, 2) then on Allplex &#8482; 2019-nCoV Assay looking for the E, N and RdRp genes. Results: Percentages of the agreement were calculated. A total of 100 samples that tested negative and 90 positives on Abbott m2000 SARS-CoV-2 were retested on Allplex &#8482; 2019-nCoV. Overall agreement was 74.74% on all samples. The specific agreement was 84% and 64.4% respectively for negative and positive samples with the RealTime SARS-CoV-2 test. A positive correlation (r ² = 0.63; p < 0.0001) was found between the two tests associated with a kappa coefficient of 0.5, thus demonstrating an acceptable concordance between the two methods. However, 48 samples gave discordant results, in particular those whose Ct (Cycle threshold) was high (low viremia). A comparison by Cts found that concordance was higher for lower Cts (high viremia) which would indicate the acute phase of the disease. Note that our study revealed a possible cross-reaction with the E gene of a Sarbecovirus. It is a subgenus of Betacoronavirus grouping coronaviruses linked to severe acute respiratory syndromes such as SARS-CoV and SARS-CoV-2. Conclusion: Our results showed good overall agreement between RT-PCR, Allplex &#8482; 2019-nCoV and Abbott RealTime SARS-CoV-2 tests in the diagnosis of COVID-19. As the concordance is low for small viremias, the RT-PCR Allplex &#8482; 2019-nCoV Assay would be better indicated during the acute and symptomatic phase of the disease.

• SARS-CoV-2
• COVID-19
• Diagnostics
• RT-PCR
• Abbott RealTime
• Seegene-Allplex

1. Introduction

The World Health Organization (WHO) was alerted for the first time to an episode of clustered cases of pneumonia of unknown etiology in Wuhan, People’s Republic of China, on December 31, 2019 [1]. The Coronavirus Disease 2019 (COVID-19), which is caused by Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), has since spread worldwide and is thus a major burden for society [2]. The clinical picture of SARS-CoV-2 infection is very broad and can range from asymptomatic infection to severe forms of infection including Acute Respiratory Distress Syndrome (ARDS) (WHO, 2020). Coronaviruses are enveloped positive single-stranded RNA viruses, classified into four genera (alpha and beta infecting humans, delta and gamma infecting animals, particularly bats and rodents). The SARS-CoV-2 virus belongs to the β-coronaviruses. It is a spherical and enveloped virus whose diameter varies between 60 and 140 nm [3] and which encodes four essential viral structural proteins, in particular proteins S, E, M and N. The latter interferes with the host’s immune response. Being an RNA virus, mutations are frequent and can generate variants of interest that allow SARS-CoV-2 to continue its spread despite the immunization of the population [4]. In recent months, variants of concern such as Alpha (first identified in the UK), Beta (originally identified in South Africa), Gamma in Brazil, Delta in India [4] or more recently the Omicron variant (first identified in Botswana) [5] have been identified. The latter has the highest number of mutations among the variants of concern [6]. These mutations, mainly protein, can have an impact on the contagiousness, the effectiveness of vaccines, the severity of the disease or even the virological diagnosis [7]. Therefore, it would be essential to update the molecular tests for SARS-CoV-2 because the probes initially designed at the start of the pandemic no longer hybridize with the target genes.

In addition, the rapid and precise identification of this pathogenic virus plays an essential role in the rapid implementation of appropriate therapies and in limiting the rate of spread of this disease. It is, therefore, important to have rapid and reliable screening tests for SARS-CoV-2 that will make it possible to track and contain the disease. The Polymerase Chain Reaction (PCR) is considered a reference test for the molecular diagnosis of viral and bacterial infections, with high sensitivity and specificity [8]. Regarding SARS-CoV-2, most of the genes targeted by these tests are the E envelope, N nucleocapsid, RdRp RNA-dependent RNA polymerase and S glycoprotein genes. The CoV genome has a variable number of Open Reading Frames (ORFs). Two-thirds of the viral RNA located mainly in the first ORF (ORF1a/b), codes for two polyproteins, pp1a and pp1b giving a total of 16 Non-Structural Proteins (NSPs). The rest of the virus genome encodes four essential structural proteins, including glycoprotein Spike (S), Envelope protein (E), Matrix protein (M) and Nucleocapsid (N), as well as several accessory proteins, that influence the host’s immune response [9]. The first quantitative test for the detection of SARS-CoV-2 was the RT-PCR validated in January 2020 by the WHO [10]. As of March 2020, several SARS-CoV-2 detection tests have received Emergency Use Authorization (EUA) from the United States Food and Drug Administration (FDA) [11]. Many commercial kits have thus become available, such as the Abbott RealTime SARS-CoV-2 test (Abbott Molecular, Plains, IL) and the Allplex™ 2019-nCoV Assay molecular test, but their performance has not yet been thoroughly evaluated by an independent party [2].

An effective fight against the COVID-19 pandemic necessarily involves the use of reliable tests with high sensitivity and specificity to limit or even avoid incorrect diagnoses, false negatives in particular. For this reason, the evaluation of diagnostic tests is of paramount importance and it is within this framework that our work falls.

Here, we present a comparative study between two molecular methods for the detection of SARS-CoV-2: the Abbott RealTime SARS-CoV-2 Assays (Des Plaines, IL) and Allplex™ 2019-nCoV (Seegene, Seoul, South Korea). The objective of this study was to comparatively determine their analytical and clinical performance for the detection of SARS-CoV-2.

2. Materials and Methods

2.1. Study Population

A total of 190 patients were tested for molecular diagnosis (RT-PCR) of SARS-CoV-2 at the molecular biology laboratory of the Armed Forces AIDS Program at the Ouakam Military Hospital (HMO) Dakar, Senegal, from February to September 2021. No distinction has been made according to age and gender.

2.2. Samples

Specimens has been collected by nasal and oropharyngeal swabs. Two swabs has been collected per patient: 1) first a pharyngeal swab by placing it respectively against the back wall of the throat and pressing 2 to 3 times against the dorsal wall of the pharynx; 2) then a nasal swab. Both 2 swabs have been discharged into viral transport media. After nucleic acid extraction, molecular testing on the Abbott m2000sp/rt platform has been performed on the nucleic acid extract. The rest of the samples were stored at −80˚C for subsequent performance of the Bio-rad CFX 96™ real-time PCR test after extraction on Zymo RNA kit™. The performance of the techniques has been evaluated by testing 100 negatives and 90 positives already tested on Abbott.

2.3. Molecular Tests

2.3.1. Abbott RealTime SARS-CoV-2

This assay has been designed for the qualitative detection of SARS-CoV-2 nucleic acid using the Abbott m2000rt/sp automated system. This test targets two genes: RdRp and N (Figure 1). An Internal Control (IC), an RNA sequence not associated with the SARS-CoV-2 target, had been introduced into each sample at the start of the extraction. It was co-amplified with the target RNA by RT-PCR, thus validating the procedure for each sample. A sample volume of 0.5 ml was extracted and an eluate volume of 40 µl was used for RT-PCR according to the protocol provided by the manufacturer (https://www.abbott.com).

2.3.2. Allplex™ 2019-nCoV

1) RNA Extraction and Nanodrop Lite Assay

Viral RNA extraction was performed using the Zymo RNA kit™ Research, USA, https://www.zymoresearch.com/ extraction kit according to the manufacturer’s protocol. The viral RNA thus extracted is analyzed using the NanoDrop™ in order to know its concentration and purity before being stored at −20˚C or amplified directly using the Allplex™ 2019-nCoV Assay.

2) Allplex™ 2019-nCoV Molecular Test on Bio-Rad CFX96™ Automaton

The Allplex™ 2019-nCoV test is a real-time RT-PCR In Vitro Diagnostic (IVD) test developed for the qualitative detection of SARS-CoV-2 viral nucleic acids in human specimens such as nasopharyngeal swab, oropharyngeal swab, anterior nasal swabs, sputum, in individuals with suspected COVID-19. 8 µl of purified nucleic acid (eluate) has been back-transcribed using 5X Real-time One-step

Buffer/Real-time One-step Enzyme into cDNA and then amplified in the CFX96™ real-time PCR system. This test includes an internal control which is composed of the phage MS2 genome. Three genes are targeted in this test: the genes, N of the nucleocapsid, E of the envelope and RdRp of the RNA-dependent RNA polymerase of the virus (Figure 1).

2.4. Statistical Tests

Data were collected using Microsoft Office Excel 2013 software. Agreement between tests was assessed using statistical kappa tests with 95% Confidence Intervals (95% CI) and an analysis of the correlation was used for comparative analysis of Ct values. These statistical analyzes were performed using IBM SPSS Statistics software version 20.0.0 (IBM Corp., Armonk, NY, USA) and Statview^®4.5 was used to generate the figures of the correlation.

3. Results

3.1. Clinical Overview

This population consisted of 99 women (52.1%) and 91 men (47.9%) of all ages. The median age was 40 years old. Of the 190 samples analyzed, 90 were positive on Abbott RealTime SARS-CoV-2, i.e. a positivity rate of 47.37%. These samples were tested again with the Allplex™ 2019-nCoV kit in order to compare the two detection methods and assess their level of concordance. On the Allplex multiplex PCR technique, 74 were positive, i.e. a positivity rate of 39% (Table 1).

Comparison of Abbott RealTime SARS-CoV-2 to Allplex™ 2019-nCoV for the detection of SARS-CoV-2, the cause of COVID-19.

In our study, on 190 samples, the two methods showed an overall agreement of 74.74%, a positive agreement of 64.4% and a negative agreement of 84% (Table 2).

A more in-depth analysis according to viral load or viremia level showed that the positive concordance was 93.3% between the two tests for samples with a

Table 1

Comparison between Abbott and Allplex

		Abbott SARS-CoV-2 RealTime			Kappa (K, 95% IC)
		Negatives	Positives	Total
Allplex™ 2019-nCoV	Negatives	84	32	116	0.50 (0.438 - 0.562)
	Positives	16	58	74
Total		100	90	190

Table 2

General and specific agreement between the two tests

Overall Agreement	74.73%
Negative Agreement	84%
Positive Agreement	64.4%

high viral load (ct < 10) but decreased with medium and low viral loads: 56.6% and 43.3% respectively for classes of 10 ≥ ct < 20, ct ≥ 20 (Table 3).

The Cohen’s Kappa coefficient of the two tests is 50% (95% CI: 43.8% - 56.2%), which shows that the concordance is average between the two techniques (Table 1).

We also looked for a correlation between the Cts obtained by the two diagnostic methods using the Pearson correlation test, a positive correlation was found (r² = 0.63, p < 0.0001) (Figure 2). This result indicates that the Cts of the two tests are correlated and vary together in the same direction (positive direction).

We also calculated the correlation for each Ct class. Thus, correlation coefficients r² = 0.14; 0.48 and 0.40 were found for the classes of Ct < 10; 10 ≤ Ct < 20 and Ct ≥ 20 in Abbott, respectively (Figures 3(a)-(c)). We would also like to note that a Ct = 40 value was assigned to the 32 samples that were initially positive with Abbott but came out negative on Allplex in order to calculate the correlation. This value is the detection threshold beyond which the sample is considered negative by the Allplex test.

Table 3

Positive and negative agreements between Abbott and Allplex™ 2019-nCoV tests based on viremia (ct)

Ct (Abbott m2000sp/rtSARS-CoV-2 RealTime)		Allplex™ 2019-nCoV		Total	Agreement
		Negative	Positive
CT < 10	Strong Positive	2	28	30	93.3%
10 ≤ CT < 20	Positive	13	17	30	56.6%
CT ≥ 20	Weak Positive	17	13	30	43.3%
Negative	Negative	84	16	100	84%

3.2. Analysis of Discordant Results

Results with the Allplex™ 2019-nCoV test are considered positive if at least one of the three target genes are detected. In total, we noted 48 discrepancies between the two techniques. Of these discrepancies, 16 were Allplex™ 2019-nCoV positive but Abbott negative and 32 Allplex™ negative but Abbott positive. A comparison according to the classes of Ct revealed that, the more the value of the Ct increased, the more the number of discrepancies increased between the two methods (Table 3). It is important to note that the two techniques studied differ by their target genes (RdRp and N for Abbott and RdRp, N and E for Allplex). The detection of this third E gene suggests the presence of a Sarbecovirus which is a subgenus of Betacoronavirus related to SARS-CoV-2 giving a positive result with this method. This is the reason why some negative samples with Abbott came out positive (detection of the E gene only) with Allplex.

4. Discussion

Since its emergence in December 2019, SARS-CoV-2 has rapidly spread around the world causing a global Coronavirus Disease 2019 (COVID-19) pandemic. It has marked the world in various ways and the response to the demand for testing has been to develop several tests for the direct diagnosis of COVID-19. Therefore, highly reliable laboratory diagnostic tests for COVID-19 are essential for case identification, patient management and contact tracing. This is how many PCR kits have been developed since the emergency authorization for the use of the SARS-CoV-2 diagnostic test. Recently, some studies have focused on comparing different SARS-CoV-2 detection kits.

Some studies comparing the Abbott RealTime SARS-CoV-2 test and other tests have been conducted and showed good agreement between them [12] [13] [14]. Others were interested in comparing the Allplex™ 2019-nCoV test and other tests [15] [16] [17], but also to their analytical performance [18] [19]. These studies showed that the Allplex test could be used in the diagnosis of COVID-19 [16], but also highlighted the genomic variability of SARS-CoV-2 that could affect these diagnostic tests [15]. However, to date, there is still no comparative analysis between these two tests.

Our results suggest that the two PCR methods present quite comparable performances (k = 50%). The Pearson correlation coefficient test showed a coefficient of r² = 0.63, thus showing a positive correlation between the two diagnostic methods. The results of the correlation are illustrated in Figure 2 showing that the values of the viral load of the two tests evolve in the same direction. However, when we look at the correlations for each Ct class, we notice that there is not a good correlation for Ct < 10 (r² = 0.14) (Figure 3(a)). Low Ct’s on Abbott gave higher Ct’s on Allplex. Both assays show differences in targets and probes. Better performance of the Abbott probes in detecting the SARS-Cov-2 genome could explain the lack of correlation between the Cts of the two assays. The Abbott kit would have better sensitivity. However, even though the two tests do not correlate well for this class of Ct, we observe that the concordance is still very high.

However, our study showed results varies based on the assay used between the two detection methods for SARS-CoV-2. First, the difference in target genes between Abbott (RdRP and N) and Allplex (RdRP, N and E). Thus, 16 samples tested negative on Abbott but emerged positive on Allplex by detection of the third E gene with Ct values > 36.00. This could suggest that these samples had low viral loads, so they were not detected by the Abbott platform. In addition, the discordant results are, for the most part, observed for samples with high Ct values, i.e. patients with low viremia and which explains why certain genes were not amplified with Allplex™. 2019-nCoV. A difference in amplification volumes in favor of Abbott (8 µl for Allplex against 40 µl for Abbott) could partially explain the fact that, for low viral concentrations, Allplex cannot detect SARS-CoV-2 genes because the volume would be insufficient. Work has also reported that this test may not consider genome variability given that it was designed at the start of the pandemic when little information on the SARS-CoV-2 genome was available [19]. Hence, SARS-CoV-2 variants might not be detected. Among the 48 discordant results, 67% (32) of the samples were positive for Abbott but negative with Allplex (median Cycle Count, CN: 22.41; RQ: 16.89 - 29.56) and 33% (16) were negative with Abbott but positive with Allplex (median CT: 38.04 [37.84 - 38.99]). Additionally, for samples with only one gene detected, the Ct was greater than 35 suggesting a low viral load. These results are in agreement with a study done in France and could explain why the other target genes were not detected [19].

Limitations of this comparative analysis include a freeze-thaw process that could impact samples by disrupting the detection of the SARS-CoV-2 viral genome. Limitations of this study also include that we did not have detailed clinical information about the patients, so the data could not be correlated with symptoms or disease course. Further investigation of patients with detailed time and symptom data and samples collected consecutively from different sites could substantiate our results.

The Allplex 2019-nCoV test is a method that can be used in symptomatic patients who may have high SARS-CoV-2 viremia.

5. Conclusion

Overall, the Allplex™ 2019-nCoV Assay exhibited significantly different performance characteristics than the Abbott RealTime SARS-CoV-2 Assay (amplification time, target genes, etc.). However, our results showed moderate concordance with a Kappa coefficient showing acceptable agreement between RT-PCR, Allplex™ 2019-nCoV and Abbott RealTime SARS-CoV-2 Assays in the diagnosis of COVID-19. With poor concordance for high Cts, the Allplex™ 2019-nCoV RT-PCR test would be more appropriate for the acute and symptomatic phase of the disease. Therefore, it would be very important to check the performance characteristics of commercially available kits before using them in routine laboratory diagnosis.

Acknowledgements

The authors are grateful to the Department of Defense HIV/AIDS Prevention Program (DHAPP) and Alliance Nationale des Communautés pour la Santé (ANCS) for their support of molecular biology equipment. We would like to extend our acknowledgement to Remi Charlebois for his revision of the manuscript.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

1. Gorbalenya, A.E., Baker, S.C., Baric, R.S., De Groot, R.J., Drosten, C., Gulyaeva, A.A., et al. (2020) Severe Acute Respiratory Syndrome-Related Coronavirus: The Species and Its Viruses—A Statement of the Coronavirus Study Group. Nature Microbiology, 5, 536-544. https://doi.org/10.1101/2020.02.07.937862

2. Van Kasteren, P.B., Van Der Veer, B., Van Den Brink, S., Wijsman, L., De Jonge, J., Van Den Brandt, A., et al. (2020) Comparison of Seven Commercial RT-PCR Diagnostic Kits for COVID-19. Journal of Clinical Virology, 128, Article ID: 104412. https://doi.org/10.1016/j.jcv.2020.104412

3. Zhu, N., Zhang, D., Wang, W., Li, X., Yang, B., Song, J., et al. (2020) A Novel Coronavirus from Patients with Pneumonia in China, 2019. The New England Journal of Medicine, 382, 727-733. https://doi.org/10.1056/NEJMoa2001017

4. Tao, K., Tzou, P.L., Nouhin, J., Gupta, R.K., De Oliveira, T., Kosakovski, S., et al. (2021) The Biological and Clinical Significance of Emerging SARS-CoV-2 Variants. Nature Review Genetics, 22, 757-773. https://doi.org/10.1038/s41576-021-00408-x

5. He, X., Hong, W., Pan, X., Lu, G. and Wei, X. (2021) SARS-CoV-2 Omicron Variant: Characteristics and Prevention. MedComm, 2, 838-845. https://doi.org/10.1002/mco2.110

6. Araf, Y., Akter, F., Tang, Y-D., Fatemi, R., Parvez, S.A., Zheng, C., et al. (2022) Omicron Variant of SARS-CoV-2: Genomics, Transmissibility, and Responses to Current COVID-19 Vaccines. Journal of Medical Virology, 94, 1825-1832. https://doi.org/10.1002/jmv.27588

7. Xu, J., Zhao, S., Teng, T., Abdalla, A.E., Zhu, W., Xie, L., et al. (2020) Systematic Comparison of Two Animal-to-Human Transmitted Human Coronaviruses: SARS-CoV-2 and SARS-CoV. Viruses, 12, Article No. 244. https://doi.org/10.3390/v12020244

8. Shen, M., Zhou, Y., Ye, J., Al-Maskri, A.A.A., Kang, Y., Zeng, S., et al. (2020) Recent Advances and Perspectives of Nucleic acid Detection for Coronavirus. Journal of Pharmaceutical Analysis, 10, 97-101. https://doi.org/10.1016/j.jpha.2020.02.010

9. Jamai Amir, I., Lebar, Z., Yahyaoui, G. and Mahmoud, M. (2020) Covid-19: Virologie, épidémiologie et Diagnostic Biologique. Option/Bio, 31, 15-20. https://doi.org/10.1016/S0992-5945(20)30178-1

10. Sheridan, C. (2020) Coronavirus and the Race to Distribute Reliable Diagnostics. Nature Biotechnology, 38, 382-384. https://doi.org/10.1038/d41587-020-00002-2

11. Smithgall, M.C., Scherberkova, I., Whittier, S. and Green, D.A. (2020) Comparison of Cepheid Xpert Xpress and Abbott ID Now to Roche Cobas for the Rapid Detection of SARS-CoV-2. Journal of Clinical Virology, 128, Article ID: 104428. https://doi.org/10.1016/j.jcv.2020.104428

12. Harrington, A., Cox, B., Snowdon, J., Bakst, J., Ley, E., Grajales, P., et al. (2020) Comparison of Abbott ID Now and Abbott m2000 Methods for the Detection of SARS-CoV-2 from Nasopharyngeal and Nasal Swabs from Symptomatic Patients. Journal of Clinical Microbiology, 58, e00798-20. https://doi.org/10.1128/JCM.00798-20

13. Moore, N.M., Li, H., Schejbal, D., Lindsley, J. and Hayden, M.K. (2020) Comparison of Two Commercial Molecular Tests and a Laboratory-Developed Modification of the CDC 2019-nCoV Reverse Transcriptase PCR Assay for the Detection of SARS-CoV-2. Journal of Clinical Microbiology, 58, e00938-20. https://doi.org/10.1128/JCM.00938-20

14. Hirschhorn, J.W., Kegl, A., Dickerson, T., Glen, W.B., Xu, G., Alden, J., et al. (2020) Verification and Validation of SARS-CoV-2 Assay Performance on the Abbott m2000 and Alinity m Systems. Journal of Clinical Microbiology, 59, e03119-20. https://doi.org/10.1128/JCM.03119-20

15. Hernández, C., Florez, C., Castaneda, S., Ballesteros, N., Martínez, D., Castillo, A., et al. (2021) Evaluation of the Diagnostic Performance of Nine Commercial RT-PCR Kits for the Detection of SARS-CoV-2 in Colombia. Journal of Medical Virology, 93, 5618-5622. https://doi.org/10.1002/jmv.27051

16. Hur, K.-H., Park, K., Lim, Y., Jeong, Y.S., Sung, H. and Kim, M.-N. (2020) Evaluation of Four Commercial Kits for SARS-CoV-2 Real-Time Reverse-Transcription Polymerase Chain Reaction Approved by Emergency-Use-Authorization in Korea. Frontiers in Medicine, 7, Article 521. https://doi.org/10.3389/fmed.2020.00521

17. Liotti, F.M., Menchinelli, G., Marchetti, S., Morandotti, G.A., Sanguinetti, M., Posteraro, B., et al. (2021) Evaluation of Three Commercial Assays for SARS-CoV-2 Molecular Detection in Upper Respiratory Tract Samples. European Journal of Clinical Microbiology & Infectious Diseases, 40, 269-277. https://doi.org/10.1007/s10096-020-04025-0

18. Degli-Angeli, E., Dragavon, J., Huang, M.L., Lucic, D., Cloherty, G., Jerome, K.R., et al. (2020) Validation and Verification of the Abbott RealTime SARS-CoV-2 Assay Analytical and Clinical Performance. Journal of Clinical Virology, 129, Article ID: 104474. https://doi.org/10.1016/j.jcv.2020.104474

19. Farfour, E., Lesprit, P., Visseaux, B., Pascreau, T., Jolly, E., Houhou, N., et al. (2020) The Allplex 2019-nCoV (Seegene) Assay: Which Performances Are for SARS-CoV-2 Infection Diagnosis? European Journal of Clinical Microbiology & Infectious Diseases, 39, 1997-2000. https://doi.org/10.1007/s10096-020-03930-8