|
 
 |
| ORIGINAL ARTICLE |
|
| Year : 2021 | Volume
: 5
| Issue : 2 | Page : 56-60 |
|
Performance evaluation of seven commercial real-time polymerase chain reaction assays for SARS-COV-2 detection
Mouna Mohamed ElJilani, Mohamed Abdusalam, Adel Abdalla, Tarek Dalyoum, Salah Ahmad, Inas Alhudiri, Adam Elzagheid
Department of Genetic Engineering, Biotechnology Research Center, Tripoli, Libya
| Date of Submission | 17-Jan-2021 |
| Date of Acceptance | 06-Jun-2021 |
| Date of Web Publication | 23-Jul-2021 |
Correspondence Address:
Prof. Adam Elzagheid
Department of Genetic Engineering, Biotechnology Research Center, Alfornaj, Tripoli
Libya
 Source of Support: None, Conflict of Interest: None  | 1 |
DOI: 10.4103/ljms.ljms_5_21
Background/Aim: Since the emergence of the severe acute respiratory syndrome-coronavirus-2 (SARS-COV-2) pandemic in Wuhan, China, several efforts are being focused on the development of fast and reliable diagnostic molecular tests. Real-time polymerase chain reaction (RT-PCR) based assay on respiratory specimens was recommended by the World Health Organization as the gold standard for early diagnosis of infection spread. Our study aimed to compare the analytical performance of seven commercially available RT-PCR assays. Materials and Methods: A total of 33 nasopharyngeal swabs were analyzed by: Da An, PerkinElmer, Norgen, Prestige, PhoenixDX, Bio-Speedy, and Xpert Xpress RT-PCR assays. Sensitivity and detection rates of SARS-COV-2 target genes were analyzed. Results: Da An and Xpert Xpress assays showed the highest detection rate and percentage for SARS-COV-2 target genes; (16/33) 48.5%, followed by PerkinElmer and Norgen kits (13/33) 39.4%, (10/33) 30.3%, respectively. However, Prestige, PhoenixDx, and Bio-speedy displayed the same performance (6/33) 18.2%. Conclusion: The analyzed assays showed inconsistent analytical performance. Overall, findings reported in our study may not be applicable to other RT-PCR assays or thermocyclers; analytical sensitivities and positive-negative cut-off values should be locally validated.
Keywords: Assays, detection, real-time polymerase chain reaction, SARS-COV-2
How to cite this article:
ElJilani MM, Abdusalam M, Abdalla A, Dalyoum T, Ahmad S, Alhudiri I, Elzagheid A. Performance evaluation of seven commercial real-time polymerase chain reaction assays for SARS-COV-2 detection. Libyan J Med Sci 2021;5:56-60 |
How to cite this URL:
ElJilani MM, Abdusalam M, Abdalla A, Dalyoum T, Ahmad S, Alhudiri I, Elzagheid A. Performance evaluation of seven commercial real-time polymerase chain reaction assays for SARS-COV-2 detection. Libyan J Med Sci [serial online] 2021 [cited 2023 Mar 31];5:56-60. Available from: https://www.ljmsonline.com/text.asp?2021/5/2/56/322208 |
| Introduction | |  |
Coronavirus disease 2019 (COVID-19) is a highly infectious disease caused by severe acute respiratory syndrome-coronavirus-2 (SARS-CoV-2). The disease was initially reported in December 2019 in Wuhan, China, and rapidly spread worldwide.[1] COVID-19 patients are the major source of infection transmission through their respiratory droplets, in addition to direct or indirect contact. The clinical manifestations of the disease (COVID-19) range widely from mild upper respiratory tract infection to more severe and critical cases.[2] Asymptomatic and presymptomatic infected persons are potential sources of infection as well.[3],[4],[5]
Due to the rapid spread of the virus, several efforts are being focused on the development of fast and reliable diagnostic molecular tests. Real-time polymerase chain reaction (RT-PCR) based assay on respiratory specimens was recommended by the World Health Organization (WHO) as the gold standard for early diagnosis and containment of infection spread.[6] The laboratory-developed tests relied primarily on the detection of SARS-CoV-2 nucleocapsid (N) genes, envelope (E), and RNA-dependent RNA polymerase as target genes, however, open reading frame 1 a/b (ORF1a/b), and the gene encoding spike (S) protein were included afterward.[7],[8]
A large number of RT-PCR kits and molecular platforms are being manufactured and brought to the market with limited validation on clinical samples. Some suspected patients manifested typical clinical pneumonia symptoms, but they were negative on RT-PCR testing.[9] This highlights the question of whether there are differences in the RT-PCR detection assays and false negatives.[10]
Understanding the analytical performance and evaluation of RT-PCR assays is crucial for valid interpretation of the results, and accordingly for making the right decisions regarding the clinical care and control measures. The aim of this study was to compare the analytical performance of seven commercial RT-PCR assays.
| Materials and Methods | | |
Clinical specimens
A total of 33 nasopharyngeal swabs in viral transport media were collected in November 2020 from suspected individuals and delivered to the laboratory of the Biotechnology Research Center (BTRC) in Tripoli. Written informed consent was not required for these types of studies because RNA remnants from nasopharyngeal swabs specimens were ordinarily kept for the purpose of routine laboratory testing after completion of standard of care testing.
Commercial real-time polymerase chain reaction assays studied
Seven commercial RT-PCR assays selected for this study: Da An® detection kit for 2019-Novel Coronavirus (2019-nCOV) RNA (PCR-Fluorescence Probing), V4, Da An® Gene Co., Ltd. of SunYat-sen University,[11] PerkinElmer® SARS-CoV-2 Real-time RT-PCR Assay, V1.0 (PerkinElmer, Inc.),[12] Norgen® 2019-nCoV TaqMan RT-PCR Kit (Norgen Biotek Corp.),[13] Prestige® COVID-19 Lyophilized Nucleic Acid Kit, V3, (Prestige Diagnostics UK Ltd, Co.), PhoenixDx® SARA-Cov-2 Multiplex in vitro diagnostic, V1.0, (Procomcure Biotech GmbH),[14] Bio-Speedy® Direct RT-quantitative PCR SARS-CoV-2, V201031-1 (Bioeksen R and D Technologies),[15] GeneXpert Xpert Xpress SARS-CoV-2 (Cepheid®) and includes both a sample processing control and a probe check control.[16] All assays are based on TaqMan technology. The target genes of each test and reaction conditions are summarized in [Table 1]. | Table 1: Characteristics of seven severe acute respiratory syndrome coronavirus 2 real-time polymerase chain reaction assays
Click here to view |
Nucleic acid extraction and real-time polymerase chain reaction
RNA was extracted from the specimens using NuActor® Viral RNA automatic extractor, UGENECELL Inc., South Korea. All steps followed the manufacturer's recommended protocol. RT-PCR was performed on the Rotor-Gene® Q RT PCR-cycler, QIAGEN, UK. All reaction procedures were set according to the reagents' specifications of the manufacturers. During each run, both positive and negative controls were added to ensure that proper PCR responses were not subjected to carryover contamination. Consequently, the test results were interpreted as either negative or positive according to the manufacturers' cycle threshold (Ct) cut-off.
Amplification and detection ability
Each target gene was analyzed based on Ct values and amplification curves obtained during RT-PCR. Then, the detection rate (number of positive results/total number of samples) was used to evaluate the detection ability of the seven assays.
Statistical analysis
GraphPad Prism 5.0 (GraphPad Software, San Diego, California, USA) was used for statistical analysis.
| Results | | |
Clinical performance
To compare the accuracy of the seven assays, 33 clinical samples were examined. Assays sensitivity of each assay was measured according to the number and percentage of the positive samples. Both Da An and Xpert Xpress showed the highest detection rate (16/33) 48.5% detection percentage, followed by PerkinElmer and Norgen kits (13/33) 39.4%, (10/33) 30.3%, respectively. However, the other kits: Prestige, PhoenixDx, and Bio-speedy displayed the same performance (6/33) 18.2%, [Table 2]. | Table 2: Detection ability of the seven real-time polymerase chain reaction assays
Click here to view |
Detection ability
Our data showed that there was the variability of reagents, PCR target genes, and reaction conditions between the different commercial RT-PCR methods for the identification of SARS-CoV-2. Da An kit and Xpert Xpress showed the highest number of positive samples followed by PerkinElmer kit. However, PerkinElmer and Da An kits were able to detect sample No. 6 with the lowest Ct values (17.2 and 18.2) respectively. On the other hand, Xpert Xpress was able to detect sample No. 8 with the highest Ct Value (43.2). Prestige, PhoenixDx, and Bio-Speedy kits were able to detect only six positive samples with the lowest Ct value (22.3) with the Bio-Speedy kit [Table 3]. Positive and negative controls showed typical amplification with all evaluated tests. [Figure 1] and [Figure 2] show amplification curves for the seven assays. [Figure 3] shows the relationship between the Ct values and N viral target gene for Da An, PerkinElmer, Prestige, PhoenixDX, Bio-Speedy, and Xpert Xpress, N2 gene fragment for Norgen kit, and N and ORF1a/b together in one channel for PhoenixDX kit. | Figure 1: Quantitation data for (a)Da An (a), (b)PerkinElmer, (c)Bio-Speedy, (d)Norgen, (e)Prestige, and (f)PhenixDX real-time polymerase chain reaction detection kits by Rotor-Gene® Q real-time polymerase chain reaction-cycler
Click here to view |
 | Figure 2: Examples of quantitation data for Xpert Xpress severe acute respiratory syndrome-coronavirus-2 real-time polymerase chain reaction detection assay
Click here to view |
 | Figure 3: Cycle threshold values of the values of the real-time polymerase chain reaction assays
Click here to view |
| Discussion | | |
SARS-COV-2 spread rapidly, leading to an outbreak worldwide, and due to its severity, it was declared by WHO a pandemic on March 11, 2020.[17] The first officially confirmed case of SARS-COV-2 in Libya was declared by the National Center for Disease Control (NCDC) on March 24, 2020.[18] There was growing concern in the first 3 months of the pandemic in Libya as a result of the civil war at the time and the lack of resources. In May, the average daily testing was <50 tests intended for suspected cases that were reported to the NCDC of Libya. Moreover, the absence of local laboratories in other cities contributed to delaying the diagnosis and management, therefore, fears of an outbreak were present.[19] The competent authorities prompted to think about developing a comprehensive plan that includes training national medical personnel on RT-PCR technology throughout the country to increase the number of samples with taking into account providing reliable molecular diagnostic assays.
The worldwide outbreak was followed by the characterization of the SARS-CoV-2 whole viral genome within weeks of its discovery, which permitted the development of multiple RT-PCR methods.[20] Several papers have been published comparing the efficiency of variable RT-PCR assays.[20],[21],[22],[23],[24],[25],[26],[27] However, with the emergence of new detection methods, their performance has not yet been systematically evaluated.[24] The present study was conducted at the laboratory of BTRC in Tripoli and compared seven commercially available RT-PCR assays for the detection of SARS-CoV-2 in clinical samples. The positive identification rate for the tested assays varied from 6 to 16 out of 33 samples. Da An and Xpert Xpress showed the best performance (16/33), followed by PerkinElmer (13/33), then Norgen (10/33). The results of the other three kits (Prestige, PhoenixDX, and Bio-Speedy) were consistent (6/33). Xpert Xpress assay was different from the other methods since it was an automated instrument and integrated sample preparation, nucleic acid extraction, and amplification. In terms of viral target genes, all compared tests target two viral gene fragments except the Bio-Speedy kit which target only the N gene. Da An, PekinElmer, Prestige, and PhoenixDX target the ORF1a/b and N genes, while Norgen targets two loci of The N gene (N1 and N2) and Xpert Xpress targets the N2 and E genes, in addition, PhoenixDX was the only kit that target a combination of ORF1a/b and N in single amplification. Norgen kit required two separate wells per sample and has the lowest throughput. This study showed inconsistent analytical sensitivity, although, all RT-PCR assays performed satisfactorily regarding positive and negative controls. In RT-PCR, the Ct value is a semi-quantitative measurement of viral load and inversely proportional to the quantity of the target gene. In other words, a high Ct value indicates a low viral load, and vice versa[24] and the higher the sensitivity of the assay, the greater its ability to detect low viral load. The present study offered a side by side comparison using the same extraction methodology (NuActor® Viral RNA automatic extractor), and the same thermocycler (Rotor-Gene® Q RT PCR-cycler) excluding the automated system.
Importantly, the analytical sensitivity and efficiency of RT-PCR assays rely on many factors other than the analytical sensitivity of the molecular assay including the sensitivity of primer-probe set and target genes. These factors are, for instance, sample type, collection and transportation conditions, well-trained laboratory staff, and the accuracy of the thermocycler.[24] As a result, findings reported in our study may not be applicable to other RT-PCR assays or thermocyclers; analytical sensitivities and positive-negative cut-off values should be locally validated.
The study has some limitations. First, the clinical specimens were obtained from the same region and geographical underrepresentation may affect the results. Second, evaluation with a paired nucleic acid extraction and nucleic acid amplification were excluded from the study. We recommend conducting further studies covering a larger geographical area and with a larger sample size to obtain more accurate results.
| Conclusion | | |
As pandemic COVID-19 continues to spread, laboratory testing is indispensable, and it is suitable to state that RNA detection remains the method of choice for COVID-19 diagnosis as long as these tests have acceptable results. Furthermore, understanding the interpretation rules and the limits of the RT-PCR assays is fundamental and should be taken into account. Our findings could help guide the choice of molecular methods in diagnostic laboratories authorized to purchase the reagents during the pandemic.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, et al. A novel coronavirus from patients with pneumonia in China, 2019. N Engl J Med 2020;382:727-33.  |
| 2. | Hoehl S, Rabenau H, Berger A, Kortenbusch M, Cinatl J, Bojkova D, et al. Evidence of SARS-CoV-2 infection in returning travelers from Wuhan, China. N Engl J Med 2020;382:1278-80. |
| 3. | Bai Y, Yao L, Wei T, Tian F, Jin D, Chen L, et al. Research letter: Presumed asymptomatic transmission of COVID-19. JAMA 2020;323:1406-7. |
| 4. | Furukawa NW, Brooks JT, Sobel J. Evidence supporting transmission of severe acute respiratory syndrome coronavirus 2 while presymptomatic or asymptomatic. Emerg Infect Dis 2020;26:e201595. |
| 5. | |
| 6. | |
| 7. | Corman VM, Landt O, Kaiser M, Molenkamp R, Meijer A, Chu DK, et al. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro Surveill 2020;25:2000045. |
| 8. | |
| 9. | Zhang JJ, Dong X, Cao YY, Yuan YD, Yang YB, Yan YQ, et al. Clinical characteristics of 140 patients infected with SARS-CoV-2 in Wuhan, China. Allergy 2020;75:1730-41. |
| 10. | van Kasteren PB, van der Veer B, van den Brink S, Wijsman L, de Jonge J, van den Brandt A, et al. Comparison of seven commercial RT-PCR diagnostic kits for COVID-19. J Clin Virol 2020;128:104412. |
| 11. | |
| 12. | |
| 13. | |
| 14. | |
| 15. | |
| 16. | |
| 17. | |
| 18. | Elhadi M, Momen AA, Ali Senussi Abdulhadi OM. A COVID-19 case in Libya acquired in Saudi Arabia. Travel Med Infect Dis. 2020;37:101705. doi: 10.1016/j.tmaid.2020.101705. |
| 19. | Elhadi M, Msherghi A. COVID-19 and civil war in Libya: The current situation. Pathog Glob Health 2020;114:230-1. |
| 20. | Uhteg K, Jarrett J, Richards M, Howard C, Morehead E, Geahr M, et al. Comparing the analytical performance of three SARS-CoV-2 molecular diagnostic assays. J Clin Virol 2020;127:104384. doi: 10.1016/j.jcv.2020.104384. |
| 21. | Dust K, Hedley A, Nichol K, Stein D, Adam H, Karlowsky JA, et al. Comparison of commercial assays and laboratory developed tests for detection of SARS-CoV-2. J Virol Methods 2020;285:113970. doi: 10.1016/j.tmaid.2020.101705. |
| 22. | LeBlanc JJ, Gubbay JB, Li Y, Needle R, Arneson SR, Marcino D, et al. Real-time PCR-based SARS-CoV-2 detection in Canadian laboratories. J Clin Virol 2020;128:104433. |
| 23. | Lu Y, Li L, Ren S, Liu X, Zhang L, Li W, et al. Comparison of the diagnostic efficacy between two PCR test kits for SARS-CoV-2 nucleic acid detection. J Clin Lab Anal 2020;34:e23554. |
| 24. | Wangn M, Chen D, Wu W, Tang W, Kan L, Zong Z, et al. Analytical performance evaluation of five RT-PCR kits for severe acute respiratory syndrome coronavirus 2. J Clin Lab Anal 2020;35:e23643. |
| 25. | Farfour E, Lesprit P, Visseaux B, Pascreau T, Jolly E, Houhou N, et al. The Allplex 2019-nCoV (Seegene) assay: Which performances are for SARS-CoV-2 infection diagnosis? Eur J Clin Microbiol Infect Dis 2020;39:1997-2000. |
| 26. | Ong D, Claas E, Breijer S, Vaessen N. Comparison of the GeneFinder™ COVID-19 Plus RealAmp Kit on the sample-to-result Platform ELITe In Genius to the national reference method: An added value of N gene target detection. J Clin Virol 2020;132:104632. doi:10.1016/j.jcv.2020.104632. |
| 27. | Etievant S, Bal A, Escuret V, Brengel-Pesce K, Bouscambert M, Cheynet V, et al. Performance assessment of SARS-CoV-2 PCR assays developed by WHO Referral Laboratories. J Clin Med 2020;9:1871. |
[Figure 1], [Figure 2], [Figure 3]
[Table 1], [Table 2], [Table 3]
|
Search Pubmed for