Introduction

The COVID-19 pandemic was announced by the WHO on 11 March 2020, only a few months after its etiological factor, the novel coronavirus SARS-CoV-2, had been identified. There is currently no specific treatment for this potentially fatal disease, but efforts around the world are being made towards combining antiviral and supportive therapy [1]. The accurate and immediate diagnosis is crucial for hampering the spreading of the disease and enabling its early treatment. The gold standard for laboratory diagnosis of COVID-19 is detection of viral RNA in the respiratory tract by means of a nucleic acid amplification test (NAAT), such as real-time RT-PCR [2]. However, the results of this test may be affected by improper sampling and mishandling of the specimen, leading to false negative results. Also, the sensitivity of the molecular methods is the highest in the first 7 days from the symptoms onset and gradually decreases afterwards [3, 4]. Simultaneously, the sensitivity of anti-SARS-CoV-2 antibody detection between 8 and 13 days from the symptoms onset is reported to be 90% [3]. Hence, it has been proposed to employ serological testing as an auxiliary means of COVID-19 diagnosis [5]. Apart from diagnosis, serological testing is used in the epidemiological investigations – determining seroprevalence, previous exposure and contact tracing. In the future, antibody detection and quantification may be useful during convalescent plasma treatment, vaccine evaluation and possibly identifying individuals who are immune to the disease [6, 7].

Currently no harmonized global guidelines or recommendations on the use of serological tests in COVID-19 are available. A thorough validation of the commercial methods on clinical samples is still lacking. Simultaneously, there are multiple assays available on the market and the demand from both clinicians and patients is recognized [8]. Generally, automatized immunoassays seem to be better validated, but in many locations they may not be available and hence the rapid immunochromatography assays (lateral flow immunoassays – LFIAs), suited to point-of-care testing, may also be useful. As shown by several instances, the performance of LFIA tends to be doubtful [9, 10]. However, this may differ between various producers and should be further elucidated.

The aim of this study is therefore to compare the results obtained with different LFIAs with the results of the chemiluminescent immunoassay (CLIA) method (MAGLUMI) and automated enzyme-linked immunosorbent assay (ELISA) (Euroimmun). This may shed some light on the clinical utility of the lateral flow assays and facilitate the selection of serological assays.

Material and methods

Results

Out of the control serum samples none tested positive for anti-SARS-CoV-2 IgM or IgG. Two patients tested positive for anti-SARS-CoV-2 IgA (Euroimmun). Those patients were asked for an additional nasopharyngeal swab which was tested with real-time RT-PCR and came out negative. Further, the repeated testing after 2 weeks showed no seroconversion – the IgA remained positive with a similar antibody ratio and IgGs were still absent. This could imply that IgA results were falsely positive.

The rest of the patients from group A tested negative in all immunoassays performed (Table III).

Case subgroup B1 consisted of 19 patients who tested negative for both IgM and IgG in CLIA. For these patients a few discordant results, mostly in Euroimmun ELISA, were observed (Table III). High concordance between the comparative method and LFIAs was observed. Three of the tested LFIAs had 100% concordance with the comparative CLIA method in IgG testing and four in IgM testing. In 1 case the sample tested positive only in AllTech LFIA both in IgG and IgM. In this subgroup, 1 case should be considered inconclusive in IgG testing, having positive results in ELISA and two of the tested LFIAs and negative results in CLIA and the other three LFIAs.

Surprisingly, the tested ELISA (Euroimmun) had higher discordance with CLIA than the LFIAs. In discordance with CLIA, ELISA produced positive results in IgG in 3 cases and in 5 cases in IgA testing.

Case subgroup B2 consisted of 7 patients who in the comparative CLIA method tested positive in IgG but negative in IgM. One patient was positive only in IgG MAGLUMI, suggesting a false positive MAGLUMI result. The remaining 6 cases of IgG positive results were confirmed by most of the other tests (Table III).

In this subgroup, the highest concordance with the CLIA method was observed for LFIAs produced by SafeCare and AllTech (6 out of 7 concordant results for IgG and 7 out of 7 for IgM), whereas the RealyTech LFIA produced contrary results in 4 out of 7 cases in both antibody classes. Euroimmun ELISA was concordant with CLIA in the IgG test (in 6 cases) but completely discordant in IgM (in 6 out of 7cases).

Subgroup B3 consisted of 6 patients for whom positive results for both IgM and IgG were obtained. In this subgroup 100% concordance for IgG results was observed between the methods. Full agreement of IgM results was also observed between CLIA, ELISA and one of the tested LFIAs (RealyTech). The other LFIAs tested produced negative results in 2, 3, or even 4 cases (RealyTech), as shown in Table III.

The summarized agreements between the results of the tested methods in the whole group of 52 subjects are shown in Table IV. The highest agreement with the MAGLUMI method was found for Watmind cassettes (98%) in IgG and for SafeCare cassettes in IgM. The lowest agreement was observed for RealyTech and Euroimmun for IgG (92.3%), and for Euroimmun for IgM (75%). The low agreement for Euroimmun ELISA is due to the low concordance with MAGLUMI negative results – suspected false positive Euroimmun IgA results.

Discussion

There is a scarcity of published data comparing the serological assays for anti-SARS-CoV-2 antibodies. There is one paper analyzing the concordance between MAGLUMI CLIA IgG and IgM to Euroimmun ELISA IgG and IgA [11]. Lippi et al. elegantly compared the results obtained with those methods in patients with suspected COVID-19. Their report pointed to the concordance of MAGLUMI and Euroimmun, calculated to be 88% for IgG and 90% for IgM/IgA. Therefore, the authors attested to the substantial degree of concordance between those methods at their respective cut-offs. Our results confirmed the above with respect to IgG. However, the concordance of MAGLUMI IgM and Euroimmun IgA reported in the current study is much lower (75%).

The weaker correlation of the early phase results between CLIA and ELISA, in comparison to the IgG results, may stem from the difference in the exact analyte tested. It has been reported that the anti-SARS-CoV-2 IgA and IgM seroconversion is similar, with both antibody classes detectable at 3–6 days from the symptom onset [12, 13]. Therefore, IgA antibodies testing instead of IgM in SARS-CoV-2 has been postulated as a means of limiting false positive results caused by the high cross-reactivity of IgM antibodies [14]. However, our results indicate the contrary – there seem to be more false positive results in the IgA assay. The only positive results observed in this study in the control group (healthy volunteers) were in fact IgA in Euroimmun ELISA. These patients were further tested with real-time RT-PCR (nasopharyngeal swab) and the results were negative. Further, the repeated testing after 2 weeks showed no seroconversion – the IgA remained positive with a similar antibody ratio and IgGs were still absent. Also, in the case of subgroup B1, a few patients with IgM and IgG negative in CLIA tested positive solely in IgA ELISA. Hence, isolated positive IgAs in those patients were probably false positive results. It must however be noted that Lassaunière et al. reported high specificity of Euroimmun ELISA – 93% and 96% for Euroimmun IgA and IgG ELISAs, respectively [9]. Nonetheless, those authors also outlined some cross-reactivity of both IgA and IgG Euroimmun ELISAs, mainly in samples containing antibodies to more than one respiratory virus, but also associated with the presence of adenovirus, dengue virus and human coronavirus HKU1 antibodies [9]. Therefore, any isolated anti-SARS-CoV-2 IgA result should be verified by real-time RT-PCR testing, and that may put additional financial strain on the health care providers. A recent paper concerning the validation of the Euroimmun ELISA assays directly states that the IgG assay displays nearly optimal diagnostic accuracy and no obvious gains result from IgA serology [15].

Apart from the suspicion of cross-reactivity of the Euroimmun IgA assay, the high number of IgA positive results in group B2, with positive IgG and negative IgM in MAGLUMI, could also indicate longer persistence of IgA in comparison to IgM. The kinetics of IgM and IgA antibody production in SARS-CoV-2 are not fully elucidated yet and could also be influenced by an individual’s immune response. It would be of interest to check whether the disappearance of IgM or IgA antibodies accompanies the disappearance of the virus and indicates the contagious status of a patient. Our data also revealed a few positive IgG Euroimmun ELISA results that were not confirmed in other methods. The study design does not allow us to draw a definite conclusion whether Euroimmun ELISA is less specific and therefore produces false positive results or it is more sensitive and the results obtained with the other methods tested are false negative. To resolve that, real-time RT-PCR data from these patients and information on antibody seroconversion over time would be needed.

Our study proves that the results obtained with the tested LFIAs are comparable with the results of the CLIA test. However, the agreement is reduced when positive samples are analyzed separately, and the results obtained for IgM positive patients are more discordant than those for IgG positive patients. A more thorough assessment of LFIA IgM performance could be made if the number of IgM positive samples was higher. Also, our cohort lacked patients with isolated IgM positive results, and such cases would shed more light on the utility of the tested methods with respect to IgM assessment.

The performance of different LFIAs varied. Among the tested immunochromatography tests, Watmind seems to be the most accurate for IgG detection, showing 100% concordance for negative CLIA results and 92% concordance for CLIA positive results. The 1 case in which there was discordance was probably false positive in CLIA. However, Watmind LFIA does not prove accurate when it comes to IgM detection. In this case the concordance with CLIA was 98% for negative results but only 50% for positive results.

A similar issue was observed for SelfCare and AllTech LFIAs. Whereas IgG detection was comparable to that of CLIA, the concordance of IgM results was substantially worse, decreasing to 33% for IgM positive by AllTech.

The highest overall concordance between CLIA and LFIA in IgM results was found for the SelfCare test (96.1%), but the other kits tested also showed agreement of more than 90%. Again, the concordance of IgM positive results between CLIA and SelfCare was substantially lower (67%).

The worst performing LFIAs in our study, RealyTech for IgG and AllTest for IgM, still had overall concordance with CLIA of 92.3 and 90.3%, respectively. It must be noted, though, that these kits failed to discern IgG positive patients in 4 out of 13 patients (RealyTech) and IgM positive patients in 4 out of 6 cases (AllTest). False negative results obtained with the tested LFIA devices are of high concern, as they may lead to COVID-19 cases being missed and the possible spread of the disease.

Other reports on the performance of LFIA kits published thus far are equally cautious about the wide use of these tests [9, 10]. The United Kingdom’s National COVID Scientific Advisory Panel has evaluated the performance of in-house ELISA and 9 undisclosed LFIA devices using COVID-19 confirmed cases samples and pre-pandemic control samples. The authors reported high sensitivity of IgG ELISA after 10 days from the symptom onset. The performance of the tested LFIAs was not sufficient to guide their recommendation for individual patient applications. The sensitivity of the tested LFIAs observed in the above study, as compared to ELISA positive, ranged from 65% to 85%, and specificity from 93% to 100%. The authors reported no evidence of differences in sensitivity or specificity between the tested LFIAs [9]. A Danish study assessed three commercial ELISAs and 6 LFIAs. The samples tested were obtained from confirmed COVID-19 cases and pre-pandemic control samples. Although the authors provided some rankings for the accuracy of LFIA devices, they outline that the performance of the different devices varies notably. The observed concordance of the tested LFIAs with Euroimmun ELISA was between 73 and 90%. Thus, the authors warn that results obtained with different methods for a given patient may not be unanimous and suggest that ELISA should be used to confirm LFIA results [9].

Our study is not free from limitations. The small size of the cohort does not allow us to make any final recommendations as to better/worse performing assays. Also, the results reported here are only a comparison of the methods. Since no clinical data or RT-PCR results are available, it is not possible to verify clinical sensitivity and specificity of the tested methods. The suspected false positive results obtained with Euroimmun IgA ELISA also need further investigation on a larger group of patients, preferably with SARS-CoV-2 status verified by real-time RT-PCR.

In conclusion, the results obtained in this study provide evidence for high overall concordance between the comparative CLIA method and different LFIAs (93.8%) not inferior to the other automated method – ELISA. This could justify the use of LFIAs in some settings, where automated assays are not available, provided that some limitation with regards to false negative results is taken into consideration. Also, the agreement between CLIA and different LFIAs varied significantly between the kits, but the small cohort tested does not allow us to unequivocally recommend any of the producers.

The concordance of the results obtained with different methods seems to be dependent on the population tested. In healthy volunteers, where the results are expected to be negative, the concordance was the highest (98.6%). Also, in patients who tested positive in both classes (IgM and IgG) in CLIA, the agreement between the methods was higher (91.6%) than in the subgroup with only IgG positive (71.4%).

The concordance was also higher in IgG testing, in comparison to IgM testing, which might be an outcome of cross-reactivity of antibodies of this class.