Coronavirus detection tests: quality or reliability?

Much has been said and written about the quality of corona testing in recent days. Difficult terms such as sensitivity, specificity and reliability are used here, and percentages are mentioned, generally saying “the higher the number, the better the test”. I will try to explain to you that determining the sensitivity of a test is often difficult (if not sometimes impossible).

Sensitivity depends less on the quality of a test, but more on the condition of the person taking the test. By making an “intelligent” selection of the research group, we can produce all the numbers for a sensitivity between 0 and 100%. “Experts” may use it in the media as they see fit and / or in their own interest.

What do we want to measure with a SARS-CoV-2 test?

The most important question is: what is the test for and what does it measure? For laboratory research on COVID-19, there are generally two types of tests:

  1. Tests to detect the virus and determine if a person is currently infected with the virus. For this purpose, viral RNA or specific viral proteins (also called viral antigens) can be measured in a sample (saliva, cheek or throat mucus, feces, blood). If it is positive, a follow-up examination can possibly be carried out to definitively establish the diagnosis of COVID-19, for example a computed tomography of the lungs (this test in combination with the clinical picture is considered as a reference).
  2. Tests to demonstrate the immune status against the virus in people who have (consciously or not) suffered from the disease.
    Antibodies to the corona virus can be detected in this sample (almost always from blood). These are specific corona IgM and IgG antibodies. If it is positive, it is assumed that the person in question has been infected with the corona virus and has developed immunity. Whether this is enough to stop having an infection, this test cannot say for sure, but I will give some suggestions on this below. There is no follow-up research with any other method to definitively determine if someone has actually gone through the disease (so there is no gold standard).

COVID-19 disease test

As mentioned above, RNA or antigen tests are usually used for this. Detection of virus specific RNA (virus genetic material by PCR method) or virus specific antigen or protein (with Elisa sandwich method) are so specific that a positive test usually indicates corona infection. These tests have a very high degree of specificity.

However, the problem is sampling. For example, if you take cheek mucus that contains little or no viral particles (with minor complaints, the number of viral particles is often low and / or the virus travels fairly quickly from the mouth and throat to the lungs; with severe complaints, it stays detectable much longer in the cheek or throat mucus), then an RNA or antigen test is negative. But because the virus is in the lungs, it is a corona infection.

In this case, we are talking about a false negative test (False Negative in English with FN for short). You could also say that the sensitivity of the test is too low to detect COVID-19 in all cases; the sensitivity of the test is low (in fact the test is not too insensitive, but the choice of the sample material is not good). This is why corona virus RNA and antigen tests have limited sensitivity. By testing more often in highly suspected people, the chances of a positive result increase.

There is evidence that around 30% of people with COVID-19 undergo a false negative test with this (specific) RNA and antigen tests for a decrease in cheek mucus. For samples taken with a throat swab, this percentage of error appears to be negatively lower. The sensitivity (of the initial test) is around 70% in this case and even lower according to some studies.

Specificity and sensitivity: two concepts to deepen

The sensitivity of a test is the percentage of correct positive test results (in English True Positive for short TP) in a group of people with a proven disease (determined with certainty by a reference method). We therefore expect a positive test result. If a negative test result is nevertheless obtained in this group of people, this is considered as a false negative (in English False Negative abbreviated by FN)

The specificity of a test is the percentage of correct negative test results (in English True Negative; TN) in a group of people who are proven not to have the disease (established with certainty by the method of l ‘gold standard). We therefore also expect a negative test result. If a positive test result is nevertheless obtained, it is considered to be false positive (in English False Positive abbreviated FP).

Suppose now that a study was conducted in a group of patients admitted to the hospital in which COVID-19 disease was diagnosed with a CT scan and a clinical image (gold standard). When this diagnosis is made, there is a serious activity of the disease, otherwise they will not be hospitalized. In this group of 100 patients, 99 are shown (as an example) as having a positive test result (TP) and 1 as having a negative test result (FN). The sensitivity of this test is 99% (99 / (99 + 1)).

Now the same test is used in a group of people who come to the doctor with very mild complaints (no fever and only a dry cough and sore throat). Of the 100 people tested, 20 were found to have a positive result (for example). It is advisable to quarantine them at home due to the presence of the corona virus. After a few days, 10 people, who had previously tested negative, still arrive at the hospital’s emergency department and now appear to have corona infection. Now that they have gotten sicker, the RNA test seems to be positive. This research suggests that 10 people were missed (FN) and 20 were correctly tested (TP) and the sensitivity of the test is 67% (20 / (20 + 10).

The sensitivity of the test thus increased from 99 to 67% and therefore seems to depend on the group of people studied or rather, depending on the severity of the disease of these people. If the viral load is high enough, the RNA test will certainly give a positive result, but if it is very low (or even absent where the sample is taken), it may fall below the critical range of the test measurement range. We call this critical limit the lower detection limit of a test. At least 10 viral particles must be present in a swab (for example) to obtain a positive test result. Less than 10 gives a negative result. This number 10 is also called the lower limit of detection of the test or functional sensitivity and is often confused with the clinical sensitivity as described above.

Clinical sensitivity therefore depends not only on the quality of the test (including functional sensitivity), but also on the severity of the disease in the population studied (or rather on the concentration of the test substance in the sample).

The choice of an RNA or antigen test to measure the RNA of the virus in a swab (mucus) is therefore an understandable choice for hospitalized patients, but for random patients with slight complaints or without complaint in the general population or among health personnel, is not a wise choice.

Since the PCR techniques used can only specifically detect RNA from the current COVID-19 virus, the specificity of these RNA tests is very high (around 100%). A positive result is most likely due to the corona virus. The first epidemic corona virus in China (SARS-Cov1 virus) has 75% RNA identical to the current virus, which is certainly not enough to give a positive result in the current PCR tests of the SARS-Cov2 virus .

Test to demonstrate the passage of COVID-19

After passing through COVID-19, the virus particles will quickly disappear once recovery begins. An RNA test will no longer help. Because the body is able to cleanse the viral particles by means of a very ingenious defense system, we can use tests that measure the concentration of proteins specific to the virus of this defense system, the so-called antibodies, also called antibody.

Broadly speaking, the defense system can be divided into two main groups:

  1. The non-specific defense system that comes into action first during a virus attack. We have cells that recognize and eat (phagocytosis) bacteria and viruses, such as granulocytes and monocytes. However, the capacity is limited and if the number of viral particles increases after a few days, this system will not be able to cope.
  2. The specific immune system comes second at the request of the above phagocytes. We have B lymphocytes capable of producing very specific proteins (antibodies) which adapt exactly to the proteins of the virus, like a key and a lock. In the first days, approximately 5 days after the onset of the first complaints, the production of the so-called IgM antibodies begins; this is called seroconversion. These are very large protein molecules that can bind to several viral proteins (antigens) at the same time and are quickly recognized by the liver as an antibody-antigen complex and are broken down.

In this way, large amounts of virus particles can be cleaned up quickly. After about 10 days, a second immune system goes into action. Based on knowledge of the protein structure of viral proteins that has accumulated in the body over the past 10 days by very specific lymphocytes, new antibodies are produced (IgG antibodies) with a degree of specificity and d much higher affinity against viral proteins.

This process is called class change. This IgG key fits even better in the lock and does not fall out. When the antibody-IgG antigen complex is formed, it is quickly eliminated and broken down by various cells in the body. The effectiveness with which this system can be activated determines the course of the disease after about 10 days.

With relatively simple and inexpensive tests, we can measure these virus-specific antibodies in the blood. The price for this type of test varies from 10 to 30 euros. This contrasts with the RNA tests mentioned above, which cost around 200 euros.

We now know what proteins the virus makes (see image above). These proteins are made by the virus on the basis of viral RNA which functions as a template. The corona virus RNA has 30,000 letters and can produce 29 proteins (the human genome has more than 3,000 billion letters). The proteins outside the virus, the so-called Spike proteins or S proteins, give a crown appearance, hence the name “corona virus”. The viral particle contains many nucleocapsid proteins or N proteins, which protect RNA. In addition, there are 27 other sneaky proteins that make the virus so virulent.

The effectiveness with which specific antibodies against this virus can be produced is, among other things, hereditary (therefore, more serious diseases occur in some families than in others) and is determined by the individual’s state of health. When the quality of the antibodies is not optimal and the viral load remains high, more antibodies are often produced, but the result is not necessarily better.

Patients who have entered intensive care usually have much higher IgG antibody titers. The effectiveness with which specific antibodies against this virus can be produced is, among other things, hereditary (therefore, more serious diseases occur in some families than in others) and is determined by the individual’s state of health. When the quality of the antibodies is not optimal and the viral load remains high, more antibodies are often produced, but the result is not necessarily better.

Patients who have entered intensive care usually have much higher IgG antibody titers. The effectiveness with which specific antibodies against this virus can be produced is, among other things, hereditary (therefore, more serious diseases occur in some families than in others) and is determined by the individual’s state of health. When the quality of the antibodies is not optimal and the viral load remains high, more antibodies are often produced, but the result is not necessarily better. Patients who have entered intensive care usually have much higher IgG antibody titers.

We now know that almost all people make IgM and IgG antibodies against protein S and against protein N during corona infection, but not all to the same extent, not at the same rate and at the same rate. IgM antibody production begins approximately 5 days after the onset of symptoms, but in some people it does not begin until 8 days. One of them then reaches a maximum peak after a few days, while the other takes 14 days.

Thus, approximately 14 days after the onset of the disease, everyone infected with the virus must have produced IgM antibodies (more than 98% with a proven viral infection). In some people who have not had any complaints, there is a chance that the virus has been (briefly) present, but the antibody concentration is very low, below the analytical detection limit of the test. We will judge this as negative, perhaps unjustified (FN); but it may also have no meaning in terms of immunity and is rightly negative (TN). There are indications that in a large part of this group, the test still becomes positive after 3 weeks.

There is no gold standard to definitively determine whether there has actually been a viral infection; formally, the sensitivity of an antibody test in people without complaints cannot be calculated. A small group of people with underlying disease and / or using drugs that negatively affect the immune system (for example corticosteroids) can produce so few IgM antibodies that they cannot be measured. This can lead to false negative results. You will feel that the maximum sensitivity of the IgM test will decrease accordingly. It is therefore now clear that sensitivity diminishes previous tests in the disease process. The sensitivity of the IgM test goes from 95% (14 days after the start of complaints) to around 30% (5 days after the start of complaints).

The best period is between about 14 days and 3 months after the onset of complaints, after which the concentration of IgM antibodies gradually decreases again. However, if there have been no complaints, we cannot use this calendar and we should keep in mind that a recent infection (within the last 14 days) may be missed. The sensitivity of the IgM test goes from 95% (14 days after the start of complaints) to around 30% (5 days after the start of complaints). The best period is between about 14 days and 3 months after the onset of complaints, after which the concentration of IgM antibodies gradually decreases again.

However, if there have been no complaints, we cannot use this calendar and we should keep in mind that a recent infection (within the last 14 days) may be missed. The sensitivity of the IgM test goes from 95% (14 days after the start of complaints) to around 30% (5 days after the start of complaints).

The best period is between about 14 days and 3 months after the onset of complaints, after which the concentration of IgM antibodies gradually decreases again. However, if there have been no complaints, we cannot use this calendar and we should keep in mind that a recent infection (within the last 14 days) may be missed.

The same story applies to IgG antibodies. However, these come into action a little later and are produced only after about 8-10 days after the onset of the disease and peak after about 4 weeks. IgG antibodies hardly decrease after that, unlike IgM antibodies, and are detectable for a long time, probably more than a year. These IgG antibodies also form the memory of COVID-19 disease. If enough IgG antibodies of sufficient quality are present, a subsequent virus attack can be effectively combated. The body will then produce the correct IgG antibodies in large quantities directly from memory cells, so the virus has no chance of causing a serious infection.

However, most tests do not measure the exact amount or affinity of the antibodies, but only indicate their presence or absence (so-called qualitative tests). But in general, we can say with reasonable certainty that the presence of IgG antibodies protects against future corona infection; this does not apply if only IgM and no IgG antibodies are detectable.

As with the IgM test, the sensitivity of the IgG test decreases the earlier we test during the course of the disease (if there have been symptoms). About 1 week after the onset of symptoms, the sensitivity of the IgG test is 40%, after 2 weeks it increases to 80% and after 3 weeks to 98%

Tests based on the measurement of IgG and IgM antibodies use purified recombinant viral protein S or protein N to capture antibodies in the blood. These proteins are placed in a test cassette or an ELISA plate, thanks to a coating process with which the blood comes into contact. There is a strong link between these blocked viral proteins and antibodies in the blood. This complex can be colored, which makes the antibodies demonstrable. Some suppliers use protein S and others use protein N and sometimes this is not known. This means that not all tests show the same result.

We distinguish two types of tests: in laboratories, Elisa (or related) tests are used and in your practice, you use rapid point-of-care tests. Both are based on the same principles. However, with ELISA tests, the washing steps are performed in the interim, which reduces the non-specific response. This allows elisa tests to measure lower concentrations and measure positive IgM and IgG values ​​a little earlier in the disease process than rapid tests. In the end, after 3 weeks, the sensitivity between the two techniques will be very different. The rapid test also shows that the sensitivity in patients whose crown is proven (RNA test, CT scan and hospitalization of 3 weeks) is approximately 95 to 98%.

The specificity of these immunoassays is approximately 95 to 99%. To establish this, serum from healthy people was used, which was taken long before the corona attack. This means that if you have not had a corona virus infection (with the exception of the last 3 weeks and with very low titers), the test is also negative with more than 95% certainty. This applies to both the Elisa test and the rapid test (rapid test) that we will both be using.

What we need to keep in mind ?

In the literature, various antibody tests are often compared to the serum of patients who have had a recent infection. And then it appears that an ELISA test has a (slightly) higher sensitivity than a rapid test. This will no doubt be widely measured by the media in the near future. But from the above, you can understand that this type of research should be considered critically. Sensitivity is less determined by the quality of the test but more by the time the test is performed.

In addition, antibody tests, especially rapid tests, will be rejected by experts based on the sensitivity of the sensors in the future, while this discussion for RNA tests for cheek mucus, with a questionable sensitivity (see above), has never been conducted. Political and financial interests can play a role here.

Of course, worse tests may also appear on the market. Based on our experience with the suppliers of these types of kits, we have selected two appropriate tests; an Elisa test for laboratory use (for the home collection test) and a rapid test for “professional use only” as a point-of-care test. After a positive internal validation, we can deliver the tests. Both tests have a CE / IVD certificate and are authorized on the European market. There are currently no approved home tests for consumers.