Social Question

JLeslie's avatar

Can you help with some math, more specifically calculating statistics?

Asked by JLeslie (65477points) February 5th, 2021

Several days ago I was postulating that the efficacy of the Moderna vaccine might be inaccurate if it was tested in places with low infection rates. I know one place it was tested was where I live, and while the trial was happening there was a very low amount of cases here, so not very much chance to catch it. The person I was talking to said it wouldn’t matter since they used a placebo also, but I think it would. I am checking with the collective to see if I am thinking about this correctly.

It turns out that J&J was tested here also, but I think it was more recently and also J&J was tested in Latin America and South Africa where cases were high.

Yesterday I was watching TV and Gupta or Fauci (I cannot remember which) said similar to what I am saying that, J&J might have a lower efficacy rate, because they were tested in places with high infections.

So my question is, am I correct that if the infection rate in the community is very low, the efficacy rate of the vaccine might be overestimated?

Observing members: 0 Composing members: 0

18 Answers

Zaku's avatar

Yes, there are two independent probabilities combining, which is a multiplicative effect.

A competent assessment should take that into account, or at least say that the results are inconclusive due to not having enough data. So a good professional assessment would do that, but journalists or unprofessional people might easily mis-read statistics.

For example, if in environment A, the chance of getting sick from COVID in a certain period of time is A, and if in environment B, the chance of getting sick from it in the same period of time is B, and the chance that a vaccine will prevent COVID sickness is V, then the chance of someone with the vaccine getting sick from COVID in area A is proportional to A / V, and the chance of them getting infected in area B is B / V.

That is, if the chance of an UN-vaccinated person getting sick from COVID in area A is X times as high as in area B, then the vaccinated person would also have X times greater chance of getting sick from COVID in A than THEY would in B.

I.e. It is not meaningful to say what an effectiveness rate is, without saying what the context is. In the above, the rate V would be the rate of effectiveness for someone being deliberately exposed to ha very high amount of contagious COVID virus. If looking at actual infection rates for vaccinated people, their environment absolutely does need to be taken into account. After all, if you assessed the effectiveness of drinking 7-UP on subjects in Australia during a period where no one there caught COVID, you’d see 0% infections.

There are some statistical details about vaccine effects that I don’t know that might somewhat affect how that works, but that’s the basic gist of it.

Tropical_Willie's avatar

I understood they measured for antibodies before vaccination (to rule out they person having had COVID-19) and 14 days after second vaccination. The efficacy was for the antibody production and how many caught COVID-19 from the two groups of people The two groups are in the same population group; they are side by side. .

Wish you could remember who and where you heard that, sounds like Gupta.

JLeslie's avatar

No one was deliberately exposed.

Regarding how many caught it, if in the community, if the person only had a .1% chance of catching it in their community, then if 2,000 are in the study 2 people would catch it with no vaccine. Let’s say in another city the chance is 1.5% then out of 2,000 people 30 people would catch. If in both cities both placebo groups had zero people catch it would the efficacy be 100%? What if one person in each placebo group caught, if the efficacy the same for both test locations?

Tropical_Willie's avatar

Here’s the New England Journal of Medicine article from yesterday,

Placebo and vaccinated people are in the same city . . .so they would encounter the the same people on a day to day basis.

flutherother's avatar

30,000 people took part in the Moderna trials of which 95 people caught Covid. Of those given a placebo 90 became ill and of those given the vaccine 5 became ill. All 11 severe cases of the virus were in the placebo group. This is pretty clear proof that the vaccine works.
Running the trials in a community with double the infection rate would be like running a trial with twice as many people. You could have a little more confidence in the results but not much. It is the law of diminishing returns.

doyendroll's avatar

@JLeslie, another good question of yours, in the numerical examples you quoted the observations are too small for valid statistical inference.

This video explains how efficacy is calculated

LostInParadise's avatar

I have just a limited understanding of statistics, but this is my reasoning. The Poisson distribution can be used for events with low probability. The variable used is λ = np, where n is the sample size and p is the probability of an event, and the confidence interval can be calculated from λ. To have the same value of λ, it for a lower value of p, it would require a larger value of n, a greater sample size, to achieve the same level of confidence in the results.

crazyguy's avatar

I have questioned the calculation before. I tried to find it, but given Fluther’s non-functional search engine, I gave up.

However, I remember the gist of my statement.

The way I understand it, efficacy was calculated as follows:

Number of placebo volunteers who caught covid/ total number of patients that caught covid.

Since different people have different exposures, the resulting calculation is somewhat inaccurate. This was supposed to be offset by having the end point defined as so many actual infections in large sample sizes. I would feel better about the calculation if a sample were deliberately exposed to covid. Then you would test every one of them to see how many vaccinated people caught it compared to how many placebo people caught it. However, this would require essentially quarantining a large number of volunteers for weeks.

I believe something like this is being done in England.

To me, the reliability of the data we do have is adequate.

JLeslie's avatar

@crazyguy I had heard that England was going to do deliberate exposures too, but I don’t know if it was ever done. I heard it a while ago, maybe 6 months ago. It didn’t sound wise since we didn’t have very reliable treatments, but of course it would be a better way to test for efficacy. I wonder if any testing was done?

crazyguy's avatar

@JLeslie Not yet, but it is scheduled. here is a link:

crazyguy's avatar

@doyendroll That is a very good explanation of how efficacy is calculated. My simpler formula gives the same result provided the total number of study participants is split 50–50 between placebo and vaccine.

JLeslie's avatar

@crazyguy Thanks for finding that article. Is it with Astra Zeneca? It wasn’t clear to me. AZ is mentioned regarding testing children. AZ has a different method for the vaccine than any of our vaccines.

crazyguy's avatar

@JLeslie The article does not say which vaccine.

For the first part of the study, trying to understand how the virus works, my understanding is that the volunteers will have the covid virus squirted up their noses without being vaccinated.

Tropical_Willie's avatar

”. . . my understanding is that the volunteers will have the covid virus squirted up their noses without being vaccinated”

Source please @crazyguy

JLeslie's avatar

@crazyguy I wonder what variant they will use? I’m very interested in this study.

crazyguy's avatar

@JLeslie I suspect it will be a vaccine that has proved effective against the UK variant, and is more English than American. My bet is AstraZaneca.

JLeslie's avatar

@crazyguy So you think they will use just one variant? The UK variant, to infect the participants.

crazyguy's avatar

@JLeslie The infection virus shall be the same old one:

Initially, the study will use the virus that has been circulating in the UK since the pandemic began in March, which is of low risk to healthy adults, to deliberately infect volunteers.

Answer this question




to answer.
Your answer will be saved while you login or join.

Have a question? Ask Fluther!

What do you know more about?
Knowledge Networking @ Fluther