Statistical Thoughts

[This is the English translation of an  article  originally published on SIAF Community, the Italian scientific platform for forensic and insurance medicine] Suppose we flip a coin and ask what the probability of getting heads is. Our experience and what we learned in school suggest that the answer is 50%, since there is one “favorable” case (heads) out of two possible outcomes (heads or tails). This is a very common form of reification : confusing a theoretical abstraction - in this case, the probabilistic model that assigns a 50% chance to heads - with an empirical property of the real world, as if that 50% were an intrinsic characteristic of the coin or of the physical act of flipping it. Contrary to what one might think, this almost automatic answer already assumes a very specific statistical model, based on at least two main assumptions about the process that generates the outcomes (where an outcome is obtaining 'heads' or 'tails'). Assumption of the competent man...

[This is the English translation of an article originally published on SIAF Community, the Italian scientific platform for forensic and insurance medicine] Statistical methods are used in medical research to estimate the effects of treatments or health conditions in populations. For example, when testing the effectiveness of a new treatment that has shown very positive results in preclinical animal studies, researchers draw from the target clinical population a small group of patients - called a statistical sample . The ideal goal is to administer the treatment to the patients in the sample to estimate its effectiveness in the entire population. The process by which the effect observed in the sample is “transported” to the wider population is called inference . However, even when the best inferential methods available are used, a perfect transport is factually impossible . The reasons range from the many uncertainties involved in conducting a clinical study to the variability of cond...

I invite all of you to try to forget major aspects of your life for a moment, including your work, your values, and all the related ethical considerations. Let's also try to define dis-misinformation. One possible way to do it comes from Spitzberg [1]: “[...] any message or a set of messages that represent a meaning complex discrepant from or incompatible with a sender’s intent and/or a relatively informed or expert consensual evidentiary state.” Therefore, sharing any paper that contains even a single improper methodological aspect or unclear communicative content, regardless of the motivations behind the sharing, generates a certain type of dis-misinformation. As alien observers, we must then try to establish varying degrees of impact of dis-misinformation (infodemiological objective). This means methodologically studying both the modes of propagation and the consequences of such propagation, in order to identify the forms of infodemic (information overabundance, which includes d...

PREMISE I am not a professional statistician, so I would like this blog to be understood as a space for sharing my reflections rather than a collection of "lessons" (which is why I named it "Statistical Thoughts" rather than "Statistical Certainties"). Of course, here I try to express my opinion in a convincing - but hopefully honest - way! ARGUMENTATION Here I critique (I welcome any potential rebuttals) the concept of  'null hypothesis of no effect or association' as an equivalent version of 'chance is the only factor at play.' Indeed, within a frequentist-inferential statistical model, chance is always the only factor at play! A P-value is a number calculated based on other numbers (the observed test statistic and degrees of freedom) that have no memory of how they were generated nor are "aware" of the practical meaning we assign to the numbers we put in the corresponding formulas (e.g., average values, standard deviations, sa...

The other day, a colleague asked me an important question. When the hypothesis under consideration is not the null hypothesis of no effect (e.g., the difference between two mean values is 50 and not 0), a P-value equal to 1 indicates that—assuming chance is the only factor at play—we would obtain a "result" greater than or equal to the observed one 100% of the time in numerous future equivalent experiments. So, how can chance - assuming it operates alone - always generate a greater "result" than the observed experimental one? The apparent problem is easily solved by noting that the so-called "result" (or, better, the "statistical result") does not refer to the magnitude of the effect under examination but to the value of the test statistic. If we fix the hypothesis "difference = Δ = 50" and observe a difference of 50, then the test statistic (i.e., the statistical result) turns out to be 0! And what is the probability of obtaining a t...

Reading one of the many wonderful insights by Professor Andrew Gelman on the Columbia University blog (see https://statmodeling.stat.columbia.edu/2023/04/14/4-different-meanings-of-p-value-and-how-my-thinking-has-changed ), I have developed a series of thoughts that I believe - based on my current knowledge and abilities - might be potentially useful. Original comments, part 1 Andrew Gelman: << Definition 1. p-value(y) = Pr(T(y_rep) >= T(y) | H), where H is a “hypothesis,” a generative probability model, y is the observed data, y_rep are future data under the model, and T is a “test statistic,” some pre-specified specified function of data. [...] >> April 14, 2023 9:14 AM Christian Hennig: << By the way, on misinterpretation that bugs me and seems to come up often is the idea that there is a “true” unobserved p-value potentially different from the observed one in case the model doesn’t hold. Not so. The p-value measures the relation between the data and a specifie...