I just completed a road trip to and from Colorado by myself — giving me nearly 20 hours of prime selfish podcast listening. I spent most of the precious time on science and philosophy podcasts and the experience drew me back to a draft of this post I started last month.

False dichotomies. They seem so natural that we rarely question them — until we become so painfully aware that they are impossible to ignore. My sensitivity to false, or artificial, dichotomies is rooted in their overuse and abuse in interpretation of statistical results in science. I saw it, felt it, and fought against it on a daily basis for nearly twenty years. Dichotomies have a comforting simplicity to them. A simplicity that blinds us to recognizing them as artificial, much less a sort of problem we need to deal with.

The one motivating this post — a person is either an expert or not an expert — is one that I was blind to until rather recently, so I have found it very useful to reflect on. And, I am well aware that I am blind to many others — I just wish I knew what they were. The reflection gives me empathy for those who simply to do not see the false dichotomies running through our use of statistical results in science. We just don’t know enough to know what we don’t know. Or, we aren’t aware enough to know what we aren’t aware of.

So, let’s dig shallowly into this expert vs. non-expert dichotomy — just to get us thinking. Journalism relies on expertise, and expertise is often communicated to the audience through people labeled as experts. This implies that somewhere in time a person moves from the bucket containing the non-experts to the bucket containing the experts. It’s not clear when or how one gets dumped, or climbs, from the non-expert bucket into the expert bucket. Does my bucket labeled Experts on a specific topic contain the same people you would put in your Expert bucket? Well, the overlap in our bucket compositions probably depends on our levels of expertise on the topic. If we are both equally ignorant and generally share views on the topic, then maybe we can agree on who should end up in the expert bucket (after some discussion of criteria). But, if you are incredibly knowledgeable on a topic that I am not, then our buckets are likely going to differ dramatically. This bucket picture has no acknowledgement of the continuum of expertise or the arbitrariness and subjectivity inherent in the split.

Do we need the expert label? What purpose does it really serve?

Luckily, the announcement in journalism that the person interviewed is “an expert” is typically accompanied by a description of their qualifications on the topic. This is usually a resume or CV type list of education and accomplishments. This list, even if superficial on some level, is the information we need. Based on it, we can weigh our trust in what the person has to say and the amount of influence we might let it have over our own thinking — that is, we can weigh how seriously to consider it relative to our own knowledge and experiences. The label of expert does not need to be overlaid on the statement of qualifications; it detracts from encouraging the audience from making their own judgment about the trustworthiness of information conveyed.

Even if you agree the label is probably not needed, you might be asking — but does it really matter? Is there any harm done? Is it a habit that’s worth breaking? I do think there is harm done to trust in journalism, and science journalism in particular.

Consider a person in the audience with little knowledge on the subject — hearing the interviewee labeled as “an expert” is likely what they hear loudest, and it’s used to justify not spending much energy evaluating the person’s specific qualifications. It comes across as if the journalist is saying “you should trust me that you should trust this person,” rather than encouraging the audience to critically evaluate the situation on their own. The labeling of “expert” adds an extra psychological layer — to critically evaluate the situation, the audience has to have experience some distrust in the journalist who just declared the person “an expert”. Leaving off the label of expert and simply presenting the person’s qualifications avoids unnecessary complications of human psychology and encourages critical thinking.

Now, let’s go to the other side of the expertise spectrum. Suppose a person in the audience has a deep knowledge of a subject and hears a journalist label a person as an expert on the topic. Because it is an area the audience member has their own expertise in, the label may put the audience member into hyper-critical mode, particularly if they find the expert leaves out or misrepresents important information or views (which I argue is often how a person with great expertise feels when they hear someone else talk about their area of expertise). This feeling leads to unnecessary frustration and a feeling of distrust in the journalist who assigned the label of expert. This elicits stronger and different feelings relative to trust than if the interviewee’s qualifications were simply described as they exist. There is really no need for the journalist to declare a person an expert — all they have to do is provide the qualifications and reasons why they chose to interview them and let the reader/listener judge for themselves.

So, what is the goal of going the extra step to label a person as an expert? I really can’t think of a good reason — except… how good it feels when we are the one labeled as “the expert”.

Could curbing our use of the label help maintain or build public trust in science journalism? Maybe I’m reaching here, but I think it’s a small and doable thing that, when combined with other small things, can make a difference.

I had this happen to me today while writing another post. It got me thinking enough that I decided it was worth a post of its own.

https://www.quickanddirtytips.com/education/grammar/dilemma-or-dilemna

I guess it has been awhile since I needed to write the word dilemma — even after two years full of experiencing dilemmas of various magnitudes. The desire to spell dilemma as “dilemna” is deeply engrained in my brain. When I prep myself to spell it, I explicitly split it into three syllables to remind myself (wrongly!) that it is “di – lem – na” and not simply “di- lemma.” In my brain, it feels exactly like I’m appealing (correctly!) to a useful trick to remind myself of less-than-intuitive spellings, like “con – science” to spell conscience. I would swear I was taught that trick — would I really just make up a wrong spelling, a trick to remember it, and the convincing feeling that I was “taught it”? But, like Grammar Girl, I have no evidence that I actually was taught this in school (though I would love to be able to scroll back through my elementary school spelling tests for the chance of finding some). I also must admit that spelling has never been my forte and I do tend to repeatedly misspell the same words.

I can’t help but thinking more about how this phenomena shows up in the use of statistical methods. As a collaborative statistician, I used to get the “But I was taught…” comment in response to my suggesting an approach that differed from discipline defaults and norms. I often wondered what that statement really meant — “taught” in terms of a formal class, “taught” from copying what was done by others? What were the credentials of the “teachers”? It’s quite possible I saw the misspelled word “dilemna” in print early in life and it stuck. I feel like I was “taught it,” but really I was copying the work of someone without sufficient knowledge of how to spell the word. I probably trusted because it superficially looked legitimate and correct, without a second thought at the qualifications of the author.

While this is an understandable oversight for a kid learning how to spell, it is not as understandable for scientists with multiple degrees doing serious work. However, to the defense of the scientists — I don’t think it is at all clear how to judge qualifications relative to advice about data analysis and statistical inference, so this is not a simple problem to fix. Much of the information available now has had decades of being passed through teachers who themselves only drew on the knowledge of the teachers before them — a long distance from the original work and thoughts. It is a long game of telephone and I can’t help but appeal to one of my favorite quotes from RA Fisher in 1955. And, I think it is safe to say the suffering of Science is far worse now than it was 65 years ago.

The History of Science has suffered greatly from the use by teachers of second-hand material, and the consequent obliteration of the circumstances and the intellectual atmosphere in which the great discoveries of the past were made.

RA Fisher 1955 (in an introduction to papers Experiments in Plant Hybridisation by Gregor Mendel, edited by J.H. Bennett and published in Edinburgh by Oliver & Boyd, p.6.)

This question sneaks its way into my thoughts nearly every day — dressed in many different outfits. A reasonable answer is far bigger a single blog post and realistically depends on the day. But, starting somewhere feels important. Much of my life’s story is wrapped up in my career as a statistician — like it or not.

When people ask the “What do you do?” question and I answer “I’m a statistician” — it has always evoked a confusing set of emotions for me. I feel proud to have earned a master’s and PhD in Statistics, I feel my love of research and science, I like the feeling that most people are surprised and can’t relate to it … but underlying a surface of positive feelings is a sense of dread. Dread that I kept hidden for so many years — a voice saying “yeah, I’m a statistician, but I loath most parts of it and have no idea how to navigate my feelings around it or communicate them in an effective way — even to myself.” I was stuck.

To be clear, I do not loath the statistician profession in general. I think it is incredibly important and valuable to science — I just have been unable to find a comfortable seat for myself within it. It’s like all the chairs were made for people much taller, shorter, wider, and narrower than myself. This is not a new feeling. In retrospect, it started 21 years ago, before I even started studying Statistics for the sake of Statistics. I was a graduate student in another discipline — simply trying to figure out how we should and shouldn’t rely on statistical inference for the science I was trying so hard to learn how to do.

I now find it very ironic that the discomfort I felt about how I was being taught to rely on statistical methods is what pushed me out of that science to study Statistics. I thought that was my way out of the discomfort — to face it head on and get to the point I understood it so deeply that I would feel comfortable using it and my work would be helping others navigate that discomfort. I would come to Statistics from science and a passion for research — a very different path from that of students with mathematics degrees who seek to pair it with a more practical degree for the sake of future jobs. There were times when I became so focused on statistical theory and my classes (not really connected to doing science in other disciplines) that the discomfort lessened — but every time I leaned back into Science, there is was again. The more knowledge I gained, the more the discomfort morphed and grew. The gut feeling of discomfort became a discomfort I could attach reasons to — and over time I realized my naive gut feeling had actually been too mild, rather than too exaggerated. Over the next 20 years, it grew in intensity and form — from a feeling akin to annoyance to an almost unbearable weight. A weight that led me to daily tears, extreme stress, and ultimately to a career crisis. The very reason for pursuing my career was also killing it.

I struggled to effectively communicate to others what was happening to me. No one seemed to get it at the level I was feeling it. I mostly got variations on — “Yeah, everyone has parts of their job they don’t like — it’s just life” — and came away feeling like a negative complainer that I didn’t want to be. I had held other jobs that definitely had their downsides — like cleaning human toilets and cow stalls — but they didn’t make me feel like I was feeling about being a statistician. The first time I felt I successfully communicated the feeling to someone else was with this explanation: imagine that to do your job each day requires you to bend far enough away from your professional ethics comfort zone that it causes you real pain. I finally realized that part of my problem was accepting that it is okay for me to have a different comfort zone than others — that understanding my boundaries and staying within them does not need to be interpreted as a negative reflection on others. I worried that my saying it out loud would be taken as if I was judging the locations of the comfort zones of others, but I am completely comfortable with different scientists having different comfort zones– it is a positive because it pushes all of us to self reflect and justify our stances. What was not positive, was for me to hide the boundaries of my own zone and to feel pressured to work outside of them to get paid. And I guess that’s why I find myself here — writing a blog and a book proposal and not getting paid (yet!?).

Many stories of middle aged career changes stem from a lack of meaning and passion for the work one is doing. This is the story people assumed I was telling — but that isn’t my story. I am passionate about understanding statistical inference and how we can use it in science — I just cannot sit comfortably with much of current practice. I don’t want to leave my career as a statistician behind — I want to use my 20 years to morph it beyond a practicing statistician to facilitating needed discussions about how and why we are using statistical methods and results.

I thought I could do it as a practicing applied statistician, but I have finally accepted that that option is untenable to me personally. I can now say it openly — and I cannot adequately describe the relief that comes with that. I will no longer attempt to walk the impossible line between holding true to my understandings and beliefs and playing the arbitrary games almost required to succeed in current scientific culture. I could not find a balance that left me feeling whole at the end of the day.

I still want to be a statistician. I still am a statistician. I want to engage with scientists — not to help you with your statistical analysis under a set of arbitrary and misguided rules about how it should be done– but to use my expertise and experiences to motivate and facilitate discussion about how we are, and could be, relying on statistical inference to support science.

This morning I was reading the Science Needs Story blog by Randy Olson and reflecting on where to take my writing. It is easy for me to stay entangled in the weeds of specific mis-uses of Statistics in practice — because I lived among them for two decades. But I left the life of a practicing statistician to break out of those weeds and I want to connect my experiences with those nasty weeds to the whole pond — and its larger ecosystem. I do get glimpses of the pond, in moments of clarity, but the weeds keep a strong grip on me. Randy Olson’s call to “Make Science Human” was good motivation for poking my head above the surface today.

Each day, in the news and in my interactions with people around me, I see stark parallels between larger issues confronting humanity and the roots of our mis-uses of Statistics: our longing to simplify complex situations by buying into the use of false dichotomies; our longing to be pretend we are being “objective”; our blind trust in sophisticated black box techniques or processes we don’t understand; our longing for personal and professional success relative culturally engrained, but clearly flawed, criteria; and our extreme discomfort with uncertainty and change.

As I work to find smaller self-contained narratives around this topic, these realizations are simultaneously depressing and inspiring to me. The “Make Science Human” idea resonates with me on so many frequencies — beyond those intended by Olson. The truth is Science is human and we need accept and embrace that — to ultimately do better science. As much as we try (and I agree with the trying!) to keep Science separate from the challenges and faults of human experience, Science is done by humans within a human constructed culture. How can we expect it not to suffer from many of the same things that creep into other human cultures, processes, institutions, etc. Convincing ourselves that it sits above our faults is in fact one of our faults — and it ultimately it hurts Science, which just spreads the hurt.

Where does Statistics come into this? I see its methods and procedures being used as if they take the human out of Science, as if they add a magic level of objectivity, as if they are not subject to the common faults of humans. This is naive and the consequences are finally getting noticed.

I love Science. I believe in the potential of statistical inference. And, I believe in the value of acknowledging and embracing the inherently human aspects of how we “do science.”

Statistical inference is built upon layers of assumptions. My colleague, Katie Banner, is giving a presentation today on some of our joint work and she came up with a great way to coarsely describe the layers of assumptions: those we are aware of and talk about; those we are aware of, but rarely talk about; and those we are not aware of (and therefore also do not talk about!).

Those on the surface seem obvious and make their way into even introductory courses — e.g., normality, linearity, constant variance, and independence in the context of linear regression. As we go down through the layers, they are increasingly ignored and accepted as implicit parts of the process — rarely acknowledged or discussed. But perhaps “ignored” is not fair in the deeper levels — I think it is more a reflection of honest ignorance. Many (most?) people using or relying on statistical inference are simply not aware of the layers or the need to peel them back. I have alluded to some of the deeper layers in other posts — such as the often automatic reliance on models for means and even the decision to use probability as the basis for inference (and all the assumptions that accompany that huge statement). These are things I can’t stop thinking about — as I struggle to figure out how to communicate the importance and associated problems to other scientists. But today, I am staying on the surface, as there are plenty of challenges there as well.

Assumptions are not met. Stating that an assumption is “met” implies that the assumption has been checked and concluded to be true. In 1000’s of Statistics courses around the world, students are being taught to use this wording and I believe it has substantial negative impacts on science and contributes to a lack of critical thinking to back up inferences.

Let’s take “normality” since that is an assumption many people are aware of from intro stats classes. The validity of statistical results may rest on the assumption that the errors are normally distributed (based on the Gaussian probability distribution model). In practice, it is common to use plotting techniques (that have their own problems and limitations), such as boxplots or Normal Q-Q plots, to “check” the assumption. If things “look okay” then the students are often taught to say “the assumption of normality is met.” This is a false statement. No amount of justification could ever convince me to believe it. The errors around a mean do not arise from a normal distribution — we just hope to model them as if they do and the validity of our inferences depends on the degree to which the “as if” approach is reasonable. The “checking” is really an assessment of how severely violated the assumption is. We know it is violated, but how severely is the violation? This is not a yes or no question, it is a question of degree of severity. The answer must be a justification — based on plots of the data, based on knowledge of the measurement and population, based on the study design, and based on knowledge of the robustness of the method to violations of the assumption. Is this easy? No. But assessing assumptions should not be an exercise is deciding whether they are met, or not. This is just another in the long list of false dichotomies that have become associated with the use of statistical methods in practice.

If you are a researcher, or a teacher of statistics, please do not treat assessment of an assumption as a yes or no question. Do not let your students ever write or mutter the phrase “the assumption of _______ is met.” I know from experience that it is possible to teach and report on research without falling into the trap of this false dichotomy.

Assumptions are not met. Period. We must discuss the severity of the violation in non-trivial way to assess the reasonableness of the suggested model.

In all my formal education as a statistician, I have no specific memories of professors or other more experienced statisticians bringing up the potential limitations of averages explicitly. The fact that we average and model means was presented as an implicit fact of how we should do things, not as something open to scrutiny and question. You can see it in the math and proofs, but there is a serious disconnect between how we learn the theory and tying that to questioning what we do in practice to address real problems. This gap has always seemed huge to me — in fact, I moved from being a graduate student in biological sciences to a graduate student in Statistics because the gap felt so massive. It has taken me almost 20 years of circling and trying to put words to what has always felt so uncomfortable about how we teach, talk about, and attempt to use statistical inference. It still feels like a jigsaw puzzle fresh out of the box — this blog is like quickly examining tiny pieces of the puzzle so I can start to make sense of how they fit together into a coherent story I can tell. I want to turn the years of struggle into glimpses of clarity that I can share. Which leads me to what I sat down to say —- I don’t think I fully realized, or at least fully acknowledged, how prominent the puzzle piece focused around our use of averages is. I want to put this out there now while it feels so obvious.

In years of trying my best to teach foundations of statistical inference and common statistical methods, I landed at a few basic messages around this topic that can be loosely summarized as: (1) look at and understand your raw data before aggregating, (2) box plots (and such) do not count as visualizing the raw data — that’s already aggregating, and (3) we should only average things that we are convinced are inherently measuring the same thing.

Using “things” and “thing” in the same sentence never felt pretty, but for some reason it seemed to get the point across. The problem is that I usually talked about it in the context of higher level modeling and not in the initial decision to even base inferences on means to begin with. So, while I think it was helpful and better than nothing, I should have tried harder to go deeper. It was like trying to help someone make ethical decisions about how to spend a pot of money when the money was stolen to begin with. I just wasn’t hitting the discussion at the right level. Why not? The best I can come up with is that my students and their advisors had very specific expectations about what they would get out of their graduate education in Statistics — and those expectations were conditional on the assumption that means represent the magic quantity of interest. We understood that we were educating so students could immediately participate as integral players in the current popular culture of averages. The problem (or one of them) is that it’s incredibly rare for scientists, including statisticians, to explicitly think about that conditions underlying their models, beyond “checking” higher level assumptions in a stale and automatic fashion. We have to start asking what a mean really means in a particular context and what an average might really represent for a particular set of data.

I am very aware that the reason this piece of the puzzle looks so clear and large to me right now is because I am still reading The End of Average by Todd Rose. There are so many things that resonate and help put words and context to my gut feelings of discomfort and academic frustrations. Over the last few years as a statistical consultant, I had many conversations in very different contexts with scientists about what the average calculated from the data (or mean in a model) could reasonably represent and whether that was really what the scientist was after. These were conversations typically met with some interest, but also (at least in my perception) annoyance. Annoyance that I was holding up the process of just proceeding under the status quo. Annoyance that I was making things more difficult when I was supposed to making things easier. The question always arises: “What else is there?”

On that note, I want to share a few quotes from Rose’s book and his discussion of the averagarianism in our society. These resonated with me last night in the context of statistical inference and science. And, this won’t be the last you hear from me on averages.

The primary research method of averagarianism is aggregate, then analyze: First, combine many people together and look for patterns in the group. Then, use these group patterns (such as averages and other statistics) to analyze and model individuals.(21) The science of the individual instead instructs scientists to analyze, then aggregate: First, look for pattern within each individual. Then, look for ways to combine these individual patterns into collective insight.

Pg 69, The End of Average by Todd Rose, HarperCollins, Reference 21 is given in Notes as Rose et al. Science of the Individual, pg 152-158.

The mathematics of averagarianism is known as statistics because it is the math of static values — unchanging, stable, and fixed values.

Pg 68-69, The End of Average by Todd Rose, HarperCollins

Note: The above quote is one I want to come back to later. I see his point, but it presents a limited view of the field of Statistics (if one reads it as if it is Statistics with an upper case ‘S’ — see my previous post on statistics vs Statistics). I think there are paths forward within a broader view of Statistics.

“What you are proposing is anarchy!” (16) This sentiment was perhaps the most common reaction among psyshometricians and social scientists whenever Molenaar showcased the irreconcilable error at the heart of averagarianism. Nobody disputed Molenaar’s math. In truth, it’s fair to say that many of the scientists and educators whose professional lives were affected by the ergodic switch did not follow all the details of the ergodic theory. But even those who understood the math and recognized that Molenaar’s conclusions were sound still expressed the same shared concern: If you could not use averages to evaluate, model, and select individuals, well then … what could you use?

This practical retort underscores the reason that averagarianism has endured for so long and become so deeply ingrained throughout society…

Pg 66 The End of Average by Todd Rose, HarperCollins. Reference 16 is given in Notes as “Molenaar, interview, 2014.”

The above quote is consistent with my experiences. Methods based on averages are available, easy, convenient, and take little creativity — and they are expected in our scientific culture. Justification for using averages is simply not demanded — though justification for use of anything but averages is incredibly difficult to sell.

This is one I thought I would do earlier in the posts. I don’t even remember where I first heard someone distinguish between statistics with a lower-case ‘s’ and Statistics with an upper case ‘S’. I apologize if I should be giving credit to someone. I have been using the distinction in conversations, presentations, and teaching for about 10 years. It hasn’t lost its relevance. In my experiences, the distinction is lost on most students, most researchers I have worked with, and even a lot of people with degrees in Statistics. And I think the distinction does matter — not only for the discipline of Statistics, but also for scientists and the public.

Lower-case ‘s’ statistics are simply the numbers we calculate from data, such as an average, a sample median, a minimum, a maximum, etc. All that is needed for little ‘s’ statistics are some numbers and a calculator. Anyone with basic math knowledge can calculate them and report them. Classic example are summaries of census data and sports data. statistics provide an easy way to summarize a lot of numerical data into a single number. My daughter came home from school last week saying they have designated a statistician for their gym class — the person who records and summarizes basic data on 6th grade ultimate frisbee competitions. I tried to embrace and appreciate the acknowledgment and inclusion of the word “statistician” into education of our children, but if I’m really honest, I find it demeaning to my education, my expertise, and potentially harmful to science because it is setting wrong stereotypes and expectations as their first introduction. Maybe what they are describing is a little ‘s’ statistician, but that just gets to be too complicated. They are describing a technician whose job is data entry and data summarizing. They are not describing a scientist. (Another blog post is coming titled An -ologist, not an -ician)

Upper-case ‘S’ Statistics, while it did evolve from the foundation of calculating lower-case ‘s’ statistics, is the scientific field devoted to the study of the whole process of statistical inference. What I mean by statistical inference and even the process of statistical inference will come in more detail later. The point here is that it is not simply the process of calculating little ‘s’ statistics.

In practice, the words are used interchangeably — in ways that are confusing and indistinguishable. It is truly unfortunate that the discipline of Statistics shares a name with little ‘s’ statistics. I used to believe it was really just unfortunate for the Statisticians who struggle to educate people about what they actually do, but I now believe it is unfortunate for science in general. The reasons we struggle to raise awareness about what things are included in our discipline and expertise are related to mis-understandings and mis-uses of statistical methods in practice — like treating statistical inferences as if they are reports of statistics from censuses.

As you will probably hear me say many times in this blog — you can discount it as semantics (as many people have and continue to do), but I do not believe it is just semantics. There are consequences to our use of words.

The wording used to describe the process of statistical inference is incredibly important. It is not uncommon to hear or see words and phrases like the following in writing and presentation about statistical approaches: the best, the correct, wrong, assumptions are met, the answer, determine, the right approach, etc.

These words are unrealistic, misleading, and usually unjustifiable. Why do we use them?

That’s never an easy question — but I think the most superficial answer is because it’s what we’ve seen modeled by others. And, there is a worry that less strong wording will come across as weak and unconfident — and then the person with the strong words will get the publication, get the promotion, get the grant funding… — you get it. But, this post isn’t about all that could explode from that last sentence. So, I’m letting go of the why for now and just want something much simpler – acknowledgment that we use such words when they are not warranted. Let’s admit it, embrace it, and call each other on it.

The focus has to change from right to reasonable. You are choosing an approach that you think it reasonable for a given problem, and it is your job to justify that opinion to others. Others can then make their own decision about whether your arguments lead them to also conclude it is a reasonable approach.

Reasonable allows space for multiple approaches and perspectives that right does not allow any space for. Think about how this can improve science, as well as humanity.

My paper in American Scientist from 2013:

The cover of the August 2019 issue of the American Statistical Association’s (ASA’s) magazine Significance:

After writing my last post about the Average American, I finally started reading a book that’s been on my bedside table for awhile: The End of Average by Todd Rose. I am only through the first chapter, but seems promising for helping with the conversation — for anyone who harbors a slight unwillingness to let go of the idea that averages are, almost always, the quantity of interest. It also reminded me to go back and re-read Simon Raper’s articles in the American Statistical Association’s (ASA’s) magazine Significance from December 2017.

In the last decade, I have had numerous discussions with researchers where I broach the subject — Are you sure a mean score, or difference in mean scores, is really what you’re interested in? What would an average score really represent in this context? Such questions were typically met with blank and confused looks — like… Duh. What else is there? Why wouldn’t I want averages? Occasionally, the person would seem relieved and acknowledge that they really weren’t interested in means, but… that was usually accompanied by a perceived need to use the most common statistical techniques (e.g., t-tests, anova, regression, etc.) — because their career depending on it. Our statistical approaches and education are so mean-focused, particularly for those who take only a one or two semester course, that the universe of possibilities just seems very small and all about averages and means. And perhaps scarier than the perceived size of the statistical methods universe is the fact that there is typically no explicit recognition that by relying on the common linear model methods you ARE implicitly making the assumption that what you care about is means/averages. In my opinion, this is something that researchers should have to explicitly justify before basing inferences on estimating means and changes in means. Is it easy? no. Does the fact it’s not easy mean (no pun intended) we just shouldn’t expect it? no.

Can we stop being so nice about mean(s)? Means are incredibly nice mathematically — but how much should this dictate our reliance on them in practice? I am all for convenience and simplicity, if justified — but it should be justified for reasons other than convenience and simplicity. It clearly is a convenient and a logical starting place for statistical inference from a mathematical perspective. Unfortunately, we don’t often get any farther.

A default mode of operation is to compute averages first — before looking at the individual data. In fact, a huge portion of my time as a statistician has been convincing people to think first and plot data first before aggregating (including averages), and I know I am not alone in this. Over the last decade, I don’t think resistance to abandoning averages has wained — in fact, it may have increased (purely personal speculation here) — possible because more people can push the buttons to carry out statistical methods based on averages and each year adds to that inherently cultural expectation. There is no cultural expectation to question it or have to justify it. To be fair, on a rare occasion, people were thankful that they got a statistician to give them permission to not aggregate and rely on averages.

These general points and concerns have been raised repeatedly by others, but I have to say — when you’re out there trying to be a statistician, it sure doesn’t feel like others have made the points before. The message isn’t spreading, or at least seeping in, as quickly as it needs to. Like nearly all the issues I will write about in this blog, the unfortunate reality is that change means more work and challenge — not less.

I end this post, though definitely not my last on the subject of means and averages, with couple of quotes and the link to one of the pieces by Simon Raper (who’s writing I always appreciate).

https://www.significancemagazine.com/science/571-an-average-understanding

Eventually, the angst felt by many intellectuals of the nineteenth century regarding probability and statistics gave way to agnosticism by the early twentieth. Probability became simply accepted as the logic of uncertainty, without worrying about what precisely the word really meant. As a result, few moderns recognize that the statistical “reality” applies to populations, but not necessarily to the individuals within those populations.

Weisberg, H. (2014). Willful Ignorance: The Mismeasure of Uncertainty

“Some people must be average, you might insist, as a simple statistical truism. This book will show you how even this seemingly self-evident assumption is deeply flawed and must be abandoned.”

Rose, T. (2016). The End of Average: Unlocking Our Potential By Embracing What Makes Us Different

References:

• Weisberg, H. (2014). Willful Ignorance: The Mismeasure of Uncertainty. John Wiley & Sons, Inc., New Jersey, USA.
• Rose, T. (2016). The End of Average: Unlocking Our Potential By Embracing What Makes Us Different. Harper Collins, New York, USA.
• Raper, S. (2017). The shock of the mean. Significance, 14: 12-17. doi:10.1111/j.1740-9713.2017.01087.x
• Raper. S. (2017). An average understanding. Significance, https://www.significancemagazine.com/science/571-an-average-understanding