The truth hidden behind the numbersNumbers are generally considered a powerful tool for presenting objective facts. However, statistics can be cleverly manipulated to lead to unintended interpretations. Depending on how statistical data is presented and divorced from context, its impression can change dramatically. In modern society, where information is abundant, it is extremely important not to take presented numbers at face value, but to deeply understand the intentions and methodologies behind them.Statistical graphs that manipulate impressionsManipulation of graph axesBy manipulating the scale of a graph's axes, the same data can produce dramatically different visual effects. For example, when the vertical axis range is set narrowly, even slight changes are emphasized as major fluctuations. If a company's quarterly sales increased by only 1%, but the graph's vertical axis is set from 0% to 2%, it would give the impression that sales skyrocketed. Conversely, if the vertical axis is set from 0% to 100%, that 1% increase becomes almost imperceptible. In this way, the choice of axes alone can exaggerate or underestimate the importance of data.Inappropriate graph format selectionPie charts are suitable for showing proportions of a whole, but are unsuitable when comparing many items. For example, even if multiple product lines' sales trends are shown in a pie chart, it's difficult to grasp each product's growth rate or relative changes. In such cases, bar graphs or line graphs more clearly convey changes over time and comparisons between items. Additionally, three-dimensional graphs that don't actually show data relationships, or graphs with unnecessary decorations, are sometimes used to create visual impact. These aim to give a specific impression rather than accurately conveying information.Misleading averagesThe average is a representative statistic showing the central tendency of data, but using it without understanding its types and characteristics can cause misunderstandings. Particularly when data distribution is skewed, the common "average" may not represent reality.The pitfall of the arithmetic meanWhat we typically think of when we hear "average" is, in most cases, the arithmetic mean. This is calculated by adding all data together and dividing by the number of data points. The arithmetic mean is widely used because it's easy to calculate and understand, but it has a major weakness: it is strongly influenced by outliers—values that are significantly distant from other data.Let's consider a concrete example. Below is monthly salary data for 10 employees working at an IT startup:Employee A: ¥300,000 Employee B: ¥300,000 Employee C: ¥300,000 Employee D: ¥300,000 Employee E: ¥300,000 Employee F: ¥350,000 Employee G: ¥350,000 Employee H: ¥400,000 Employee I: ¥500,000 Employee J (CEO): ¥3,000,000Calculating the arithmetic mean of these 10 monthly salaries: (300,000 + 300,000 + 300,000 + 300,000 + 300,000 + 350,000 + 350,000 + 400,000 + 500,000 + 3,000,000) yen / 10 people = 6,100,000 yen / 10 people = ¥610,000.If this company announced "our employees' average monthly salary is ¥610,000," outsiders might get the impression that "many employees receive fairly high salaries." However, in reality, 9 out of 10 employees receive far less than the average monthly salary of ¥610,000. In this case, the outlier of the CEO's extremely high monthly salary greatly inflates the arithmetic mean, creating a number far removed from most employees' reality.The reality shown by the medianWhen data has significant skew or outliers like this, the median shows an "average" closer to reality. The median is the value positioned exactly in the middle when data is arranged in ascending (or descending) order. When the number of data points is even, the median is the arithmetic mean of the two central values.Arranging the employee salary data above in ascending order:¥300,000, ¥300,000, ¥300,000, ¥300,000, ¥300,000, ¥350,000, ¥350,000, ¥400,000, ¥500,000, ¥3,000,000In this case, since the number of data points is 10 (even), we take the arithmetic mean of the 5th and 6th values (¥300,000 and ¥350,000 respectively) as the median. Median = (¥300,000 + ¥350,000) / 2 = ¥325,000.This median of ¥325,000 is very close to the salary level most employees receive, and reflects reality far better than the arithmetic mean of ¥610,000. While the arithmetic mean speaks to "overall wealth including some wealthy individuals," the median suggests "the typical income level where most people are positioned."Using the modeFurthermore, the value that appears most frequently in data is called the mode. In the example above, the monthly salary of ¥300,000 appears 5 times, making it the mode. The mode is also useful in showing "the most common value" in a group, but when data has large variation or multiple peaks, its meaning as a representative value can diminish.In this way, even when the word "average" is simply used, by understanding which type of average it refers to and what shape the data distribution takes, you can discern the truth of the message the numbers convey.Confusing correlation with causationStatistical data often shows that two events occur simultaneously—that is, a correlation. However, just because there's a correlation doesn't mean one is the cause of the other—that is, there isn't necessarily a causal relationship. Ignoring this distinction is a major cause of drawing incorrect conclusions.For example, suppose a correlation is observed where ice cream sales and drowning accidents both increase in summer. Even if data from a summer month shows ice cream sales increased 20% from the previous month while drowning accidents increased 15%, concluding from this data alone that "eating ice cream makes you more likely to have a drowning accident" is clearly wrong. In reality, both are phenomena caused by a common factor—"rising temperatures"—and ice cream consumption itself is not a direct cause of drowning accidents.Additionally, even if data shows that graduates of a certain university have high average annual incomes, this doesn't necessarily mean only that the university's educational quality is high. It's necessary to consider the possibility that students entering that university already have high potential abilities or privileged family backgrounds. Thus, it's important to identify the true factors behind apparent correlations and the existence of confounding variables.Sampling bias in statisticsWhen conducting statistical surveys, we extract and analyze a portion from the entire survey target (population). This extracted portion is the sample. However, depending on how this sample is selected, results may not accurately reflect the population's characteristics. This is sampling bias.A famous example is the 1936 U.S. presidential election prediction failure by "Literary Digest" magazine. This magazine sent questionnaires by mail to its readers and predicted Landon's overwhelming victory based on responses. Specifically, they compiled 2.4 million responses and predicted Landon would win with 57% versus Roosevelt's 43%. However, Roosevelt actually won in a landslide with 60.8% of the vote. The cause of this failure was that the questionnaire recipients were extracted from telephone directories and automobile registration lists. In America at that time, only relatively wealthy people could own telephones and automobiles, so the opinions of the general public suffering during the economic downturn were not adequately reflected. This is a典型的な example of how sampling biased toward the wealthy greatly distorted election result predictions.Self-selection bias is also a common problem. This is bias that arises when survey participants choose to participate themselves. For example, in a product satisfaction survey, if only people who are very satisfied or very dissatisfied tend to respond, neutral opinions are underrepresented.To avoid such biases, it's essential to extract samples using statistically appropriate methods such as random sampling or stratified sampling.Misunderstanding statistical significanceIn scientific papers and news articles, we often see the expression "a statistically significant difference was found." This means that the observed effect can be said to exist with some degree of reliability, rather than being due to chance. However, being "statistically significant" doesn't necessarily mean being "practically important."For example, suppose a new drug statistically significantly improves patients' symptoms compared to an existing drug. This shows that the improvement is not a chance result, but whether the degree of improvement is substantial enough to significantly impact patients' quality of life is a separate issue. Even if symptom improvement is only 1%, in a large-scale trial with 10,000 subjects, it could be detected as a statistically significant difference. However, whether that 1% improvement is truly meaningful to patients needs to be considered from clinical judgment and cost-effectiveness perspectives. If that drug is significantly more expensive than the existing drug, whether it's worth the cost for just a 1% improvement is debatable.Also, attention must be paid to a practice called p-hacking. This is the act of trying various analysis methods or adding data until statistically significant results are obtained. This increases the risk of accidentally finding relationships that don't actually exist.Specific examples of p-hackingP-hacking refers to the act of adjusting data analysis or collection methods until favorable results are obtained (in most cases, statistically significant results, meaning p-values below 0.05), whether intentionally or unconsciously by researchers. This creates the danger that effects that don't actually exist, or have only minimal effects, are reported as "scientific discoveries."Example 1: Repeatedly analyzing while collecting data Suppose a researcher begins an experiment with 100 students to study the effectiveness of a new learning method. Even though data collection is incomplete, the researcher analyzes once when data from 50 people is gathered. At that point, no statistically significant difference is found. So they add more data and analyze again with 75 people's data, but again no significant difference appears. However, when analyzing again after collecting all 100 people's data, the p-value happens to be 0.048, yielding a statistically significant result. The researcher publishes this result as a paper, but in reality, this learning method may not have had a clear effect—the significant value may have simply appeared by chance from repeatedly reanalyzing data. Properly, analysis should be done only once with a predetermined sample size.Example 2: Reporting only favorable indicators from among multiple ones In a study verifying the effectiveness of a health supplement, the research team measured multiple health indicators in participants: "weight," "body fat percentage," "blood pressure," "cholesterol levels," "mood," etc. Suppose they didn't clearly define before starting research which indicator changes would be considered "effective." After the study ended, when all indicators were analyzed, no statistically significant changes were found in "weight," "body fat percentage," "blood pressure," or "cholesterol levels," but only "mood" showed statistically significant improvement with a p-value of 0.03. The researchers published a paper emphasizing only this "mood improvement" and advertised "our health supplement has mood-improving effects." However, because they're concealing that many other indicators showed no change, this result may mislead about the supplement's overall effectiveness. Selecting and reporting only items that happened to be significant from among many measurement items is a典型的な example of p-hacking.Example 3: Arbitrary exclusion of outliers Suppose in a clinical trial of a new drug, researchers compared symptom improvement between a patient group given the drug and a control group given a placebo. Initial analysis showed no statistically significant difference between the groups. However, reviewing the data, the researcher noticed several patients in the drug group whose symptoms extremely worsened. Speculating these patients' symptoms likely worsened due to other diseases rather than drug side effects, they excluded those patients' data as "outliers" and reanalyzed. This time, a statistically significant improvement effect was detected, so the researcher published the results. While excluding outliers from analysis isn't always wrong, arbitrarily excluding outliers to obtain statistical significance constitutes p-hacking.As these examples show, p-hacking often has researcher bias or publication pressure in the background. To address this problem, pre-registration of research is an effective measure—publicly announcing in advance what hypotheses will be tested and what data collection and analysis methods will be used.Incidents related to statistics manipulationMany incidents where statistical manipulation had major social impact have occurred in the past. Among them, the debate over "mortality rates of children born in London" associated with Thomas Bayes, also known as the "father of Bayesian statistics," is a classic case showing how statistics can manipulate people's perceptions and lead to incorrect conclusions.In 18th-century London, with population growth, child mortality rates in urban areas became a major problem. According to the official records of the time called "Bills of Mortality," the overwhelming majority of children born in London died within a few years of birth. This number was shocking, and many people concluded that London's environment was extremely unhealthy for children. Some argued that births in London should be avoided.However, Thomas Bayes and others challenged this data. What they pointed out was that the calculation method for this "mortality rate of children born in London" had fundamental statistical bias. Specifically, this mortality table included not only children born in London, but also seriously ill children brought to London from rural areas for medical treatment. These children were often already in poor health, having fallen ill before coming to London or during the journey, so naturally their mortality rate was high.In other words, within the category of "children born in London," two groups with different characteristics were mixed: "healthy children born and raised in London" and "children transferred to London with health problems." The latter group's high mortality rate made it appear as though London's overall child mortality rate was abnormally high.Bayes and others argued that if purely measuring the mortality rate of "children born in London," the number should be far lower than what was published. Their analysis, by revealing the context hidden behind simple numbers and biases in data collection methods, helped overturn the mistaken perception widespread among people of the time that "London is a dangerous place to raise children."This incident demonstrates the importance of critically questioning what numbers mean and how they were collected when statistical data is provided. It teaches us that even seemingly objective numbers can, unintentionally or intentionally, give completely different impressions depending on the definitions, aggregation methods, or data selection behind them.How to identify statistical deceptionTo avoid being deceived by statistical data, it's essential to be conscious of several important points. Not taking presented numbers and graphs at face value, but exploring the information and intentions behind them, is the first step toward accurate understanding.Verify the source and purpose of dataFirst, check where the statistical data comes from. Credibility varies greatly depending on whether it's data from trusted research institutions, government agencies, or objective survey organizations, versus data published by specific interest groups or companies. In the latter case, special attention is needed because they may present only favorable data or guide interpretation to make their products or services appear advantageous. Also consider what purpose the data was compiled and published for.Examine data definitions and measurement methodsEven with the word "average," its meaning differs greatly depending on whether it's the arithmetic mean or median. Also, when looking at "unemployment rate" statistics, direct comparison is dangerous because the definition of "unemployed person" may differ by country or institution. For example, fine definitional differences like whether people working even one hour per week for income are excluded from the unemployed can greatly impact numbers. The higher the transparency of how data was collected and measured, the higher the reliability.Confirm comparison targets and periodsIt's also important what the presented numbers are compared to. For example, if there's a report that "sales increased 20%," the meaning changes greatly depending on whether it's compared to the previous month, the same period last year, or increase relative to a specific target. Also, if the data period is extremely short, temporary fluctuations may be emphasized, making it insufficient for reading long-term trends. Consider the possibility that only convenient periods are being cherry-picked for comparison.Examine sample size, bias, and how questions were askedWhen looking at survey results, check how many people (sample size) were surveyed and how those people were selected (sample bias). Results from a small, biased sample cannot represent the whole. Also important is whether the way questions were asked was leading respondents in a particular direction. Leading questions like "What do you think of this wonderful new product?" make it difficult to obtain objective opinions.Distinguish correlation from causationJust because two events occur simultaneously doesn't mean one causes the other. This is one of statistics' most common traps. Even if data shows "correlation between a product's sales increase and disease incidence rate increase," you need to carefully judge whether that correlation is mere coincidence or caused by a third hidden factor.Look for missing informationPay attention not only to presented information, but also to information that may be intentionally hidden. For example, if only success cases are emphasized while failure cases or inconvenient data are never presented, it may be distorting the full picture. Consider whether balanced information is provided and data is presented from multiple perspectives.Statistics are powerful tools that provide essential information for decision-making. However, without the ability to understand and correctly interpret that power, there's also risk of being led to incorrect conclusions. Having critical thinking and an attitude of exploring the story behind presented numbers are the most important keys to not being deceived by statistics.ReferencesHow to Lie with Statistics: An Introduction to Statistics for Avoiding Deception, Joel BestHow to Lie with Statistics, Darrell HuffHow to Make the World Add Up: Ten Rules for Thinking Differently About Numbers, Tim HarfordStatistics That Actually Lie, Takashi KadokuraPeople Often Lie with Statistics, Jean-Paul FitoussiData Analysis Lies, Mario Riva, Patrizia RivaIntroduction to Statistical Thinking, Masahiro KaminagaSix Statistics That Changed the World, Gavin Whitehead