Statistics Terms and Concepts for Everyone

Key statistics terms and concepts in layperson’s terms!

Statistics has a reputation of being confusing, and it can be hard to truly understand your organization’s data without the knowledge of these foundational concepts. These terms and concepts are integral for day-to-day meetings, business reviews, and product discussions for tech, non-tech, and executive leaders alike. I even had to do a big refresher on these as an analyst intern. I hope this guide makes learning them as simple as possible! I will create another blog on the application of some of these concepts using actual data.

Control and Treatment Groups

A control group either receives a placebo, does not receive any treatment, or receives a standard treatment and serves as the baseline. A treatment group does receive one or more experimental treatments in the study. The control group’s purpose is to serve as the comparison to the treatment group, to see if there was any effect.

A/B Test (Also Called Split Test)

A/B tests compare two different versions of something, such as an existing webpage and that same webpage with an additional enhancement. Users will be randomly assigned to each webpage version. It is not the same thing as control and treatment groups, as control and treatment groups can be used in other experiment types besides A/B tests such as a medical trial.

Null Hypothesis

There is no relationship between the variables being measured.

P-Value

A p-value is the probability that your data occured by random chance (or that the null hypothesis is true/there is no relationship between the variables measured). A general rule of thumb is that the p-value is statistically significant if it is less than 0.05, though the p-value fluctuates based on the sample size and the effect size and should be determined as statistically significant based on the specific experiment parameters.

Confidence Interval

A Confidence Interval is a range of values that has a likelihood of containing the true value. For example, a 95% Confidence Interval would mean that we are 95% confident that the true value lies within the specified range. The most popular Confidence Intervals used are 90%, 95%, and 99%; they are often used for forecasts and chosen based on how large of a margin of error is acceptable (95% would mean a 5% margin of error, etc.).

Percentiles

A percentile is a comparison of one specific value to the rest of the values in a group. The percentage itself showcases the percentage of values falling below that specific value. For example, if a classroom of students took a test and one student (let’s call them Bobby) was in the 80th percentile, Bobby’s test score was greater than or equal to 80% of the classroom’s test scores. That also means that 20% of the classroom scored greater than or equal to Bobby’s score.

Standard Deviation

It is a calculation of how far the data points tend to be from the mean/average. The calculation is the average of squared deviations from the mean/average. A low standard deviation means the data is very close to the mean/average, while a high standard deviation means the data is further from the mean/average. It is often represented by the symbol “σ”.

T-Test

A t-test is one of the most common statistical tests performed. It simply compares the means of two groups, and is popular to use in hypothesis testing with a smaller sample size (you will want to use one of the different tests below if you have a large sample size).

Z-Test

A z-test is another very common statistical test, for larger sample sizes. It either compares a specified value and the mean of a group, or it compares the means of two groups when the variance/standard deviation is known.

Chi-Square Test (Also χ2 Test)

A chi-square test is very popular for showcasing whether two variables are independent of one another.

Two-Sample Kolmogorov-Smirnov Test

This is a very useful statistical test to show if two sample groups come from the same distribution.