Descriptive Statistics

Descriptive statistics are methods used to summarize and describe the main features of a dataset. Instead of looking at individual data points, we use these techniques to get a clear overview of the data's characteristics. This is a crucial first step in any data analysis process. Think of it like taking a snapshot of your data before you dig deeper. The main types of descriptive statistics we will cover today are measures of central tendency and measures of dispersion.

These measures tell us where the 'center' of the data lies. The three most common measures are:

• Mean: The average. Calculated by summing all the values in a dataset and dividing by the number of values. It is sensitive to outliers (extreme values).

  • Example: Dataset: 2, 3, 3, 5, 7. Mean = (2+3+3+5+7) / 5 = 4.

• Median: The middle value when the data is sorted in ascending order. Less sensitive to outliers.

  • Example: Dataset: 2, 3, 3, 5, 7. Median = 3.

• Mode: The value that appears most frequently in the dataset. A dataset can have no mode, one mode, or multiple modes.

  • Example: Dataset: 2, 3, 3, 5, 7. Mode = 3.

Let's apply this in code:

import statistics

data = [2, 3, 3, 5, 7]

# Calculate mean
mean_value = statistics.mean(data)
print(f"Mean: {mean_value}")

# Calculate median
median_value = statistics.median(data)
print(f"Median: {median_value}")

# Calculate mode
mode_value = statistics.mode(data)
print(f"Mode: {mode_value}")

#If there is no mode the statistics.mode(data) will throw an error.
#Example
#data_no_mode = [1, 2, 3, 4, 5]
#print(statistics.mode(data_no_mode)) # Throws StatisticsError

These measures describe how spread out the data is. They tell us about the variability or consistency of the data.

• Range: The difference between the highest and lowest values in the dataset. Simple but easily affected by outliers.

  • Example: Dataset: 2, 3, 3, 5, 7. Range = 7 - 2 = 5.

• Variance: A measure of how far each number in the dataset is from the mean. Calculated as the average of the squared differences from the mean. Higher variance means greater spread. The steps are:

  1. Calculate the mean.
  2. Subtract the mean from each data point.
  3. Square each of these differences.
  4. Sum the squared differences.
  5. Divide by the number of data points (for population variance) or number of data points minus one (for sample variance).
     * Example (simplified): Dataset: 2, 3, 3, 5, 7. Mean = 4. Variance = ( (2-4)^2 + (3-4)^2 + (3-4)^2 + (5-4)^2 + (7-4)^2 ) / 5 = 4

• Standard Deviation: The square root of the variance. It provides a more interpretable measure of spread because it's in the same units as the original data. A higher standard deviation means the data is more spread out.

  • Example: If Variance = 4, then Standard Deviation = √4 = 2.

Let's use code:

import statistics

data = [2, 3, 3, 5, 7]

# Calculate range (this requires basic math)
range_value = max(data) - min(data)
print(f"Range: {range_value}")

# Calculate variance
variance_value = statistics.variance(data)
print(f"Variance: {variance_value}")

# Calculate standard deviation
std_dev_value = statistics.stdev(data)
print(f"Standard Deviation: {std_dev_value}")

The best measure to use depends on your data and what you want to learn. The mean is great for general understanding, but can be skewed by outliers. The median is robust to outliers, making it a better choice for data with extreme values, like income or house prices. The mode helps you identify the most common value. The standard deviation helps you understand the data's variability, and variance is also a useful value but less intuitive than standard deviation.