Descriptive statistics

The essentials

Pandas has built-in functions that summarise data in a Series or DataFrame. For a DataFrame they work column by column by default (axis = 0) and return one value per column; with axis = 1 they work row by row. They ignore NaN values unless told otherwise.

Key points

• max(), min(): largest and smallest values.
• count(): number of non-NaN values.
• sum(): total of the values.
• mean(): average. median(): middle value after sorting.
• mode(): most frequent value(s); it returns a Series because there may be more than one mode.
• quantile(q): the value below which a fraction q of the data lies. quantile(0.25) is the first quartile, quantile(0.5) is the median and quantile(0.75) is the third quartile.
• var(): variance; std(): standard deviation. By default Pandas uses the sample formula, dividing by n − 1. For 2, 4, 4, 4, 5, 5, 7, 9, var() gives 32 ÷ 7 ≈ 4.571 and std() ≈ 2.138.
• describe(): gives count, mean, std, min, quartiles and max in one table.
• Example: for df with P = [3, 1] and Q = [4, 2], df.max() gives P 3, Q 4, while df.max(axis=1) gives 4 and 2 (row-wise).

Exam tips

• Note the difference from the statistics module's pvariance(), which divides by n.
• For a Series with a NaN, count() is smaller than size and mean() uses only the non-NaN values.

Common mistakes

• Expecting mode() to return a single number.
• Forgetting that a DataFrame function returns column-wise results by default.