I just received my copy of Introduction to statistical inference, by Jack C. Kiefer (Springer, 1987).¹ After having read the first two chapters I wonder: How come I didn’t start with that book when I was studying elementary statistics!

I recently had to give a short series of lectures on statistics (at a very introductory level) where I started with an illustration of a coin tossing experiment (starting at slide #7). Well, now that I have read Kiefer’s first chapter I feel like the take-away message was already there:

A typical problem in probability theory is of the following form: A sample space and underlying function are specified, and we are asked to compute the probability of a given chance event. (…) In a typical problem of statistics it is not a single underlying probability law which is specified, but rather a class of laws, any of which may possibly be the one which actually governs the chance device or experiment whose outcome we shall observe. We know that the underlying probability law is a member of this class, but we do not know which one it is. The object might then be to determine a “good” way of guessing, on the basis of the outcome of the experiment, which of the possible underlying probability laws is the one which actually governs the experiment whose outcome we are to observe.

And the ensuing discussion (including chapter 2) drives the reader toward the specification of a statistical problem based on a simple coin experiment.