Guide to sampling error

Introduction

BARB viewing data are results from a sample survey, and are therefore subject to sampling variation (also known as sampling error). Published viewing figures represent an estimate of the actual size of the audience, and may be thought of as the best estimate that can be made from a range of possible values. The lower and upper limits of this range of values can be defined in a confidence interval.

Sampling variation depends on a number of factors, including sample size, weighting, survey design and whether or not data is averaged over time. The confidence limits given in the spreadsheet are the result of modelling work carried out by RSMB, and aim to account for all these factors. The spreadsheet shows approximate 95% limits, meaning that there is roughly a 95% chance that the true audience figure lies between the lower and upper limits of the confidence interval.

Results are given at the Network and ITV Area level for twenty target audiences, and are available for the following types of audience estimate:

  • Single minutes or commercial breaks
  • Single minutes or breaks averaged over 4 weeks
  • Half-hour programmes
  • Half-hour programmes averaged over 4 weeks

How to use the system

Please note that the spreadsheet will only function if macros are enabled. If your browser does not prompt you to enable the macros and the spreadsheet is blank when you open it you will need to save the file to your desktop – right click the link and choose ‘Save Target As’.

Open / Download the Sampling Error spreadsheet (Subcribers only)

The area and type of audience estimate for which a confidence interval is desired are specified on the ‘Inputs’ sheet. After clicking on the ‘Get Sampling Errors’ button, the ‘Sampling Error Output’ sheet then gives estimated 95% confidence limits for a number of different levels of reported audience across the twenty target groups.

The table on the ‘Sampling Error output’ sheet is in three sections. The top section shows example reported audiences expressed as a TVR at the top, and then as an equivalent in 000s for the twenty target audiences below. The middle section gives estimated 95% confidence limits in 000s for these reported audiences. The lower section gives the same confidence limits expressed as a TVR.

Example 1

For a single minute reported on the Network, the 95% confidence interval for a reported audience for Men of 2 TVR is 2 ± 0.5, i.e. 1.5 – 2.5 TVR. This is equivalent to a reported Network Men audience of 482,000 and 95% confidence interval of 482,000 ± 118,000, i.e. 364,000 – 600,000.

Example 2

For a half-hour programme averaged over 4 weeks reported on the Network, the 95% confidence interval for a reported audience of 0.5 Individuals TVR is 0.5 ± 0.1, i.e. 0.4 – 0.6 TVR. This is equivalent to a reported programme audience of 292,000 with 95% confidence interval of 292,000 ± 64,000, i.e. 228,000 – 356,000.

Note on confidence limits for small audiences reported on small samples

The confidence limit estimates given by the system assume that sampling errors follow the Normal probability distribution. For very small audiences reported on small sample sizes this assumption becomes tenuous, as illustrated by the following example.

Example 3

For a single minute or commercial break in the Yorkshire region, the system gives the 95% confidence interval for a reported audience of 2 TVR for Housewives with Children as 2 ± 2.7 TVR. Clearly it is impossible for the lower confidence limit to be less than zero. However the estimate for the upper limit may still be reasonable, so the estimated confidence interval in this case can be given as approximately 0 to 4.7 TVR. Note that in this case the reported audience (2 TVR) is no longer in the middle of the interval, but should still be regarded as the appropriate audience estimate.