11 Categories and Population Proportions (Ch. 11)

In Ch. 11 Drennan looks at standard deviation and standard error of proportions (using the example, from Ch.9, of the 100 projectile points of which 13 are obsidian). We can make a series of functions for these calculations, partly for practice with functions and partly to follow the logic of generating confidence intervals around proportions.

make a function for standard deviation of a proportion (\(s = \sqrt{pq}\)), where p = proportion and q = 1-p.
make a function for standard error of a proportion (\(SE = \frac{\sigma}{\sqrt{n}}\)), where since the population standard deviation (\(\sigma\)) is unknown, we use the sample standard deviation (s, calculated in Step 1) and n = sample size.

SDP <- function(p, q) {sqrt(p*q)}  # function for standard deviation of a proportion
SDP(p = .13, q = .87)

## [1] 0.3363034

SEP_v1 <- function(s, n) {s/sqrt(n)}   # function for standard error of a proportion
SEP_v1(s = .3363, n = 100)

## [1] 0.03363

#combine these functions and calculate q in the function
SEP <- function(p, n) {(sqrt(p*(1-p))) / sqrt(n)}
SEP(p = .13, n = 100)

## [1] 0.03363034

The result (.0336) can be used as 1 standard error range - i.e. we can be about 66% confident that 13% ± 3.4% is the population proportion of obsidian points.

We might want to be more than 66% confident, and in any case it probably is more practical to be able to calculate the confidence interval for a specific desired confidence level (e.g., “I’d like to know the proportion with 90% certainty.”). To that end we can build another function, where we adjust an error range for a desired confidence interval using Student’s t (using qt() as in Ch.9). Here SEP = standard error proportion, ci = desired confidence interval, and n = sample size. Confusingly, we now

SEPconf <- function(SEP, ci, n) {SEP * (qt((ci+1)/2, df = (n-1)))} 
SEPconf(SEP = .0336, ci = .95, n = 100)

## [1] 0.06666969

Note that our function requires the standard error proportion as input. We can make something more intuitive to use by instead including the calculation for SEP (see above) in the function. Here we calculate the SEP adjusted for the desired confidence interval, where p = proportion, n = sample size, and ci = desired confidence interval.

SEPcon <- function(p, n, ci) {((sqrt(p*(1-p)))/sqrt(n)) * (qt((ci+1)/2, df = (n-1)))}   
SEPcon(p = .13, n = 100, ci = .95)

## [1] 0.0667299

…and why not include the finite population corrector (see Ch.9)? First we revisit our function to calculate the standard error of the mean with a finite population correction (fpc): for the standard error of the mean we used (\(SE = \frac{\sigma}{\sqrt{n}}\sqrt{1 - {\frac{n}{N}}}\)), where s is the sample standard deviation (since the population standard deviation is unkown), n = sample size, and N = population size.

Because we are dealing with proportions, when we substitute s for \(\sigma\), we use \(s = \sqrt{pq}\). We can then include that in a combined function that calculates standard error of a proportion with a finite population correction, where p = proportion, n = sample size, N = population size, and ci = desired confidence interval: \[SE = \sqrt{\frac{pq}{n}(1 - \frac{n}{N})}\].

fpc <- function(s, n, N) {s/sqrt(n)*sqrt(1-n/N)}  
SEPfin <- function(p, n, N, ci) {((sqrt(p*(1-p)))/sqrt(n)) * sqrt(1-(n/N))*(qt((ci+1)/2, 
                                                                             df = (n-1)))} 
#Drennan's house door example (p141):
SEPfin(p = .353, n = 17, N = 24, ci = .9)

## [1] 0.1092883

As with estimating population means, we can reverse-engineer the process, and calculate how large a sample we need in order to estimate population proportions with a particular degree of confidence. The basic equation (\(n=\left(\frac{\sigma t}{ER}\right)^2\)) is the same as for population means; we just use the standard deviation of a proportion for \(\sigma\). As above we have to use the sample standard deviation (s) instead of the population standard deviation (\(\sigma\)), and as above for the standard deviation of a proportion \(s=\sqrt{pq}\). As Drennan (p142) notes, if we know nothing about the proportion in the population of the target category, the most conservative estimate results from assuming that it makes up 50% of the population.
So, we build a function for \(n=\left(\frac{\sqrt{pq}t}{ER}\right)^2\), where p = proportion expected in the population, t = the t statistic for a desired confidence interval, ER = desired confidence interval. In other words, we specify the population proportion we expect and the confidence with which we’d like to be able to estimate the proportion, and our function will return the necessary sample size.

SampSiz <- function(p, ci, ER) {round(((sqrt(p*(1-p)) * (qt(((ci+1)/2), 
                                                  df=Inf)))/ER)^2, 0)}
SampSiz(p = .5, ci = .95, ER = .05)

## [1] 384