3 The Spread or Dispersion of a Batch (Ch. 3)
## X....Context Weight
## Length:25 Min. : 7.60
## Class :character 1st Qu.: 9.80
## Mode :character Median :11.30
## Mean :11.85
## 3rd Qu.:12.90
## Max. :28.60#Note that our column names (for 'flakes') have gone a bit funny
#but those are easy to fix
colnames(flakes) #examines existing ## [1] "X....Context" "Weight"colnames(flakes) <- c("Context", "Weight")
#overwrites; note that these change only in our R object, *not* in the original .txt file. If you want to write out an R object as file for use elsewhere, that’s easily accomplished:
Remember some of the techniques we’ve previously used to explore data; they’re applicable here also.
#subset flakes by pit
Pit1 <- subset(flakes, Context == "Pit 1")
Pit2 <- subset(flakes, Context == "Pit 2")
#reorder by weight
Pit1 <- Pit1[order(Pit1$Weight, decreasing = T),]
Pit2 <- Pit2[order(Pit2$Weight, decreasing = T),]Back to exploring data as Drennan does:
## flakes$Context: Pit 1
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 7.600 9.875 11.100 12.325 12.075 28.600
## ------------------------------------------------------------
## flakes$Context: Pit 2
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 7.80 9.80 11.30 11.42 13.50 14.30## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 7.600 9.875 11.100 12.325 12.075 28.600## [1] 7.6 28.6## [1] 2.2Note that quartiles can be calculated in various ways - see ?IQR - so will vary for small samples. The result for Pit 1 will not quite match Drennan’s.
## [1] 3.7The functions var() and sd() will give you the variance and the standard deviation. You can also calculate these from scratch following Drennan.
To recreate Drennan’s Table 3.2, we can take advantage of the ease with which R lets us repeat an operation for an entire vector.
First we calculate the deviations from the mean (wrapping that calculation in round() in order to round the result to 2 decimal places). Note that by specifying a new column name (“Deviations”) we create that column and populate it at the same time.
Pit2$Deviations <- round((Pit2$Weight - mean(Pit2$Weight)), 2)
#Then calculate the squared deviations, using the new "Deviations" column
Pit2$SquaredDeviations <- round(((Pit2$Deviations)^2), 2)
Pit2 #Our Pit2 object now resembles Drennan's Table 3.2## Context Weight Deviations SquaredDeviations
## 22 Pit 2 14.3 2.88 8.29
## 15 Pit 2 14.1 2.68 7.18
## 23 Pit 2 13.6 2.18 4.75
## 16 Pit 2 13.5 2.08 4.33
## 18 Pit 2 12.0 0.58 0.34
## 21 Pit 2 11.5 0.08 0.01
## 13 Pit 2 11.3 -0.12 0.01
## 25 Pit 2 10.9 -0.52 0.27
## 20 Pit 2 10.6 -0.82 0.67
## 14 Pit 2 9.8 -1.62 2.62
## 17 Pit 2 9.7 -1.72 2.96
## 24 Pit 2 9.3 -2.12 4.49
## 19 Pit 2 7.8 -3.62 13.10If you want to make sure that our calculations match Drennan’s, calculate the bottom row (we won’t add these values to Pit2, but can easily inspect them)
## [1] 11.41538## [1] -0.06## [1] 49.02## [1] 4.085## [1] 4.08641#by taking the square root of the variance we can derive the standard deviation
sqrt(sum(Pit2$SquaredDeviations) / (nrow(Pit2) - 1)) ## [1] 2.021138## [1] 2.021487Calculating the trimmed standard deviation requires deriving a Winsorized batch. This is not available in Base R (that is, R as loaded), but is available in the psych package.
This uses a slightly different method of Winsorizing than Drennan (replacing the trimmed values with the quartiles defined by the trim), so to approximate his results we use a 9% rather than a 5% trim. We can then, following Drennan, calculate the Winsorized variance.
# uncomment the below if needed
Pit1$Winsorized <- winsor(Pit1$Weight, trim = .09)
Pit1$WinsorizedDeviations <- round((Pit1$Winsorized - mean(Pit1$Winsorized)), 2)
Pit1$WinsorizedSquaredDeviations <- round(((Pit1$WinsorizedDeviations)^2), 2)
#calculate Winsorized variance
sum(Pit1$WinsorizedSquaredDeviations) / (nrow(Pit1) - 1)## [1] 3.374545#and the trimmed standard deviation
sqrt(((nrow(Pit1) - 1) * sum(Pit1$WinsorizedSquaredDeviations) /
(nrow(Pit1) - 1)) / (nrow(Pit1) - 3))## [1] 2.030873## [1] 1.304231The data you’ll need for the Ch.3 Practice is available as a .txt file. Before we explore the data, let’s use it to explore how to index data in R using the [ operator.
When using [, the comma separates row and column indices, so [2,3] specifies ‘Row 2, Column 3’; [,-1] specifies all rows, and drops (‘-’) column 1.
nanxiong <- read.delim("data/Drennan_datasets/Nanxiong.txt", header = T)
head(nanxiong) #use `head()` to look at the data without printing the whole table## Row Period Area
## 1 1 Early Bronze Age 1.8
## 2 2 Early Bronze Age 1.0
## 3 3 Early Bronze Age 1.9
## 4 4 Early Bronze Age 0.6
## 5 5 Early Bronze Age 2.3
## 6 6 Early Bronze Age 1.2When you examine the file you’ll note that it has an extraneous leading column (row numbers). It’s not doing any harm, but provides an opportunity to learn how to index data in R. Remove that extraneous column by selecting the other columns using the [ operator.
# '[' selects from an R object
nanxiong[,c(2,3)] # columns 2 and 3
#or
nanxiong[,2:3] # columns 2 through 3
#or
nanxiong[,-1] # not column 1
#or, alternatively by column name
nanxiong <- nanxiong[c("Period","Area")]You could also specify this when importing the file
## Period Area
## 1 Early Bronze Age 1.8
## 2 Early Bronze Age 1.0
## 3 Early Bronze Age 1.9
## 4 Early Bronze Age 0.6
## 5 Early Bronze Age 2.3
## 6 Early Bronze Age 1.2
## 7 Early Bronze Age 0.8
## 8 Early Bronze Age 4.2
## 9 Early Bronze Age 1.5
## 10 Early Bronze Age 2.6
## 11 Early Bronze Age 2.1
## 12 Early Bronze Age 1.7
## 13 Early Bronze Age 2.3
## 14 Early Bronze Age 2.4
## 15 Early Bronze Age 0.6
## 16 Early Bronze Age 2.9
## 17 Early Bronze Age 2.0
## 18 Early Bronze Age 2.2
## 19 Early Bronze Age 1.9
## 20 Early Bronze Age 1.1
## 21 Early Bronze Age 2.6
## 22 Early Bronze Age 2.2
## 23 Early Bronze Age 1.7
## 24 Early Bronze Age 1.1
## 25 Late Bronze Age 10.4
## 26 Late Bronze Age 5.9
## 27 Late Bronze Age 12.8
## 28 Late Bronze Age 4.6
## 29 Late Bronze Age 7.8
## 30 Late Bronze Age 4.1
## 31 Late Bronze Age 2.6
## 32 Late Bronze Age 8.4
## 33 Late Bronze Age 5.2
## 34 Late Bronze Age 4.5
## 35 Late Bronze Age 4.1
## 36 Late Bronze Age 4.0
## 37 Late Bronze Age 11.2
## 38 Late Bronze Age 6.7
## 39 Late Bronze Age 5.8
## 40 Late Bronze Age 3.9
## 41 Late Bronze Age 9.2
## 42 Late Bronze Age 5.6
## 43 Late Bronze Age 5.4
## 44 Late Bronze Age 4.8
## 45 Late Bronze Age 4.2
## 46 Late Bronze Age 3.0
## 47 Late Bronze Age 6.1
## 48 Late Bronze Age 5.1
## 49 Late Bronze Age 6.3
## 50 Late Bronze Age 12.3
## 51 Late Bronze Age 3.9Note that you would read this command as “select columns 2 through 3 of this file”, which is subtly different than [,c(2,3)] - “select columns 2 and 3”.