## Tuesday, November 12, 2013

### Quick correction in simulating Typical Error

Quick correction in simulating Typical Error

# Quick correction in simulating Typical Error

I am awaiting review and opinions/critiques from statistics wizzard Will Hopkins for my How to visualize test change scores for coaches, but even on his first look he pointed to one small error I did in simulating typical error for vertical jump. It is not huge error, but it is an error in representing typical error in measurement.

### What I did wrong?

This is what I did wrong (for this example I will generate 1000 athletes/samples)

# Load library for random names generation for athlete
library(randomNames)

# Simulate vertical jump scores for 1000 players, with an average of 55cm
# and SD of 6cm We are going to repeat the assessment on two days in the
# same conditionins and add some 'noise'
set.seed(505)

reliabilityTest1CM_old <- rnorm(1000, mean = 55, sd = 6)
reliabilityTest2CM_old <- reliabilityTest1CM_old + runif(1000, min = -2, max = 2)

# Let's name those 1000 athletes
reliabilityPlayerNames_old <- randomNames(1000)

# Put the data in data frame
reliabilityData_old <- data.frame(Athlete = reliabilityPlayerNames_old, Test1 = reliabilityTest1CM_old,
Test2 = reliabilityTest2CM_old)

# Calcualte the differences between tests
reliabilityData_old <- within(reliabilityData_old, Diff <- Test2 - Test1)

Here are our first 10 athletes from the table

Athlete Test1 Test2 Diff
1 O'Connor, Joshua 48.27 49.04 0.76
2 Matthews, Tisa 47.31 46.94 -0.37
3 Obi, Victoria 42.76 42.88 0.12
4 Leal, Alexandra 49.37 48.09 -1.28
5 Salazar, Ana 51.94 53.61 1.68
6 Nunez-Medina, Makeyla 52.52 53.17 0.65
7 Shin, Nicholas 49.54 49.81 0.28
8 Smith, Bruce 55.84 57.57 1.73
9 Bayarsaikhan, Gabriel 58.39 58.10 -0.29
10 Yarangsy, Miles 52.41 52.13 -0.27

What I did wrong is that I have added uniform noise to the first test

reliabilityTest1CM_old <- rnorm(1000, mean=55, sd=6)

reliabilityTest2CM_old <- reliabilityTest1CM_old + runif(1000, min=-2, max=2)

What I should have done to simulate this more methodically and correctly is to create real test scores and add normal “noise” (TE) to BOTH of them. Here is the right code

set.seed(505)

# Create the 'REAL' scores in vertical jump with mean=55 and SD=6 for 1000
# athletes
realVerticalJumpCM <- rnorm(1000, mean = 55, sd = 6)

# Now add 'noise' (TE) to both reliability tests. In this case noise has
# mean=0 and SD=1. If mean of the TE would be >0 then that would be a
# constant error.

reliabilityTest1CM <- realVerticalJumpCM + rnorm(1000, mean = 0, sd = 1)
reliabilityTest2CM <- realVerticalJumpCM + rnorm(1000, mean = 0, sd = 1)

# Let's name those 1000 athletes
reliabilityPlayerNames <- randomNames(1000)

# Put the data in data frame
reliabilityData <- data.frame(Athlete = reliabilityPlayerNames, Test1 = reliabilityTest1CM,
Test2 = reliabilityTest2CM)

# Calcualte the differences between tests
reliabilityData <- within(reliabilityData, Diff <- Test2 - Test1)
Athlete Test1 Test2 Diff
1 Martinez, Kuan-Hsuen 48.77 47.65 -1.12
2 Earvin, Colton 47.38 46.84 -0.54
3 Vigil, Analee 44.16 43.63 -0.53
4 Bruce, Victoria 49.55 49.44 -0.11
5 Slechter, Joshua 51.76 52.26 0.50
6 Kaeser, Lnay 52.35 54.24 1.89
7 Shibles, Kevin 50.82 49.11 -1.70
8 Russell, Jonathan 54.21 56.85 2.64
9 Chavez, Selena 58.96 58.21 -0.75
10 Saldivar, Ebonnie 51.09 52.33 1.24

Here is the histogram of difference scores between test1 and test2 in both methods

par(mfrow = c(1, 2))

hist(reliabilityData_old\$Diff, 30, col = "red", main = "Old method with uniform error",
xlab = "Vertical Jump difference [cm]")

hist(reliabilityData\$Diff, 30, col = "blue", main = "Correct method wih normal error",
xlab = "Vertical Jump difference [cm]")

Let's calculate TE for both simulations. I need to copy paste our function for TE calculus first.

typicalError <- function(data, test1 = "Test1", test2 = "Test2", na.rm = TRUE) {
# Remove the missing values
if (na.rm)
data <- na.omit(data)

# Calculate the difference from named collums
diff <- data[[test2]] - data[[test1]]

return(sd(diff)/sqrt(2))
}

# Calculate Typical Errors for both simulations
TE_old <- typicalError(reliabilityData_old)
TE_correct <- typicalError(reliabilityData)

cat("Simulation #1 with uniform error\nTE=", round(TE_old, 3), "\n")
## Simulation #1 with uniform error
## TE= 0.822
cat("\nSimulation #2 with normal error\nTE=", round(TE_correct, 3), "\n")
##
## Simulation #2 with normal error
## TE= 0.987

As you can see, TE in correct method of simulation (TE=0.987) is VERY close to SD=1 of our normally distributed noise we have added to the “real” vertical jump.

I hope I haven't made any errors now… But Will will notice :)