Permutation/Randomization Tests

Two Sample Problem: Difference in Means

Job Satisfaction Example

(Data Source: Pallini & Pesarin, 1992)

Data Entry

> extroverted <- scan()

1: 66 57 81 62 61 60 73 59 80 55 67 70

13:

Read 12 items

> introverted <- scan()

1: 64 58 45 43 37 56 44 42

9:

Read 8 items

Randomization Test

> x <- compare(extroverted, introverted, alternative="greater",

+              method="perm")

Requested size 125970 is too large
        ... reset to 10000 
Requested permutation test is changed to randomization test!

> # Note that alternative hypothesis is one of "two.sided" (default),

> # "greater", and "less" and can be specified by

> # the first few character(s) that uniquely identify

> # the choice (i.e., partial matching).

> # Also note that method is one of "randomization" (default),

> # "permutation" and "bootstrap". Again, partial matching is possible.

> # Moreover, there are choose(12+8,12)=125970 possible permutations

> # which exceeds default maximum allowable size of

> # 10000. The function then changes it to a randomization

> # test.

>

> summary(x) # summarizes the test

Two Samples Comparison
======================

Sample 1: extroverted
Sample 2: introverted 
Comparison: mu[1]-mu[2] 
H0: mu[1]-mu[2]=0  vs.  H1: mu[1]-mu[2]>0 
Method used: randomization test

Test statistic: 17.29167 
P value: 0.00025 

(based on 10000 random permutations)

> # Reject H0

> plot(x, density=T) # histogram with kernel density curve

The plot is shown below (also available PDF and PS versions):

(CheckPermu1.png here)

Permutation Test

> # Now, perform permutation test by specifying Max.size

> # as choose(20,8). Note that this will generate a large

> # object.

> x <- compare(extroverted, introverted, alternative="greater",

+              method="perm", Max=choose(20,8))

> summary(x)

Two Samples Comparison
======================

Sample 1: extroverted
Sample 2: introverted 
Comparison: mu[1]-mu[2] 
H0: mu[1]-mu[2]=0  vs.  H1: mu[1]-mu[2]>0 
Method used: permutation test

Test statistic: 17.29167 
P value: 0.0003929507 

(based on 125970 all possible permutations)

> plot(x, density=T)

The plot is shown below (also available PDF and PS versions):

(CheckPermu2.png here)

Bootstrap Test

> # Now, perform bootstrap test.

> x <- compare(extroverted, introverted, alternative="greater",

+              method="boot", size=10000)

> summary(x)

Two Samples Comparison
======================

Sample 1: extroverted
Sample 2: introverted 
Comparison: mu[1]-mu[2] 
H0: mu[1]-mu[2]=0  vs.  H1: mu[1]-mu[2]>0 
Method used: bootstrap test

Test statistic: 17.29167 
P value: 7e-04 

(based on 10000 bootstrap samples)

> plot(x, density=T)

The plot is shown below (also available PDF and PS versions):

(CheckPermu3.png here)


Feb. 21, 2008