matched pair test vs two sample t test

[magomago]

So I'm trying to wrap around the differences between these two tests and I Think I got it…but I’d like some feedback from ATHT
(FYI: This is not for school. This is for my own personal development, and I realized that back when I took stats in college I didn't pay attention, and I didn't really need to use it again after that course. Now I find myself more and more interested so Ive been going over my original stats book for several months and have come across this section once more)

Matched pairs: Looking to compare a response where the test subject (be it a person, a cell, a plant, animal, material) is the control. I apply my first stimulus, wait for the response, and then apply my second stimulus (obviously one needs to be able to assess that the stimulus has completely worn off, or randomize the order in which the stimuli has given, etc. etc.). Since I am using the same subject and taking the differences of the stimuli, I am ideally only looking at the change in responses to that subject. An important factor is I’m using the SAME POPULATION to perform by test.

Two Sample T Test: I want to compare two different populations with the same stimulus/method applied and then compare their responses. I apply the first method/stimulus to the first population, and then the second method/stimulus to the second population. These are then compared. The number of samples doesn’t have to be the same, but my dof will always be dictated by the smaller number if i'm using tables and not going to calculate the parameters that follow though. The big key factor is that I’m using two independent samples.

I tried finding explanations online and actually found this interesting sheet - are my assessments of each test correct?
http://www.stat.uiowa.edu/~rdecook/s39/handouts/Matched_Pair_or_2-sample.pdf
1. Matched Pair Test; I'm assuming the 20 rats are all the special lab rats that are supposed to essentially be the same; hence, we can view each 'pairing' as a distinct unit. That way, we get 10 pairs and can compare the responses of each, and look at the difference of the responses, which would 'subtract' out any response brought about the actual rats. One sided since we are looking for a ">" response
2. Two Sample. Sampling two different populations that are completely independent of eachother and measuring the density of the cells. It is two sided because we just want to look for a difference --> Ho: mean_site1_density = mean_site2_density, Ha: mean_site1_density ~= mean_site2_density

3. Two sample b/c we are looking at two different populations for comparison. They have no effect on the other (well they might depenending on any interaction and their work, but lets assume they don’t). IT is one sided because the question is that higher IQ income > Lower IQ Income
4. Matched Pair One Sided Test. The plots are picked randomly across the field, but we can assume that splitting up the plot does not change the fact that each one is homogenous unto itself.
5. Matched pairs one sided test; group drawn from the same population, and if they matched it right, they would have matched these healthy white males based on age and other factors in order to try to remove lurking variables as much as possible. Since we are looking for a specific change (higher calcium reduces blood pressure) in one direction it is one sided
6. Matched Pairs two sided. Same subject is used which is the big hint, and since they are looking for a CHANGE in response, without a specific direction, it becomes a two sided test

 

[Mark R]

1. Two-sample. Each rat only receives 1 drug. The matched-pairs are only suitable where the method specifically matches - e.g. 2 measures are performed on the same subject, or 2 subjects are specifically selected to be 'matched' for everything you aren't interested in (age, sex, height, favorite color, etc.) The experiment design here just says 2 groups. 1-tailed

2. Matched pair - you're interested only in intestinal site, and have 'matched' everything else by using the same horse. - 2 tailed (just looking for difference)

3.TS - 1 tailed (looking for higher)

4.MP - 1 tailed

5. TS - 1 tailed. No matching is performed. Just because 2 groups are obtained from 1 population doesn't mean that you can directly compare person 1 in group A to person 1 in group B. Matched pairs assumes that person 1 in group A is directly comparable to person 1 in group B (e.g. they are the same person, or you have specifically selected person B so that their characteristics - that you aren't interested in - match A).

6. MP - 2 tailed.

 

[CycloWizard]

1. Two-sample. Each rat only receives 1 drug. The matched-pairs are only suitable where the method specifically matches - e.g. 2 measures are performed on the same subject, or 2 subjects are specifically selected to be 'matched' for everything you aren't interested in (age, sex, height, favorite color, etc.) The experiment design here just says 2 groups. 1-tailed

2. Matched pair - you're interested only in intestinal site, and have 'matched' everything else by using the same horse. - 2 tailed (just looking for difference)

3.TS - 1 tailed (looking for higher)

4.MP - 1 tailed

5. TS - 1 tailed. No matching is performed. Just because 2 groups are obtained from 1 population doesn't mean that you can directly compare person 1 in group A to person 1 in group B. Matched pairs assumes that person 1 in group A is directly comparable to person 1 in group B (e.g. they are the same person, or you have specifically selected person B so that their characteristics - that you aren't interested in - match A).

6. MP - 2 tailed.

These.

 

[magomago]

So I can start to see where my confusion lies...

(1) My mistake is I assumed that the lab rats are indeed matched for everything else, when the reality is that there is still enough variation to say "We can't do a matched test, but must do a Two Sample Test".

This makes me say that unless you literally use the same subject to test upon, even 'matching' for everything we aren't interested in (age, sex, height, etc) should still result in a 2 sample test. Is that a valid assumption?

(2) Aren't i looking at completely different population of cells though? One at two different intestinal sites?

(5) Okay so TS...but if we assumed they matched for all characteristics they weren't interested in, then it could be a MP test?

 

[CycloWizard]

So I can start to see where my confusion lies...

(1) My mistake is I assumed that the lab rats are indeed matched for everything else, when the reality is that there is still enough variation to say "We can't do a matched test, but must do a Two Sample Test".

This makes me say that unless you literally use the same subject to test upon, even 'matching' for everything we aren't interested in (age, sex, height, etc) should still result in a 2 sample test. Is that a valid assumption?

Sort of. If it said that such matching had been used, you should use the matched pair test. Instead, it uses language that indicates only that the 20 rats were split into 2 populations. Generally, matching can only be used to distinguish between treatment effects when there is no intrinsic difference between samples. For example, if you have one person but are testing their right and left hands under different conditions, that would be a matched pair test. If you were testing whether the environment two identical twins are raised in has any effect on how they turn out, that could qualify as a matched pair study.

(2) Aren't i looking at completely different population of cells though? One at two different intestinal sites?

That's what you're testing - whether there is a difference between the two populations. If the t-test is negative, then there is no statistical difference between the two populations and they are statistically identical.

(5) Okay so TS...but if we assumed they matched for all characteristics they weren't interested in, then it could be a MP test?

Probably not because there could be hidden variables which would be missed. That is essentially the point of matching - to eliminate the effects of unknown variables. In the twin study, we know that the genetics are identical, so there are no unknown genetic factors. That leaves only environmental factors (treatments) to differentiate between the two samples.

 

[Mark R]

(2) Aren't i looking at completely different population of cells though? One at two different intestinal sites?

They are different only because of site - they are matched pairs. The point is that for each test subject you have a pair of results, where only one thing has been manipulated (the site). If the results you get easily fall into 'pairs' where it is the difference between the members of each pair that interests you - you can use the matched pairs test.

E.g. goldilocks wants to know if eating porridge from the left side of the bowl has a better temperature than porridge from the right of the bowl.

Let's say she has 10 bowls (which may be of 10 different sizes). She could choose 5 to test the right, and 5 to test the left. She could then use an unmatched test.

Alternatively, she could test the left and the right of each bowl - she now has 10 pairs of results. The matched pairs test could be used here, and would give a more reliable result than the unmatched test above.

The key point of the matched pairs test, is that the pairs matter. The test compares Bowl 1 -L to Bowl 1 -R , then Bowl 2L to Bowl 2R, etc. It therefore only works if you have a clear reason for pairing (e.g. 2 measures from the same bowl, 2 sites from teh same animal, 2 drugs given to the same patient, etc.). Inherently paired data will always have the same number of measures in each group, no matter how the experiment is performed.

The unpaired test compares mean of R Bowls to mean of L Bowls. If the data isn't inherently paired, then you have to use this.

(5) Okay so TS...but if we assumed they matched for all characteristics they weren't interested in, then it could be a MP test?

You can't assume. If the experiment says that they are matched for all characteristics you aren't intersted in, then you can use a matched test.

E.g. let's say I want to know if black swans have a different wing span to white swans. I have a very detailed database of white swans, with thousands of measurements. However, black swans are rare, and I've only been able to catch 10.

I could compare my 10 black swans against my whole population of 1000 white swans, using an unmatched test.

Alternatively, I could measure the weight, sex, height, neck length, etc. of my black swans - and then search my database to find a 'perfect match' white swan for each of my black swans. I could then analyses my paired data using a matched pairs test.