I'm a bit rusty in statistics

[DrPizza]

I have a relative who works in a hospital. Some company sends out surveys to patients after they've been discharged. Out of 300+ something patients on her floor over a 3 month span, they received 13 surveys back. 8 contained complaints.

Naturally, administration sees this as 62% of patients are dissatisfied with their care, and are ruining morale among their employees by being overly critical about it. The few who have some sense realize that the people most likely to respond to those surveys are the people who actually do have a complaint, while the people who were satisfied are far less likely to respond.

In the previous quarter, there were fewer complaints (5?) from an unknown number of surveys.

If any of you are less rusty in stats (the math course I hated the most, simply because there are so many people who don't understand/abuse statistics) - does this type of sampling, with people having the option to respond or not, and getting 13 responses back, have any statistical significance whatsoever?

Personally, if I were in charge of such a survey, I would have done a follow-up phone survey with some number of individuals who failed to return their surveys in order to determine if the returned surveys were representative of opinions as a whole.

 

[CSMR]

It is not an unbiased sample.
Stastical significance is a term which is used in another context.
It would be difficult to make definite conclusions from the survey. Maybe there are estimates of the likelihoods of replying to a servey if satisfied and if unsatisfied from other contexts which you could use.

 

[dkozloski]

You have a return of surveys from people with an axe to grind and nothing more. It doesn't remotely resemble an exit pole in an election. The data is worthless.

 

[Mark R]

This data is too biased to be of use - this is a major problem with questionnaire surveys, the people who return the surveys are not an accurate representation of the population as a whole.

Indeed, a non-responder rate of 96% is so poor as to be meaningless.

What would be of more value is to compare the total number of patient episodes, and the total number of complaints (assuming that both are known, and that every patient received a questionnaire for each episode). If the management are serious then they need to do this, or select an minimally-biased sample (e.g. every 10th patient throught the door).

If you do have several sets of data, then you can compare them. (But in an uncontrolled survey such as this, beware of the 'regression to the mean' fallacy - which is a frequent killer of studies that don't have a stable historical control to go on).

Even then, there may be issues of variability between sampling periods (e.g. different people may be admitted depending on the season - in Winter, more elderly people with chest disease may be admitted, whereas in Summer there may be more young asthmatics, or allergy sufferes). Unless you can reasonably say that these effects aren't significant, then comparing data from one period to another may be invalid. In a long-term follow up study you may need to perform some form of seasonal adjustment.

 

[chcarnage]

http://en.wikipedia.org/wiki/Selection_bias (self selection bias)

The responsiveness of unsatisfied customers is way higher and the data not analysable. But this is a methodological error, nothing mathematically describable.

The math starts when we look at the sub-group of the 8 complaints. Are they representative for the things that go wrong in the hospital? Perhaps somebody wants to take over, since I haven't any formulas at hand. But I've never encountered a quantitative analysis with a sample of less than 25 or 30.

http://en.wikipedia.org/wiki/Sample_size
http://en.wikipedia.org/wiki/Sampling_%28statistics%29#Sample_size

So if the management claims that 87.5% of all complaints are because of unkindliness and 12.5% because of medical malpractise, the confidence interval is way too bad for an interpretation.

(Oh and another selection bias risk is that the victims of medical malpractise are less likely to participate in a survey than the rudely treated )

 

[mozirry]

surveys from an unbiased group have shown that a survey of an unbiased group can produce genuine unbiased information

 

[gsellis]

Originally posted by: mozirry
unbiased surveys from an unbiased group have shown that a survey of an unbiased group can produce genuine unbiased information

Fixed that for you. Bad survey questions and formats are the number 1 method of screwing up results. Including the groups studying groups

 

[bpatters69]

As others have said, you don't have an un-biased sample so the results are inherantly flawed. In order to get a "good" sample, someone would need to actively take a sample of a statistically signifigant number of patients to get a "good" sample. In other words, using randomness or whatever means at hand, patients would be selected and then sampled.

 

[Paperdoc]

These results are absolutely meaningless! In fact, this whole scenario is a prime example of why you NEVER use self-selected surveys. Somebody who does not understand the limits and severe distortion of the information obtained is trying to USE that information as the basis for important decisions. Guaranteed the decisions will be wrong!!

You cannot get a true representation of the population unless the survey respondents are selected TOTALLY randomly - that is, with NO ability of the respondent to choose whether or not to participate. Now, in practice that is almost impossible to achieve. But it can be done to a very satisfactory level if the survey methodology involves active explanation and persuasion to induce every person chosen to participate honestly. You don't survey every patient, but you do ensure a high response rate by persuading all you approach to answer. Just asking everyone to answer if they feel motivated is a cheap way to get enough responses to count up, but the results are completely non-representative of the entire population of discharged patients.

 

[PlasmaBomb]

The way this was carried out wasnt valid, the sample size is small and as others have stated people who are dissapointed in some way are much more likely to take the time to complain. You could use stats to say almost anything on that - for example I can say that there is no signifigant difference between the number of satisfied and dissatisfied patients, which is quite a bit different to 68% of patients are dissatisfied. You could probably push further than that statistically too.

 

[Accipiter22]

I just wanted to say that I <3 statistics, and this thread. and all of you that posted in it.

 

[bobsmith1492]

Quick, highly-technical question: What does "I <3" mean?? By context in various places, I would guess it means "I love," but it makes no sense. It also looks like a pair of balls hanging there. ???

 

[TuxDave]

Originally posted by: bobsmith1492
Quick, highly-technical question: What does "I <3" mean?? By context in various places, I would guess it means "I love," but it makes no sense. It also looks like a pair of balls hanging there. ???

Turn your head to the sideways to the right.

 

[gsellis]

Originally posted by: TuxDave

Originally posted by: bobsmith1492
Quick, highly-technical question: What does "I <3" mean?? By context in various places, I would guess it means "I love," but it makes no sense. It also looks like a pair of balls hanging there. ???

Turn your head to the sideways to the right.

OK, and what do those have to do with it?

 

[CycloWizard]

Originally posted by: gsellis
OK, and what do those have to do with it?

It's a heart. <3

 

[gsellis]

Originally posted by: CycloWizard

Originally posted by: gsellis
OK, and what do those have to do with it?

It's a heart. <3

My 'other' right. Do it to the left and it looks like what covers the heart and tends to be gender specific.

 

[Thyme]

I wouldn't say the data is useless. You can get good information from the open-ended questions on how you can improve, regardless of the statistics.

As has been mentioned, you don't have an unbiased sample of the population of patients, but if you change the population (and thus the information you can get), you may have an unbiased sample. Still a low amount of data points so you're not likely to get very good information. Your most useful results from the surveys are the open-ended questions.