You’ve designed your study, recruited a sample and collected data. Maybe your study finds 20% of the women who gave birth at a birth center had at least a second degree tear, while 25% of the women who gave birth at a hospital had at least a second degree tear. The next question you need to ask yourself is this:
Does this difference represent a real difference, or is this difference due to the random variation I should expect when I sample and measure?
And that is the ONLY question in the research statistics will answer.
- Statistics cannot tell you if this is a clinically relevant or important difference.
- Statistics cannot tell you if this is a true representation of the perineal tear rates in either population.
- Statistics cannot tell you if the study is good
All statistics can tell you is if the results obtained from the sample differ more than by chance.
We have talked about a lot of issues leading up to this point, and some of you may have wondered why we spent so much time talking about other things. The reason is simple, statistics is only one piece of the work done to build a good study. The process of research goes something like this:
- Identify a gap in knowledge
- Ask a question
- Decide on a measure
- Design a study
- Collect data
- Summarize data
- Analyze data (this is the statistics)
- Draw conclusions
- Interpret Results and Identify a gap in knowledge
And then you do it all over again. The reason we spent so much time on the other pieces of a study is because if the other pieces are poorly put together, the statistics do not matter. Statistics are only helpful if the study itself is well designed: with a well-powered sample, a validated measure, controls for confounders and blinded when appropriate.
How Research Statistics Work
Basically, all research statistics do the same thing. They identify the difference between what is expected and what is observed, to measure the likelihood a difference actually exists.
The difference is always measured per the null hypothesis – or what we think is the “status quo” line of thought on the subject. If we hypothesize women who use an epidural have a longer labor than women who do not use an epidural, the null hypothesis would be that women who use an epidural have the same length of labor as women who do not use an epidural. We then use statistics to measure the difference between the two groups of women to see if the groups differ more than we expect by random variation if the null is true (if there really is no difference).
Because the difference is always measured per the null hypothesis, you do not “prove” anything with research. You either reject the null or fail to reject the null. If you reject the null, it means the data shows a difference that is more than expected by chance if the null were true — which means it is unlikely the null is true. If you fail to reject the null, it means the data does not show a difference that is more than expected by chance if the null is true — but this is not saying the null is true. Why? Because there could be many other ways the null is not true, even if in this sample with this measure the null cannot be rejected.
Over time, research builds to help identify boundaries — maybe sometimes women with epidurals have longer labors, but other times they do not (for example, depending on if oxytocin is used or with particular mixtures of epidural medications) — that allow us to be confident in our understanding. But no individual piece of research provides that confidence.
The Birth Workers Survey
As an example, let us hypothesize that women who are doulas are less likely to have cesarean deliveries than women who are not doulas. Don’t worry if you don’t understand everything yet, we will get to it. Our hypothesis and null hypothesis would look like this:
Hypothesis: Women who work as doulas have a lower rate of cesarean delivery than women who do not work as doulas.
Null: Women who work as doulas have the same rate of cesarean delivery as women who do not work as doulas.
Overall, 28 respondents had given birth, and of those 3 (11%) gave birth via cesarean delivery. Of 21 women who reported working as a doula, 2 (5%) gave birth via cesarean delivery; of 7 women who reported not working as a doula, 1 (14%) gave birth via cesarean delivery. The chi-square value is 1.102 with a p value of o.576 which means the difference between the two groups falls within the expected range of variation under the null hypothesis. Therefore, we fail to reject the null hypothesis and this data does not provide support for the hypothesis that doulas are less likely to have cesarean deliveries than women who are not doulas.
Now that we have that background, next week we will start looking at the specific statistical tests to better understand what they mean and when they should be used.