When you find a mean outside the range or margin you had expected for samples from a specific population, you may wonder whether that mean is really coming from the same population. This is considered to be an issue of testing for significance which is the topic of this chapter.

What does testing for significance entail? Say that you had expected a mean of 33 but in fact observed or measured a mean of 35.3. Is this difference significant? In other words, is this difference likely to be the mere result of random sampling? Or is the actual difference (measured in SE units as z- or t-values) beyond the critical difference that you take as a borderline case for being random? If the latter is the case, you would consider this sample to be from a different population, which usually means that some specific treatment affected the sample and had a significant impact.

When dealing with testing for significance, the term hypothesis comes into play:

• The null hypothesis states that the difference between observed and expected is the outcome of randomness: "Results may vary."

• The alternative hypothesis states that the difference is caused by a real difference in the underlying sample (caused by the factor under investigation).

• There are two possible outcomes in testing for significance:

  • When the actual z- or t-value is less than the critical z- or t-value, the null hypothesis is accepted. Conclusion: The difference is (very) likely a matter of...