Hypothesis Testing


We need to compare two distributions of data

            The two samples will always be different

Sampling error and measurement error are unavoidable


Statistics are used to distinguish between trivial and meaningful differences


A statistical test works much like a jury in criminal trial

            Both situations operate without absolute proof

                        You may make wrong decision

            A jury trial is a test for guilt, while a statistical test is a test for differences

                        You do not stand trial to show you are innocent

                        You do not design experiment unless you expect an effect

Both situations consider one of two mistakes more serious

                        Finding innocent man guilty in a jury trial

                        Concluding a difference was "real" when it was not


A statistical test

            Is based on the data

            Begins by assuming "no difference" (= null hypothesis)

            It is difficult to reject the null hypothesis

                        Must be > 95% differences observed did not occur by chance

                        Now you accept the alternative hypothesis

Science progress by rejecting null hypotheses


To carry out most kinds of statistical tests

            Use data to calculate test statistic

            Look up the critical value of the test statistic

            Compare to see if the test statistic is in the rejection region