For the new semester and the new blogging life, I chose the topic that I found interesting, “Is the term “approaching significance” cheating?” (cheating sounds fun!) 😛
When the result of a statistical test is significant, it means that it is unlikely to occur by chance therefore the testing hypothesis is valid/true.(at the critical value/ significant level of 5%=0.05)
For example, the research report I have done in semester 1 on reaction time, which used a paired t-test compare the 2 different conditions revealed a significant difference. (Between narrow variability distribution and wide variability distribution) In this case, simple easy straight forward! We can just stated there is a significant difference between the two, no hassle at all.
However, if the researchers/investors spent a lot of money and time for one particular research and are hoping for a definite yes or no (in the worst case), but when they come to the results, it is just ~0.001 away from being significant, (I think it is worse than being not significant) it would be a total nightmare. And as mentioned they already invested a lot on it and possibly cannot afford another re-test anymore, which I think it is acceptable to say it is approaching significant. And it does not involved cheating as approaching means “close to” therefore people should be able to understand.
In conclusion, if the term “approaching significant” is not used by purpose (to confuse people), it would be adequate and acceptable. But in most of the cases when researchers really want to reveal significant difference and the result is just an “approaching significant”, it would be better to do a re-test in order to support the result. (Therefore it won’t be an error or may even get a significant difference at the re-test which support the testing hypothesis)