Is the term “approaching significance” cheating?

For the new semester and the new blogging life, I chose the topic that I found interesting, “Is the term “approaching significance” cheating?”  (cheating sounds fun!) 😛

When the result of a statistical test is significant, it means that it is unlikely to occur by chance therefore the testing hypothesis is valid/true.(at the critical value/ significant level of 5%=0.05)

For example, the research report I have done in semester 1 on reaction time, which used a paired t-test compare the 2 different conditions revealed a significant difference. (Between narrow variability distribution and wide variability distribution) In this case, simple easy straight forward! We can just stated there is a significant difference between the two, no hassle at all.

However, if the researchers/investors spent a lot of money and time for one particular research and are hoping for a definite yes or no (in the worst case), but when they come to the results, it is just ~0.001 away from being significant, (I think it is worse than being not significant) it would be a total nightmare. And as mentioned they already invested a lot on it and possibly cannot afford another re-test anymore, which I think it is acceptable to say it is approaching significant. And it does not involved cheating as approaching means “close to” therefore people should be able to understand.

In conclusion, if the term “approaching significant” is not used by purpose (to confuse people), it would be adequate and acceptable. But in most of the cases when researchers really want to reveal significant difference and the result is just an “approaching significant”, it would be better to do a re-test in order to support the result. (Therefore it won’t be an error or may even get a significant difference at the re-test which support the testing hypothesis)

This entry was posted in Psychology weekly blog. Bookmark the permalink.

7 Responses to Is the term “approaching significance” cheating?

  1. dybanneediu says:

    I think this topic is similar to the ethics of removing outliers. But when it comes to this, I think your result is either significant or not. If, as you say it’s ~0.001 away from being significant, given this extra 0.001, it would only just be significant and therefore still not an amazing result. Stats isn’t purely about the numbers, it’s about what the data shows. For something to be overwhelmingly significant then there should be more confidence in the result than “approaching significance”. Manipulating the data or carrying out different statistical tests until one says it’s significant seems wrong to me. You shouldn’t have to do that. If someone is doing it then it’s for their benefit and satisfaction, and not a true reflection of the result if you have to hunt down something that works for what you want. 🙂

  2. Pingback: Homework for my TA | dy bannee diu

  3. PAW! 😀 so -0.049 is approaching the significance? If that will go through I would go with -0.048 is approaching the “approaching the significance”. If I found -0.049 I would state how it is, and if some one want to do the retest, GO AND DO, AND ALSO PAY ME FOR WHAT I ALLREADY FOUND! Retest will probably will cost them(company) double. I would probably find the bigger population, like find another 20 participants, test them, and check results again. Becouse cheating is totally not fun. Its bad. Comment done – find more fresh and crunchy participants, test them, check results. Or you also can manipulate stimuli, so they would be stronger, stronger effect size. But totally, think about your study design before testing more carefully. Do you really measure what you want? (or all of what you do is “donuts”)

  4. Pingback: TA WORK! POOOOW! POOOOOOW! pow « glebskrecetovs

  5. psuc15 says:

    You bring up a valid point of the frustration such a small decimal can impact upon the effectiveness of a research investigation. The blog provides an insight into an idea maybe the importance of a significance testing is potentially given too much authorisation within the research field. This could be a sub content to investigate within your blog providing an idea as to whether approaching significance would be considered cheating if there was less importance placed on significance.
    Through outline the basics of approaching significance you present your reader with background knowledge of the issue at hand and allow them to consider for themselves whether they regard approaching significance as cheating. Being Psychologists in training ourselves this topic allows students to empathise with the researchers on this matter and possible apply some empathy into overlooking this idea of cheating through understanding the rigors of research investigation.
    Like within society there are set rules to provide law and order, significance testing provides this within research studies ensuring that an effective method of qualification of research is presented without the need to question validity. Looking at how this could affect the general population or applying this method to other situations could be another aspect of increasing your view on the conscientious level of accepting this method.
    Although you provide an unbiased argument through consideration of both sides of the topic there doesn’t seem to be clear indication of which side you stand within this argument. Therefore adding evidence to support your views could be another method of adding validity to not only your argument but also to allow your readers to take this into consideration and potentially providing a different view in which they did not originally consider. Through the method of examples and evidence you thoroughly cover the topic at hand and provide validity and reliability for your argument.
    In conclusion I found your blog informative although I think you need to include different sub categories of expanding your views as well as providing evidence to add justification of your clear hard work and time dedicated to taking this issue into consider. Well done and I look forward to reading more.

  6. Pingback: Blog 1 Comments for TA « psuc15

  7. poeywycheung says:

    I have done more research on the topic after the comments, that Shaver ( 1993 ) stated the term highly significant is silly, and so is” approaching significance”. Shaver suggested everything should be stated clearly, only 2 answer, YES or NO. Thompson ( 1993 ) argues that sometimes the data are obviously sufficient for an estimate. The more importantly is how we decide on our research/test, whether or not we should add the adjective statistical.

    And yes, I agree with glebskrecetovs that cheating is actually not fun but again it is still depending on how a (group) of researcher(s) act on the result.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s