tag:blogger.com,1999:blog-3240271627873788873.post1599162088667893395..comments2017-04-26T07:56:16.758-04:00Comments on Doing Bayesian Data Analysis: Equivalence testing (two one-sided test) and NHST compared with HDI and ROPEJohn K. Kruschkehttp://www.blogger.com/profile/17323153789716653784noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-3240271627873788873.post-41974045978795764812017-02-21T09:43:42.450-05:002017-02-21T09:43:42.450-05:00That's a disappointing reply, Daniel. You shou...That's a disappointing reply, Daniel. You should give it more thought. The bottom line is that, frequentists offer two solutions where bayesian offer only one. These are additional degrees of freedom provided by the frequentist approach (and as we know since Simmons et al. 2011, DoF are bad). Furthermore, I'm not aware of any guidelines on what to do when the frequentist tests disagree. As always when frequentist methods provide useless results, the researcher is left alone to divine the practical significance from the inconclusive result and is given all the blame for the inevitable failure. matushttp://simkovic.github.ionoreply@blogger.comtag:blogger.com,1999:blog-3240271627873788873.post-12026761415620952402017-02-17T04:34:44.174-05:002017-02-17T04:34:44.174-05:00I am not sure I understand
The first paragraph rea...I am not sure I understand<br />The first paragraph reads: <br />"First, the TOST procedure cannot be used to ACCEPT anything. TOST *rejects* values that fall outside it.", <br />and the second begins with <br />"When NHST rejects the null, and TOST accepts equivalence, that is one of the four possible outcomes when you combine NHST and TOST".<br /><br />I understand the first quote to mean that we can use the NHST framework only to reject statistical hypothesis (that’s what I learned in school) but the second quote suggests that rejecting outside values after all means accepting equivalence.<br />I agree that the root of the problem is that TOST and NHST accept/reject different things, but I wouldn’t frame it as statistical vs practical significance, because the TOST procedure doesn’t care about what is a practically meaningful difference, it is just a procedure, just as NHST. The difference is that NHST rejects a point-hypothesis whereas TOST rejects an interval-hypothesis, and as long the point lies within the interval, it is no big surprise that one procedure can reject the point whereas the other accepts the interval.<br /><br />The real problem, in my view, is that the frequentist framework needs different approaches for accepting and rejecting the hypothesis that there is no difference between to parameters. This is cumbersome, leads to many misunderstandings, and can lead to (apparently) contradictory conclusions. In contrast, things are more intuitive in then Bayesian framework, where one can use Bayes factors to accept or reject point-hypotheses, or the ROPE approach to accept or reject interval-hypotheses. The Bayesian framework is also clearer about when we should stay undecided.guidohttp://www.blogger.com/profile/10981583489689200030noreply@blogger.comtag:blogger.com,1999:blog-3240271627873788873.post-12255886411574973682017-02-17T03:51:33.430-05:002017-02-17T03:51:33.430-05:00Seems you don't disagree on substance, just us...Seems you don't disagree on substance, just using the the term "accept" and the term "conflict". Instead of "accept" you say "statistically equivalent" and instead of "conflict" you say "both statistically equivalent and statistically different". Okay. But HDI+ROPE never leads to that combination.John K. Kruschkehttp://www.blogger.com/profile/17323153789716653784noreply@blogger.comtag:blogger.com,1999:blog-3240271627873788873.post-89751983541494564202017-02-17T01:50:53.587-05:002017-02-17T01:50:53.587-05:00Hi John, very nice post! But there is no conflict ...Hi John, very nice post! But there is no conflict here. First, the TOST procedure can not be used to ACCEPT anything. TOST *rejects* values that fall outside it. TOST allows you to conclude the data you have observed is unlikely, if there was an effect as large as the equivalence bounds. <br /><br />When NHST rejects the null, and TOST accepts equivalence, that is one of the four possible outcomes when you combine NHST en TOST, and it is a strength, not a conflict. It means that you can conclude the data is unlikely, if the null was true, but the data is also unlikely, if there was an effect you'd care about. It means the effect is statistically significant AND statistically equivalence. See Figure 1 in my paper on TOST: https://osf.io/preprints/psyarxiv/97gpc/<br /><br />This prevents common misinterpretations of p-values, where you mistake statistical significance for practical significance. But NHST and TOST are two different tests, completely orthogonal, and the result of one can not be in conflict with the result of the other. Daniel Lakenshttp://www.blogger.com/profile/18143834258497875354noreply@blogger.com