How to report a Bayesian analysis: Teaching why

An interesting recent article emphasized that asking students to generate what should be reported from a Bayesian analysis also gets them to think about why things need to be reported, which in turn can get students to understand better how Bayesian analysis works and what it means. In this blog post I briefly summarize the article, compare it with the recommendations for reporting a Bayesian analysis in Doing Bayesian Data Analysis, and suggest that there are some important points for reporting that might be difficult for beginning students to generate on their own.

The article is: Pullenayegum, E. M., Guo, Q., & Hopkins, R. B. (2012). Developing critical thinking about reporting of Bayesian analyses. Journal of Statistics Education, 20(1). It's available at this link. Here are some excerpts:

[Abstract:] Whilst there are published guidelines on reporting of Bayesian analyses, students should also be encouraged to think about why some items need to be reported whereas others do not. We describe a classroom activity in which students develop their own reporting guideline. ... [Section 1:] Since Bayesian analyses are far less widely used in medicine [than frequentist analyses], it is difficult for students to gauge what is typical. Thus, teaching students how to report Bayesian analyses in the medical literature is an important component of a course in Bayesian biostatistics. When teaching reporting, we need to avoid training students to follow a set of rules unthinkingly. Rather we need to focus on helping students to think critically about what is needed. That is, we need to teach not just the “what”, but also the “why”. ... [Section 4.4:] Two of us (RBH and QG) participated in the first implementation of the exercise as students. As the exercise progressed, we found ourselves thinking as readers, reviewers or editors rather than as the statistician or author. ... [Section 5:] The value of the activity is in the learning process rather than the guideline itself.

The exercise itself consists of students anonymously generating candidate items for inclusion, then having the items collated onto a master list of candidates, then rating the items for importance and discussing why they are important. (See the article for details about the group dynamic.) Importantly, but perhaps not emphasized enough in the article, the instructor can (and did) anonymously contribute items to the list during the first round. This strikes me as a very useful exercise for students, and I may try some variation of it myself next time I teach my course.

The article reports the lists of items generated by two different classes. The lists included points about the prior distributions, the likelihood function, the analysis technique, and the results. Please see Table 1 of the article for details. Although the emphasis of the article was on getting students to think about why items should or need not be reported, rather than on the actual list created, it is still informative to see what items the students settled on, and whether anything went missing that might be considered to be important.

In particular, it's natural for me to compare the student-generated lists with my own recommendations in Doing Bayesian Data Analysis (Ch. 23, pp. 620-622):

There is notable overlap with items in the student-generated lists, but the first two "essential" points raised in DBDA were not emphasized by the students. These points are (1) Motivate the use of Bayesian (non-NHST) analysis, and (2) Clearly describe the model and its parameters. Both of these points are about reporting the forest before reporting the trees. Both of these points are about contextualizing the analysis and giving it meaning. In particular, the second point entails the idea that the model is a particular description of data, selected from the space of all possible models because it is meaningful in the particular context. Finally, the "essential" points in DBDA are meant to be reported in sequence. This temporal, sequential order of points is important for the comprehensibility of the report. For example, the likelihood function and its parameters must be explained before the prior and its parameters can be explained. (And the structure of the data must be explained before the likelihood function can be described.)

I think that all these issues ---motivating use of Bayesian analysis, describing the model and parameters, and reporting in a particular sequential order--- are about understanding the larger context in which statistical analysis resides, and this larger context might be relatively difficult for beginning students to apprehend. Therefore instructors who adopt the exercise, of students generating guidelines for reporting, may want to seed the candidate list with these sorts of items, or at least be sure that they are brought up during discussion.