Thursday, January 17, 2013

Uniform R code for opening, saving graphs in Windows and Mac OS

[See follow-up here.]

A big frustration when trying to create R code that works across Windows and Mac OS is that the R commands for opening graphics windows, and for saving their contents, are different in the two operating systems. In Windows, the functions windows() and savePlot() do the job nicely. But in Mac OS, equivalent commands are X11(,type="cairo") and either savePlot() or dev.copy2* depending on the desired format.

In this post I present two simple utility functions for opening graphics windows and saving them that operate the same for Windows and Mac OS. (You'd think someone would have done this already, but it appears not?) The basic sequence of commands is this:

openGraph(...)  # open a graphics window
plot(...)       # create graph in the window
saveGraph(...)  # save to a file in desired format

I have tested the functions on recent Windows and Mac OS machines, running from RStudio desktop. I would like to know if users encounter problems with the functions when running in a different configuration. I am told that the functions also work on Linux.

Here is an R script that defines the functions and then calls them with several different file formats. Please copy and paste the entire script into R and give it a try. Let me know how it goes.

Update January 29, 2013: The functions defined below work robustly and they are now in the program openGraphSaveGraph.R in the program repository. Several other programs have been modified to use these functions, and the others will be modified eventually.

#------------------------------------------------------------------------
# Modified 2013-Jan-22 8:55pm EST

openGraph = function( width=7 , height=7 , ... ) {
  if ( .Platform$OS.type != "windows" ) { # Mac OS, Linux
    X11( width=width , height=height , type="cairo" , ... )
  } else { # Windows OS
    windows( width=width , height=height , ... )
  }
}

saveGraph = function( file="saveGraphOutput" , type="pdf" , ... ) {
  if ( .Platform$OS.type != "windows" ) { # Mac OS, Linux
    if ( any( type == c("png","jpeg","jpg","tiff","bmp")) ) {

      sptype = type
      if ( type == "jpg" ) { sptype = "jpeg" }
      savePlot( file=paste(file,".",type,sep="") , type=sptype , ... )     
    }
    if ( type == "pdf" ) {
      dev.copy2pdf(file=paste(file,".",type,sep="") , ... )
    }
    if ( type == "eps" ) {
      dev.copy2eps(file=paste(file,".",type,sep="") , ... )
    }
  } else { # Windows OS
    file=paste(file,".",type,sep="") # force explicit extension
    savePlot( file=file , type=type , ... )
  }
}

graphics.off()
for ( typeVal in c("eps","pdf","jpg","jpeg","png") ) {
  openGraph(width=3,height=3)
  plot(1:4,c(1,3,2,4),type="o",main=bquote(typeVal==.(typeVal)))
  saveGraph( file="openGraphSaveGraphOutput", type=typeVal )
}
 

#------------------------------------------------------------------------

Monday, January 7, 2013

Bayesian multiple linear regression with interaction: Erratum

Background: In regression models, the posterior distribution on the regression coefficients and intercept can be strongly correlated, causing some MCMC samplers to become inefficient as they "bang into the walls" of the diagonal ridge in parameter space. There are various ways to address this problem, but one simple way is by standardizing the data (or at least subtracting out the mean), and then converting the MCMC-derived parameters back to the original scale of the data. This approach was taken in the book (DBDA) for using BUGS, and continued, vestigially, in the programs that were converted to JAGS even though JAGS is often less susceptible to the problem.

The news in this post is that a reader, Francis Campos, caught an error in the formulas for the translation from standardized data to original-scale data, for the particular case of the multiple linear regression model with two predictors and a multiplicative interaction. Specifically, part of Equation 17.5, at the top of p.472, has two subscripts reversed, and should be changed as shown in red here:
Correction to Eqn. 17.5, top of p. 472
It turns out that the error was not only in the printed equation. It was also in the programs MultiLinRegressInterBrugs.R and MultiLinRegressInterJags.R, and this error caused the graphical output to be wrong as well. The corrected programs are now available at the program repository, and the corrected figures are included below. (In a separate program, I verified the new results simply by removing the standardization of the data and the transformation back to original scale. JAGS handles the unstandardized, original-scale data very efficiently.) Fortunately, nothing conceptual in the discussion in the book needs to be changed. All the conceptual points still apply, even though the graphical details have changed as shown here:
Top of Fig. 17.9, p. 473, corrected.
Bottom of Fig. 17.9, p. 473, corrected.
Figure 17.10, p. 474, corrected.
My thanks to Francis Campos for finding the error and alerting me to it!

Friday, January 4, 2013

Bayesian estimation in a new web app!

A wonderful new web app is now available for doing Bayesian estimation of two groups. Just enter the data for each group, click the "go" button, and watch the MCMC sample emerge, revealing the posterior distribution for the means, standard deviations, and normality, along with the difference of means, difference of standard deviations, and effect size. This web app was created by Rasmus Bååth and is available at http://www.sumsar.net/best_online/. Give it a try!
 
The web app is fabulous for quickly and easily seeing the main results of a Bayesian estimation of parameters for two groups. There is no need to install R and JAGS and then invoke R and run commands. Instead, just paste in your data and click a button!

Here (below) is a screen shot from January 03, 2013, using the data from the main example of BEST. More comments about the app continue after the screen shot.


The original implementation of BEST, available from http://www.indiana.edu/~kruschke/BEST/, provides some more functionality than in the app (at the time of this post), but only by taking the effort to install (free) software and to interact with the R programming language. The original BEST software implements ROPEs for the decision rule and shows a posterior predictive check in the form of a smattering of credible model distributions superimposed on data histograms. It also shows correlations of parameters in the posterior. The original BEST software also does power computations. Another benefit of dealing with the BEST software is that it is a gateway to the full spectrum of Bayesian data-analysis programs written in R and JAGS/BUGS. But for quickly and easily showing the primary results from a Bayesian analysis of two groups, this web app is brilliant!

Big thanks to Rasmus Bååth for creating this awesome web app! If you like it, let him know -- his e-mail is linked in the lower right "About" section on the web app. The URL again is http://www.sumsar.net/best_online/

Friday, December 21, 2012

Another reason to use JAGS instead of BUGS

BUGS is the pioneering software that made MCMC available to so many of us, but it has some problems with robustness that are not suffered by the subsequent software JAGS. Readers of DBDA tell me of some new problems running models in BUGS, which I have not been able to solve in BUGS, but the same model runs fine in JAGS. The purpose of this post is to point out the particular problem and the complete solution in JAGS. (If you have a solution for BUGS, please comment.)

I converted all the BUGS programs to JAGS versions back on 01 January 2012. Please see the previous post for information about JAGS.

A reader reports this problem:
I [the reader] have received this error several times:
"update error for node <theta[66]> algorithm conjugate beta updater error argument two  has too small a value "
...  In particular, I [the reader] noticed for exercise 12.3, if I remove the 0's from the data, I do not get the error. 
When I try running the program from Exercise 12.3 with BUGS, I also get the error, although I did not get any error a few years ago when I created the program and reported its results in the solutions manual. I have tried to solve the problem by arbitrarily censoring/truncating the distributions in BUGS (as I did in some other programs in the book), but without success in this application.

I quickly re-wrote the BUGS+BRugs program for JAGS+rjags, going step-by-step through the guidelines from this previous post. The resulting program, using the identical model statement, runs fine in JAGS and produces results identical to those in the solutions manual. Here is the complete program and results:


graphics.off()
rm(list=ls(all=TRUE))
fileNameRoot="Exercise12.3.A.JAGS" # for constructing output filenames

require(rjags)

if ( .Platform$OS.type != "windows" ) {
  windows <- function( ... ) X11( ... )
}


#------------------------------------------------------------------------------
# THE MODEL.

modelstring = "
model {
   for ( subjIdx in 1:nSubj ) {
      # Likelihood:
      z[subjIdx] ~ dbin( theta[subjIdx] , N[subjIdx] )
      # Prior on theta: Notice nested indexing.
      theta[subjIdx] ~ dbeta( a[cond[subjIdx]] , b[cond[subjIdx]] )
   }
   for ( condIdx in 1:nCond ) {
      a[condIdx] <- mu[condIdx] * kappa[condIdx]
      b[condIdx] <- (1-mu[condIdx]) * kappa[condIdx]
      # Hyperprior on mu and kappa:
      mu[condIdx] ~ dbeta( Amu , Bmu )
      kappa[condIdx] ~ dgamma( Skappa , Rkappa )
   }
   # Constants for hyperprior:
   Amu <- 0.35 * 2
   Bmu <- 0.65 * 2
   Skappa <- pow(meanGamma,2)/pow(sdGamma,2)
   Rkappa <- meanGamma/pow(sdGamma,2)
   meanGamma <- 10
   sdGamma <- 10
}
" # close quote for modelstring
# Write model to a file:
writeLines(modelstring,con="model.txt")

#------------------------------------------------------------------------------
# THE DATA.

nCorrOfSubj = c( 8, 4, 6, 3, 1, 4, 4, 6, 4, 2, 2, 1, 1, 4, 3 ,3 ,2 ,6, 3, 4,
                 2, 1, 1, 3, 2, 7, 2, 1, 3, 1, 0, 2, 4, 2, 3, 3, 0, 1, 2, 2 )
CondOfSubj = c( rep(1,20) , rep(2,20) )
nTrlOfSubj = rep(10,40)
nSubj = length(CondOfSubj)
nCond = length(unique(CondOfSubj))

# Specify the data in a form that is compatible with BRugs model, as a list:
dataList = list(
 nCond = nCond ,
 nSubj = nSubj ,
 cond = CondOfSubj ,
 N = nTrlOfSubj ,
 z = nCorrOfSubj
)

#------------------------------------------------------------------------------
# INTIALIZE THE CHAINS.


#------------------------------------------------------------------------------
# RUN THE CHAINS.

parameters = c("mu","kappa","theta")     # The parameter(s) to be monitored.
adaptSteps = 500              # Number of steps to "tune" the samplers.
burnInSteps = 1000            # Number of steps to "burn-in" the samplers.
nChains = 3                   # Number of chains to run.
numSavedSteps=50000           # Total number of steps in chains to save.
thinSteps=1                   # Number of steps to "thin" (1=keep every step).
nIter = ceiling( ( numSavedSteps * thinSteps ) / nChains ) # Steps per chain.
# Create, initialize, and adapt the model:
jagsModel = jags.model( "model.txt" , data=dataList , # inits=initsList ,
                        n.chains=nChains , n.adapt=adaptSteps )
# Burn-in:
cat( "Burning in the MCMC chain...\n" )
update( jagsModel , n.iter=burnInSteps )
# The saved MCMC chain:
cat( "Sampling final MCMC chain...\n" )
codaSamples = coda.samples( jagsModel , variable.names=parameters ,
                            n.iter=nIter , thin=thinSteps )
# resulting codaSamples object has these indices:
#   codaSamples[[ chainIdx ]][ stepIdx , paramIdx ]


#------------------------------------------------------------------------------
# EXAMINE THE RESULTS.

source("plotPost.R")

mcmcChain = as.matrix( codaSamples )

# Extract parameter values and save them.
mu = NULL
kappa = NULL
for ( condIdx in 1:nCond ) {
   mu = rbind( mu , mcmcChain[, paste("mu[",condIdx,"]",sep="") ] )
   kappa = rbind( kappa , mcmcChain[, paste("kappa[",condIdx,"]",sep="") ] )
}
save( mu , kappa , file=paste(fileNameRoot,"MuKappa.Rdata",sep="") )
chainLength = NCOL(mu)

# Histograms of mu differences:
windows(width=7,height=7)
plotPost( mu[1,]-mu[2,] , xlab=expression(mu[1]-mu[2]) , main="" ,
          breaks=20 , compVal=0 , ROPE=c(-.1,.1) )
savePlot( file=paste(fileNameRoot,"MuDiffs.jpg",sep="") , type="jpg" )

# Scatterplot of mu,kappa in each condition:
windows(width=7,height=7)
muLim = c(min(mu),max(mu))
kappaLim = c(min(kappa),max(kappa))
mainLab="Posterior"
thindex = round( seq( 1 , chainLength , len=300 ) )
plot( mu[1,thindex] , kappa[1,thindex] , main=mainLab ,
      xlab=expression(mu[c]) , ylab=expression(kappa[c]) , cex.lab=1.75 ,
      xlim=muLim , ylim=kappaLim , log="y" , col="red" , pch="1" )
points( mu[2,thindex] , kappa[2,thindex] , col="blue" , pch="2" )
savePlot( file=paste(fileNameRoot,"Scatter.jpg",sep="") , type="jpg" )





Tuesday, November 27, 2012

Shrinkage in multi-level hierarchical models


In a previous post and in a video, I used baseball data to illustrate some ideas about shrinkage and multiple comparisons in hierarchical models. Here I take it a step further, to multi-level hierarchical models, to illustrate shrinkage of estimates within different levels of the model.

Background: Consider the sport of baseball, in which players have opportunities at bat during a season of games, and occasionally hit the ball. In Major League Baseball, players typically hit the ball on about 24% of their opportunities at bat. That ratio, of hits (H) divided by opportunities at bat (AB), is called the "batting average" of each player. Players also must play a position in the field when the other team is at bat. Some fielding positions are specialized and those players are expected to focus on those skills, not necessarily on hitting. In particular, pitchers are not expected to be strong hitters, and catchers might also not be expected to focus so much on hitting. (But I'm no baseball aficionado, so please forgive any inadvertent insults or errors! My thanks to colleagues Ed Hirt and Jim Sherman for tutoring me in some of the basics of baseball rules.)

Our goal is to estimate the underlying probability of getting a hit by each player, based on their hits H, at bats AB, and primary position. The model will put each player's estimated hitting ability (i.e., probability of getting a hit) under a distribution for the primary position, and put those position distributions under an overarching distribution across all positions. Here is a diagram of the hierarchical model. It should be scanned from the bottom up, as described next.
The data are indicated at the bottom of the diagram, with Hi being the number of hits for player i, and ABi being the number of at-bats for player i. The parameter θi in the binomial distribution is the underlying probability of getting a hit for player i. The θi are assumed to come from a beta distribution, but there is a distinct beta distribution for each type of field position: pitcher, catcher, 1st base, 2nd base, 3rd base, shortstop, left field, center field, and right field. The subscript pp(i) indicates the primary position of player i. Thus, each of the nine position-specific beta distributions has a mean μpp and a consistency (like precision) κpp. (Now I notice a small error in the diagram: The ellipses labeled with pp(i) should instead be labeled with only pp.) Moving up the diagram, the nine position-specific means μpp come from a beta distribution, with a a mean μμpp that expresses the central tendency across all positions.At the top level, the priors are vague and noncommittal.

The data are from 948 players in the 2012 regular season of Major League Baseball. Thanks to Matthew Maenner for posting a comment explaining how to download the data. The model simultaneously estimates 948 + 9*2 + 4 = 970 parameters.

Results:


Let's first consider the estimates of hitting ability for different positions. At right are the marginal posterior distributions for the μpp parameters for the positions of catcher and pitcher. The blue distributions show the credible values of the parameters, and the red crosses on the x-axis indicate the ratio in the data of total hits divided by total at bats for all players in that position. Notice that the modes of the posterior are not centered on the red crosses. Instead, the modal estimates are shrunken toward the middle between the pitchers (who tend to have low batting averages) and the catchers (who tend to have higher batting averages). The lowest panel shows the distribution of differences between catchers and pitchers. The distribution falls far above zero, so we can say with high credibility that catchers hit better than pitchers. (Click on the picture to see it enlarged.)


At right is a comparison of two individual players with same record, 1 hit in 3 at bats, but playing different positions, namely catcher and pitcher. Notice that the red crosses are at the same place on the x-axes for the two players, but notice the radically different estimates of their underlying probability of getting a hit, because of the different positions they play. The data from all the other catchers tells the model that catchers tend to have theta values around 0.24. Because this particular catcher has so few data to inform his estimate, the estimate from the higher-level distribution dominates. The same is true for the pitcher, but the higher-level distribution says that pitchers tend to have theta values around 0.13. The resulting distribution of differences, in the lowest panel, suggests that these two players may have credibly different hitting abilities, even though their hits and at-bats are identical! In other words, because we know the players play these particular different positions, we can infer that they probably have different hitting abilities. (Click on the picture to see it enlarged.)


Here is a comparison of two indivdiual players, both of whom are pitchers, with seemingly quite different batting averages of 18/61 and 4/61, as marked by the red crosses on the x-axis. But the posterior estimates of their hitting probabilities are not so different. Notice the dramatic shrinkage of the estimates toward the average of players who are pitchers. In the lower panel, we see that a difference of zero is credible. (Click on the picture to see it enlarged.)


Here is a comparison of two indivdiual players with different records but both playing Center Field. The larger amount of individual data (593 and 120 at bats) allows somewhat less shrinkage than the previous example, so that the estimated difference excludes zero (but still overlaps the particular ROPE used here). Notice also the difference in widths of HDIs for the two players: There is a narrower HDI for the player with more data. (Click on the picture to see it enlarged.)


Here are two players from Right Field with huge numbers of at-bats and nearly the same (impressive!) batting average. The HDI of the difference falls almost entirely within the ROPE, so we accept that the players have identical probability of getting a hit (for practical purposes). (Click on the picture to see it enlarged.)







Summary: With multi-level hierarchical models, there is shrinkage within each level. In this model, there was shrinkage of the position parameters toward each other, as illustrated by the pitcher and catcher distributions. And there was shrinkage of the player parameters within each position, as shown by various examples above. The model also provided some strong inferences about player abilities based on position alone, as illustrated by the estimates for individual players with few at bats.

Monday, November 12, 2012

Video: Bayesian Methods Interpret Data Better

Another 15 minute talk, (to be) presented at the Psychonomic Society Special Session on Improving the Quality of Psychological Science. This talk briefly discusses sequential testing, the goal of achieving precision (as opposed to rejecting a null value), and multiple comparisons. The video can be found here.

Sunday, November 11, 2012

Video: Bayesian estimation supersedes the t test

Bayesian estimation supersedes the t test in 15 minutes of video.  
For the complete info, see the article and software at this site.