Increasingly, statisticians are faced with the problem of identifying interesting
subsets of predictors from among a large number of candidates. Existing methods
for variable selection, such as stochastic search algorithms, tend to explore
the model space too slowly in large dimensions. Shotgun stochastic search (SSS)
algorithms have been proposed as an efficient alternative. As current SSS algorithms
rely on conjugacy, they are not appropriate for generalized linear models
without use of approximation methods. This article compares the frequently used
Laplace approximation with two alternatives based on Variational Bayes methods.
The comparison is illustrated using several simulated data examples and
an application to the problem of predicting conception using data on timing of
intercourse in the menstrual cycle. This application also illustrates the problem
of selection of interactions.