Schedule

Readings

None

In class

• March 29th Notes

Readings

None

Readings

• BDA, Chapter 1
• Review probability. See this Probability Resource.
• Review R. I suggest reading R for Data Science.
• Frank Harrell, “My Journey From Frequentist to Bayesian Statistics“

In class

• April 3rd Notes

Readings

None

In class

• April 5th Notes

Readings

None

Readings

None

In class

• April 10th notes

Readings

• BDA3, Ch. 2: “Single Parameter Models”
• BDA3, Ch. 10: “Introduction to Bayesian

Readings

None

Additional Resources

• Richard McElreath, “Markov Chains: Why Walk When You Can Flow?“
• Chi Feng, Interactive MCMC Demo
• Getting started with RStan
• Michael Clark, Bayesian Basics

Readings

None

In class

• Replicating Nunn-Wantchekon (2011)

Readings

None

In class

• Bayesplot vignettes

Readings

None

Readings

None

Summary

Lectured on separation in binomial models, and using the Student-t distribution as a model for heteroskedasticity and robust regression.

In class

• Notes on Separation
• Notes on Robust Regression
• Notes on Heteroskedasticity

Readings

None

In class

• Lab canceled

Readings

None

Summary

Started working on Assignment 1

Readings

None

Summary

Lectured on posterior predictive checks (BDA Ch. 6) and model checking (BDA Ch. 7), with a discussion of generalization error.

In class

• Notes on Model checking
• Notes on Model comparison

Readings

• BDA3, Ch. 6: “Model Checking” BDA3, Ch. 7: “Evaluating, Comparing, and Expanding Models”
• Gabry, J., D. Simpson, A. Vehtari, M. Betancourt, and Gelman, A. “Visualization in Bayesian workflow.” (2017)
• Vehtari and Gabry (2018) “Using the loo package (version >= 2.0.0).” R package vignette.
• Vehtari, Gelman, and Gabry (2017) “Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC” Statistics and Computing
• Gabry, Jonah. “Graphical posterior predictive checks using the bayesplot package” 2018-03-30. R package vignette.

Additional Resources

• Gelman, Meng, and Stern. “Posterior Predictive Assessment of Model Fitness Via Realized Discrepencies.” Statistica Sinica. 1996.
• Kruschke. “Posterior predictive checks can and should be Bayesian: Comment on Gelman and Shalizi, ‘Philosophy and the practice of Bayesian statistics’.” British Journal of Mathematical and Statistical Psychology. 2013.

In class

• May 3rd notes - comparing Poisson and Negative Binomial models

Readings

None

Summary

• Reviewed schedule
• Lecture on model comparison, selection, and averaging

In class

• Lecture slides on Model Selection, Comparison, Averaging

Readings

• Yao, Vehtari, Simpson, and Gelman. “Using Stacking to Average Bayesian Predictive Distributions.” Bayesian Analysis. 2018.
• Vehtari and Gabry. “Bayesian Stacking and Pseudo-BMA weights using the loo package“

Additional Resources

• Vehtari, A. and Ojanen, J. (2012). “A survey of Bayesian predictive methods for model assessment, selection and comparison.” Statistics Survey.
• Kass and Raftery. (1993) Bayes Factors. Journal of the American Statistical Association
• Hoeting, Madigan, Raftery, and Volinsky (1995) “Bayesian model averaging: a tutorial.” Statistical Science
• Tiago M. Fragoso, Francisco Louzada Neto, Bayesian model averaging: A systematic review and conceptual classification
• Montgomery and Nyhan (2010) “Bayesian Model Averaging: Theoretical Developments and Practical Applications” Political Analysis
• BMA R package
• BMS R package

Summary

Bias-variance tradeoff. Bayesian hierarchical models and shrinkage. The Efron (1970) baseball hit example.

In class

• Shrinkage and Hierarchical Models

Readings

• BDA3, Ch. 5. “Hierarchical Models”
• James, Gareth, Daniela Witten, Trevor Hastie and Robert Tibshirani. An Introduction to Statistical Learning with Applications in R. 2014. PDF here. Sec 2.2: “Assessing Model Accuracy”’
• Bob Carpenter, Jonah Gabry and Ben Goodrich. “Hierarchical Partial Pooling for Repeated Binary Trials.” Stan Case Studies 2017-01-19.

In class

• May 10th R script - regularized regression from a machine learning perspective, via glmnet

Readings

None

Summary

Bayesian shrinkage and regularization for regression models.

In class

• Shrinkage and Regularized Regression

Readings

• ISLR Sec 6.2 “Shrinkage Methods”
• BDA3. Sec 14.6 “Regularization and dimension reduction for multiple predictors”
• Stan Modeling Language Reference. Ch 9. “Regression Models”
• http://mc-stan.org/users/documentation/case-studies/qr_regression.html
• https://cran.r-project.org/web/packages/rstanarm/vignettes/lm.html
• Polson, Nicholas and James G. Scott (2010) “Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction” Bayesian Statistics
• Juho Piironen and Aki Vehtari. 2017. “Sparsity information and regularization in the horseshoe and other shrinkage priors“

Summary

Continued discussion of shrinkage in regression models. VAariable selection in regression models.

Readings

• Juho Piironen and Aki Vehtari (2017). Comparison of Bayesian predictive methods for model selection. Statistics and Computing, 27(3):711-735. doi:10.1007/s11222-016-9649-y. online.
• projpred Quick Start
• Hahn and Carlos M. Carvalho “Decoupling shrinkage and selection in bayesian linear models: a posterior summary perspective”. 2015. Journal of the American Statistical Association

Summary

More on Bayesian variable shrinkage and variable selection in regression models.

Readings

None

Summary

Continuation of Bayesian shrinkage in regression and variable selection methods.

In class

• Shrinkage and Regularized Regression notes

Readings

None

Summary

Bayesian hierarchical/multi-level regression models.

Readings

• BDA3 Ch. 15: “Hierarchical Linear Models”
• Gabry and Goodrich. Estimating Generalized (Non-)Linear Models with Group-Specific Terms with rstanarm 2018.
• Fonnesbeck. A Primer on Bayesian Multilevel Modeling using PyStan. 2016. Stan Case Studies. This uses the Python interface to Stan, but the Stan models are the same, and so is the description. See this for R code of the Radon model.

Additional Resources

• Stegmueller. “How many countries for multilevel modeling? a comparison of frequentist and Bayesian approaches.” American Journal of Political Science. 2013.
• Rendon. “Fixed and Random Effects in Classical and Bayesian Regressions” Oxford Bulletin of Economics and Statistics. 2012.
• Clark and Linzer. “Should I use fixed or random effects?” Political Science Research and Methods 2014.
• Bates. lme4: Mixed-effects modeling with R 2010. The R package lme4 estimates mixed-effects models using MLE-type methods. This book is also a good introduction to these methods.
• Burkner. “Advanced Bayesian Multilevel Modeling with the R package brms” and “brms: An R Package for Bayesian Multilevel Models using Stan” The brms package is an alternative to the rstanarm package. Like that package it uses Stan to estimate the models, but provides a user-friendly front-end.
• Gelman and Hill. *Data Analysis Using Regression and Multilevel/Hierarchical Models” 2006. An accessible introduction to these models.

Readings

• Richard McElreath, Multilevel Regression as Default (blog post)
• Richard McElreath, Metamorphosis and the Multilevel Model (blog post)

Additional Resources

• Statistical Rethinking, Chapter 12 (free sample pdf online)

Summary

• Bayesian Measurment models and item-response models
• Using MAP and variational inference to approximate a posterior distribution.

Readings

• Stan Modeling Language User’s Guide and Reference Manual: Stan Version 2.17.0, Sec. 9.11 “Item-Response Theory Models” (p. 142).
• Jackman, Simon. (2009) Bayesian Analysis for the Social Sciences, Ch. 9 “Bayesian Approaches to Measurment”. This chapter is a good overview of some common, basic measurement mdoels. Ignore the Gibbs/MCMC estimation details due to the datedness of the source.
• Clinton, Joshua, Simon Jackman, and Douglas Rivers. 2004. “The Statistical Analysis of Roll Call Data” This was the first application of Bayesian statistical methods to estimating ideal points in political science. There have been many innovations since then, but since this article was written when many were unfamiliar with Bayesian stats, it will provide an good introduction to some of the issues.
• Quinn, Kevin. (2004) “Bayesian Factor Analysis for Mixed Ordinal and Continuous Responses” Political Analysis Ignore the Gibbs sampling method, but note how the different outcomes can be modeled.
• Kruschke. Bayesian Iterm Response Theory in JAGS. Ignore the JAGS model. This is a quick introduction to Bayesian IRT models.
• edstan package vignette. This package provides pre-programmed Stan models for common item-response theory models.

Summary

Draft of research project is due at the start of class. Review and collaboratively work on research projects.

Readings

None

Summary

Review and work on research projects.

Readings

None

Summary

Review and work on research projects.

Readings

None