These papers, books, and essays serve as a shared basis of knowledge for the MIT Probabilistic Computing Project.

### Non-Technical Background (Required)

- The Structure of Scientific Revolutions by Thomas Kuhn. [Amazon]
- The Innovator’s Dilemma: The Revolutionary Book That Will Change the Way You Do Business by Clayton Christensen. [Amazon]
- Epigrams on Programming by Alan Perlis. [html] [Turing award page][Foreword to SICP]
- The Effective Executive: The Definitive Guide to Getting the Right Things Done by Peter F. Drucker.[Amazon]
- Technology and Courage by Ivan Sutherland. [pdf]
- Mathematical Writing by Don Knuth. [pdf]
- You and Your Research by Hamming [html]
- Whirlwind I Computer [Wikipedia]
- Worse Is Better by Richard Gabriel [html]
- Who Inspired the Leamer-Rosenthal Prizes? Part II by Ed Leamer [html]
- The Laws of Medicine: Field Notes From an Uncertain Science by Siddhartha Mukherjee [Interview][Amazon]
- The War of Art: Winning the Inner Creative Battle by Steven Pressfield [Amazon]
- Study of Telemedicine Finds Misdiagnoses of Skin Problems by Melinda Beck [html]
- Many Analysts, One Dataset: Making Transparent How Variations in Analytical Choices Affect Results by Silberzahn et al. [pdf]
- What Is It Like To Be A Bat? by Thomas Nagel [pdf]
- When U.S. air force discovered the flaw of averages [html]
- The origins of SageMath – creating a viable open source alternative to Magma, Maple, Mathematica, and Matlab [pdf]
- Politics and the English Language by George Orwell [html]
- The Sciences of the Artificial by Herbert Simon [Amazon]
- Building Robust Systems an essay by Gerald Jay Sussman [pdf]

### Technical Background (Required)

- Mathematics: Its Content, Methods and Meaning: Chapter 11. The Theory of Probability by A. N. Kolmogorov. [Amazon]
- Structure and Interpretation of Computer Programs by Harold Abelson, Gerald Jay Sussman, and Julie Sussman. [Amazon] [pdf]
- Classifier Technology and the Illusion of Progress by David J. Hand. [pdf]
- Machine Learning: The High-Interest Credit Card of Technical Debt by Sculley et al. [pdf]
- (Skim) On Computable Numbers and the Entscheidungsproblem by Alan Turing. [pdf]
- (Skim) Noncomputable Conditional Distributions by Ackerman, Freer, and Roy [pdf]
- Natively Probabilistic Computation; PhD Dissertation by Vikash Mansinghka, 2009 [pdf]
- Design of LISP-Based Processors by Steele and Sussman, 1979 [pdf]
- Lisp: A Language for Stratified Design by Abelson and Sussman, 1987 [pdf]
- Steps Toward Artificial Intelligence by Minsky, 1961 [pdf]
- On Chomsky and the Two Cultures of Statistical Learning by Peter Norvig [html]
- Statistical Modeling: The Two Cultures by Leo Breiman [pdf]
- Critique of Paper by “Deep Learning Conspiracy” (Nature 521 p 436) by Jürgen Schmidhuber [html]
- Pages 3-31 (Introducton) from Artificial Intelligence A Modern Approach by Russell and Norvig [pdf]
- Confessions of a Used Programming Language Salesman by Erik Meijer [pdf]

### Probabilistic Programming (Required)

- VentureScript Tutorial. [html]
- BayesDB Documentation. [html]
- Picture: an imperative probabilistic programming language for scene perception. Kulkarni, T., Kohli, P., Tenenbaum, J., Mansinghka, V. [Under review at CVPR.] [[webpage]]43
- Bayesian Statistics Without Tears: A Sampling-Resampling Perspective by Smith and Gelfand. [pdf]
- When are probabilistic programs probably computationally tractable? Freer, C., Mansinghka, V., Roy, D. [NIPS Workshop on Monte Carlo Methods for Modern Applications.], 2010. [pdf]
- Bayesian Optimization as a Probabilistic Meta-Program; MEng thesis by Ben Zinberg, 2015 [pdf]

### Probabilistic Programming (Optional, Outdated)

- Venture: a general purpose probabilistic programming platform with efficient stochastic inference. Mansinghka, V., Radul, A. [preprint available as arXiv, number 1404:0099] [[pdf]]47
- A stochastic programming perspective on nonparametric Bayes. Roy, Mansinghka, Goodman and Tenenbaum. [ICML Nonparametric Bayes Workshop.], 2008. [Oral] [[pdf]]48

### Foundations of Probabilistic Modeling (Required)

- Bayesianism and Causality, or, Why I am Only a Half-Bayesian by Judea Pearl. [pdf]
- Why I Am Not a Bayesian by Clark Glymour. [pdf]
- Chapters 1-3 of Probability Theory: The Logic of Science by E. T. Jaynes. [pdf] [Amazon]
- Chapter 28 (Model Comparison and Occam’s Razor) from Information Theory, Inference, and Learning Algorithms by David MacKay. [webpage]

### Standard Library of Modeling Techniques (Required)

- Chapter 45 (Gaussian Processes) from Information Theory, Inference, and Learning Algorithms by David MacKay. [webpage]
- Unifying Rational Models of Categorization via the Hierarchical Dirichlet Process by Griffiths, Canini, Sanborn, and Navarro. [pdf]
- Discovering latent classes in relational data by C. Kemp, T. L. Griffiths, and J. B. Tenenbaum. [pdf]
- Topics in Semantic Representation by Griffiths and Steyvers. [pdf]
- Finding Scientific Topics by Griffiths and Steyvers. [pdf]
- Continuous Time Dynamic Topic Models by Wang, Blei, and Heckerman. [pdf]
- Practical Bayesian Optimization of Machine Learning Algorithms by Snoek, Larochelle, and Adams.[webpage]
- A Unifying Review of Linear Gaussian Models by Roweis and Ghahramani. [pdf]

### Standard Library of Inference Techniques (Required)

- Composable Probabilistic Inference with Blaise by Keith A. Bonawitz. [pdf]
- Chapters 5-8 of Venture: a general purpose probabilistic programming platform with efficient stochastic inference. Mansinghka, V., Radul, A. [preprint available as arXiv, number 1404:0099] [[pdf]]47
- An Introduction to MCMC for Machine Learning by Andrieu, De Freitas, Doucet, and Jordan. [pdf]
- The Markov Chain Monte Carlo Revolution by Persi Diaconis. [pdf]
- Convergence of Sequential Monte Carlo-based Sampling Methods by Huggins and Roy. [pdf]

### Useful Examples (Optional)

- MrBayes: Bayesian Inference of Phylogenetic Trees by Huelsenbeck and Ronquist. [pdf] [webpage]
- FastSLAM: FastSLAM: A Factored Solution to the Simultaneous Localization and Mapping Problem. [pdf]
- First few chapters of Bayesian Spectrum Analysis and Parameter Estimation by Bretthorst. [pdf]
- Recovering Intrinsic Images from a Single Image by Tappen, Freeman, and Adelson. [pdf]
- Bayesian Inference for PCFGs via Markov chain Monte Carlo by Johnson, Griffiths and Goldwater. [pdf]
- Global Seismic Monitoring as Probabilistic Inference by Arora, Russell, Kidwell, and Sudderth. [pdf]

### Automatic Model Discovery (Optional)

- CrossCat: A Fully Bayesian Nonparametric Method for Analyzing Heterogeneous, High Dimensional Data by Mansinghka et al. [Journal of Machine Learning Research, accepted pending revision, available internally]
- Structure Discovery in Nonparametric Regression through Compositional Kernel Search by Duvenaud, Lloyd, Grosse, Tenenbaum, Ghahramani. [pdf]
- Exploiting compositionality to explore a large space of model structures by Grosse et al. [pdf]
- Model Selection in Compositional Spaces by Grosse. PhD Thesis. [pdf]
- The discovery of structural form by Kemp and Tenenbaum. [pdf]
- Structured priors for structure learning by Mansinghka, Kemp, Tenenbaum, and Griffiths [pdf]
- Chapter 28 (Model Comparison and Occam’s Razor) from Information Theory, Inference, and Learning Algorithms by David MacKay. [webpage]

### Reference Material on Approximate Inference (Optional)

- Probabilistic Inference using Markov chain Monte Carlo Methods by Neal. [pdf]
- Hamiltonian Monte Carlo tutorial by Neal. [pdf]
- Exact and approximate inference via systematic stochastic search by Mansinghka et al. [pdf]
- Factor graphs and the sum-product algorithm by Kschischang, Frey, and Loeliger. [pdf]
- An Architecture for Parallel Topic Models by Smola and Narayanamurthy. [pdf]
- LeCun, Bottou, Orr and Muller: Efficient BackProp, in Orr, and Muller (Eds), Neural Networks: Tricks of the trade. [pdf]
- Black Box Variational Inference by Ranganath, Gerrish and Blei. [pdf]