Learning Deep Boltzmann Machines

Code provided by Ruslan Salakhutdinov
Permission is granted for anyone to copy, use, modify, or distribute this program and accompanying programs and documents for any purpose, provided this copyright notice is retained and prominently displayed, along with a note saying that the original programs are available from our web page. The programs and documents are distributed without any warranty, express or implied. As the programs were written for research purposes only, they have not been tested to the degree that would be advisable in any important application. All use of these programs is entirely at the user's own risk.

How to make it work:

1. Create a separate directory and download all these files into the same directory
2. Download from http://yann.lecun.com/exdb/mnist the following 4 files:
   • train-images-idx3-ubyte.gz
   • train-labels-idx1-ubyte.gz
   • t10k-images-idx3-ubyte.gz
   • t10k-labels-idx1-ubyte.gz
3. Unzip these 4 files by executing:
   • gunzip train-images-idx3-ubyte.gz
   • gunzip train-labels-idx1-ubyte.gz
   • gunzip t10k-images-idx3-ubyte.gz
   • gunzip t10k-labels-idx1-ubyte.gz
   If unzipping with WinZip, make sure the file names have not been changed by WinZip.
4. Download code_DBM.tar which contains 16 files OR
   download each of the following 16 files separately:
   • demo_small.m Main file for training and fine-tuning a toy DBM model.
   • demo.m Main file for training and fine-tuning a DBM model (reproduces results of the DBM paper).
   • converter.m Converts raw MNIST digits into matlab format.
   • rbm.m Training RBM with binary hidden and binary visible units.
   • rbm_l2.m Training 2nd layer RBM with binary hidden and visible units.
   • dbm_mf.m Joint training of all layers in a DBM.
   • mf.m Implements mean-field inference.
   • backprop.m Backpropagation for fine-tuning a DBM.
   • mf_class.m Helper function used by backprop.m
   • CG_MNIST.m Conjugate Gradient optimization for fine-tuning a DBM.
   • CG_MNIST_INIT.m Conjugate Gradient optimization for fine-tuning a DBM (training top-level parameters, while holding low-level parameters fixed).
   • makebatches.m Creates minibatches for DBM training.
   • testerr.m Computes misclassification error on the MNIST dataset.
   • dispims.m Displays progress during DBM training.
   • minimize.m Conjugate gradient code.
   • README.txt
5. For the toy experiment, run demo_small in matlab.
6. Run demo in Matlab. This is a long version of training a Deep Boltzmann Machine, which can take over 2 days. The code also uses sparsity to improve model performance.
7. Make sure you have enough space to store the entire MNIST dataset on your disk.

I did not try to optimize this code, but if you find bugs, please e-mail me.

Contact Information