This is the home page for the course CS 256: Neural Computation, offered by the Department of Computer Science at the University of Vermont, Spring 2008.
(N.B., the content of this page changes frequently.)

General Information:

Catalogue description: Artificial neural networks, their computational capacity and limitations, and the algorithms used to train them. Statistical capacity, convergence theorems, reinforcement learning, generalization.

(Further details appear in the syllabus.)

• Class meets on Tuesdays and Thursdays in Room 223 Votey, 3:35 – 4:45 p.m.
• Office Hours: Tuesdays, 10:00 – 11:00 a.m., and Thursdays, 2:00 – 3:15 p.m., in Room 353 Votey (or by appointment).

Handouts:

Most handouts are distributed in pdf format, a page description language supported by Adobe Acrobat. If you do not have Acrobat Reader for your personal computer, you can download it for free from Adobe.

• Syllabus [pdf] 1/14/2008, 2 pages.
• The latex template should generate this typeset document.

Reading Assignments:

• RA1: Read Chapter 3 of M. Minsky, Computation: Finite and Infinite Machines, Prentice-Hall, NJ 1967, available on course reserve at the Bailey-Howe Library. Due: Tuesday, 22 Jan 08.
• RA2: Read the following two journal articles:
  1. John Hopfield, "Neural networks and physical systems with emergent collective computational abilities", Proceedings of the National Academy of Sciences of the USA, 79 1982, pp. 2554–2558.
  2. John Hopfield, "Neurons with graded response have collective computational properties like those of two-state neurons," Proceedings of the National Academy of Sciences of the USA, 81 1985, pp. 3088–3092
  For your convenience, both appear in Anderson and Rosenfeld, ed., Neurocomputing, vol. 1, MIT Press, 1988, which is on reserve.
• RA3: R. J. McEliece, E. C. Posner, E. R. Rodemich, and S. S. Venkatesh, "The Capacity of the Hopfield Associative Memory," IEEE Transactions on Information Theory, 33, 1987, pp. 461–483. On reserve.
• RA4: J. J. Hopfield and D. W. Tank, " 'Neural' Computation of Decisions in Optimization Problems," Biological Cybernetics, 52, 1985, pp. 141–152.
• RA5 D. A. Ackley, G. E. Hinton, T. J. Sejnowski, "A learning algorithm for Boltzmann Machines," Cognitive Science, 9, 1985, pp. 147–169, also contained in J. A. Anderson and E. Rosenfeld, ed., Neurocomputing, MIT Press, Cambridge, MA, 1988, pp. 368–649.

Homework Assignments:

• HW1: Problems set HW1 is due in class on Thursday, January 31, 2008.
• HW2 was a separate handout, and collected on Feb. 13.
• HW3 Problem 1. Do Hopfield and Tank's choices for the the weight matrix and threshold biases for the Traveling Salesman Problem (TSP), always lead to a stationary fixed point? Explain. Problem 2. Simulate Hopfield and Tank's method for solving the TSP. How well does it work?. Due, Thursday, Feb. 28.

Lecture Notes [.pdf] and Mathematica Notebooks [.nb]:

• Introduction [pdf] 14 Jan. 2008.
• McCulloch-Pitts Networks [pdf] 24 Jan. 2008.
• Hopfield Networks 1 [pdf, hopfield.nb] 24 Jan. 2008.
• Probability Theory [pdf] 4 Feb. 2008.
• Hopfield Networks 2 [pdf] 25 Feb. 2005.
• Perceptrons [pdf] 8 Mar. 2005.
• LMS Algorithm [pdf] 9 Mar. 2005.
• Backpropagation Algorithm [pdf] 1 Apr. 2005.
• Capacity of an LTU [pdf] 22 Apr. 2008.

External Resources:

• International Society for Bayesian Analysis
• www.boosting.org, a compilation of research on boosting algorithms in machine learning.
• Caltech's Ph.D. program on Computation and Neuron Systems (CNS).
• CiteSeer: a search engine for publications in computer and information science.
• COLT: Computational Learning Theory
• John Hopfield's home page.
• IEEE Computational Intelligence Society supports the IEEE Transactions on Neural Networks (Search on IEEE XPlore)
• IEEE Information Theory Society supports the IEEE Transactions on Information Theory (Search on IEEE XPlore)
• International Neural Network Society (INNS).
• Journal of Machine Learning Research.
• Kernal-Machines.org.
• The LaTeX Project home page.
• Teuvo Kohonen's home page.
• Machine Learning, a journal published by Kluwer.
• MLNet: Machine Learning Network Online Information Service.
• Neural Computation, published by MIT Press.
• NeuroCOLT: Neural Networks and Computational Learning Theory
• Nils J. Nilsson's home page.
• NIPS and NIPS Online: A DjVu archive of Volumes 1–13 of Advances in Neural Information Processing Systems.
• RIKEN Brain Science Institute, Laboratory for Mathematical Neuroscience.
• Rich Sutton's web site and his 1988 paper on the temporal difference algorithm.
• Richard A. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, MA, 1998.
• Gerald Tesauro, Temporal difference learning and TD-Gammon, Communications of the ACM, 38(3), 1995, pp. 58–68.
• A facsimile of Alan M. Turing's 1948 report, Intelligent Machinery. An annotated version appears in B. Jack Copeland, ed., The Essential Turing, Clarendon Press, Oxford, 2004, pp. 395 – 432.
• UCI Machine Learning Repository.
• Bernard Widrow's home page.