On the Linearity of Bayesian Classifiers

Sun, 17 Jul 2016 14:19:40 +0530

In his book, Neural Networks - A Comprehensive Foundation, Simon Haykin has an entire section (3.10) dedicated to how perceptrons and Bayesian classifiers are closely related when operating in a Gaussian environment. However, it is not until the end of the section that Haykin mentions that the relation is only limited to linearity. What is interesting about this is that a Perceptron can produce the same classification “model” as a Bayesian classifier, provided that the underlying data is drawn from a Gaussian distribution. This post is an experimental verification of that.

A Geometric Proof of the Perceptron Convergence Theorem

Wed, 06 Jul 2016 14:45:55 +0530

The last time I studied neural networks in detail was five years ago in college. I did touch upon backpropagation when Andrew Ng’s machine learning MOOC was offered on Coursera for the first time, but beyond that I’ve only dabbled with them through keras. Then recently, when I read about Coursera’s imminent decision to pull down much of their freely available material (you can read a rant about it here), I went on a downloading spree (many thanks to the wonderful coursera-dl). Of all the courses I downloaded, the one that caught my eye was Geoffrey Hinton’s course on Neural Networks for Machine Learning. Because of that and the fact that there were some computer vision projects going on at work, I decided to dive right in.

Perceptron on Jaidev's Blog

On the Linearity of Bayesian Classifiers

A Geometric Proof of the Perceptron Convergence Theorem