Thoughts on data science, statistics and machine learning.
Playing With the Konmari Method
I heard about the bestseller The Life-Changing Magic of Tidying Up at a SciPy talk about deculttering your data science project. The speakers admitted they hadn’t read it - they were simply trying to point out that tidying up your space and tidying up your software project are both similar.
I’ve been married and living with my wife for about a year now. After we moved into “our own home” last year, we have both undergone major role reversals when it comes to tidying up. I was never accustomed to spaces larger than a single bedroom, so I never cared enough to sort or declutter my space as long as my desk and bed were clean. As for my wife, she never owned too much stuff (between the two of us, I’m the hoarder) and therefore never had to make a chore out of tidying up. Now that I live in a fairly spacious apartment, even a little clutter looks very conspicuous to me. My wife generally agrees with me about tidying up, but she’s not anal about it. I have been having arguments about the clutter in my house with her for a long time now. Since both of us jokingly say that I’m the wife in this relationship (by the way, I would be proud to be one), tidying up and decluttering is often left up to me. I lost no time in buying Marie Kondo’s book and diving right in.
On the Linearity of Bayesian Classifiers
In his book, Neural Networks - A Comprehensive Foundation, Simon Haykin has an entire section (3.10) dedicated to how perceptrons and Bayesian classifiers are closely related when operating in a Gaussian environment. However, it is not until the end of the section that Haykin mentions that the relation is only limited to linearity. What is interesting about this is that a Perceptron can produce the same classification “model” as a Bayesian classifier, provided that the underlying data is drawn from a Gaussian distribution. This post is an experimental verification of that.
A Geometric Proof of the Perceptron Convergence Theorem
The last time I studied neural networks in detail was five years ago in college. I did touch upon backpropagation when Andrew Ng’s machine learning MOOC was offered on Coursera for the first time, but beyond that I’ve only dabbled with them through keras. Then recently, when I read about Coursera’s imminent decision to pull down much of their freely available material (you can read a rant about it here), I went on a downloading spree (many thanks to the wonderful coursera-dl). Of all the courses I downloaded, the one that caught my eye was Geoffrey Hinton’s course on Neural Networks for Machine Learning. Because of that and the fact that there were some computer vision projects going on at work, I decided to dive right in.
Understanding Allen Downey's Solution to the M&M Problem
Allen Downey makes a very good case for learning advanced mathematics through programming (Check the first section of the preface of Think Bayes, titled “My theory, which is mine”). But before the reader can hit paydirt with using the Bayes theorem in programming, Downey makes you go through some elementary problems in probability, which have to be solved by hand first, if you expect to have a clear enough understanding of the concept. I can vouch for this way of learning complex concepts. The way I learnt the backpropagation algorithm (and its derivation), was with a pen, paper and a calculator.