coding beacon

[programming & visualization]

Monthly Archives: April 2015

Shiny: Extensions

Machine Learning Models of My Personal Interest (work in progress)

Convolutional Neural Networks

Convolutional Neural Networks (CNN) are biologically-inspired variants of MLPs. From Hubel and Wiesel’s early work on the cat’s visual cortex [Hubel68], we know the visual cortex contains a complex arrangement of cells. These cells are sensitive to small sub-regions of the visual field, called a receptive field. The sub-regions are tiled to cover the entire visual field. These cells act as local filters over the input space and are well-suited to exploit the strong spatially local correlation present in natural images.

Additionally, two basic cell types have been identified: Simple cells respond maximally to specific edge-like patterns within their receptive field. Complex cells have larger receptive fields and are locally invariant to the exact position of the pattern.

The animal visual cortex being the most powerful visual processing system in existence, it seems natural to emulate its behavior. Hence, many neurally-inspired models can be found in the literature. To name a few: the NeoCognitron [Fukushima], HMAX [Serre07] and LeNet-5 [LeCun98], which will be the focus of this tutorial.

alt text

Random Forests


Andrew Ng: Deep Learning

Self-Taught Learning and Unsupervised Feature Learning

Choosing GPU Hardware for Neural Network Modeling (draft)

Hardware Interaction Model


It is not yet clear to me whether whether specific NVidia hardware makes a lot of difference. More on this later. In the mean time, here’s a library that claims to need Fermi/Tesla or equivalent (“Fermi-generation GPU (GTX 4xx, GTX 5xx, or Tesla equivalent required.”) What the reason is, I can only guess.

OpenCL (mentioned in the second half of the page)

More on this later

GPU Use with Neural Networks (Other Sources)

Open Source Data Mining Software: Orange

The website ( describes Orange as follows: “Open source data visualization and analysis for novice and experts. Data mining through visual programming or Python scripting. Components for machine learning. Add-ons for bioinformatics and text mining. Packed with features for data analytics.

I am currently evaluating various front-ends for modeling various project workflows and visual programming is very appealing for this purpose. Orange’s environment looks as follows:

orange0b_scrshotsOrange is already included in the list of data mining software I compiled back in 2013