Convolutional Neural Networks
Convolutional Neural Networks (CNN) are biologically-inspired variants of MLPs. From Hubel and Wiesel’s early work on the cat’s visual cortex [Hubel68], we know the visual cortex contains a complex arrangement of cells. These cells are sensitive to small sub-regions of the visual field, called a receptive field. The sub-regions are tiled to cover the entire visual field. These cells act as local filters over the input space and are well-suited to exploit the strong spatially local correlation present in natural images.
Additionally, two basic cell types have been identified: Simple cells respond maximally to specific edge-like patterns within their receptive field. Complex cells have larger receptive fields and are locally invariant to the exact position of the pattern.
The animal visual cortex being the most powerful visual processing system in existence, it seems natural to emulate its behavior. Hence, many neurally-inspired models can be found in the literature. To name a few: the NeoCognitron [Fukushima], HMAX [Serre07] and LeNet-5 [LeCun98], which will be the focus of this tutorial.
Andrew Ng: Deep Learning
Self-Taught Learning and Unsupervised Feature Learning
Hardware Interaction Model
It is not yet clear to me whether whether specific NVidia hardware makes a lot of difference. More on this later. In the mean time, here’s a library that claims to need Fermi/Tesla or equivalent (“Fermi-generation GPU (GTX 4xx, GTX 5xx, or Tesla equivalent required.”) What the reason is, I can only guess.
http://deeplearning.net/software/theano/tutorial/using_gpu.html (mentioned in the second half of the page)
More on this later
GPU Use with Neural Networks (Other Sources)
The website (http://orange.biolab.si/) describes Orange as follows: “Open source data visualization and analysis for novice and experts. Data mining through visual programming or Python scripting. Components for machine learning. Add-ons for bioinformatics and text mining. Packed with features for data analytics.”
I am currently evaluating various front-ends for modeling various project workflows and visual programming is very appealing for this purpose. Orange’s environment looks as follows:
Orange is already included in the list of data mining software I compiled back in 2013 https://wordpress.com/post/55540359/77.