coding beacon

[programming & visualization]

Tag Archives: neural networks

Machine Learning Models of My Personal Interest (work in progress)

Convolutional Neural Networks

http://deeplearning.net/tutorial/lenet.html#lenet

Convolutional Neural Networks (CNN) are biologically-inspired variants of MLPs. From Hubel and Wiesel’s early work on the cat’s visual cortex [Hubel68], we know the visual cortex contains a complex arrangement of cells. These cells are sensitive to small sub-regions of the visual field, called a receptive field. The sub-regions are tiled to cover the entire visual field. These cells act as local filters over the input space and are well-suited to exploit the strong spatially local correlation present in natural images.

Additionally, two basic cell types have been identified: Simple cells respond maximally to specific edge-like patterns within their receptive field. Complex cells have larger receptive fields and are locally invariant to the exact position of the pattern.

The animal visual cortex being the most powerful visual processing system in existence, it seems natural to emulate its behavior. Hence, many neurally-inspired models can be found in the literature. To name a few: the NeoCognitron [Fukushima], HMAX [Serre07] and LeNet-5 [LeCun98], which will be the focus of this tutorial.

http://stackoverflow.com/questions/1313336/convolutional-neural-network-how-to-get-the-feature-maps?rq=1

alt text


Random Forests

Slides: http://www.cs.ubc.ca/~nando/540-2013/lectures.html

Andrew Ng: Deep Learning

Self-Taught Learning and Unsupervised Feature Learning


Choosing GPU Hardware for Neural Network Modeling (draft)

Hardware Interaction Model

CUDA

It is not yet clear to me whether whether specific NVidia hardware makes a lot of difference. More on this later. In the mean time, here’s a library that claims to need Fermi/Tesla or equivalent (“Fermi-generation GPU (GTX 4xx, GTX 5xx, or Tesla equivalent required.”) What the reason is, I can only guess.

https://code.google.com/p/cuda-convnet/

OpenCL

http://deeplearning.net/software/theano/tutorial/using_gpu.html (mentioned in the second half of the page)

More on this later

GPU Use with Neural Networks (Other Sources)

http://deeplearning.net/software/theano/tutorial/using_gpu.html