Contents

## Advanced topics Self-taught learning Learning feature hierarchies (Deep learning) Scaling up Self-taught learning

Tabback, Tom, On-Air Personality and General Manager has reference to this Academic Journal, PHwiki organized this Journal Advanced topics Outline Self-taught learning Learning feature hierarchies (Deep learning) Scaling up Self-taught learning

This Particular University is Related to this Particular Journal

Supervised learning Cars Motorcycles Semi-supervised learning Unlabeled images (all cars/motorcycles) Self-taught learning Unlabeled images (r in addition to om internet images)

Self-taught learning Sparse coding, LCC, etc. f1, f2, , fk Use learned f1, f2, , fk to represent training/test sets. Using f1, f2, , fk a1, a2, , ak If have labeled training set is small, can give huge per as long as mance boost. Learning feature hierarchies/Deep learning Why feature hierarchies pixels edges object parts (combination of edges) object models

Deep learning algorithms Stack sparse coding algorithm Deep Belief Network (DBN) (Hinton) Deep sparse autoencoders (Bengio) [Other related work: LeCun, Lee, Yuille, Ng ] Deep learning with autoencoders Logistic regression Neural network Sparse autoencoder Deep autoencoder Logistic regression Logistic regression has a learned parameter vector q. On input x, it outputs: where Draw a logistic regression unit as:

Neural Network String a lot of logistic units together. Example 3 layer network: x1 x2 x3 +1 +1 Layer 1 Layer 3 Layer 3 Neural Network x1 x2 x3 +1 +1 Layer 1 Layer 2 Layer 4 +1 Layer 3 Example 4 layer network with 2 output units: Neural Network example [Courtesy of Yann LeCun]

Training a neural network Given training set (x1, y1), (x2, y2), (x3, y3 ), . Adjust parameters q ( as long as every node) to make: (Use gradient descent. Backpropagation algorithm. Susceptible to local optima.) Unsupervised feature learning with a neural network Autoencoder. Network is trained to output the input (learn identify function). Trivial solution unless: Constrain number of units in Layer 2 (learn compressed representation), or Constrain Layer 2 to be sparse. a1 a2 a3 Training a sparse autoencoder. Given unlabeled training set x1, x2, Unsupervised feature learning with a neural network Reconstruction error term L1 sparsity term a1 a2 a3

Unsupervised feature learning with a neural network x4 x5 x6 +1 Layer 1 Layer 2 x1 x2 x3 x4 x5 x6 x1 x2 x3 +1 Layer 3 a1 a2 a3 Unsupervised feature learning with a neural network x4 x5 x6 +1 Layer 1 Layer 2 x1 x2 x3 +1 a1 a2 a3 New representation as long as input. Unsupervised feature learning with a neural network x4 x5 x6 +1 Layer 1 Layer 2 x1 x2 x3 +1 a1 a2 a3

Unsupervised feature learning with a neural network x4 x5 x6 +1 x1 x2 x3 +1 a1 a2 a3 +1 b1 b2 b3 Unsupervised feature learning with a neural network x4 x5 x6 +1 x1 x2 x3 +1 a1 a2 a3 +1 b1 b2 b3 Unsupervised feature learning with a neural network x4 x5 x6 +1 x1 x2 x3 +1 a1 a2 a3 +1 b1 b2 b3

Unsupervised feature learning with a neural network x4 x5 x6 +1 x1 x2 x3 +1 a1 a2 a3 +1 b1 b2 b3 New representation as long as input. Unsupervised feature learning with a neural network x4 x5 x6 +1 x1 x2 x3 +1 a1 a2 a3 +1 b1 b2 b3 Unsupervised feature learning with a neural network x4 x5 x6 +1 x1 x2 x3 +1 a1 a2 a3 +1 b1 b2 b3 +1 c1 c2 c3

Unsupervised feature learning with a neural network x4 x5 x6 +1 x1 x2 x3 +1 a1 a2 a3 +1 b1 b2 b3 +1 c1 c2 c3 New representation as long as input. Use [c1, c3, c3] as representation to feed to learning algorithm. Deep Belief Net Deep Belief Net (DBN) is another algorithm as long as learning a feature hierarchy. Building block: 2-layer graphical model (Restricted Boltzmann Machine). Can then learn additional layers one at a time. Restricted Boltzmann machine (RBM) Input [x1, x2, x3, x4] Layer 2. [a1, a2, a3] (binary-valued) MRF with joint distribution: Use Gibbs sampling as long as inference. Given observed inputs x, want maximum likelihood estimation: x4 x1 x2 x3 a2 a3 a1

Other resources Workshop page: http://ufldl.stan as long as d.edu/eccv10-tutorial/ Code as long as Sparse coding, LCC. References. Full online tutorial.

## Tabback, Tom On-Air Personality and General Manager

Tabback, Tom is from United States and they belong to KAZM-AM and they are from Sedona, United States got related to this Particular Journal. and Tabback, Tom deal with the subjects like Entertainment Programming; Music Programming; News Programming

## Journal Ratings by Johnson & Wales University

This Particular Journal got reviewed and rated by Johnson & Wales University and short form of this particular Institution is US and gave this Journal an Excellent Rating.