A Deep Learning course taught by Yann LeCun, a pioneer of convolutional neual networks and Facebook's Chief AI scientist, has been made available online for free.

This is courtesy of the New York University's Center for Data Science or NYU CDS for short, where Yann LeCun taught the course last Spring under the code name DS-GA 1008.It is based on Python/Pytorch wich code can be run online on Jupyter Notebooks.

A quick look at the curiculum shows that the course is cutting edge and covers most deep learning techniques;supervised/self-supervised learning, embedding methods, metric learning, convolutional and recurrent nets and all that with practical application on computer vision, natural language understanding and speech recognition.

The 15-week curriculum in full:

Week 1

• Motivation of Deep Learning, and Its History and Inspiration
• Evolution and Uses of CNNs and Why Deep Learning?
• Problem Motivation, Linear Algebra, and Visualization

Week 2

• Introduction to Gradient Descent and Backpropagation Algorithm
• Computing gradients for NN modules and Practical tricks for Back Propagation
• Artificial neural networks (ANNs)

Week 3

• Visualization of neural networks parameter transformation and fundamental concepts of convolution
• ConvNet Evolutions, Architectures, Implementation Details and Advantages.
• Properties of natural signals

Week 4

• Linear Algebra and Convolutions

Week 5

• Optimisation Techniques I
• Optimisation Techniques II
• Understanding convolutions and automatic differentiation engine

Week 6

• Applications of Convolutional Network
• RNNs, GRUs, LSTMs, Attention, Seq2Seq, and Memory Networks
• Architecture of RNN and LSTM Model

Week 7

• Energy-Based Models
• SSL, EBM with details and examples
• Introduction to autoencoders

Week 8

• Contrastive Methods in Energy-Based Models
• Regularized Latent Variable Energy Based Models
• Generative Models - Variational Autoencoders

Week 9

• Discriminative Recurrent Sparse Auto-Encoder and Group Sparsity
• World Models and Generative Adversarial Networks
• Generative Adversarial Networks

Week 10

• Self-Supervised Learning - Pretext Tasks
• Self-Supervised Learning - ClusterFit and PIRL
• The Truck Backer-Upper

Week 11

• Activation and loss functions (part 1)
• Loss Functions (cont.) and Loss Functions for Energy Based Models
• Prediction and Policy learning Under Uncertainty (PPUU)

Week 12

• Deep Learning for NLP
• Decoding Language Models
• Attention and the Transformer

Week 13

• Graph Convolutional Networks I
• Graph Convolutional Networks II
• Graph Convolutional Networks III

Week 14

• Deep Learning for Structured Prediction
• Graphical Energy-based Methods
• Overfitting and regularization

Week 15

• Inference for latent variable Energy-Based Models
• Training latent variable Energy-Based Models (EBMs)

On the course's main website one can find the whole course material in video, text and pdf format,while the recorded lectures can be collectively found as a playlist on YouTube.

The Jupyter notebook repository resides on Github,which material has been made available to 10 other languages apart from English.Those resources aside, the course is very organized as it also has a forum on Reddit.

Caveat, it's not a course for beginners in machine learning; it requires having taken the NYU DS-GA 1001 Intro to Data Science, which examines how data science methods can be used to improve decision-making. It studies the fundamental principles and techniques of data science under real-world examples and cases in order to place data science techniques in context, to develop data-analytic thinking,
and to illustrate that proper application is as much an art as it is a science. Of course, this one again is based on Python and its associated data analysis libraries. The DS-GA 1001 is not free, however, and is part of of the Master’s Degree in Data Science. However having taken part in any other machine learning course would do too, for example Andrew Ng's ever-popular free course on Coursera or an alternative introduction to data science such as Introduction to Data Science from IBM on edX.

Closing up, I just need to say that what differentiates this course from others is that Yann LeCun is an authority in this field, which is backed by his exquisite credentials. In short Yann LeCun is currently a Silver Professor of Data Science, Computer Science, Neural Science, and Electrical and Computer Engineering at New York University, while also a VP and Chief AI Scientist at Facebook. Along with Geoffrey Hinton and Yoshua Bengio, he was awarded the 2018 ACM Turing Award with the citation describing the three of them as the "Fathers of Deep Learning".

Back in 2016 I  covered his interview by Udacity's Sebastian Thrun in "Facebook's Yann LeCun On Everything AI", in which he provided his opinion on General Artificial intelligence :

LeCun's talk began with the general truth that in order for machines to show common sense, they first need to be able to understand the way the world works and that's only going to happen under the state of unsupervised learning and not under the currently employed supervised learning that uses humans to annotate the data that machines work with. Key to this process is granting the machine the ability to predict, something that his team at Facebook works on with its video prediction software.

His opinion on the view that General Artificial Intelligence will someday grow out of proportion and outsmart us humans, was inconclusive; for the time being, though, he sees no reason for concern.

But also he also enumerated the qualifications required in order to get into his team of researchers and engineers. In a sentence, researchers require PhDs, engineers need MScs or BScs. That's quite something.

The session was been recorded for those unable to catch it live, and is still available online on Udacity's Facebook page for everyone to enjoy.

More Information

DS-GA 1008 · SPRING 2020 · NYU CENTER FOR DATA SCIENCE
Deep Learning (with PyTorch) GitHub

Deep Learning (with PyTorch) YouTube

r/NYU_DeepLearning

Related Articles

Hinton, LeCun and Bengio Receive 2018 Turing Award

Facebook's Yann LeCun On Everything AI

Yann LeCun Recruited For Facebook's New AI Group

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info