The content of CS224n Natural Language Processing with Deep Learning, a graduate level, one-semester course originally provided to Stanford University Computer Science students, has been made available for free to anyone in a self-paced version.

The main focus of CS224n is about investigating the fundamental concepts and ideas in natural language processing (NLP) under a deep learning approach, looking to convey the understanding of both the algorithms available for processing linguistic information as well as the underlying computational properties of natural languages.

The material offered is the one of class of 2021, so it is up to date. In any case the concepts don't change, and this course leans heavily towards them.

Normally, there would be lot of demanding prerequisites in order to attend and graduate from it:

Proficiency in Python
All class assignments are in Python (using NumPy and PyTorch). But it's still ok if you have a lot of programming experience but in a different language (e. g. C/C++/Matlab/Java/Javascript).

College Calculus, Linear Algebra
Students should be comfortable taking (multivariable) derivatives and understanding matrix/vector notation and operations.

Basic Probability and Statistics
Knowledge in the basics of probabilities, gaussian distributions, mean, standard deviation, etc.

Foundations of Machine Learning
If you already have basic machine learning and/or deep learning knowledge, the course will be easier; however it is possible to take CS224n without it.

In the free version, however, you get access to the course's resources and the full YouTube playlist of the recorded lectures without those limitations. That way you get a taste of what it would have been like taking the class as a Stanford student. Especially comforting when you consider that for attending there's a fee of $4,056 - $5,408.

Meet main instructor Christopher Manning in this extract from the first lecture:

But first of all why get into NLP in the first place?

NLP tries to make sense out of textual data, which is much more difficult than doing the same with numerical data. Applications of NLP are everywhere because people communicate almost everything in language: web search, advertising, emails, customer service, language translation, virtual agents, medical reports, etc. Many organizations are looking to integrate NLP into their workflows and products they provide such as translation, speech recognition and chatbots. Sounds like a good career move.

An example of such an NLP driven application is the WantWords Reverse Dictionary, which in contrast to a regular (forward) dictionary that provides definitions for query words,returns words semantically matching the query descriptions. What are the use cases of a tool like this?

• Solve the tip-of-the-tongue problem, the phenomenon of failing to retrieve a word from memory
• Help new language learners
• Help word selection (or word dictionary) anomia patients, people who can recognize and describe an object but fail to name it due to neurological disorder

Without further ado let's take a look at the course's syllabus:

1-Introduction and Word Vectors
2-Neural Classifiers
3-Backprop and Neural Networks
4-Dependency Parsing
5-Language Models and RNNs
6-Simple and LSTM RNNs
7-Translation, Seq2Seq, Attention
8-Final Projects; Practical Tips
9-Self- Attention and Transformers
10-Transformers and Pretraining
11-Question Answering
12-Natural Language Generation
13-Coreference Resolution
14-T5 and Large Language Models
15-Add Knowledge to Language Models
16-Social & Ethical Considerations
17-Model Analysis and Explanation
18-Future of NLP Deep Learning
19-Low Resource Machine Translation
20-BERT and Other Pre-trained Language Models 

This syllabus encompasses a number of cutting edge topics:

• Computational properties of natural languages
• Coreference, question answering, and machine translation
• Processing linguistic information
• Syntactic and semantic processing
• Modern quantitative techniques in NLP
• Neural network models for language understanding tasks

As far as the course goes, it progresses in incremental difficulty, from word-level and syntactic processing to question answering and machine translation.There's also a final project where students apply a complex neural network model to a large-scale NLP problem. By that time, students' are expected to have gained a firm understanding of implementing, training, debugging, and extending neural network models for a variety of language understanding tasks.

This course offering is yet another addition to an increasing catalog of College courses being made available to the general public for free. That's due to the pandemic; probably the only positive outcome out of it.

Three other courses that we've examined fall into this category are :

Cornell's CS 6120 Advanced Compilers

Yann LeCun’s Deep Learning Course Free From NYU

Nottingham's University Functional Programming in Haskell

Add CS224n Natural Language Processing with Deep Learning to this list.

More Information

CS224n: Natural Language Processing with Deep Learning

YouTube Playlist

WantWords

Related Articles

Cornell's CS 6120 Advanced Compilers

Yann LeCun’s Deep Learning Course Free From NYU

Nottingham's University Functional Programming in Haskell

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info