Data Science: Transformers for Natural Language Processing from Udemy

Ever wondered how AI technologies like OpenAI ChatGPT, GPT-4, Gemini Pro, Llama 3, DALL-E, Midjourney, and Stable Diffusion really work? In this course, you will learn the foundations of these groundbreaking applications.

Hello friends.

Welcome to Data Science: Transformers for Natural Language Processing.

Ever since Transformers arrived on the scene, deep learning hasn't been the same.

Machine learning is able to generate text essentially indistinguishable from that created by humans
We've reached new state-of-the-art performance in many NLP tasks, such as machine translation, question-answering, entailment, named entity recognition, and more
We've created multi-modal (text and image) models that can generate amazing art using only a text prompt
We've solved a longstanding problem in molecular biology known as "protein structure prediction"

In this course, you will learn very practical skills for applying transformers, and if you want, detailed theory behind how transformers and attention work.

This is different from most other resources, which only cover the former.

The course is split into 3 major parts:

Using Transformers
Fine-Tuning Transformers
Transformers In-Depth

PART 1: Using Transformers

In this section, you will learn how to use transformers which were trained for you. This costs millions of dollars to do, so it's not something you want to try by yourself.

We'll see how these prebuilt models can already be used for a wide array of tasks, including:

text classification (e.g. spam detection, sentiment analysis, document categorization)
named entity recognition
text summarization
machine translation
question-answering
generating (believable) text
masked language modeling (article spinning)
zero-shot classification

This is already very practical.

If you need to do sentiment analysis, document categorization, entity recognition, translation, summarization, etc. on documents at your workplace or for your clients - you already have the most powerful state-of-the-art models at your fingertips with very few lines of code.

One of the most amazing applications is "zero-shot classification", where you will observe that a pretrained model can categorize your documents, even without any training at all.

PART 2: Fine-Tuning Transformers

In this section, you will learn how to improve the performance of transformers on your own custom datasets. By using "transfer learning", you can leverage the millions of dollars of training that have already gone into making transformers work very well.

You'll see that you can fine-tune a transformer with relatively little work (and little cost).

We'll cover how to fine-tune transformers for the most practical tasks in the real-world, like text classification (sentiment analysis, spam detection), entity recognition, and machine translation.

PART 3: Transformers In-Depth

In this section, you will learn how transformers really work. The previous sections are nice, but a little too nice. Libraries are OK for people who just want to get the job done, but they don't work if you want to do anything new or interesting.

Let's be clear: this is very practical.

How practical, you might ask?

Well, this is where the big bucks are.

Those who have a deep understanding of these models and can do things no one has ever done before are in a position to command higher salaries and prestigious titles. Machine learning is a competitive field, and a deep understanding of how things work can be the edge you need to come out on top.

We'll look at the inner workings of encoders, decoders, encoder-decoders5, ChatGPT, and GPT-4 (for the latter, we are limited to what OpenAI has revealed).

We'll also look at how to implement transformers from scratch.

As the great Richard Feynman once said, "what I cannot create, I do not understand".

SUGGESTED

What's inside

Learning objectives

Apply transformers to real-world tasks with just a few lines of code
Fine-tune transformers on your own datasets with transfer learning
Sentiment analysis, spam detection, text classification
Ner (named entity recognition), parts-of-speech tagging
Build your own article spinner for seo
Generate believable human-like text
Neural machine translation and text summarization
Question-answering (e.g. squad)
Zero-shot classification

Understand self-attention and in-depth theory behind transformers
Implement transformers from scratch
Use transformers with both tensorflow and pytorch
Understand bert, gpt, gpt-2, and gpt-3, and where to apply them
Understand encoder, decoder, and seq2seq architectures
Master the hugging face python library
Understand important foundations for openai chatgpt, gpt-4, dall-e, midjourney, and stable diffusion
Show more
Show less

Apply transformers to real-world tasks with just a few lines of code
Fine-tune transformers on your own datasets with transfer learning
Sentiment analysis, spam detection, text classification
Ner (named entity recognition), parts-of-speech tagging
Build your own article spinner for seo
Generate believable human-like text
Neural machine translation and text summarization
Question-answering (e.g. squad)
Zero-shot classification
Understand self-attention and in-depth theory behind transformers
Implement transformers from scratch
Use transformers with both tensorflow and pytorch
Understand bert, gpt, gpt-2, and gpt-3, and where to apply them
Understand encoder, decoder, and seq2seq architectures
Master the hugging face python library
Understand important foundations for openai chatgpt, gpt-4, dall-e, midjourney, and stable diffusion
Show more
Show less

Syllabus

Welcome

Introduction

Outline

Getting Setup

Get Your Hands Dirty, Practical Coding Experience, Data Links

How to use Github & Extra Coding Tips (Optional)

Where to get the code, notebooks, and data

Are You Beginner, Intermediate, or Advanced? All are OK!

How to Succeed in This Course

Temporary 403 Errors

Beginner's Corner

Beginner's Corner Section Introduction

From RNNs to Attention and Transformers - Intuition

Sentiment Analysis

Sentiment Analysis in Python

Text Generation

Text Generation in Python

Masked Language Modeling (Article Spinner)

Masked Language Modeling (Article Spinner) in Python

Named Entity Recognition (NER)

Named Entity Recognition (NER) in Python

Text Summarization

Text Summarization in Python

Neural Machine Translation

Neural Machine Translation in Python

Question Answering

Question Answering in Python

Zero-Shot Classification

Zero-Shot Classification in Python

Beginner's Corner Section Summary

Suggestion Box

Fine-Tuning (Intermediate)

Fine-Tuning Section Introduction

Text Preprocessing and Tokenization Review

Models and Tokenizers

Models and Tokenizers in Python

Transfer Learning & Fine-Tuning (pt 1)

Transfer Learning & Fine-Tuning (pt 2)

Transfer Learning & Fine-Tuning (pt 3)

Fine-Tuning Sentiment Analysis and the GLUE Benchmark

Fine-Tuning Sentiment Analysis in Python

Fine-Tuning Transformers with Custom Dataset

Hugging Face AutoConfig

Fine-Tuning with Multiple Inputs (Textual Entailment)

Fine-Tuning Transformers with Multiple Inputs in Python

Fine-Tuning Section Summary

Named Entity Recognition (NER) and POS Tagging (Intermediate)

Token Classification Section Introduction

Data & Tokenizer (Code Preparation)

Data & Tokenizer (Code)

Target Alignment (Code Preparation)

Create Tokenized Dataset (Code Preparation)

Target Alignment (Code)

Data Collator (Code Preparation)

Data Collator (Code)

Metrics (Code Preparation)

Metrics (Code)

Model and Trainer (Code Preparation)

Model and Trainer (Code)

POS Tagging & Custom Datasets (Exercise Prompt)

POS Tagging & Custom Datasets (Solution)

Token Classification Section Summary

Seq2Seq and Neural Machine Translation (Intermediate)

Translation Section Introduction

Things Move Fast

Aside: Seq2Seq Basics (Optional)

Model Inputs (Code Preparation)

Model Inputs (Code)

Translation Metrics (BLEU Score & BERT Score) (Code Preparation)

Translation Metrics (BLEU Score & BERT Score) (Code)

Train & Evaluate (Code Preparation)

Train & Evaluate (Code)

Translation Section Summary

Question-Answering (Advanced)

Question-Answering Section Introduction

Exploring the Dataset (SQuAD)

Exploring the Dataset (SQuAD) in Python

Using the Tokenizer

Using the Tokenizer in Python

Aligning the Targets

Aligning the Targets in Python

Applying the Tokenizer

Applying the Tokenizer in Python

Question-Answering Metrics

Question-Answering Metrics in Python

From Logits to Answers

From Logits to Answers in Python

Computing Metrics

Computing Metrics in Python

Train and Evaluate

Train and Evaluate in Python

Question-Answering Section Summary

Transformers and Attention Theory (Advanced)

Theory Section Introduction

Basic Self-Attention

Self-Attention & Scaled Dot-Product Attention

Attention Efficiency

Attention Mask

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Develops transformer applications, models, and theory, which are core skills for data scientists and machine learning engineers

Taught by Lazy Programmer Inc. and Lazy Programmer Team, who are recognized for their work in data science and machine learning

Examines cutting-edge AI technologies such as ChatGPT, DALL-E, and GPT-4, which are highly relevant in the field of data science and machine learning

Provides hands-on labs and interactive materials for practical experience in applying transformers

Builds a strong foundation for beginners in the concepts and applications of transformers

Explicitly advises students to take other courses first as prerequisites

Career center

Learners who complete Data Science: Transformers for Natural Language Processing will develop knowledge and skills that may be useful to these careers:

Natural Language Processing Engineer

Natural Language Processing Engineers design and develop systems to enable computers to work with and understand human language. The course goes over Sentiment Analysis, Masked Language Modeling, Named Entity Recognition, Text Summarization, Neural Machine Translation, Question Answering, and Zero-Shot Classification, all key skills used by NLP Engineers.

See salaries and explore the career path for Natural Language Processing Engineer

Machine Learning Engineer

Machine Learning Engineers use their knowledge of AI, computer science and probability and statistics to design, build, train, and maintain machine learning systems. This course goes over Transformers and Attention which are fundamental concepts for an MLE.

See salaries and explore the career path for Machine Learning Engineer

Data Analyst

Data Analysts analyze data to help businesses make better decisions. This course teaches skills like Sentiment Analysis, Masked Language Modeling, Named Entity Recognition, Text Summarization, Neural Machine Translation, Question Answering, and Zero-Shot Classification, all useful skills for a Data Analyst.

See salaries and explore the career path for Data Analyst

Software Engineer

Software Engineers design, develop, test, and maintain software systems. The course goes over Tensorflow, PyTorch, and Hugging Face, which are frameworks used to develop transformers. This course may be useful to gain the understanding needed to build transformer based software.

See salaries and explore the career path for Software Engineer

Data Scientist

Data Scientists use their knowledge of math, statistics, computer science, and business to solve problems using data. As the course teaches Attention Theory, Model Inputs, Tokenization, and Translation Metrics, which are essential concepts for a Data Scientist to build better models, it may be useful to take.

See salaries and explore the career path for Data Scientist

Research Scientist

Research Scientists conduct research and advance knowledge in a specific field. As the course dives into the theory of transformers and self-attention, it may be useful for a Research Scientist.

See salaries and explore the career path for Research Scientist

AI Engineer

AI Engineers design, build, and maintain AI systems. This course may be useful as it dives into transformer architecture and theory.

See salaries and explore the career path for AI Engineer

Technical Writer

Technical Writers create documentation for software, hardware, and other technical products. This course teaches the fundamentals of how transformer models work, which would be useful for a technical writer to coherently explain to others.

See salaries and explore the career path for Technical Writer

Quantitative Analyst

Quantitative Analysts use their knowledge of math, statistics, and computer science to analyze financial data. This course teaches intermediate skills such as how to fine-tune a transformer model, which could be useful for building more advanced financial modeling systems.

See salaries and explore the career path for Quantitative Analyst

Data Engineer

Data Engineers design, build, and maintain data pipelines. This course dives deep into the transformers model, which could be useful for building out more effective data pipelines.

See salaries and explore the career path for Data Engineer

DevOps Engineer

DevOps Engineers work to bridge the gap between software development and IT operations. As a DevOps Engineer would need to work with Data Engineers, Data Scientists, and Software Engineers, this course may be useful for learning about the technologies they use, such as Tensorflow, PyTorch, and Hugging Face.

See salaries and explore the career path for DevOps Engineer

Cloud Architect

Cloud Architects design and manage cloud computing systems. This course may be useful as it teaches how to use transformers, which are often deployed in the cloud.

See salaries and explore the career path for Cloud Architect

Business Analyst

Business Analysts use their knowledge of business and technology to help organizations improve their performance. This course teaches valuable skills like sentiment analysis, text summarization, and question-answering, all of which may be used to analyze business data.

See salaries and explore the career path for Business Analyst

Product Manager

Product Managers are responsible for planning and developing products. This course may be useful as it teaches Sentiment Analysis, Masked Language Modeling, Named Entity Recognition, Text Summarization, Neural Machine Translation, Question Answering, and Zero-Shot Classification, all skills that relate to understanding and improving user experience with tech products.

See salaries and explore the career path for Product Manager

Information Security Analyst

Information Security Analysts protect computer systems and networks from attacks. It is possible that this course could be useful in learning to identify and mitigate risks associated with transformer models.

See salaries and explore the career path for Information Security Analyst

Reading list

We've selected two books that we think will supplement your learning. Use these to develop background knowledge, enrich your coursework, and gain a deeper understanding of the topics covered in Data Science: Transformers for Natural Language Processing.

Deep Learning with Python, Second Edition

Save

Provides a broad overview of deep learning, including transformers. It is written for beginners with no background in machine learning.

Deep Learning with Python, Second Edition

Paperback

Deep Learning with Python, Second Edition

Kindle Edition

Speech and Language Processing. An Introduction to...

Save

Provides a comprehensive overview of speech and language processing, including transformers. It is written for advanced undergraduates and graduate students with some background in machine learning.

Speech and Language Processing An Introduction to...

Paperback

Data Science

Transformers for Natural Language Processing

What's inside

Learning objectives

Syllabus

Good to know

Save this course

Activities

Career center

Reading list

Share

Similar courses