Mastering Generative AI: LLM Architecture & Data Preparation from edX

The demand for gen AI is forecast to grow over 46% annually by 2030 (Source: Statista). AI engineers and developers, data scientists, machine learning engineers, and other AI professionals with gen AI skills are highly sought-after. This course builds in-demand skills in large language model (LLM) architecture and data preparation employers are looking for.

During the course, you’ll learn about real-world applications using generative AI. You’ll gain insights into gen AI architectures and models, such as recurrent neural networks (RNNs), transformers, generative adversarial networks (GANs), variational autoencoders (VAEs), and diffusion models. You’ll use different training approaches for each model. Plus, you’ll explore LLMs such as generative pre-trained transformers (GPT) and bidirectional encoder representations from transformers (BERT).

Additionally, you’ll gain a detailed understanding of the tokenization process, tokenization methods, and the use of tokenizers for word-based, character-based, and subword-based tokenization. You’ll get hands-on experience using data loaders for training generative AI models, using PyTorch libraries, and generative AI libraries in Hugging Face. Plus, you’ll implement tokenization and create an NLP data loader.

If you’re looking to master gen AI LLM architecture and data preparation, ENROLL TODAY and get ready to power up your resume with skills employers need!

Prerequisites: To enroll for this course, a basic knowledge of Python and PyTorch and an awareness of machine learning and neural networks would be an advantage, though not strictly required.

What's inside

Learning objectives

Job-ready generative ai architecture and data science skills in two weeks, plus practical experience and an industry-recognized credential employers value.
The difference between generative ai architectures and models, such as rnns, transformers, vaes, gans, and diffusion models.
How llms such as gpt, bert, bart, and t5 are used in language processing.

How to implement tokenization to preprocess raw textual data using nlp libraries such as nltk, spacy, berttokenizer, and xlnettokenizer.
How to create an nlp data loader using pytorch to perform tokenization, numericalization, and padding of text data.

Job-ready generative ai architecture and data science skills in two weeks, plus practical experience and an industry-recognized credential employers value.
The difference between generative ai architectures and models, such as rnns, transformers, vaes, gans, and diffusion models.
How llms such as gpt, bert, bart, and t5 are used in language processing.
How to implement tokenization to preprocess raw textual data using nlp libraries such as nltk, spacy, berttokenizer, and xlnettokenizer.
How to create an nlp data loader using pytorch to perform tokenization, numericalization, and padding of text data.

Syllabus

Reading: Basics of AI Hallucinations

Reading: Overview of Libraries and Tools

Module 1: Generative AI Architecture

Video: Overview of AI Engineering with LLMs Professional Certificate

Traffic lights

Read about what's good

what should give you pause

and possible dealbreakers

Develops skills in large language model (LLM) architecture and data preparation, which are highly sought-after by employers in the AI field

Explores LLMs such as generative pre-trained transformers (GPT) and bidirectional encoder representations from transformers (BERT), which are essential for modern language processing tasks

Provides hands-on experience using data loaders for training generative AI models, using PyTorch libraries, and generative AI libraries in Hugging Face

Requires a basic knowledge of Python and PyTorch, which may necessitate additional learning for individuals without prior experience in these areas

Examines tokenization methods and the use of tokenizers for word-based, character-based, and subword-based tokenization, which are standard techniques in NLP

Presented by IBM, which is recognized for its contributions to artificial intelligence and its development of AI technologies and platforms

Reviews summary

Practical foundation in llm architecture & data prep

According to learners, this course provides a solid introductionpositive to Large Language Model (LLM) architecturesneutral and essential data preparation techniquespositive. Many students highlight the hands-on labs using PyTorch and Hugging Facepositive as particularly valuable, offering practical experiencepositive with concepts like tokenizationneutral and data loadersneutral. While praised for its relevance to current AI trendspositive and job market, some note that a basic understanding of Python and machine learning is highly recommendedwarning to fully grasp the material, suggesting the prerequisites might be slightly understated for those completely new to the field.

Provides a solid foundational understanding.

"This is a great course for someone looking for a foundational understanding of LLM architectures and data handling."

"It doesn't go super deep into complex theory, but it gives you a solid base to build upon."

"I feel prepared to tackle more advanced topics after completing this course."

"A good stepping stone into the world of generative AI and LLMs."

Topics are current and career-relevant.

"The course content is highly relevant to the current trends in AI and machine learning."

"I feel this course gives me a good head start on understanding the technology behind LLMs, which is very in-demand."

"Learning about different model architectures and data prep is exactly what I needed for my career goals in AI."

"The focus on LLMs like GPT and BERT is timely and useful for anyone entering the field."

Course effectively introduces key LLM topics.

"This course provided a clear overview of different generative AI architectures like Transformers, GANs, and VAEs."

"I now have a much better grasp of the tokenization process and why it's crucial for NLP tasks."

"The explanations on data loaders and how they work in PyTorch were very helpful."

"It covered the fundamental concepts of LLMs and their underlying architecture well for an introductory course."

Hands-on labs are frequently praised.

"The labs, especially the ones involving Hugging Face and PyTorch, were incredibly useful for getting practical experience."

"I found the lab exercises on tokenization and creating data loaders to be the most valuable part, really solidifying my understanding."

"The hands-on coding and projects are the strongest part of the course for me, allowing me to apply what I learned immediately."

"Putting the concepts into practice through the labs made the material much clearer and more applicable."

Prior knowledge is beneficial, maybe required.

"While they say prerequisites aren't strictly required, I think having some Python and ML background is pretty essential to keep up."

"Learners without prior knowledge of Python and PyTorch might find some sections challenging."

"I struggled a bit with the coding parts because my Python wasn't as strong as it needed to be."

"It would be easier if you already know the basics of neural networks before starting this."

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Mastering Generative AI: LLM Architecture & Data Preparation with these activities:

Review Neural Network Fundamentals

Show steps

Solidify your understanding of neural network basics to better grasp the complexities of LLM architectures.

Browse courses on Neural Networks

Show steps

Review online resources on neural networks.
Work through introductory tutorials on building simple neural networks.
Familiarize yourself with common activation functions and loss functions.

Natural Language Processing with PyTorch

Show steps

Study a book on NLP with PyTorch to enhance your practical skills in implementing LLM data preparation techniques.

View Natural Language Processing with PyTorch: Build... on Amazon

Show steps

Read the chapters on tokenization and data loaders.
Run the code examples provided in the book.
Adapt the code examples to your own projects.

Implement Basic Tokenization Methods

Show steps

Practice implementing tokenization techniques to gain hands-on experience with data preparation for LLMs.

Show steps

Write code to implement word-based tokenization.
Write code to implement character-based tokenization.
Experiment with different tokenization libraries like NLTK and spaCy.

Four other activities

Expand to see all activities and additional details

Show all seven activities

Create a Blog Post on Tokenization Techniques

Show steps

Write a blog post explaining different tokenization methods to reinforce your understanding and share your knowledge with others.

Show steps

Research different tokenization techniques (word-based, character-based, subword-based).
Write a clear and concise explanation of each technique.
Include examples of how each technique is used in practice.
Publish your blog post online.

Deep Learning (Adaptive Computation and Machine Learning series)

Show steps

Study a comprehensive textbook on deep learning to gain a deeper understanding of the underlying principles of LLMs.

View Deep Learning on Amazon

Show steps

Read the chapters on recurrent neural networks and transformers.
Review the sections on training deep learning models.
Work through the exercises to solidify your understanding.

Build a Simple Text Generator

Show steps

Create a basic text generation model to apply your knowledge of LLM architecture and data preparation.

Show steps

Choose a simple LLM architecture (e.g., a basic RNN).
Prepare a small dataset of text data.
Train your model to generate text.
Evaluate the performance of your model.

Contribute to a Hugging Face Transformers Project

Show steps

Contribute to an open-source project related to Hugging Face Transformers to gain real-world experience with LLMs.

Show steps

Explore the Hugging Face Transformers repository on GitHub.
Identify a bug or feature request that you can contribute to.
Submit a pull request with your changes.
Respond to feedback from the project maintainers.

Career center

Learners who complete Mastering Generative AI: LLM Architecture & Data Preparation will develop knowledge and skills that may be useful to these careers:

Natural Language Processing Engineer

A natural language processing engineer develops systems that enable computers to understand and process human language, and this course is highly relevant to such a position. The course focus on tokenization, data loaders, and large language models such as BERT and GPT is central to the work of a natural language processing engineer. This course also provides a foundation for working with different types of neural networks, which is another essential aspect of a natural language processing engineer’s work. Understanding the nuances of generative AI models helps build sophisticated and effective applications.

See salaries and explore the career path for Natural Language Processing Engineer

Machine Learning Engineer

A machine learning engineer develops and implements machine learning models, and this course helps build a foundational understanding of generative AI architectures and data preparation techniques needed for such work. This role often involves creating and optimizing models for specific applications, requiring familiarity with various model types from recurrent neural networks to diffusion models, all of which are covered in this course. Understanding tokenization processes and creating data loaders, another component of this course, further equips an aspiring machine learning engineer for handling textual data effectively.

See salaries and explore the career path for Machine Learning Engineer

Artificial Intelligence Engineer

An artificial intelligence engineer designs, develops, and deploys AI systems, and this course provides a practical understanding of generative AI model architectures and data handling. The course's focus on large language models such as GPT and BERT directly applies to many AI applications, where natural language processing is key. The deep dive into tokenization methods and data loaders, as presented in this course, is particularly crucial to an AI engineer working with textual datasets. Understanding how different models like GANs and VAEs function will help in choosing the right model for the job.

See salaries and explore the career path for Artificial Intelligence Engineer

Data Scientist

Data scientists analyze complex data to extract insights and build data-driven solutions, and this course provides a practical background in generative models and data preparation. A data scientist often works with textual data, and the tokenization and data loading skills taught here are vital. This course may be particularly helpful for a data scientist using large language models, as it covers generative pre-trained transformers and other architectures. Familiarity with generative adversarial networks, variational autoencoders, and diffusion models, taught in this course, is also useful for a data scientist.

See salaries and explore the career path for Data Scientist

Data Engineer

A data engineer designs, builds, and manages data infrastructure, and this course helps build their understanding of how generative AI models are made ready for use. The process of creating data loaders is important to how generative AI models are trained. The knowledge of tokenization methods and preprocessing textual data is key to this role. The course explains how to use PyTorch and Hugging Face libraries. These libraries are often used by data engineers when preparing data. This course is extremely helpful for a data engineer working with generative AI.

See salaries and explore the career path for Data Engineer

Research Scientist

A research scientist in the field of artificial intelligence advances the state of the art in generative models, and this course introduces key concepts in the field. The course's detailed exploration of generative AI architectures, including RNNs, transformers, GANs, VAEs, and diffusion models, may be particularly valuable. A research scientist will find the training approaches for these models, as well as the handling of large language models such as GPT and BERT, extremely relevant. This course helps build a strong understanding of tokenization and data loaders. An advanced degree is typically required for this role.

See salaries and explore the career path for Research Scientist

AI Trainer

An AI trainer is needed to help others understand and use AI effectively, and this course gives a good foundation for someone looking to guide others in using generative AI. The course’s content on generative AI architectures and models, along with LLMs, offers a broad conceptual overview. Further, the discussion of data preparation, including tokenization and data loaders, empowers this role to better guide users of these systems. The practical skills taught help an AI trainer guide users on data preprocessing and using tools from Hugging Face.

See salaries and explore the career path for AI Trainer

AI Research Analyst

An AI research analyst analyzes the latest trends and developments in artificial intelligence, and this course helps build a foundation in generative AI models. The course's overview of various generative AI architectures and models may be useful when evaluating new AI technologies. The focus on large language models like GPT and BERT is directly relevant to the current state of the field. This course gives an aspiring AI research analyst the knowledge to understand how data is preprocessed using tokenization and data loaders.

See salaries and explore the career path for AI Research Analyst

Computational Linguist

A computational linguist develops computational models of human language, and this course may be useful for analyzing the architecture of large language models. This role requires a deep understanding of tokenization methods, which is a major component of this course. The course’s inclusion of tokenization with NLTK, spaCy, BertTokenizer, and XLNetTokenizer may be particularly helpful. Learning about how these models like BERT, GPT, and others are used in language processing is also valuable to a computational linguist. An advanced degree is often required for this position.

See salaries and explore the career path for Computational Linguist

AI Consultant

An AI consultant provides expert advice on implementing artificial intelligence solutions in businesses. This course may be helpful for a consultant needing a solid understanding of how generative AI operates. The course's focus on understanding generative AI models like RNNs, transformers, generative adversarial networks, variational autoencoders, and diffusion models, paired with large language models like GPT and BERT is foundational. The practical knowledge of tokenization and data loading that an AI consultant receives from this course will help them propose practical and realistic solutions.

See salaries and explore the career path for AI Consultant

Data Analyst

A data analyst interprets data to identify trends and business opportunities, so this course, although it does not train someone directly for the role of a data analyst, may be useful for some data analysts who work with textual data using generative AI models. Tokenization, tokenization methods, and data loaders are key skills that a data analyst could learn by taking this course. The detailed exploration of data preprocessing techniques, including tokenization and data loading, may be particularly beneficial. A data analyst gaining familiarity with LLMs could enhance their capabilities in handling text-based datasets.

See salaries and explore the career path for Data Analyst

AI Product Manager

An AI product manager guides product development for AI and machine learning products, and this course may be useful for understanding the technical aspects of developing generative AI products. This course introduces generative AI architectures and models, as well as large language models. An AI product manager will also find this course helpful for understanding the data preparation process from tokenization to making data available for models. This course helps build a basic foundation for a non-technical product manager responsible for AI-related products.

See salaries and explore the career path for AI Product Manager

Software Developer

A software developer designs, develops, and tests software applications. This course may be useful for software developers who wish to understand the intricacies of generative AI models. A software developer working with AI applications can learn from this course how to use different training approaches for models. This course covers important concepts, such as tokenization and data loaders. This would help a software developer to understand how models are prepared to take in data. This course may help a developer understand how to integrate AI functionalities into applications.

See salaries and explore the career path for Software Developer

Solutions Architect

A solutions architect designs and implements IT solutions based on business needs. This course may be helpful for a solutions architect wanting to implement AI solutions with generative AI. Understanding how to use generative AI architectures and models, including RNNs, transformers, GANs, VAEs, and diffusion models, as discussed in this course, helps build a foundation in this area. The course also provides insight into tokenization and data loaders, which is necessary for a deeper understanding of AI implementation. This course helps build familiarity with generative AI architecture.

See salaries and explore the career path for Solutions Architect

Technical Writer

A technical writer produces documentation for technical products and services. This course may be useful for a technical writer who works in AI or machine learning, as it would help them learn the terminology and concepts in this space. The course explores generative AI architectures, models, and large language models. It also provides practical skills in implementing tokenization and creating data loaders. The concepts and technical language learned in this course, would help a technical writer be more effective in their work. A technical writer may find this helpful when preparing documentation for software or systems that use generative AI.

See salaries and explore the career path for Technical Writer

Mastering Generative AI

LLM Architecture & Data Preparation

What's inside

Learning objectives

Syllabus

Traffic lights

Save this course

Reviews summary

Practical foundation in llm architecture & data prep

Activities

Career center

Reading list

Share

Similar courses