GCP: Complete Google Data Engineer and Cloud Architect Guide from Udemy

What's inside

Learning objectives

Deploy managed hadoop apps on the google cloud
Build deep learning models on the cloud using tensorflow

Make informed decisions about containers, vms and appengine
Use big data technologies such as bigtable, dataflow, apache beam and pub/sub

Deploy managed hadoop apps on the google cloud
Build deep learning models on the cloud using tensorflow
Make informed decisions about containers, vms and appengine
Use big data technologies such as bigtable, dataflow, apache beam and pub/sub

Syllabus

You, This Course and Us

Course Materials

Introduction

Theory, Practice and Tests

Lab: Setting Up A GCP Account

Lab: Using The Cloud Shell

Important! Delete unused GCP projects/instances

Compute

About this section

Compute Options

Google Compute Engine (GCE)

Lab: Creating a VM Instance

More GCE

Lab: Editing a VM Instance

Lab: Creating a VM Instance Using The Command Line

Lab: Creating And Attaching A Persistent Disk

Google Container Engine - Kubernetes (GKE)

More GKE

Lab: Creating A Kubernetes Cluster And Deploying A Wordpress Container

App Engine

Contrasting App Engine, Compute Engine and Container Engine

Lab: Deploy And Run An App Engine App

Storage

Storage Options

Quick Take

Cloud Storage

Lab: Working With Cloud Storage Buckets

Lab: Bucket And Object Permissions

Lab: Life cycle Management On Buckets

Fix for AccessDeniedException: 403 Insufficient Permission

Lab: Running A Program On a VM Instance And Storing Results on Cloud Storage

Transfer Service

Lab: Migrating Data Using The Transfer Service

gcloud init

Lab: Cloud Storage ACLs and API access with Service Account

Lab: Cloud Storage Customer-Supplied Encryption Keys and Life-Cycle Management

Lab: Cloud Storage Versioning, Directory Sync

Cloud SQL, Cloud Spanner ~ OLTP ~ RDBMS

Cloud SQL

Lab: Creating A Cloud SQL Instance

Lab: Running Commands On Cloud SQL Instance

Lab: Bulk Loading Data Into Cloud SQL Tables

Cloud Spanner

More Cloud Spanner

Lab: Working With Cloud Spanner

Just wanted to send along an important note for anyone learning a cloud technology like GCP - please be sure to delete your projects, instances and in general to free up your resources after you are done using them. Resources like BigTable, Cloud Spanner are pretty expensive - if you happen to create one, then forget to free it up, you could be hit with real sticker shock when you get your next invoice.

Just something important to keep in mind if you are new to using pay-as-you-go technologies:-)

Hadoop Pre-reqs and Context

BigTable ~ HBase = Columnar Store

BigTable Intro

Columnar Store

Denormalised

Column Families

BigTable Performance

Getting the HBase Prompt

Lab: BigTable demo

An important note for anyone learning a cloud technology like GCP - please be sure to delete your projects, instances and in general to free up your resources after you are done using them. Resources like BigTable, Cloud Spanner are pretty expensive - if you happen to create one, then forget to free it up, you could be hit with real sticker shock when you get your next invoice.

Just something important to keep in mind if you are new to using pay-as-you-go technologies:-)

Datastore ~ Document Database

Datastore

Lab: Datastore demo

BigQuery ~ Hive ~ OLAP

BigQuery Intro

BigQuery Advanced

Lab: Loading CSV Data Into Big Query

Lab: Running Queries On Big Query

Lab: Loading JSON Data With Nested Tables

Lab: Public Datasets In Big Query

Lab: Using Big Query Via The Command Line

Lab: Aggregations And Conditionals In Aggregations

Lab: Subqueries And Joins

Lab: Regular Expressions In Legacy SQL

Lab: Using The With Statement For SubQueries

Dataflow ~ Apache Beam

Data Flow Intro

Apache Beam

Lab: Running A Python Data flow Program

Lab: Running A Java Data flow Program

Lab: Implementing Word Count In Dataflow Java

Lab: Executing The Word Count Dataflow

Lab: Executing MapReduce In Dataflow In Python

Lab: Executing MapReduce In Dataflow In Java

Lab: Dataflow With Big Query As Source And Side Inputs

Lab: Dataflow With Big Query As Source And Side Inputs 2

Dataproc ~ Managed Hadoop

Data Proc

Lab: Creating And Managing A Dataproc Cluster

Lab: Creating A Firewall Rule To Access Dataproc

Lab: Running A PySpark Job On Dataproc

Lab: Running The PySpark REPL Shell And Pig Scripts On Dataproc

Lab: Submitting A Spark Jar To Dataproc

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Provides a deep dive into the Google Cloud Platform with detailed explanations and practical demos, making it suitable for learners seeking comprehensive knowledge

Focuses on the latest version of TensorFlow, a widely adopted deep learning framework, empowering learners to build advanced machine learning models

Offers a practical approach with numerous hands-on labs that allow learners to apply their knowledge and build their practical skills

Covers a wide range of topics, from compute and storage to big data and machine learning, providing a well-rounded understanding of Google Cloud Platform

Taught by Loony Corn, a reputable provider specializing in cloud training, adding credibility to the course

Suitable for intermediate learners who have some prior knowledge of cloud computing or machine learning

Reviews summary

Data engineer & cloud architect guide

Learners say this course provides a comprehensive and clear guide for becoming a Google Data Engineer and Cloud Architect.

Instructor uses humor to make learning enjoyable.

"even with some humor"

Explanations are well organized and easy to follow.

"Extremely comprehensive and clear"

Helpful for passing the Data Engineer exam.

"I passed the Data Engineer test, due largely to this course."

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in GCP: Complete Google Data Engineer and Cloud Architect Guide with these activities:

Review core concepts in computer science

Show steps

A strong foundation in computer science concepts is essential for excelling in this course. This activity allows you to refresh your knowledge of topics such as data structures, algorithms, and software design patterns.

Show steps

Review notes or online resources on data structures and algorithms
Go through examples and practice problems

Review key concepts in Hadoop

Show steps

Hadoop is a fundamental technology in the realm of big data. Before diving into the course, it's beneficial to brush up on its key concepts to ensure a solid foundation.

Browse courses on Hadoop

Show steps

Review basic Hadoop terminology and concepts
Understand Hadoop ecosystem components such as HDFS and MapReduce

Read 'Designing Data-Intensive Applications' by Martin Kleppmann

Show steps

This book provides a comprehensive overview of data-intensive systems, covering topics such as data storage, processing, and analysis. It offers valuable insights and best practices that can enhance your understanding of GCP technologies.

View Secret Colors on Amazon

Show steps

Read chapters 1-3 to understand data modeling and data storage
Read chapters 5-7 to explore data processing and analysis techniques

Five other activities

Expand to see all activities and additional details

Show all eight activities

Explore Google Cloud Platform (GCP) tutorials

Show steps

GCP provides a wealth of tutorials that cover a wide range of topics. By following these tutorials, you can gain practical experience and deepen your understanding of GCP services.

Show steps

Complete the 'Getting Started with Cloud Functions' tutorial
Complete the 'Introduction to Datastore' tutorial
Complete the 'Building a React App with Cloud Storage' tutorial

Practice creating and deploying Kubernetes clusters

Show steps

Creating and deploying Kubernetes clusters is a practical skill often tested on Kubernetes certification exams. By practicing, you can improve your proficiency and prepare for real-world scenarios.

Browse courses on Kubernetes

Show steps

Create a Kubernetes cluster using Google Kubernetes Engine (GKE)
Deploy a simple application to the cluster
Scale the application by adding replicas

Join a GCP study group or discussion forum

Show steps

Engaging with peers can provide valuable insights, help you resolve challenges, and reinforce your understanding. Joining a study group or forum specific to GCP can foster collaboration and enhance your learning.

Show steps

Join a relevant GCP community on platforms like Reddit or Discord
Participate in discussions and ask questions

Build a machine learning model using TensorFlow on GCP

Show steps

Hands-on experience is critical for proficiency in machine learning. This activity allows you to apply your knowledge of TensorFlow and GCP to solve a practical problem.

Browse courses on TensorFlow

Show steps

Choose a dataset and define your model
Train the model using TensorFlow on GCP
Evaluate and deploy the model

Contribute to open-source GCP projects

Show steps

Participating in open-source projects related to GCP can provide hands-on experience, expose you to real-world scenarios, and contribute to the broader community. It's a valuable way to deepen your understanding and develop your skills.

Show steps

Explore open-source GCP projects on platforms like GitHub
Identify a project that aligns with your interests
Make a meaningful contribution, such as fixing a bug or adding a feature

Career center

Learners who complete GCP: Complete Google Data Engineer and Cloud Architect Guide will develop knowledge and skills that may be useful to these careers:

Data Engineer

Data Engineers are responsible for designing and managing data pipelines. They work with data scientists and other stakeholders to understand data needs, and then design and implement systems to collect, process, and store data. This course provides a comprehensive overview of the Google Cloud Platform (GCP), including compute, storage, big data, and machine learning technologies. It can help you develop the skills you need to become a successful Data Engineer.

See salaries and explore the career path for Data Engineer

Cloud Architect

Cloud Architects design and manage cloud computing systems. They work with customers to understand their business needs, and then design and implement cloud solutions that meet those needs. This course provides a comprehensive overview of the GCP, including compute, storage, big data, and machine learning technologies. It can help you develop the skills you need to become a successful Cloud Architect.

See salaries and explore the career path for Cloud Architect

Machine Learning Engineer

Machine Learning Engineers design and develop machine learning models. They work with data scientists and other stakeholders to understand machine learning needs, and then design and implement models that can solve real-world problems. This course covers the fundamentals of machine learning, as well as how to use TensorFlow, Google's open-source machine learning library. It can help you develop the skills you need to become a successful Machine Learning Engineer.

See salaries and explore the career path for Machine Learning Engineer

DevOps Engineer

DevOps Engineers are responsible for building and maintaining software systems. They work with developers and operations teams to ensure that systems are reliable, scalable, and secure. This course covers the fundamentals of DevOps, as well as how to use Google Cloud technologies to build and maintain cloud-based systems. It can help you develop the skills you need to become a successful DevOps Engineer.

See salaries and explore the career path for DevOps Engineer

Data Scientist

Data Scientists use data to solve business problems. They work with data engineers and other stakeholders to understand data needs, and then design and implement data analysis solutions. This course provides a comprehensive overview of the GCP, including compute, storage, big data, and machine learning technologies. It can help you develop the skills you need to become a successful Data Scientist.

See salaries and explore the career path for Data Scientist

Software Engineer

Software Engineers design, develop, and maintain software systems. They work with customers and other stakeholders to understand software needs, and then design and implement software solutions that meet those needs. This course covers the fundamentals of software engineering, as well as how to use Google Cloud technologies to build and maintain cloud-based software. It can help you develop the skills you need to become a successful Software Engineer.

See salaries and explore the career path for Software Engineer

Systems Engineer

Systems Engineers design, integrate, and maintain computer systems. They work with customers and other stakeholders to understand system needs, and then design and implement systems that meet those needs. This course covers the fundamentals of systems engineering, as well as how to use Google Cloud technologies to build and maintain cloud-based systems. It can help you develop the skills you need to become a successful Systems Engineer.

See salaries and explore the career path for Systems Engineer

IT Manager

IT Managers plan, implement, and manage IT systems. They work with customers and other stakeholders to understand IT needs, and then design and implement IT solutions that meet those needs. This course provides a comprehensive overview of the GCP, including compute, storage, big data, and machine learning technologies. It can help you develop the skills you need to become a successful IT Manager.

See salaries and explore the career path for IT Manager

Network Engineer

Network Engineers design, implement, and maintain computer networks. They work with customers and other stakeholders to understand network needs, and then design and implement networks that meet those needs. This course covers the fundamentals of network engineering, as well as how to use Google Cloud technologies to build and maintain cloud-based networks. It can help you develop the skills you need to become a successful Network Engineer.

See salaries and explore the career path for Network Engineer

Security Engineer

Security Engineers design, implement, and maintain computer security systems. They work with customers and other stakeholders to understand security needs, and then design and implement security solutions that meet those needs. This course covers the fundamentals of security engineering, as well as how to use Google Cloud technologies to build and maintain cloud-based security systems. It can help you develop the skills you need to become a successful Security Engineer.

See salaries and explore the career path for Security Engineer

Data Analyst

Data Analysts collect, clean, and analyze data. They work with data scientists and other stakeholders to understand data needs, and then design and implement data analysis solutions. This course provides a comprehensive overview of the GCP, including compute, storage, big data, and machine learning technologies. It can help you develop the skills you need to become a successful Data Analyst.

See salaries and explore the career path for Data Analyst

Database Administrator

Database Administrators design, implement, and maintain databases. They work with customers and other stakeholders to understand database needs, and then design and implement databases that meet those needs. This course covers the fundamentals of database administration, as well as how to use Google Cloud technologies to build and maintain cloud-based databases. It can help you develop the skills you need to become a successful Database Administrator.

See salaries and explore the career path for Database Administrator

Product Manager

Product Managers are responsible for the development and management of products. They work with customers and other stakeholders to understand product needs, and then design and implement products that meet those needs. This course provides a comprehensive overview of the GCP, including compute, storage, big data, and machine learning technologies. It can help you develop the skills you need to become a successful Product Manager.

See salaries and explore the career path for Product Manager

Business Analyst

Business Analysts help businesses improve their operations. They work with stakeholders to understand business needs, and then design and implement solutions that improve business outcomes. This course provides a comprehensive overview of the GCP, including compute, storage, big data, and machine learning technologies. It can help you develop the skills you need to become a successful Business Analyst.

See salaries and explore the career path for Business Analyst

Project Manager

Project Managers plan, implement, and manage projects. They work with customers and other stakeholders to understand project needs, and then design and implement projects that meet those needs. This course provides a comprehensive overview of the GCP, including compute, storage, big data, and machine learning technologies. It can help you develop the skills you need to become a successful Project Manager.

See salaries and explore the career path for Project Manager