Practice Exams | AWS Certified Data Engineer - Associate

Preparing for AWS Certified Data Engineer Associate DEA-C01? This is THE practice exams course to give you the winning edge.

These practice exams have been co-authored by Stephane Maarek and Abhishek Singh who bring their collective experience of passing 20 AWS Certifications to the table.

The tone and tenor of the questions mimic the real exam. Along with the detailed description and “exam alert” provided within the explanations, we have also extensively referenced AWS documentation to get you up to speed on all domain areas being tested for the DEA-C01 exam.

We want you to think of this course as the final pit-stop so that you can cross the winning line with absolute confidence and get AWS Certified. Trust our process, you are in good hands.

All questions have been written from scratch. And more questions are being added over time.

Quality speaks for itself

The data engineer has identified the root cause of the sluggish performance as the excessive number of partitions in the S3 bucket, leading to increased Athena query planning times.

What are the two possible approaches to mitigate this issue and enhance query efficiency (Select two)?

1. Transform the data in each partition to Apache ORC format

2. Compress the files in gzip format to improve query performance against the partitions

3. Perform bucketing on the data in each partition

4. Set up an AWS Glue partition index and leverage partition filtering via the GetPartitions call

5. Set up Athena partition projection based on the S3 bucket prefix

What's your guess? Scroll below for the answer.

Correct: 4,5.

Explanation:

Correct options:

Set up an AWS Glue partition index and leverage partition filtering via the GetPartitions call

When you create a partition index, you specify a list of partition keys that already exist on a given table. The partition index is sub list of partition keys defined in the table. A partition index can be created on any permutation of partition keys defined in the table. For the above sales_data table, the possible indexes are (country, category, creationDate), (country, category, year), (country, category), (country), (category, country, year, month), and so on.

Let's take a sales_data table as an example which is partitioned by the keys Country, Category, Year, Month, and creationDate. If you want to obtain sales data for all the items sold for the Books category in the year 2020 after 2020-08-15, you have to make a GetPartitions request with the expression "Category = 'Books' and creationDate > '2020-08-15'" to the Data Catalog.

If no partition indexes are present on the table, AWS Glue loads all the partitions of the table and then filters the loaded partitions using the query expression provided by the user in the GetPartitions request. The query takes more time to run as the number of partitions increases on a table with no indexes. With an index, the GetPartitions query will try to fetch a subset of the partitions instead of loading all the partitions in the table.

Overview of AWS Glue partition index and partition filtering:

Reference Image

via - Reference Link

Set up Athena partition projection based on the S3 bucket prefix

Processing partition information can be a bottleneck for Athena queries when you have a very large number of partitions and aren’t using AWS Glue partition indexing. You can use partition projection in Athena to speed up query processing of highly partitioned tables and automate partition management. Partition projection helps minimize this overhead by allowing you to query partitions by calculating partition information rather than retrieving it from a metastore. It eliminates the need to add partitions’ metadata to the AWS Glue table.

In partition projection, partition values, and locations are calculated from configuration rather than read from a repository like the AWS Glue Data Catalog. Because in-memory operations are usually faster than remote operations, partition projection can reduce the runtime of queries against highly partitioned tables. Depending on the specific characteristics of the query and underlying data, partition projection can significantly reduce query runtime for queries that are constrained by partition metadata retrieval.

Overview of Athena partition projection:

Reference Image

via - Reference Link

Incorrect options:

Transform the data in each partition to Apache ORC format - Apache ORC is a popular file format for analytics workloads. It is a columnar file format because it stores data not by row, but by column. ORC format also allows query engines to reduce the amount of data that needs to be loaded in different ways. For example, by storing and compressing columns separately, you can achieve higher compression ratios and only the columns referenced in a query need to be read. However, the data is being transformed within the existing partitions, this option does not resolve the root cause of under-performance (that is, the excessive number of partitions in the S3 bucket).

Compress the files in gzip format to improve query performance against the partitions - Compressing your data can speed up your queries significantly. The smaller data sizes reduce the data scanned from Amazon S3, resulting in lower costs of running queries. It also reduces the network traffic from Amazon S3 to Athena. Athena supports a variety of compression formats, including common formats like gzip, Snappy, and zstd. However, the data is being compressed within the existing partitions, this option does not resolve the root cause of under-performance (that is, the excessive number of partitions in the S3 bucket).

Perform bucketing on the data in each partition - Bucketing is a way to organize the records of a dataset into categories called buckets. This meaning of bucket and bucketing is different from, and should not be confused with Amazon S3 buckets. In data bucketing, records that have the same value for a property go into the same bucket. Records are distributed as evenly as possible among buckets so that each bucket has roughly the same amount of data. In practice, the buckets are files, and a hash function determines the bucket that a record goes into. A bucketed dataset will have one or more files per bucket per partition. The bucket that a file belongs to is encoded in the file name. Bucketing is useful when a dataset is bucketed by a certain property and you want to retrieve records in which that property has a certain value. Because the data is bucketed, Athena can use the value to determine which files to look at. For example, suppose a dataset is bucketed by customer_id and you want to find all records for a specific customer. Athena determines the bucket that contains those records and only reads the files in that bucket.

Good candidates for bucketing occur when you have columns that have high cardinality (that is, have many distinct values), are uniformly distributed, and that you frequently query for specific values.

Since bucketing is being done within the existing partitions, this option does not resolve the root cause of under-performance (that is, the excessive number of partitions in the S3 bucket).

With multiple reference links from AWS documentation

Instructor

My name is Stéphane Maarek, I am passionate about Cloud Computing, and I will be your instructor in this course. I teach about AWS certifications, focusing on helping my students improve their professional proficiencies in AWS.

I have already taught

I'm delighted to welcome Abhishek Singh as my co-instructor for these practice exams.

Welcome to the best practice exams to help you prepare for your AWS Certified Data Engineer Associate exam.

• You can retake the exams as many times as you want

• This is a huge original question bank

• You get support from instructors if you have questions

• Each question has a detailed explanation

• Mobile-compatible with the Udemy app

• 30-days money-back guarantee if you're not satisfied

We hope that by now you're convinced. . And there are a lot more questions inside the course.

Happy learning and best of luck for your AWS Certified Data Engineer Associate DEA-C01 exam.