We may earn an affiliate commission when you visit our partners.
Cristina Gheorghisan

The aim of this course is to cover the essentials techniques and tools for reverse engineering and malware analysis. As the title suggests, we will cover only the Windows environment in this class, since it is by far the most used and abused. We try to provide a complete picture for the starting reverse engineer but in the same time relevant for the more advanced analysts.

There are always multiple ways to do a task. We will insist more on "Why?"s instead of "How?"s since we consider that it's more important to understand WHAT we're trying to achieve and WHY.

Read more

The aim of this course is to cover the essentials techniques and tools for reverse engineering and malware analysis. As the title suggests, we will cover only the Windows environment in this class, since it is by far the most used and abused. We try to provide a complete picture for the starting reverse engineer but in the same time relevant for the more advanced analysts.

There are always multiple ways to do a task. We will insist more on "Why?"s instead of "How?"s since we consider that it's more important to understand WHAT we're trying to achieve and WHY.

There are no pre-requisites for this class other that a Windows virtual machine and the will to learn. All the tools discussed here are freely available online. Analyses are demonstrated on a Windows 8.1 virtual machine.

Don't worry. Neither professional programming experience nor assembly language knowledge are required to benefit from the course. If you know these already, it would be helpful when we’ll look at identifying encryption algorithms and bypass anti-virtualisation checks. The concepts will be explained clearly and additional resources will be recommended.

Some programming experience will definitely be beneficial, however, the focus will be mostly on understanding the technique, the tools and their most useful features. When needed, source code written in C or Python will be provided.

To get the most out of this course, we recommended to try all the exercises and assignments that follow the lectures and whenever a concept or idea is not fully explained or clearly understood, either reach out with questions on the Q&A section or consult online resources.

Enroll now

What's inside

Learning objectives

  • How to build a virtual environment for malware analysis
  • How to identify and bypass anti-virtualisation techniques
  • How to confidently use debuggers and disassemblers
  • Be able to analyse processes and perform low-level api monitoring
  • How to monitor changes to the registry and the file system
  • Learn how to intercept and investigate network traffic
  • Execute samples inside a sandbox, extract and analyse artefacts
  • Become comfortable with tools for static analysis
  • How to identify packers automatically and manually

Syllabus

Introduction
Building a virtual environment
Setup
Setup Practice
Read more
Hardening - VM Detection

Play with few VM detection tools and understand how they work.

Make your virtual machine resilient against being detected, using WMI and Registry tricks.

Anti-VM and Anti-Debug
Anti-VM and Anti-Debug Practice 1
Anti-VM and Anti-Debug Practice 2
Anti-VM and Anti-Debug Practice 3
Anti-VM and Anti-Debug Practice 4
Analysis Tools
Monitor Process Activity
Monitor Process Activity Practice 1
Monitor Process Activity Practice 2
Monitor Process Activity Practice 3
Monitor Process Activity Practice 4

In this quiz we'll see how we can spot malicious processes using only Process Explorer.

Monitor API Calls
Monitor API Calls Practice 1
Monitor API Calls Practice 2
Monitor API Calls Practice 3
Monitor API Calls Assignment
Monitor Registry and File System
Monitor Registry and File System Practice 1
Monitor Registry and File System Practice 2
Monitor Registry and File System Assignment
Monitor Network Activity
Monitor Network Activity Practice 1
Monitor Network Activity Practice 2
Monitor Network Activity Practice 3
Monitor Network Activity Practice 4
Monitor Network Activity Practice Assignment
Sandboxing
Sandboxing Practice 1
Sandboxing Practice 2
Static Analysis
Static Analysis Practice 1
Static Analysis Practice 2
We will quickly recap what we’ve learnt so far and recommend resources and materials to continue your study.
Next steps

Save this course

Save Reverse Engineering Essentials to your list so you can find it easily later:
Save

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Reverse Engineering Essentials with these activities:
Review Assembly Language Fundamentals
Strengthen your understanding of assembly language, which is helpful for reverse engineering and malware analysis, especially when identifying encryption algorithms and anti-virtualization checks.
Browse courses on Assembly Language
Show steps
  • Review basic assembly instructions (MOV, ADD, SUB, JMP).
  • Practice reading and interpreting simple assembly code snippets.
  • Understand the role of registers and memory in assembly.
Read 'Reverse Engineering for Beginners'
Reinforce your understanding of reverse engineering fundamentals with a beginner-friendly guide.
View Alter Ego on Amazon
Show steps
  • Read the introductory chapters on reverse engineering concepts.
  • Work through the examples and exercises in the book.
  • Focus on understanding the basic principles of disassembly and debugging.
Read 'Practical Malware Analysis'
Enhance your understanding of malware analysis techniques, which are directly applicable to the course content.
Show steps
  • Read the chapters on static and dynamic analysis.
  • Follow the examples and try them on your own.
  • Take notes on key concepts and techniques.
Four other activities
Expand to see all activities and additional details
Show all seven activities
Practice Debugging with OllyDbg/x64dbg
Improve your debugging skills, which are essential for reverse engineering and malware analysis.
Show steps
  • Download and install OllyDbg or x64dbg.
  • Load a simple executable and step through the code.
  • Set breakpoints and examine memory and registers.
  • Practice debugging different types of programs.
Write a Blog Post on Anti-VM Techniques
Solidify your understanding of anti-virtualization techniques by researching and explaining them in a blog post.
Show steps
  • Research common anti-VM techniques.
  • Write a blog post explaining how these techniques work.
  • Include code examples or screenshots to illustrate your points.
  • Publish your blog post online.
Create a Yara Rule Set for Malware Detection
Develop practical skills in malware detection by creating a set of Yara rules based on your analysis of malware samples.
Show steps
  • Research and understand the syntax and structure of Yara rules.
  • Analyze several malware samples to identify unique characteristics.
  • Write Yara rules to detect these characteristics.
  • Test your Yara rules against a collection of malware samples.
Analyze a Sample Malware
Apply the knowledge and skills learned in the course to analyze a real-world malware sample.
Show steps
  • Obtain a malware sample from a reputable source.
  • Perform static and dynamic analysis of the malware.
  • Document your findings in a report.
  • Present your analysis to your peers.

Career center

Learners who complete Reverse Engineering Essentials will develop knowledge and skills that may be useful to these careers:

Reading list

We've selected two books that we think will supplement your learning. Use these to develop background knowledge, enrich your coursework, and gain a deeper understanding of the topics covered in Reverse Engineering Essentials.
Comprehensive guide to understanding and analyzing malware. It covers static and dynamic analysis techniques, debugging, and disassembly, which are all essential skills for reverse engineering. It provides practical examples and step-by-step instructions, making it an excellent companion for the course, especially for the hands-on exercises.
Provides a gentle introduction to reverse engineering concepts and techniques. It covers topics such as disassembly, debugging, and code analysis. It is particularly useful for beginners who are new to the field and want to build a solid foundation. It good reference for understanding the basics of reverse engineering.

Share

Help others find this course page by sharing it with your friends and followers:

Similar courses

Similar courses are unavailable at this time. Please try again later.
Our mission

OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.

Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.

Find this site helpful? Tell a friend about us.

Affiliate disclosure

We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.

Your purchases help us maintain our catalog and keep our servers humming without ads.

Thank you for supporting OpenCourser.

© 2016 - 2025 OpenCourser