If you've been using debuggers to reverse-engineer programs and wish you had a better understanding of Assembly Language, or if you're just looking to learn Assembly Language in a fun and exciting way, then this course is for you. Embark on a journey to master the intricacies of x64 assembly language with this specialized course, designed for those aiming to elevate their reverse engineering skills. This is the second course in this series after the first one: Assembly Language Programming for Reverse Engineering. In that earlier course the focus was on x86 basics where we covered 32-bit assembly. In this course, we continue the journey by learning x64 (64-bit assembly). In the second half of this course, we will apply all the knowledge from the first course, as well as new insights from this course, to hands-on reverse-engineering exercises with executable binaries. Taking the first course is preferable but not mandatory. You could just jump straight into this x64 course and learn the differences with x86 as you go along.
If you've been using debuggers to reverse-engineer programs and wish you had a better understanding of Assembly Language, or if you're just looking to learn Assembly Language in a fun and exciting way, then this course is for you. Embark on a journey to master the intricacies of x64 assembly language with this specialized course, designed for those aiming to elevate their reverse engineering skills. This is the second course in this series after the first one: Assembly Language Programming for Reverse Engineering. In that earlier course the focus was on x86 basics where we covered 32-bit assembly. In this course, we continue the journey by learning x64 (64-bit assembly). In the second half of this course, we will apply all the knowledge from the first course, as well as new insights from this course, to hands-on reverse-engineering exercises with executable binaries. Taking the first course is preferable but not mandatory. You could just jump straight into this x64 course and learn the differences with x86 as you go along.
While traditional assembly language courses focus on writing code from scratch using assemblers like NASM or FASM, they often leave a knowledge gap when it comes to applying that expertise to real-world reverse engineering. This course is tailored to fill this gap, providing you with the knowledge to modify and extend the functionality of existing 32-bit and 64-bit applications.
Utilizing the powerful x64dbg debugger, we'll bypass the traditional assemblers to teach you assembly language in the context it's most used in the field: directly within the debugging environment. This is also known as the hacker's perspective to assembly language. This hands-on approach ensures that you learn by doing, which is critical for effectively reverse engineering and manipulating software.
This black art is not widely taught and there are no existing courses elsewhere that put together coherently all the knowledge of assembly and reverse engineering in one place. More often than not, courses on Assembly and Reverse Engineering are offered separately. Many courses on Reverse Engineering don't cover the background assembly language from a software hacker's perspective. Moreover, almost all Assembly Language courses focus on writing programs from scratch rather than modifying existing programs to add new functionality with your own code using a debugger like x64dbg.
In this course, you will learn to:
- Navigate and utilize the x64dbg debugger
- Analyze and alter x64 executable files to inject custom code.
- Harness data, executable and memory segments to expand program capabilities.
- Develop new functions within existing applications for added functionality.
- Understand the x64 Microsoft Calling conventions and Stack Frames.
- Directly manipulate memory data segment
- Bypass string encryption
- Deep tracing to retrieve data and passwords
- Use Python to patch process memory
- Modify packed programs without unpacking
and more . . .
Features of this course:
This course is oriented towards practical applications
No lengthy, dull theoretical lectures
First half of this course: Learn x64 Assembly Language through the x64dbg debugger
Second half of this course: Practice reversing, tracing, extracting data, memory hacking, and modifying executable binaries
Learning Objectives:
By the end of this course, you will have a thorough understanding of x64 assembly language from a reverse engineer's perspective, a skill set that is rare and highly sought after in fields like cybersecurity, malware analysis, and software development.
Whether you're a security researcher, a malware analyst, a student of software security, or a programmer looking to deepen your understanding of software internals, this course is your stepping stone to becoming proficient in the 'black art' of assembly language and reverse engineering.
Enroll now to gain this competitive edge and take your skills to the next level. Let's unravel the complexities of x64 together. I look forward to guiding you through every step of this exciting journey. See you inside.
Welcome and introduction to the course
The importance of using virtual machines
Why using a debugger to learn assembly language rather than common assemblers like NASM, FASM or MASM
Intel CPU compatibility mode (32-bit) and 64-bit mode, addressing 64-bit registers and parts thereof
Write simple move instructions.
This is to test your understanding and ability to write your first assembly instructions from within x64dbg.
How write to memory and read from it.
Test your understanding of initialized data and uninitialized data segments of memory
How to create strings in memory and access it.
Using the BSS segment to create variables
Using XOR to zero out a register
Learn the ADD instructions
Learn how ADD Instructions work in assembly
Create a new hollowed out template for use in this course.
MOV using partial registers
How to access parts of a register
Basic PUSH and POP
An exercise on PUSH and POP
PUSH and POP exercise
Practical on MOV
XCHG Instructions
XCHG Memory
Learn how to use the INC and DEC on registers and memory
Learn how to use the NEG (Negate) Instruction
How to use ADD and SUB instructions
A quick conceptual introduction to the Carry Flag (CF), Overflow Flag (OF), Sign Flag (SF), and Zero Flag (ZF) in x64 assembly programming
How to interpret the CF flag
How the OF flag works with signed operations.
How to interpret and use the SF flag register.
Learn how the ZF flag works.
Understand bitwise logical operations
Understand how AND Operations works and how to use them
Understand how OR Operations work and how to use them
Understand how XOR operation works and the significance of XOR
Learn how the Logical Operations affect the SF and ZF registers
Practical on NOT Operation
Introduction to the two types of jumps: Unconditional and Conditional Jumps
Learn how JMP works and its 3 forms, address, register and memory.
How the TEST instruction works and its significance in testing RAX values after function calls.
How the CMP instruction how and its use in testing for register values.
Learn about the most common conditional jumps in x64dbg, i.e. JE/JNE and JB/JAE and their significance.
Understanding how CMP is used to compare signed numbers and the use of JG, JL and their counterparts JGE and JLE
Learn how to implement the IF structure in assembly using TEST, CMP together with JE and JNE instructions
Learn how to implement IF-ELSE statements in assembly by using CMP and JG
How to implement multiple IF tests within assembly.
How to implement WHILE loops in assembly inside x64dbg
How to implement DO-WHILE loops in assembly inside x64dbg.
Introduction to how to insert an array directly into memory and access it.
How to loop through an array use base address and offsets.
Learn how LEA is used to calculate effective addresses and also how to use it to do simple arithmetic.
Learn how to implement MUL in x64dbg
Learn how DIV instructions are implemented inside x64dbg
Learn how to implement IMUL inside x64dbg
How to implement PUSH and POP inside x64dbg
What are stackframes and how to use them to create functions
Understanding Microsoft x64 Calling Conventions
A practical on how to implement a 4 arguments function call with no local variables
A practical hands-on on how to implement a 4 arguments function call with with 4 local variables
A hands-on on how to implement a 5 argument function call
Learn how to implement a Function Call with 6 arguments
Practical on implementing a 7 argument function call.
Introduction to the practicals and some tips as well as installing DiE
Preliminary basic skills including how to check the exe file type
Use the String Search method to phish for passwords.
How to reverse a jump and do a simple surface patch.
How to use the Comment Tracing method to analyze register values and control flow logic
How to directly access memory to modify password and patch the exe file
How to enable a greyed-out disable Register button
How to remove a Nag Message Box
Patching to accept any serial key and also to modify the message box
Deep trace into inner function calls to identify username
Do deep tracing into inner function calls to uncover the password
Understand the significance of the EAX register in relation to function call returns and step into function calls to trace it.
Identify the correct jump to patch in order to reverse the logic so that the right message is shown
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.