Intel x86-64 Assembly

Xeno Kovah
January, 2024

Abstract

Intel processors have been a major force in personal computing for more than 30 years. An understanding of low level computing mechanisms used in Intel chips as taught in this course serves as a foundation upon which to better understand other hardware, as well as many technical specialties such as reverse engineering, compiler design, operating system design, code optimization, and vulnerability exploitation.

This class serves as a foundation for the follow on “x86-64 OS Internals” class. It teaches the basic concepts and describes the hardware that assembly code deals with. It also goes over many of the most common assembly instructions. Although x86 has hundreds of special purpose instructions, students will be shown it is possible to read most programs by knowing only around 20-30 instructions and their variations.

This class can be customized to be delivered in a 2-day form, and/or mixed with other x86-64 classes that depend on it, such as x86-64 OS Internals, or Intel x86-64 Firmware Attack & Defense.

If your engineers need to know how things work at a low level, reach out to us with info about how many students you’d like us to teach, and where.

Full class outline
  • Introduction
    • About this class
    • Optional refresher - binary to hex to decimal
    • Optional refresher - two’s complement negative numbers
    • Optional refresher - C data type sizes
    • Background - Endianness
  • Computer Registers
    • Memory hierarchy
    • x86-64 general purpose registers
    • Intel recommended register conventions
  • Your First Instruction
    • No-Operation (nop)
  • The Stack
    • Overview
    • New instructions - push & pop
    • RoX0r Arcade - The RSP Game
        - Level 1 - Canonical orientation, rbp at top, rsp at bottom
        - Level 2 - Canonical orientation, rbp and rsp move around
        - Level 3 - Canonical or flipped orientation, rbp and rsp move around
  • Calling Functions
    • CallASubroutine1.c - New Instructions - call, ret, mov, add, sub
    • Stack frame single-step slideware walkthrough
    • Stack frame single-step VisualStudio walkthrough
    • RoX0r Arcade - Dark Mathemagic
        - Level 1 - Basic mov, add, sub w/o calculator
        - Level 2 - mov, add, sub w/ calculator
  • Local Variables
    • SingleLocalVariable.c
    • Mystery Listery 3 Solved!
    • ArrayLocalVariable.c, New Instructions - imul, movsx, movzx
    • StructLocalVariable.c
    • RoX0r Arcade - Dark Mathemagic
        - Level 1 - movzx
        - Level 2 - movsx
  • Function Parameter Passing
    • Pass1Parameter.c
    • TooManyParameters.c
    • Mystery Listery 2 Solved!
    • 64-bit Calling Conventions
    • 32-bit Calling Conventions
    • Mystery Listery 1 Solved!
    • SpecialMaths.c, New instructions - lea
    • RoX0r Arcade - Dark Mathemagic
        - Level 1 - lea
  • Control Flow
    • GotoExample.c, New instructions - jmp
    • IfExample.c, New instructions, “jcc”, cmp
    • IfExample2.c
    • SwitchExample.c
    • RoX0r Arcade - 1 step forward, 3 steps back - Jumps
        - Level 1 - jz/je, jnz/jne
        - Level 2 - ja/jnbe, jna/jbe, jb/jnae, jnb/jae
        - Level 3 - jg/jnle, jng/jle, jl/jnge, jnl/jge
  • Boolean Logic
    • Refresher - Boolean logic
    • BooleanBecause.c, New instructions - and, or, not, xor
    • ForLoopNoRet.c, New instructions - inc, dec
    • BitmaskExample.c, New instructions - test
    • RoX0r Arcade - Dark Mathemagic
        - Level 1 - and, or, xor, not
        - Level 2 - inc, dec
    • 1 step forward, 3 steps back - Jumps w/ test
        - Level 1 - test, js, jns
  • Bit Shifting
    • ShiftExample1.c, New instructions - shl, shr
    • ShiftExample2Unsigned.c
    • ShiftExample3Signed.c, New instructions - cdq, sar
    • RoX0r Arcade - Dark Mathemagic
        - Level 1 - shl/sal, shr, sar
  • Multiply and Divide
    • MulDivExample.c, New instructions - div, idiv
    • RoX0r Arcade - Dark Mathemagic
        - Level 1 - mul + imul, 1-operand form
        - Level 2 - mul + 1, 2, and 3-operand forms of imul
        - Level 3 - div + idiv
  • CISC Delight - REPeatable Instructions
    • ArrayLocalVariable2.c, New instructions - rep stos
    • ThereWillBe0xb100d.c
    • JourneyToTheCenterOfMemcpy.c - New instructions - rep movs
    • RoX0r Arcade - DarkMathemagic - BOSS LEVEL!!!
        - YOU VS. ACCUMULATOR!!!
  • Two Paths Diverge
    • Here, if you mostly care about Windows, we will GOTO Windows Binary Debugging
    • Else, if you care about *NIX systems, you can learn about AT&T syntax and see all of the previous code again as compiled by GCC instead
  • Windows Binary Debugging
    • Windbg tutorial
    • GOTO Learning to Fish - Read The F*n Intel Manual!
  • Looking at all those examples on Linux!
    • Intel vs. AT&T assembly syntax
    • Separate videos for all of the following in Linux - CallAFunction1.c, SingleLocalVariable.c, ArrayLocalVariable.c, StructLocalVariable.c, Pass1Parameter.c, TooManyParameters.c, SpecialMaths.c, GotoExample.c, IfExample.c, IfExample2.c, SwitchExample.c, BooleanBecause.c, ForLoopNoRet.c, BitmaskExample.c, ShiftExample1.c, ShiftExample2Unsigned.c, ShiftExample3Signed.c, MulDivExample.c, ArrayLocalVariable2.c, ThereWillBe0xb100d.c, JourneyToTheCenterOfMemcpy.c
    • RoX0r Arcade - DarkMathemagic - BOSS LEVEL!!!
        - YOU VS. GNASTY ACCUMULATOR!!!
  • Learning to Fish - Read The F*n Intel Manual!
    • Learn to read the FUN manual!
  • Learning to Fish - Writing Assembly
    • Writing x86-64 assembly in Visual Studio
    • Writing x86-64 assembly in GCC
  • The Most Important Assembly Exercise You’ll Ever Do - The CMU Binary Bomb Lab!
    • Bomb Lab Intro
    • Do the Bomb Lab in either WinDbg or GDB or Ghidra (in concert with WinDbg or GDB)
  • Conclusion
Request Training Quote

Assembly Intel x86-64 x86-32 Windows Linux debugging gdb WinDbg Ghidra Reverse Engineering binary-bomb-lab
Xeno Kovah
Xeno Kovah
Dark Mentor Level X

Hacking firmware like it’s no big deal.