Lab 03 — Memory Protections: ASLR, Stack Canaries & NX¶

Overview¶

Memory corruption vulnerabilities — buffer overflows, use-after-free, format string bugs — remain among the most dangerous classes of software defects. Modern operating systems deploy three complementary mitigations that dramatically raise the cost of exploiting these vulnerabilities: ASLR (Address Space Layout Randomization), stack canaries, and the NX bit (No-eXecute, also called DEP on Windows). In this lab you will observe each mitigation in action, disable them to see the difference, trigger a live stack canary detection, and examine binary metadata to confirm whether protections are enabled.

By the end of this lab you will understand:

• How ASLR randomizes the memory layout and why that matters for exploit reliability
• What a stack canary is, where it is placed, and how the runtime detects overflow
• How the NX bit prevents code injection by marking the stack non-executable
• Why layered defenses retain value even when each individual layer can be bypassed

Prerequisites¶

• Completed Labs 01 and 02
• Chapter 3 reading on memory layout and exploitation mitigations
• Basic understanding of what a stack is and what a return address does

Part 1 — ASLR: Address Space Layout Randomization¶

Step 1.1 — Check the ASLR Setting¶

docker run --rm python:3.11-slim bash -c "cat /proc/sys/kernel/randomize_va_space"

Expected output: 2

📸 Screenshot checkpoint 03a — Capture the randomize_va_space value of 2.

Step 1.2 — Observe ASLR in Action¶

docker run --rm python:3.11-slim bash -c "
for i in 1 2 3; do
  python3 -c \"
import ctypes
buf = ctypes.create_string_buffer(16)
print(f'Run \$i: buffer_addr={hex(ctypes.addressof(buf))}')
\"
done"

Expected output: Three different hexadecimal addresses — one per iteration.

Run 1: buffer_addr=0x7f3a2c001b20
Run 2: buffer_addr=0x7f91d4000b40
Run 3: buffer_addr=0x7f5e88002a60

📸 Screenshot checkpoint 03b — Capture all three run outputs showing three different addresses.

Step 1.3 — Memory Map Shows Randomized Base Addresses¶

docker run --rm python:3.11-slim python3 -c "
with open('/proc/self/maps') as f:
    for line in list(f)[:8]:
        print(line.rstrip())
"

Run this command twice and compare the base addresses in both outputs. With ASLR enabled they will differ between runs.

📸 Screenshot checkpoint 03c — Capture /proc/self/maps output (run it twice if possible and show both — addresses should differ).

Step 1.4 — Disable ASLR and Observe Predictable Addresses¶

docker run --rm --privileged python:3.11-slim bash -c "
echo 0 > /proc/sys/kernel/randomize_va_space
for i in 1 2 3; do
  python3 -c \"
import ctypes
buf = ctypes.create_string_buffer(16)
print(f'Run \$i: buffer_addr={hex(ctypes.addressof(buf))}')
\"
done"

Expected output: Three identical addresses.

Run 1: buffer_addr=0x55a3b4001b20
Run 2: buffer_addr=0x55a3b4001b20
Run 3: buffer_addr=0x55a3b4001b20

📸 Screenshot checkpoint 03d — Capture all three identical addresses with ASLR disabled.

Part 2 — Stack Canaries¶

Step 2.1 — Compare Binaries With and Without Stack Canary¶

docker run --rm ubuntu:22.04 bash -c "
apt-get update -qq && apt-get install -y -qq gcc binutils 2>/dev/null
printf '#include <stdio.h>\nvoid greet() { char buf[8]; }\nint main() { greet(); return 0; }\n' > /tmp/prog.c
gcc -o /tmp/with_canary /tmp/prog.c -fstack-protector-strong
gcc -o /tmp/no_canary   /tmp/prog.c -fno-stack-protector
echo '=== With stack canary: __stack_chk_fail references ==='
objdump -d /tmp/with_canary | grep -c 'stack_chk'
echo '=== Without canary: references ==='
objdump -d /tmp/no_canary | grep -c 'stack_chk' || echo '0'"

Expected output:

=== With stack canary: __stack_chk_fail references ===
2
=== Without canary: references ===
0

📸 Screenshot checkpoint 03e — Capture the objdump output showing the count difference (e.g., 2 vs. 0).

Step 2.2 — Trigger Live Stack Smashing Detection¶

docker run --rm ubuntu:22.04 bash -c "
apt-get update -qq && apt-get install -y -qq gcc 2>/dev/null
printf '#include <string.h>\nvoid vuln() { char buf[8]; memset(buf,65,32); }\nint main() { vuln(); return 0; }\n' > /tmp/overflow.c
gcc -o /tmp/overflow /tmp/overflow.c -fstack-protector-strong
/tmp/overflow 2>&1 || echo 'Stack smashing detected — canary triggered!'"

Expected output:

*** stack smashing detected ***: terminated
Stack smashing detected — canary triggered!

📸 Screenshot checkpoint 03f — Capture the "stack smashing detected" termination message.

Part 3 — NX Bit (No-Execute)¶

Step 3.1 — Compare Executable vs. Non-Executable Stack¶

docker run --rm ubuntu:22.04 bash -c "
apt-get update -qq && apt-get install -y -qq gcc binutils 2>/dev/null
printf '#include <stdio.h>\nint main(){return 0;}\n' > /tmp/nx.c
gcc -o /tmp/nx_on  /tmp/nx.c
gcc -o /tmp/nx_off /tmp/nx.c -Wl,-z,execstack
echo '=== Default (NX enabled): stack NOT executable ==='
readelf -l /tmp/nx_on  | grep GNU_STACK
echo '=== execstack flag (NX disabled): stack EXECUTABLE ==='
readelf -l /tmp/nx_off | grep GNU_STACK"

Expected output:

=== Default (NX enabled): stack NOT executable ===
  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW  0x10
=== execstack flag (NX disabled): stack EXECUTABLE ===
  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RWE 0x10

📸 Screenshot checkpoint 03g — Capture both readelf outputs showing RW vs. RWE.

Step 3.2 — Observe Executable Memory Regions at Runtime¶

docker run --rm python:3.11-slim python3 -c "
with open('/proc/self/maps') as f:
    for line in f:
        if 'x' in line.split()[1]:
            print('EXEC:', line.rstrip())
" | head -8

Expected output: Only text sections of python3 and linked libraries appear as executable. The [stack] and [heap] regions are not listed — they are rw- only.

📸 Screenshot checkpoint 03h — Capture the executable memory regions output, confirming stack and heap are absent from the list.

Cleanup¶

docker system prune -f

Assessment¶

Screenshot Checklist¶

Memory Protection Comparison Table¶

Complete the following table in your lab report (20 points):

For each row, fill in all four columns based on your lab observations and the Chapter 3 reading.

Reflection Questions¶

Answer each question in 3–5 sentences. (40 points — 10 points each)

Question 1

ASLR randomizes the memory layout of each new process. Explain how this defeats a return-address-overwrite exploit that works without ASLR — specifically, what does the attacker no longer know, and why does that matter? Then explain what a brute-force against ASLR attack is, and why it is much more practical against a 32-bit address space than a 64-bit one. (Hint: calculate the approximate number of possible base addresses in each case.)

Question 2

A stack canary is a random value inserted between the local variables and the saved return address on the stack. Describe precisely where the canary value is stored at runtime (both the "template" copy and the stack copy), when the comparison happens in program execution, and what happens if the comparison fails. Why is the canary random rather than a fixed value like 0xDEADBEEF?

Question 3

The NX bit marks the stack segment as non-executable (RW instead of RWE). Name the classic exploit technique that this directly defeats, and briefly describe how that technique worked before NX was available. Then name the bypass technique attackers developed in response to NX — explain what ROP (Return-Oriented Programming) is and why it does not require injecting new code.

Question 4

ASLR, stack canaries, and NX are three independent layers of defense. A skilled attacker with (a) an information-leak vulnerability to defeat ASLR, (b) a ROP chain to defeat NX, and (c) a way to avoid triggering the canary can theoretically defeat all three simultaneously. Does this mean these mitigations are useless? Explain the concrete security value of layered defenses even when each layer can be individually bypassed — consider the attacker's cost in time, expertise, and reliability.

Grading Rubric¶