Lab 12: Backup, Recovery & Neon Branching¶
| Field | Details |
|---|---|
| Course | ITEC-445 โ Advanced Database Management |
| Week | 12 |
| Difficulty | โญโญโญ Advanced |
| Estimated Time | 80 minutes |
| Topic | Backup, Recovery & High Availability |
| Prerequisites | Lab 01 schema + all previous labs on Neon branch lab-12 |
| Deliverables | pg_dump backup, recovery simulation, Neon branch strategy, verify_lab12() PASS |
Overview¶
Data loss is career-ending. Backup and recovery skills are the most operationally critical skills in database administration. In this lab you will use pg_dump for logical backups of Neon, simulate a disaster scenario, practice point-in-time recovery using Neon's branch feature (which provides database-level time travel), and design a backup strategy with defined RTO and RPO targets.
Branch Requirement
Create branch lab-12 with the full accumulated schema from Labs 01โ11 (or at minimum Lab 01 + Lab 06 seed data). This is the database you will "back up" and "recover."
Part A โ Logical Backup with pg_dump (25 pts)¶
pg_dump creates a portable logical backup of a PostgreSQL database. It works natively with Neon.
A1. Full schema + data backup¶
# Full backup โ schema and data, custom format (compressed, supports selective restore)
pg_dump "$DATABASE_URL" \
--format=custom \
--compress=9 \
--schema=fsu \
--file=fsu_backup_$(date +%Y%m%d_%H%M%S).dump \
--verbose
# Verify the backup file
ls -lh fsu_backup_*.dump
A2. Schema-only backup (DDL without data)¶
# Useful for deploying to a new environment
pg_dump "$DATABASE_URL" \
--schema-only \
--schema=fsu \
--file=fsu_schema_$(date +%Y%m%d).sql
# View the schema DDL
head -80 fsu_schema_*.sql
A3. Data-only backup for specific tables¶
# Export just the students and enrollments tables as INSERT statements
pg_dump "$DATABASE_URL" \
--data-only \
--schema=fsu \
--table=fsu.students \
--table=fsu.enrollments \
--inserts \
--file=fsu_data_only_$(date +%Y%m%d).sql
wc -l fsu_data_only_*.sql # count lines to verify
A4. List backup contents¶
# pg_restore --list shows what's in the custom-format dump
pg_restore --list fsu_backup_*.dump | head -30
Part B โ Disaster Simulation & Recovery (30 pts)¶
B1. Simulate a disaster โ accidental data deletion¶
SET search_path = fsu;
-- Record the count before the disaster
SELECT 'Before disaster:' AS event, COUNT(*) AS students FROM students
UNION ALL
SELECT 'Enrollments:', COUNT(*) FROM enrollments;
-- THE DISASTER: someone accidentally deletes all CS department students
BEGIN;
DELETE FROM enrollments
WHERE student_id IN (SELECT student_id FROM students WHERE dept_id = 1);
DELETE FROM students WHERE dept_id = 1;
-- "Oops" โ how many rows deleted?
SELECT 'After disaster:' AS event, COUNT(*) AS students FROM students
UNION ALL
SELECT 'Enrollments:', COUNT(*) FROM enrollments;
-- DO NOT COMMIT โ we'll recover two ways
ROLLBACK; -- first recovery: transaction rollback
B2. Verify transaction rollback worked¶
SELECT 'After rollback:' AS event, COUNT(*) AS students FROM students;
-- Should be back to original count
B3. Simulate a committed disaster (harder recovery)¶
-- This time, commit the deletion โ simulating a real production incident
BEGIN;
DELETE FROM enrollments WHERE student_id IN (
SELECT student_id FROM students WHERE dept_id = 1 LIMIT 10
);
DELETE FROM students WHERE dept_id = 1 AND student_id IN (
SELECT student_id FROM students WHERE dept_id = 1 LIMIT 10
);
COMMIT; -- data is gone โ need to recover from backup
SELECT 'After committed delete:', COUNT(*) FROM students WHERE dept_id=1;
B4. Recovery from pg_dump backup¶
# Recover just the students table from backup
pg_restore \
--dbname "$DATABASE_URL" \
--schema=fsu \
--table=students \
--data-only \
--single-transaction \
--verbose \
fsu_backup_*.dump
After restore:
Part C โ Neon Branch-Based Point-in-Time Recovery (25 pts)¶
Neon's branch feature provides instant database-level snapshots โ equivalent to point-in-time recovery but without binary log replay.
C1. Create a "before" snapshot branch¶
# Via Neon CLI (if installed)
# Or via Neon web UI: your project โ Branches โ Create branch
# Name: lab-12-checkpoint-before-delete
# Created from: lab-12 branch at current timestamp
Via the Neon web UI: 1. Go to your project โ Branches tab 2. Click Create branch 3. Name it lab-12-checkpoint 4. Leave timestamp as "now" (captures current state) 5. Screenshot the branch creation
C2. Perform destructive changes on the main branch¶
-- On lab-12 branch: simulate more data changes
UPDATE fsu.students SET gpa = 0.00 WHERE student_id <= 20;
DELETE FROM fsu.enrollments WHERE semester = '2023SP';
SELECT 'Corrupted students:', COUNT(*) FROM fsu.students WHERE gpa = 0;
SELECT '2023SP enrollments:', COUNT(*) FROM fsu.enrollments WHERE semester = '2023SP';
C3. "Recover" by connecting to the checkpoint branch¶
# Get the checkpoint branch connection string from Neon UI
# Connection Details โ select "lab-12-checkpoint" branch
export CHECKPOINT_URL="postgresql://user:***@ep-checkpoint.neon.tech/neondb?sslmode=require"
# Verify the checkpoint has good data
psql "$CHECKPOINT_URL" -c "SELECT COUNT(*) FROM fsu.students WHERE gpa > 0;"
psql "$CHECKPOINT_URL" -c "SELECT COUNT(*) FROM fsu.enrollments WHERE semester='2023SP';"
C4. Branch comparison report¶
Write branch_comparison.sql:
-- Run this on the CORRUPTED branch (lab-12)
-- Compare with the output from the CHECKPOINT branch
SELECT
'lab-12 (corrupted)' AS branch,
COUNT(*) AS total_students,
COUNT(CASE WHEN gpa = 0 THEN 1 END) AS zero_gpa_students,
(SELECT COUNT(*) FROM enrollments WHERE semester='2023SP') AS sp23_enrollments
FROM fsu.students;
Screenshot results from both branches side-by-side.
Part D โ Backup Strategy Design (20 pts)¶
Write lab12_backup_strategy.md โ a formal backup strategy document for the Frostburg University database:
# Frostburg University Database โ Backup & Recovery Strategy
## 1. Recovery Objectives
| Scenario | RTO Target | RPO Target |
|----------|-----------|-----------|
| Accidental row deletion | < 15 min | < 1 hour |
| Schema corruption | < 30 min | < 4 hours |
| Full database loss | < 2 hours | < 24 hours |
| Ransomware/site disaster | < 4 hours | < 1 week |
## 2. Backup Schedule
| Backup Type | Frequency | Retention | Tool |
|-------------|-----------|-----------|------|
| Full logical backup | Daily (off-peak) | 30 days | pg_dump |
| Schema-only DDL | Weekly | 1 year | pg_dump --schema-only |
| Neon branch checkpoint | Before major changes | 7 days | Neon branching |
| Off-site copy | Weekly | 90 days | S3/cloud copy of dump |
## 3. Recovery Procedures (step-by-step)
### Scenario A: Accidental data deletion (< 1 hour ago)
1. ...
### Scenario B: Recovery from pg_dump backup
1. ...
## 4. Testing Schedule
- Monthly: simulate data deletion + recover from pg_dump
- Quarterly: simulate full schema recovery
- Annual: full disaster recovery drill
## 5. Neon-Specific Considerations
- Neon branches are instantaneous (copy-on-write, no clone time)
- Neon free tier retains 7 days of history for PITR
- ...
Verification¶
SET search_path = fsu;
CREATE OR REPLACE FUNCTION verify_lab12()
RETURNS TABLE(check_id TEXT, description TEXT, result TEXT, points INT) AS $$
BEGIN
RETURN QUERY SELECT '01', 'Students table has 500+ rows (recovery successful)',
CASE WHEN (SELECT COUNT(*) FROM students) >= 500 THEN 'PASS' ELSE 'FAIL' END, 10;
RETURN QUERY SELECT '02', 'CS dept students exist (dept_id=1)',
CASE WHEN (SELECT COUNT(*) FROM students WHERE dept_id=1) > 0
THEN 'PASS' ELSE 'FAIL' END, 10;
RETURN QUERY SELECT '03', 'No students have GPA=0 after recovery',
CASE WHEN (SELECT COUNT(*) FROM students WHERE gpa = 0) = 0
THEN 'PASS' ELSE 'FAIL' END, 10;
RETURN QUERY SELECT '04', 'Enrollments table has 1000+ rows',
CASE WHEN (SELECT COUNT(*) FROM enrollments) >= 1000 THEN 'PASS' ELSE 'FAIL' END, 10;
RETURN QUERY SELECT '05', '2024FA enrollments still exist',
CASE WHEN (SELECT COUNT(*) FROM enrollments WHERE semester='2024FA') > 0
THEN 'PASS' ELSE 'FAIL' END, 10;
RETURN QUERY SELECT '06', 'All departments have at least 1 student',
CASE WHEN NOT EXISTS(
SELECT dept_id FROM departments d
WHERE NOT EXISTS(SELECT 1 FROM students s WHERE s.dept_id=d.dept_id)
) THEN 'PASS' ELSE 'NEEDS_RECOVERY' END, 10;
RETURN QUERY SELECT '07', 'fsu schema intact (5 base tables)',
CASE WHEN (SELECT COUNT(*) FROM information_schema.tables
WHERE table_schema='fsu' AND table_type='BASE TABLE') >= 5
THEN 'PASS' ELSE 'FAIL' END, 10;
RETURN QUERY SELECT '08', 'grade_points view exists',
CASE WHEN EXISTS(SELECT 1 FROM information_schema.views
WHERE table_schema='fsu' AND table_name='grade_points')
THEN 'PASS' ELSE 'FAIL' END, 10;
RETURN QUERY SELECT '09', 'Indexes still present after recovery',
CASE WHEN (SELECT COUNT(*) FROM pg_indexes WHERE schemaname='fsu'
AND indexname NOT LIKE '%pkey%') >= 3
THEN 'PASS' ELSE 'FAIL' END, 10;
RETURN QUERY SELECT '10', 'schema_migrations table intact',
CASE WHEN EXISTS(SELECT 1 FROM information_schema.tables
WHERE table_schema='fsu' AND table_name='schema_migrations')
THEN 'PASS' ELSE 'FAIL' END, 10;
END;
$$ LANGUAGE plpgsql;
SELECT check_id, description, result,
CASE WHEN result = 'PASS' THEN points ELSE 0 END AS earned
FROM verify_lab12()
ORDER BY check_id;
Additional Requirement (20 pts)¶
Write an automated backup verification script verify_backup.sh that: 1. Takes a .dump file as argument 2. Creates a fresh Neon branch (or uses a test connection) to restore into 3. Runs SELECT COUNT(*) FROM fsu.students and fsu.enrollments against the restored data 4. Compares counts against a baseline expected_counts.json file 5. Exits 0 (success) or 1 (failure) based on match
This demonstrates "backup is only as good as its last tested restore."
Submission Checklist¶
- [ ]
fsu_backup_*.dumpโ pg_dump custom format backup (verify with pg_restore --list) - [ ]
fsu_schema_*.sqlโ schema-only DDL backup - [ ] Neon branch
lab-12-checkpointโ screenshot of creation in Neon UI - [ ] Screenshots: corrupted state vs checkpoint state (branch comparison)
- [ ]
lab12_backup_strategy.mdโ formal strategy document - [ ]
verify_backup.shโ additional requirement - [ ]
verify_lab12()screenshot โ all PASS (on recovered branch)
Grading¶
| Component | Points |
|---|---|
| Part A โ pg_dump (3 backup types, all files present) | 25 |
| Part B โ Disaster simulation + recovery (transaction rollback + pg_restore) | 30 |
| Part C โ Neon branch checkpoint (created, compared, screenshots) | 25 |
| Part D โ Backup strategy document (all sections, RTO/RPO defined) | 20 |
| Additional requirement โ verify_backup.sh | 20 |
| Total | 120 |