Migration · Active Directory · AWS networking
Migrating a full on-premises estate to AWS
An organization’s entire production estate, identity core included, ran on aging on-premises hypervisors and operating systems at or near end of support. It moved to a multi-AZ AWS VPC one application at a time. I was the lead engineer and owned identity and the cloud platform.
At a glance
How to read the tagsStarting point
Every production server ran on an operating system at or near end of support, on hypervisor hardware that was due for refresh. Both domain controllers were on-premises, and one of them carried unresolved USN-rollback exposure. Several independent application vendors each had their own support matrix and their own view of which operating system they’d run on.
My role
I was the lead engineer and sole owner of the identity and cloud-platform workstreams: the VPC design and address plan, the Terraform provisioning pattern and AMI strategy, and the Active Directory forest migration from design through decommission. Two network engineers handled the SD-WAN fabric and site connectivity, and I reviewed their work against the architecture. I also coordinated the application vendors’ change windows and sent the cutover communications ahead of each one. I wrote every runbook on the project, the risk register, the cutover sequence, and the go/no-go criteria.
Approach
Each application was provisioned from Terraform onto an AMI baked to its target OS, domain-joined, installed or restored with its vendor, validated against its own test plan, and cut over in its own vendor-coordinated window. One application at a time meant one rollback surface at a time.
The identity core went last on purpose. It had the largest blast radius and no vendor to share the risk. Going last also meant the bake period tested a loaded production environment instead of an empty one.
Identity cutover
One evening change window:
- Flip the VPC DHCP option set and the legacy DHCP resolver option.
- Stand up DHCP on the new controllers and repoint the relays.
- Transfer all five FSMO roles with
ntdsutil. - Make the new PDC emulator authoritative for time against the AWS time source.
- Cut fleet DNS over with one reversible script change distributed from SYSVOL. No per-endpoint touch, and revertible in minutes.
Then six days of bake with one go signal: DNS query volume on the legacy controllers trending to zero. Anything still querying them would have been a device with a hardcoded resolver, which fails silently after a demotion. Query volume reached near zero, and the first demotion ran on day seven.
Key decision
Demote both legacy controllers in one window, or stagger them?
Why
One controller carried unresolved USN-rollback exposure. A second demotion on top of an unresolved directory inconsistency could have turned a rollback into a forest recovery.
Outcome
That mattered the same day. The first controller demoted cleanly, but objects for it lingered in the Configuration partition after DC-locator had already dropped it from live DNS. Because the second demotion hadn’t fired, I could force replication, allow convergence, and run metadata cleanup against a stable directory.
What broke
DHCP failover failed with error 20010
The failover wizard copies scopes and their option references to the partner, but not the option definitions those references point to. The error pointed at the scopes; the cause was the missing option definitions.
FixDiff the option definitions across both servers and pre-create any missing ones on the partner before running the wizard.
Demoted controller still listed in the directory
DC-locator reads live DNS SRV records, while the Configuration partition still held objects for the demoted server until replication and cleanup caught up.
FixConfirm whether a tool reads live DNS or directory objects before acting on its result.
Stale errors in health checks
The check scraped event logs and reported entries from before the teardown as current state.
FixTimestamp every error against the change time before chasing it.
Incorrect egress assumption
The working model had cloud servers breaking out through the VPC Internet Gateway. A network engineer challenged it, and the evidence backed him: the same public address appeared on cloud servers and site edge devices, which only happens with one upstream breakout point.
FixThree deliverables corrected before delivery. The migration had kept the existing filtered egress path instead of adding an unfiltered one.
Conflicting subnet documentation
Planning documents, discovery tooling, and legacy device names gave three different answers about which subnet belonged to which site.
FixThe live DHCP scope configuration became the source of truth, and every document was reconciled to it.
Vendor CloudFormation ran past plan
The SD-WAN appliance shipped as vendor CloudFormation templates, and its deployment ran well past plan.
FixSchedule risk for vendor-supplied IaC is budgeted the same way as for a vendor-installed appliance.
Documentation
At closure, the client received a knowledge-transfer document covering the before state, the risk register, the per-server implementation pattern, the cutover log, and the test and rollback plans.
Both architecture diagrams were generated from Python instead of drawn, so three rounds of review feedback were code changes rather than redraws. QA was programmatic: every element bounds-checked against the canvas, and key strings extracted to confirm each edit landed. Every deliverable came from one Markdown source, so the Word and PDF versions couldn’t drift apart.
Stack
- EC2
- VPC (multi-AZ, function-tiered subnets)
- Transit Gateway
- Site-to-Site VPN
- AWS Backup
- VPC DHCP option sets
- Terraform
- CloudFormation
- PowerShell
- AD DS
- AD CS
- DNS, DHCP failover, NTP
- DFSR
- Group Policy
- Entra Connect
- SQL Server
- SD-WAN appliance
- Python (diagram generation)
Organizations, people, dates, and identifying environment details are omitted or generalized. Each figure carries a tag for the kind of evidence behind it.