A botched data centre migration has taken down DNS resolution services and email for thousands of customers of Enom – a wholesale provider of domains, email, and SSL – in an incident that began on Saturday January 15.
Scheduled for Jan 15, 2022, 18:10 PST, Enom posted – as customers began to lose services – that “our Dev team’s testing indicates the need to extend the migration period to midnight (PST), 16 January…”
Describing its “scheduled maintenance” as “completed” at midnight (PST) on Sunday (in what may have been an automated update) Enom continued to face complaints from customers later on Monday.
Hundreds appeared to have completely lost access to email and their domains after the Enom outage. The number for those affected is likely to be significantly higher than those being vociferous about it on social media.
Enom is owned by Tucows --which also owns OpenSRS, EPAG, Ascio and Hover, and claims to be the "second largest domain registrar in the world by volume."
“We are investigating reports of DNS resolution and management issues, as well as domain registration difficulties, post maintenance” Enom said via Twitter (8am PST) on Monday afternoon UK time.
“What an EPIC failure of a planned data migration. I have clients with mission critical email delivery (credit card transactions for hundreds of clients) who are dead in the water due to failed DNS propagation” one customer fumed. There were 500+ customers waiting in a rapidly climbing queue for support as The Stack published late on Monday.
“£10 says this is a BOM character introduced in the data migration of DNS records between linux based name servers with a c# processor in the middle” one observer speculated. The cause was not immediately clear.
Those affected included the Scottish Parliament which appeared to be relying on Enom for its emails.