NTP server that traveled back in time caused massive Aussie mobile outage
Jump to main content
Search
REG AD
networks
NTP server that traveled back in time caused massive Aussie mobile outage
Telstra skipped a patch, didn’t record changes, had no idea it was an accident waiting to happen
Simon Sharwood
Simon<br>Sharwood
APAC Editor
Published<br>fri 17 Jul 2026 // 05:48 UTC
Australian telco Telstra has revealed the cause of the recent incident that caused widespread connectivity problems across its mobile networks, inculding outages to Australia's 000 emergency services line, plus outages to electronic payments services and transport networks.<br>The carrier explained itself in a submission [PDF] to a Senate inquiry into outages affecting Australia’s emergency services which initially investigated an outage at Telstra's main rival, Optus. The Optus incident is linked to multiple deaths after people could not reach emergency services.<br>Telstra's submission reveals that the carrier's attempt to address a known resilience problem was the instigating incident.
REG AD
That problem was a faulty backup power feed in the chassis used to house a network time protocol (NTP) server. A few minutes before midnight on July 7, a Telstra techie started work to replace that chassis.
REG AD
By 3:38 AM on July 8, the worker had finished the job and powered up the server.<br>The NTP server included a GPS card that Telstra says “did not operate as expected” when the server came back online.
MORE CONTEXT
Telstra outage: Failed emergency services calls, train chaos, payment systems down
Aussie telco Telstra says soz after accidentally diverting traffic meant for encrypted email biz through its servers
Telstra chairman: If those darn kids can earn $5m playing Fortnite, why can't execs?
Telstra told by regulator to keep copper for an extra five years
“We now believe this occurred because of an intentional design change that had previously been made to the equipment to fix an earlier fault [which] had not been properly documented,” the submission states. “This meant the maintenance team was not aware of the way the device would behave when restarted.”<br>Telstra also admitted it had not applied a software update to the device, despite knowing of its availability in early 2026.<br>“Had that software update been completed or had the design change been properly reviewed and documented post the earlier incident, and reflected in the maintenance procedure, the outage may not have occurred,” the submission states.<br>The outage did happen because once the NTP server came online it reset its clocks to the year 2006.<br>The server then did what NTP servers do: publish that time to myriad other machines across the network.<br>Once those machines received the incorrect time, other kit on the Telstra network compared digital certificates and decided something dodgy was going on – so denied connections.
REG AD
Just one of Telstra’s three NTP servers had this problem, but enough bad timing info made it onto the carrier’s network to cause chaos.<br>The telco isolated the bad box at 7:11 AM, and by 10:30 AM had identified all network components that used the bad time information the naughty NTP box broadcast.<br>That wasn’t the end of the matter, because as valid time information rippled across the network, some equipment didn’t close IP sessions. Customer devices therefore couldn’t reconnect to the network unless rebooted.<br>The carrier has described the outage as “clearly unacceptable.”<br>“If maintenance work can trigger this kind of outage, it suggests our controls were not good enough. We are accountable for that, and our investigation will address why that design change was not documented, why the software update was not completed, and what needs to change in our controls so known risks are captured, prioritised and closed before they can affect customers.”<br>The carrier has done the usual thing by promising to conduct deep and lengthy self-reflection, submit itself to further flagellation at more inquiries, co-operate with a probe by the relevant regulator, and to do what it takes to prevent similar incidents. ®
network time protocol<br>australia<br>telstra<br>ntp<br>networks<br>outage
REG AD
personal tech
Tech support chap told angry customer to think inside the box – and solved the problem
We're not Li-ion, user thought mobile phone batteries died every few days
networks
NTP server that traveled back in time caused massive Aussie mobile outage
Telstra skipped a patch, didn’t record changes, had no idea it was an accident waiting to happen
Gobi X: Creating more energy for AI, not taking it from society
PARTNER CONTENT: How Envision is reversing the datacenter playbook by making computing chase abundant desert power, not the other way around
AI AND ML
South Korea making its own security-centric AI model
Adapting existing local LLM project for security and sovereignty purposes and hopes to one day match Mythos
PaaS + IaaS
Amazon Web Services' most vocal...