20210522 – Core Router Issue - PUBLIC

NOTICE TIME ZONE	Brisbane GMT+10
STATUS (Open/Closed)	Closed
INCIDENT START DATE	20210522
INCIDENT START TIME (HH:MM)	08:40
INCIDENT CAUSED BY A 3RD PARTY?	No
IF YES, NAME OF 3RD PARTY
OUTAGE DURATION	Approx: 50 minutes
ESTIMATED TIME TO RESOLUTION	N/A
XCRM TICKET NUMBER	Not Applicable
BRAND	XNET
PRIORITY	P1
CUSTOMERS AFFECTED	All XCLOUD, XVOICE, XRACK and XNET Customers
DESCRIPTION OF INCIDENT	Seems no traffic beyond our core router – could be a faulty piece of hardware like a switch
DESCRIPTION IMPACT	Users may have no access to services
Primary Effect	Unable to connect to services
Secondary Effect	None
Residual Effect	None expected once the problem is resolved
EVENT TIMELINE
	Saturday 08:48 22nd May 2021
08:42	Monitoring detected disruption of network
08:50	XSTRA technicians despatched to NEXTDC B1 and XSTRA Head Office Toowong
09:05	Technician arrived at NEXT DC B1
09:05	Technician arrived at XSTRA Head Office
09:32	All services restored
10:56	Closing SDN
RECOVERY & RESOLUTION	Restarted Core Router
ROOT CAUSE	At this stage we believe the router simply needed a restart after years of being on-line without issue. There could have been a DDOS attack in progress which overwhelmed the router, however that is just speculation.
CORRECTIVE & PREVENTATIVE MEASURES	XSTRA will investigate whether quarterly restarts of key infra might be a preventative measure.

Feedback