20230522 – XNET – NextDC Inter-Network is Down Between B1 & B2 - PUBLIC

NOTICE TIME ZONE	Brisbane UTC +10
STATUS (Open/Closed)	Closed
INCIDENT CAUSED BY A 3RD PARTY?	Yes
IF YES, NAME OF 3RD PARTY	NEXTDC – https://nextdc.com
INCIDENT START DATE	2023-05-22
INCIDENT START TIME (HH:MM)	03:45
ESTIMATED TIME TO RESOLUTION	N/A
INCIDENT END DATE	2023-05-22
INCIDENT END TIME (HH:MM)	10:15 am
INCIDENT TOTAL DURATION	6.5 hours –
XCRM TICKET NUMBER	Not Applicable
BRAND	XNET
PRIORITY	P1
CLIENTS AFFECTED	All Clients
DESCRIPTION OF INCIDENT	Core Hardware incident in XSTRA’s main Data Center
EVENT TIMELINE
	Monday 22nd May 2023
5:15	XSTRA XNET lead was inbound to NEXTDC to investigate a major outage as detected by XSTRA’s PRTG monitoring system. Spare equipment was collected en-route.
6:00	XSTRA day shift technicians confirmed this issues again from XSTRA’s PRTG that some hosts were down.
9:15	NextDC have identified the issue and are working on resolving it. XSTRA technicians have been able to work around the issue.
9:30	75% of the infrastructure is online with the remaining to be up within the next hour or so.
10:14	NextDC have resolved their internal switching and infrastructure issues. All services are back to normal. Closing SDN now.
RECOVERY & RESOLUTION	TBA – waiting for response from NextDC
ROOT CAUSE	NEXTDC Switch or redundant links between their Brisbane B1 & B2 data centers failed
CORRECTIVE & PREVENTATIVE MEASURES	TBA – waiting for response from NextDC

Feedback