20230602 – XVOICE – Inbound and Outbound calls reporting as disconnected - PUBLIC

NOTICE TIME ZONE	Brisbane UTC +10
STATUS (Open/Closed)	Closed
INCIDENT CAUSED BY A 3RD PARTY?	Yes
IF YES, NAME OF 3RD PARTY	AAPT
INCIDENT START DATE	2023-06-02
INCIDENT START TIME (HH:MM)	12:10
ESTIMATED TIME TO RESOLUTION	N/A
INCIDENT END DATE	2023-06-02
INCIDENT END TIME (HH:MM)	14:00
INCIDENT TOTAL DURATION	1hr 50min
XCRM TICKET NUMBER	Not Applicable
BRAND	XVOICE
PRIORITY	P1
CLIENTS AFFECTED	All Clients
DESCRIPTION OF INCIDENT	All calls via CTS not working
EVENT TIMELINE
	Friday 2nd June 2023
12:10	XSTRA Monitoring indicates communication voice switch to upstream carriers voice platform has no connectivity
12:15	XSTRA XVOICE lead confirmed all network connectivity is correct on infrastructure
12:20	XSTRA redirected al outbound calls out alternative CTS
12:21	XSTRA logged fault with upstream carrier to further investigate
13:00	Upstream responding to ticket and advised they can see calls coming in requested we further investigate our configuration
13:02	XSTRA contact account manager to provide further assistance and request diverting all calls to 2nd CTS
13:10	XSTRA attended NextDC B1 to powercycle upstream carrier hardware
13:45	Upstream completed diverting calls to 2nd CTS
14:00	XSTRA performed test calls and confirmed they are being routed to 2nd CTS
RECOVERY & RESOLUTION	All calls to use new CTS in B2
ROOT CAUSE	All calls not reaching CTS1
CORRECTIVE & PREVENTATIVE MEASURES	Carrier implemented automatic failover to CTS2

Feedback