Start Date: 09/21/2022 11:00 AM (EST) / 21 September 2022 15:00 (UTC)
Finish Date: 09/21/2022 11:40 AM (EST) / 21 September 2022 15:40 (UTC)
Description:
Delays in processing mobile terminating messages on shared short code 62346
Impacted Services:
Impacted Customers:
Cause:
On September 22nd, 2022, there were delays of processing between 2SMS and its upstream provider over the shared short code 62346. The issue was caused by a large campaign from a customer using 62346 that was requested to be sent out immediately.
As traffic on shared short codes is processed on a first in first out bases this delayed the processing of all messages over 62346 and therefore queued message transmission.
Detection:
On the 22nd September staff members were alerted to the delay in traffic processing and message requests not reaching their intended destinations in a timely fashion. Initial responses actions were taken to resolve the issue. 2SMS staff manually delayed the large non-urgent campaign messages to allow for processing of the backlog for all customers.
Corrective Actions:
2sms staff manually delayed the campaign messages to allow for the backlog of other customers traffic to be processed. Then when the backlog had cleared, the campaign messages were release in a gradual sequence to prevent further disruption.
Preventative actions:
We will be putting in automated measures to handle when a large campaign has been sent that will manage the resequencing automatically to minimize the impact to other traffic on shared short codes.