Past Incidents

10/15/2024 08:00 am - 11:59 pm EDT: QTS Critical Operations Technicians (COTs) and third-party contractors at the Suwanee, GA data center will perform important power switchovers to connect backup power to PDU Substation 1A. This will maintain N+1 redundancy while they work on updating the UPS System. During this process, there will be brief interruptions to power from the affected substation, which will temporarily put customer loads on a single power source (no backup). There is a chance for High impact to GMS equipment during this change at the Datacenter.

10/11/2024 8:00AM EST - GTS Poulsbo phones and network are down. ISP Lumen has a fiber outage in the area being worked on, no eta. Lumen Ticket #: 30382365

10/9/24 22:10 CDT - Network Outage - Cherokee Jackson, power verified, local contact is Mark Zipp 731-499-3575 - Spectrum issue which they had a tech fix. Back online 10/10/24 6:25 CDT. Total outage time 8 hours and 15 minutes.

10/8/2024 13:37 EDT - Evalon/Converge Credit Card Machine service outage.

09/04/2024 9:25 EDT: We are currently experiencing an outage with Smartsheets, affecting the ability to access Smartsheet's company wide. GMS IT is aware of the issue and is working to resolve the problem as quickly as possible.

We have been made aware that Microsoft is already processing email through IPv6 which for the security side is handled slightly different than IPv4 (current standard). As a result, remote email systems can't properly process emails and end up stuck in their security stack/tools. We are working with our vendor, Cisco, to come up to par and align/match what Microsoft is doing.

We are aware that some users have experienced disruptions with access to Syncplicity. This appears to have been caused by an automated process that unexpectedly adjusted user access.

Please rest assured that we are actively investigating the issue alongside our third-party vendors to understand the cause and resolve it as quickly as possible. Our goal is to restore full access to everyone impacted, and we'll provide updates as soon as we have more information.

9/27/24 4:11 EDT - Network Outage - Power verified on Monday 9/30/24 morning - Equipment restarted, local contact is Roger Wentz 912-486-4397 - Damage visible outside the building to the fiber coming in. AT&T Ticket number CS1042334 From AT&T. AN ACCESS SUPPLIER PROBLEM HAS BEEN DETECTED. THE TICKET HAS BEEN REFERRED TO THE ACCESS SUPPLIER FOR FURTHER ANALYSIS. AT&T DIAGNOSE HAS DETECTED THAT THE PINGS BETWEEN THE PROVIDER EDGE AND CUSTOMER EDGE ROUTERS ARE UNSUCCESSFUL. FURTHER TROUBLESHOOTING WILL BE PERFORMED BY AT&T. PLEASE CHECK TICKET FOR ADDITIONAL STATUS. Cradlepoint is being shipped to the site as a temporary solution. CP#12 - Device IMEI 350884581087817

We are aware of an issue where emails sent to USG.com are bouncing back. After investigating, we’ve found that USG’s email filtering systems are modifying the emails during their security scans, which changes the email’s source address to their system’s IP (148.163.140.163). This is causing the emails to be blocked.

We have reached out to USG and offered to work together on resolving the issue. Since we don’t manage their security systems or have access to their environment, we’re dependent on their team to address this on their end.

We will keep you informed as soon as we have updates. Thank you for your understanding and patience.

9/19/2024 12:29 EDT - WBM Oakland phones working one direction. Telware ticket: 7698

09/18/2024 10:00AM EST - Cherokee Jonesboro lost network and phones, issue caused by ISP Optimum circuit outage in the area. No ETR.

9/17/2024 12:25 EDT - TAM Bloomington: Network offline due to area power outage.

9/16/24 00:59 - Cowtown Dallas: Network outage. Due to downed fiber line.

9/12/24 22:21 EDT - Network Outage - Came back up at 9/13/24 5:56 EDT Issue was a surge protector that they are replacing. Ordered an APC UPS for them. Total outage time - 7 hours and 35 minutes

GTS Puyallup 9/12/2024 1:00PM EST - Network and phones down. Comcast is experiencing an unexpcted outage in the Puyallup, WA area due to a power outage. No etr currently.

9/11/24 3:37 EDT - Network Outage - Sam Kelley is the local contact - 256-990-9279, Networking team has reached out to AT&T

9/8/2024 23:38 - CHE Memphis: Network outage.

9/5/24 4:14 EDT - Network outage - Checked the YIAB and power was out to the switch. Had him restart the UPS. Local contact Cesil Saldivar - (443) 797-4194 Restarted the Comcast equipment. Restarted the Cienna as well. Damon needed to reload the config on the router. reference number for comcast coax cr159956177 Ticket CR159957679 Comcast sent a tech out and switched out the modem. Back online at 9/5/24 10:17 EDT for a total outage time of 6 hours and 3 minutes

9/4/2024 12:00 NEG Milford: Network Outage.

09/03/24 14:41 EDT WBM Santa Maria - Power outage / PG&E estimates power will be restored around 17:30 EDT

8/30/2024 13:35 - Kamco Manhattan: Network Outage. Spectrum maintenance.

8/24/24 12:30 EDT Outage - Restarted networking equipment, ours & ISP's. Appears to be ISP related (Metronet with a Spectrum backup). Local contact is Andy Hinman - 910-929-8132 dispatch scheduled for Fayetteville. currently 12 to 3 ticket number ett860760

Site back online 8/26/24 13:36 EDT - ISP repaired the line. Total outage time

8/28/24 7:17 EDT Network Outage, verified power and restarted our equipment and the ISP's, contact number for Jason is 8646081252. AT&T has been dispatched

8/27/24 9:01 EDT Power Outage took the network down. Power restored at 10:06 EDT for a total outage time of one hour and 5 minutes.