We're investigating elevated errors in Columbus (CMH).

All other locations and services are unaffected

Fastly Engineering has identified a third party service provider issue that caused elevated latency and increased errors for customers within the Columbus (CMH) Point of Presence (POP). Engineering has rerouted customer traffic around the impacted region and services be gradually restoring across our network. 

We will continue to monitor the actions of the third party service provider and will resolve our status post once we've seen full recovery from the provider.

Update: An earlier status post: https://www.fastlystatus.com/incident/376077 has been closed. As the elevated errors observed when access our Control Plane (manage.fastly.com) and API services has been correlated to the third party service provider issue identified within this status post.

This incident has resolved.

During this incident, customers would have experienced increased latency and elevated 5xx errors in our Columbus (CMH) POP, as well as, elevated errors when attempting to access configuration management and API services through our Control Plane from 18:40 to 21:30 UTC.

Affected customers may have experienced impact to varying degrees and to a shorter duration than as set forth above.

Fastly Engineering has confirmed that our network availability and Control Plane services have been fully restored.

Fastly will be adding capacity at our Columbus (CMH) Point of Presence (POP). End-users may observe connection resets as traffic is migrated onto new hardware, starting on 25 August 2023 at 12:00 UTC. 

Our estimated duration is 4h.

When this change is applied, customers may observe additional origin traffic as new cache nodes retrieve content from origin. Please verify that your origin access lists allow the full range of Fastly IP addresses (https://docs.fastly.com/en/guides/accessing-fastlys-ip-ranges).

Customers with any questions or concerns should contact Fastly’s Support team at https://support.fastly.com. 

In order for customers who use this POP as an Origin Shield to leverage new cache nodes for an expanded cache footprint, customers can either apply a new config version at a time of their choosing, or their shielding configuration will be updated on their behalf 7 days after this maintenance window closes.

The scheduled maintenance has been completed. 

We're currently investigating performance impacts in Europe, North America.

All other locations and services are unaffected.

This issue has been identified and a fix is being implemented. 

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We're currently investigating potential impact to performance with our CDN services.

A fix has been implemented and we are monitoring the results.

Fastly will be adding capacity at our Columbus (CMH) data center. End-users may observe connection resets as traffic is migrated onto new hardware.

In order for customers who use this POP as an Origin Shield to leverage new cache nodes for an expanded cache footprint, customers can either apply a new config version at a time of their choosing, or their shielding configuration will be updated on their behalf 7 days after this maintenance window closes.

When this change is applied, customers may observe additional origin traffic as new cache nodes retrieve content from origin. Please be sure to check that your origin access lists allow the full range of Fastly IP addresses (https://docs.fastly.com/en/guides/accessing-fastlys-ip-ranges). Customers with any questions or concerns should contact Fastly’s Support team (support@fastly.com). 

Scheduled maintenance is currently in progress. We will provide updates as necessary.

Fastly is aware of an expected critical vulnerability the OpenSSL project is expected to disclose in the near future. We are studying the currently available information surrounding this vulnerability and do not currently believe that Fastly is vulnerable. We will continue to monitor as additional information is released and will provide our customers with more information as available.

Fastly has reviewed the initial notification from OpenSSL regarding CVE-2022-3786 and CVE-2022-3602. We have analyzed the versions of OpenSSL in use at Fastly, and verified that we do not use OpenSSL 3.x. Fastly and customer usage of Fastly services are not vulnerable to CVE-2022-3786 or CVE-2022-3602.

We're currently investigating potential impact to performance with our CDN services.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We're investigating elevated errors in Columbus (CMH). All other locations and services are unaffected

A fix has been implemented and we are monitoring the results.

This incident has been resolved.