All systems operational

Asia-Pacific Regional Outage

Resolved
Major outage
Started 21 days ago Lasted about 4 hours

Affected

Maikiwi (Services)
Aria
Route Propagation
Caesar Monitoring
Event Digest (Ingress)
Event Aggregator (Ingress)
Updates
  • Resolved
    Resolved

    All outages have been restored. Please contact us as soon as possible if you deployed any custom workload with us in Osaka, otherwise, all managed services in Osaka will be seamlessly redirected to Tokyo.

  • Identified
    Identified

    We identified a repeatedly tripped internal failsafe that has been adding bogus routes to Aria's global routing table. Expect to receive inconsistent routes in the meantime as we patch Aria on-the-fly.

  • Monitoring
    Update

    We have brought all locations online except for Osaka which will be subjected to an extended maintenance period. We are also enabling a small portion of customers to interface with Caesar again to contain potential cascading failures.

  • Monitoring
    Update

    We rolled back a recent update to internal critical infrastructure in Hong Kong and Singapore and are proactively monitoring the result. Caesar Monitoring will be disabled globally in this moment and time.

  • Monitoring
    Monitoring

    We implemented a fix for Tokyo and are incrementally routing in traffic again.

  • Identified
    Update

    We have identified an additional outage in Singapore.

  • Identified
    Identified

    We are commencing a cold restart of all services in Japan and removing routes destined for Tokyo in Hong Kong.

  • Investigating
    Update

    We are currently investigating a simultaneous regional outage.

  • Investigating
    Investigating
    We are currently investigating this incident.