FASHN - [App & API] Service degradation: Increased load may result in delayed generations or occasional timeout errors. – Incident details

All systems operational

[App & API] Service degradation: Increased load may result in delayed generations or occasional timeout errors.

Resolved
Major outage
Started 3 months agoLasted about 2 hours

Affected

API

Partial outage from 8:58 AM to 11:08 AM, Degraded performance from 11:08 AM to 11:26 AM

Apps

Partial outage from 8:58 AM to 11:08 AM, Degraded performance from 11:08 AM to 11:26 AM

Web app

Partial outage from 8:58 AM to 11:08 AM, Degraded performance from 11:08 AM to 11:26 AM

Updates
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Monitoring
    UTC
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Identified
    UTC
    Identified

    We have identified delayed response times from a downstream service as the cause of elevated generation latency and intermittent timeout errors. We are working to increase capacity and return the service to normal operation.

    It is recommend to temporarily use "Fast" generation mode where possible for a significantly reduced error rate.

    This issue doesn't affect tryon-v1.5 and tryon-v1.6 API endpoints.