All Systems Operational
Uptime over the past 90 days. View historical uptime.
API Operational
90 days ago
100.0 % uptime
Today
Platform Operational
90 days ago
100.0 % uptime
Today
Cloudflare CDN/Cache ? Operational
90 days ago
100.0 % uptime
Today
Mux Streaming Service ? Operational
90 days ago
100.0 % uptime
Today
Postmark Email Service Operational
90 days ago
100.0 % uptime
Today
Firebase Web Sockets Operational
90 days ago
100.0 % uptime
Today
UK Server Region Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.

No incidents or maintenance related to this downtime.

System Metrics Month Week Day
API Response Time
Fetching
Platform Response Time
Fetching
Past Incidents
Sep 17, 2021

No incidents reported today.

Sep 16, 2021

No incidents reported.

Sep 15, 2021

No incidents reported.

Sep 14, 2021

No incidents reported.

Sep 13, 2021
Slow web socket connection
Postmortem - Read details
Sep 13, 17:33 EDT
Resolved - This incident has been resolved.
Sep 13, 17:15 EDT
Monitoring - The team has concluded the implementation of all temporary solutions on the platform.
This includes creating a timeout option on slow connecting sockets and also disabled Redis as a single connection.
The web socket chat will remain with limited chat support on the firebase instance until we add support for per instance local connection, which should happen next week.
The Redis cache team will be implementing a new permanent solution end of this week, Friday at the latest.
Sep 13, 13:07 EDT
Update - We currently fixed an autoscaling Redis instance that was not able to redirect the load to a separate system. Redis is used by InEvent to quickly balance its write operations instead of relying only on a traditional SQL database.

The following components are still affected:
Firebase Web Sockets, which covers the live chat on the Virtual Lobby.
Sep 13, 11:45 EDT
Identified - We have identified the slowness from the Firebase product.
We are deploying an alternative with the Native websockets, under Company > Tools.
We are currently deploying a fix for UI improvements on the Firebase UI console while Firebase is not responsive.
Chat may be offline while the fix is being applied, but video and streaming should work normally.
Sep 13, 10:55 EDT
Investigating - We are currently seeing a slow scalability response from the InEvent Firebase socket group. We are currently investigating the issue with Google Firebase team.
Sep 13, 09:45 EDT
Sep 12, 2021

No incidents reported.

Sep 11, 2021

No incidents reported.

Sep 10, 2021

No incidents reported.

Sep 9, 2021

No incidents reported.

Sep 8, 2021

No incidents reported.

Sep 7, 2021

No incidents reported.

Sep 6, 2021

No incidents reported.

Sep 5, 2021

No incidents reported.

Sep 4, 2021

No incidents reported.

Sep 3, 2021

No incidents reported.

← Incident History Powered by Atlassian Statuspage