Active Incident

Updated a few seconds ago

Incident Status

Operational

Components

CoreWeave Services

Locations

US-ORD1, US-LAS1, US-LGA1, US-RNO2



June 15, 2024 6:07AM UTC
[Investigating] We are actively investigating a metrics issue that is impacting the availability of our metrics. As the situation evolves, we will continue to provide our customers with updates on the matter. If you have any questions, we encourage you to reach out to support by email: [email protected]. We apologize for any inconvenience this may cause and thank you for your patience while we work to resolve the matter in a timely manner.

June 15, 2024 3:45PM UTC
[Monitoring] We believe to have identified the issue. We will begin actively monitoring the matter at this time. CoreWeave apologizes for any inconvenience this may have caused and we thank you for your patience on the matter. Thank you.

June 15, 2024 9:44PM UTC
[Monitoring] We're still actively monitoring this issue at this time. Thank you for your continued patience.

June 16, 2024 1:09AM UTC
[Monitoring] We are monitoring this issue; however, there are still intermittent periods of impact to the availability of our metrics. We will provide updates as soon as possible.

June 17, 2024 7:27PM UTC
[Monitoring] We have been monitoring our services for the last several hours, and are seeing improvements. We will continue to monitor for any further degradation, but we believe this to be in a stable state. If you are still experiencing issues, please reach out to [email protected].

CoreWeave Cloud




Operational

CoreWeave Inference




Operational

CoreWeave Shared Filesystem Volumes




Operational

CoreWeave Object Storage




Operational

CoreWeave Block Storage Volumes




Operational

CoreWeave Services




Operational

CoreWeave Premium Shared Filesystem Volumes




Operational

0

Upcoming Maintenances

7

Incidents Last 30 Days

4

Maintenances Last 30 Days

External Services

History (Last 7 days)

Description

A route-policy change will be applied to network edge in the RNO2 region. The change is necessary to ensure continued stability of network connectivity in the site. No impact is expected


Components

CoreWeave Cloud, CoreWeave Services


Locations

US-RNO2


Schedule

June 14, 2024 11:00PM - June 14, 2024 11:30PM UTC



June 14, 2024 11:00PM UTC
[Update] Emergency Network maintenance is starting.

June 14, 2024 11:30PM UTC
[Update] Emergency Network maintenance is complete.
Planned Network MaintenancePlanned Maintenance

Description

CoreWeave will be performing routine network maintenance on Friday, June 14th from 00:30 UTC 06/14 to 04.30 UTC. Scope: We will be replacing switches during this window. Impact: We do not anticipate connectivity issues, but switching capacity in and out of this environment will be reduced by 50% during this maintenance window. We will be actively monitoring the situation during this time. This activity is deemed Planned Maintenance under the CoreWeave Terms of Service Maintenance Policy (docs.coreweave.com/resources/terms-of-service/maintenance-policy


Components

CoreWeave Cloud, CoreWeave Inference, CoreWeave Shared Filesystem Volumes, CoreWeave Object Storage, CoreWeave Block Storage Volumes, CoreWeave Services, CoreWeave Premium Shared Filesystem Volumes


Locations

US-ORD1, US-LAS1, US-LGA1


Schedule

June 14, 2024 12:30AM - June 14, 2024 4:30AM UTC



June 14, 2024 12:46AM UTC
[Update] Maintenance is starting.

June 14, 2024 4:55AM UTC
[Update] Maintenance has concluded for the evening. There is remaining work that will be scheduled in a later maintenance window.
Loki MaintenancePlanned Maintenance

Description

We are performing emergency Loki Maintenance to bump Loki to 3.0 max. There will be a brief gap in logs. Reason : Loki querying performance is consistently degraded over the last week.


Components

CoreWeave Cloud, CoreWeave Inference, CoreWeave Shared Filesystem Volumes, CoreWeave Object Storage, CoreWeave Block Storage Volumes, CoreWeave Services, CoreWeave Premium Shared Filesystem Volumes


Locations

US-ORD1, US-LAS1, US-LGA1, US-RNO2


Schedule

June 12, 2024 10:00PM - June 13, 2024 2:00PM UTC



June 12, 2024 10:03PM UTC
[Update] Maintenance Started

June 13, 2024 2:41PM UTC
[Update] Maintenance is complete
Degraded Metrics PeformanceDegraded Performance

Incident Status

Degraded Performance


Components

CoreWeave Services


Locations

US-ORD1, US-LAS1, US-LGA1, US-RNO2




June 7, 2024 4:13PM UTC
[Investigating] We are actively investigating a metrics issue that is impacting the availability of our metrics. As the situation evolves, we will continue to provide our customers with updates on the matter. If you have any questions, we encourage you to reach out to support by email: [email protected]. We apologize for any inconvenience this may cause and thank you for your patience while we work to resolve the matter in a timely manner.

June 8, 2024 12:47PM UTC
[Monitoring] We believe the metrics issue has been mitigated but we are continuing to actively monitor the affected services.

June 9, 2024 3:47AM UTC
[Investigating] During monitoring of the metrics issue, we have become aware of it happening again. It is now impacting the availability of our metrics. As the situation evolves, we will continue to provide our customers with updates on the matter. If you have any questions, we encourage you to reach out to support by email: [email protected]. We apologize for any inconvenience this may cause and thank you for your patience while we work to resolve the matter in a timely manner.

June 9, 2024 8:40AM UTC
[Monitoring] We believe the metrics issue has been mitigated but we are continuing to actively monitor the affected services.

June 9, 2024 11:16PM UTC
[Investigating] We are aware of a re-occurrence of issues impacting the availability of our metrics. Our team is actively investigating and working to mitigate the issue. As the situation evolves, we will continue to provide our customers with updates on the matter. If you have any questions, we encourage you to reach out to support by email: [email protected]. We apologize for any inconvenience this may cause and thank you for your patience while we work to resolve the matter in a timely manner.

June 10, 2024 2:22PM UTC
[Monitoring] We believe the metrics issue has been mitigated but we are continuing to actively monitor the affected services.

June 11, 2024 9:02PM UTC
[Resolved] We believe this issue to be resolved and in a stable state.