HostVenom & WinterNode - Chicago Data Center Network Blips – Incident details

No further issues has been reported. Marking as resolved.

We will continue to monitor network's performance.

We're re-opening the issue based on an alert we're investigating.

We will continue to monitor network performance following recent changes and engage with our providers.

Should we observe any new network events, we'll open an event and continue to track this publicly.

We are continuing to monitor network performance following the recent mitigation changes implemented by our data center. Conversations remain ongoing with the datacenter and downstream service providers as they fine-tune routing and stability measures.

Fortunately, we have not observed any significant or widespread drops since the changes were applied. We will continue to provide updates as more information becomes available.

An automated alert has been triggered by our monitoring system regrading the temporarily connectivity loss. From our alerts, it lasts ~1 minute during each network blip.

The data center engineering team is actively working to mitigate the issue and we will continue to monitor and communicate with them to resolve the issue.

We completely understand how frustrating this can be and appreciate everyone's patience while the data center continues to work toward full resolution. We will post another update as soon as more information becomes available.

As of 4 PM Central Time yesterday, no additional automated alerts have been triggered. We will continue to monitor the fix.

Our data center has identified most of the target prefixes related to the ongoing DDoS event and is shifting network announcements away from primary carriers to help reduce impact.

While this work is still in progress, these mitigation efforts are expected to improve network stability as adjustments take effect.

To reduce notification spam, we've consolidated all WinterNode Server notices under a new component titled Chicago Data Center for the duration of this incident. It is recommended you subscribe to this incident directly to receive the latest updates regardless of component selected.

Thank you for your continued patience and understanding while this work is in progress.

We have continued to observe intermittent periods of packet loss and short network interruptions affecting our Chicago Data Center. The data center engineering team has confirmed that these disruptions are related to an ongoing large-scale DDoS event. While the attacks are not targeting HostVenom directly, they are impacting connectivity across several upstream carriers and other providers in the facility.

The data center engineering team is actively working to mitigate the issue and has implemented multiple measures to isolate the affected network paths. The scale and distributed nature of the attacks have made mitigation challenging, but we are seeing some periods of stability between brief interruptions.

We completely understand how frustrating this can be and appreciate everyone's patience while the data center continues to work toward full resolution. We will post another update as soon as more information becomes available.

We have continued to observe intermittent network blips at our Chicago Data Center. Multiple automated alerts have been triggered throughout the day, each indicating brief connectivity losses lasting approximately one minute.

Our data center has confirmed that they have observed additional short inbound traffic spikes affecting specific upstream carriers (GTT, Lumen, and Comcast). They are continuing to investigate and have made routing adjustments to help isolate the source of the issue.

At this time, we are continuing to monitor uptime closely and remain in contact with the data center team for further updates. We will post the next update once more information becomes available or if the situation changes.

The data center has confirmed they are see some sporadic issues as well on the network and are currently investigating this matter. Some of our team members are also noticing periods of slowness accessing our Client/Billing Areas and connections between our VPS/Dedicated Control Panel and our Client/Billing Area systems.

No further alerts for loss of connectivity have been triggered since our last update.

We will continue to update this incident as we receive more information. The next expected update likely between the morning and afternoon.