2025/11/27 20:05 GMT: Problem solved
Hydro66 datacenter team found out that our main uplink link was renegociated to 100Mbps speed instead of the expected 1Gbps. Thus the outgoing traffic was saturating the link. They switched the priority of our VRPP to use the other link (at the correct 1Gbps speed) and the situation is back to normal.
We will do a short maintenance window next Monday to fix the broken link. See rapidspace-Document.Operation.Maintenance.Hydro66.20251201.
2025/11/26 ~17:00 GMT: Problem identified and ticket opened
We identified that the outgoing traffic from our Hydro66 rack was very limited (arounf 2MB/s for the whole rack) which is rendering all services hosted in this rack to feel slow.
We opened an urgent ticket to our rack provider.
2025/11/26 ~09:00 GMT: Investigation Started
Our network team started to look at the issue.
2025/11/26 07:50 GMT: Initial Detection
The users started reporting abnormally slow websites served from our CDN. Our CDN team identified a problem with the underlying network.