Network Status News & Information

REPORT:

This notice is in regards to the network outage that occurred on June 27th, 2020 between the hours of approximately 2PM CEST to 5PM CEST.  

Our core routing device experienced a non-recoverable hardware failure, and went offline causing a full BGP outage.  We were notified of the failure by our monitoring system and immediately begin to work to replace the device.  It was completely replaced with in 1 hour of arriving to the data center.

We suspect that this device was responsible for causing  the partial BGP outages in the previous days leading up to this outage.  

We replaced the device with a new model Juniper device, and  we do not expect any future outages or service issues moving forward.

No power was lost to any servers in the data center.

AFFECTED SERVICES: VPS and Dedicated servers at Switzerland.

Update June 24th, 2020 23:45 CEST

Due to public works outside our Switzerland data center, there has been a fiber cut out side our data center.  Our network continues to operate, but at a reduced capacity.  Your server(s) might be experiencing some increased latency due to this issue.

We are working with our fiber provider to restore full service, and they have given us an estimated time to complete the repairs with 3 - 6 hours.

AFFECTED SERVICES: All VPS and dedicated servers in Switzerland.

One of our cPanel Shared hosting servers in UK is having a hardware issue: cphost14. We are working on resolving it already.

Downtime: unknown, still working on it...

You can follow our official announcement for latest updates on this matter: cphost14 Announcement

The VPS node in Italy is down, we are checking it now.

Server experienced unexpected network problem and is due to be fixed ASAP, our technical teem is already on it. Thank you for your patience!

Server experienced unexpected network problem and is due to be fixed ASAP, our technical teem is already on it. Thank you for your patience!

Server experienced unexpected network problem and is due to be fixed ASAP, our technical teem is already on it. Thank you for your patience!

Server experienced unexpected network problem and is due to be fixed ASAP, our technical teem is already on it. Thank you for your patience!

Our OpenVZ service (VPS) in location Italy - is experiencing network issues, we are already working with the data center technicians to resolve the issue as soon as possible.

We are performing some maintenance operations on our Shared hosting cPanel servers. The maintenance should complete in less than 1 hr.

One of our xen VPS nodes in Germany is DOWN: xenhost78.

Reason for the outage is RAID drive(s) failure on the node. Our team is already working on this incident.

Expected downtime: Unknown.

UPDATE #1: A failed disk drive has been replaced on this node. Restoration of this node operational status is ongoing and shall be completed later today.

We are performing an urgent maintenance on cPanel shared hosting server cphost20.

The server shall be back in less than 1 hr.

Excuse us for any inconvenience, caused to you by this.

The node OVZHOST77 is down for 30 mins ~ 1 hr approximately, for an urgent maintenance.

Affected services: all VPS servers on this node.

Please, excuse us for any inconvenience caused to you by this.

EMERGENCY NETWORK MAINTENANCE - SERVICE IMPACTING

Location: Los Angeles (530 W. 6th ST)
Date: Friday, April 24th 2020
Start Time: 2:00AM PST
End Time: 4:00AM PST
Expected Outage: 0-20 Minutes

Risk: Network connectivity will be degraded while we perform a software upgrade on our edge router. BGP routes will shift during that time to a redundant Edge router.

Description:

We will be performing a network maintenance in order to upgrade our edge networking infrastructure. This will be a maintenance to perform a critical software upgrade to one of our edge MX routers.

This network maintenance will benefit customers by providing improved latency/connectivity and improved reliability.

We are experiencing network issues for all services, located at Switzerland.

We are working already with the network provider to resolve it as soon as possible.

Thank you for your patience.

A maintenance has been scheduled for OpenVZ VPS Node - ovzhost73.qhoster.net

The node will be down for about 10-30 mins on 3rd of April between 13 and 14 o'clock UTC time.

A network issue has been reported, that renders shared cPanel server cphost20.qhoster.net unreachable at the moment.

The network provider is already working to resolve the issue and we hope the connectivity to the server will be restored later today.

A scheduled maintenance is coming tonight on VPS node ovzhost44. We will be performing a hardware upgrade on the node.

Expected downtime: unknown

Maintenance will start at 5 PM EST (10 PM UTC) tonight, 6th of March, 2020.

Affected VPS node: xenhost44

Reason for maintenance: Due to system instabilities on this node, we will be replacing hardware.

Expected downtime: unknown

Dear Customers,
 
Please be advised of the upcoming infrastructure activity.
 
Event Details: 
Start Time: 2020-03-07 04:00 UTC
End Time: 2020-03-07 04:30 UTC
Expected Impact: Up to 5 minutes downtime, if any
 
Activity Details:
Voxility will perform an emergency maintenance. During the timeframe you may expect a short outage of up to 5 minutes if your are using the DDoS protection service.

Affected services: All VPS in Germany

The network provider in Germany is currently down.

Reason is still unclear.

Affected services: probably all VPSs in Germany.

UPDATE: The issue has been resolved. Services in Germany are restored.

We need to perform a server checkup of one of our VPS nodes - xenhost44.qhoster.net [Florida, US]

The maintenance will take place between Friday (14 Feb) night and Saturday (15 Feb) and will take approximately 4 hrs to complete.

We are sorry for the inconvenience.

We have a RAID failure on one of our VPS nodes - xenhost51 [Lithuania (LT)] - one disk failed and one disk degraded. We are investigating the issue and will fix it soon, we will have to replace one or more disk drives on the node.

It is advisable, that all clients on it take necessary backup of their important data just in case.

Downtime can be expected while the node is taken down for the disk replacements. Further information will be provided.

If  you have questions, open a support ticket.

AFFECTED CLIENTS: All VPS and dedicated servers in Switzerland

Dear Clients,

Please note that the Network Maintenance originally scheduled for today has been postponed and rescheduled for the following:

===================================
Beginning: February 07th, 2020:  2am CEST

Ending: February 07th, 2020: 6am CEST

Total Network Outage: No more than 15 – 30  minutes
====================================

During this time period you will notice 1 to 3 network outages lasting approximately 5 – 10 Minutes each.  Your server(s) will not lose power during this maintenance period. 

====================================
This outage is required to correct several security issues as well as add additional capacity and carriers to our network.  These changes will dramatically improve the security and performance of our network and your server(s).

If you have any questions regarding this maintenance, then please respond back to this email and we will address your concerns.

Thanks for your business!

Regarding the planned maintenance,

In order to improve the quality of our network we are planning an aggregation switch replacement including a capacity upgrade. During the maintenances we will migrate our rack uplinks to the new redundant aggregation switch environment. The maintenance will be spread over 3 maintenance windows, the timeframes for these are as follows:

- Wednesday February 5th between 08:00 CET and 10:00 CET.
- Wednesday February 12th between 08:00 CET and 10:00 CET.
- Wednesday February 19th between 08:00 CET and 10:00 CET.

During the 3 maintenance windows some or all VPS services with location the Netherlands may be affected.

Type: Maintenance
Severity: High

Affected Zone: LA

Time Frame
Start Time: Thursday January 24, 2020, 12:00PM- EST
End Time: Thursday January 24, 2020,- 1:00PM EST

Expected Downtime: 60 Minutes

During the 60 minute maintenance we will be doingNetwork maintenance and Data Center power tests. During the maintenance the system will not be accessible.
We anticipate a maximum downtime of 60 minutes for any given device within this
maintenance window. Please note the systems will be powered off and
rebooted.

We have unplanned maintenance on one of our shared cPanel hosting servers: cphost09.qhoster.net (UK)

Planned downtime: ~ less than 2 hrs.

We had a temporary problem with these servers and they are back online now. it was related to the backup battery of the raid.

Thank you for your patience

One of our KVM nodes - kvm10018.qhoster.net is DOWN due to RAID issue, which is most probably on hardware level.
The incident has been reported to the data center technicians and we are waiting for them to identify and fix the issue.

Expected downtime: Unknown at this point.

Server experienced unexpected network problem and is due to be fixed ASAP, our technical teem is already on it. Thank you for your patience!

Server experienced unexpected network problem and is due to be fixed ASAP, our technical teem is already on it. Thank you for your patience!

Server experienced unexpected network problem and is due to be fixed ASAP, our technical teem is already on it. Thank you for your patience!

We will be performing network maintenance in our Hong Kong datacenter.

Start of maintenance: February 4th, 2020, 17:00 UTC
End of maintenance: February 5th, 2020, 01:00 UTC

Reason of maintenance: We will be upgrading our network hardware to next-gen hardware and upgrading the firmware of our switches.

Impact: During the replacement of the core-routers the latency can go up a little bit during migration. The replacement of the core-switches will have impact when the uplinks of the racks are moved from current core-switches to the new next-gen core-switches. The upgrade of the switches requires a reboot resulting in a few minutes of additional downtime per rack.

One of our data centers in USA is performing an unplanned maintenance.

Affected servers/nodes: cphost20, kvm10020

Expected network outage: ~ 30 minutes

Since this night we are experiencing serious DDoS attacks against one of our cPanel servers: cphost17
Clients on the affected server may experience slow website loading and other operations or lack of connectivity at some times.

We are already working with the data center to resolve this situation quickly.

We have received an urgent and unplanned maintenance notification from the data center in Italy, informing us that services could be affected (VPS) - network outages or disruptions can be expected. We hope this will not last for long.

Unplanned maintenance is being conducted from one of our providers, as a result one of our OpenVZ nodes (Russia) is affected:

Node: ovzhost42

Expected outage: 30-60 min

One of our shared hosting servers - cphost15.qhoster.net is down due to technical issues. Our support team is already working on it.

One of our VPS nodes - xenhost26 is down due to network equipment failure.

 We have received this explanation from our network provider and we expect them to fix it soon:

We've been facing a network issue with the core switch and we have been working with our on-site engineers to resolve it as soon as possible.

It will be a temporary network issue and I am really sorry for the unexpected outage caused.

Meanwhile, I appreciate your patience at this time.

One of our main servers

for VPS in Switzerland: xenhost61 is currently down.

Reason for the outage: UNKNOWN at the moment.

Status: In progress - we are working with the data center to resume services as soon as possible.

Thank you.

Node xenhost54.qhoster.net is offline due to emergency maintenance. One of the disks failed and data center technicians are currently replacing the faulty drive.

We hope there will be no data loss.

Thank you for your patience.

Hi,

there is data center problem. We are waiting for the servers to come back online.

Thank you.

Our network provider in Germany is planning some maintenance on 6th of November 2019, between 01.00 and 02.00 CET - expected downtime 5-10 mins :

Event Details:
Startzeit: 2019-11-06 01:00 CET
Endzeit: 2019-11-06 02:00 CET
Erwartete Auswirkungen: Unerreichbarkeit Ihrer Dienste für ca. 5-10 Minuten

Some of our nodes (master servers for VPS) are down at the moment. We expect this situation to be resolved very soon. We have delayed a payment to one of our providers with just 1 day and they've stopped our servers.
Now we have paid all of our invoices, but the provider has rather slow support. We hope they will see and resolve the situation very soon, bringing the servers back online.

Thank you for your patience.

There is a DDoS attack against one of our US providers, affecting servers cphost20 (cPanel server, USA) and KVM node 10020. You may experience slow connection and packets loss. We are sorry for the inconvenience.

Update: seems like the attack is not taking place anymore, however we'll continue to monitor the situation in the coming days.

Shared cPanel hosting server cphost17 is DOWN. We expect it to come online later today.

Thank you for your patience.

Currently, one of our  servers (KVM node 10020) in USA is DOWN due to network outage. We expect this to be fixed very soon!

Thank you for your patience.

Currently, one of our shared hosting (cPanel) servers in USA is DOWN due to network outage. We expect this to be fixed very soon!

Thank you for your patience.

Network issue with KVM node in Switzerland kvm10018. We expect it to be fixed soon.

We are experiencing packets loss on node ovzhost52 due to network switch change. The issue should be fixed soon.

ovzhost02 is currently down. We are investigating the issue and working to fix it ASAP.

Update: We are not able to provide VPS at location Brazil for some time. We will have this location again soon. Until then, please choose one of our other locations or turn to our support team for more information.

The MySQL service of server cphost10.qhoster.net (UK) is DOWN. We are investigating the issue!

xenhost01 is currently down. We are investigating the issue and working to fix it ASAP.

The issue will be taken care of asap. sorry for the inconvenience.

The issue will be taken care of asap. sorry for the inconvenience.

Hello,

On the 19th of February we received a scheduled maintenance notification from our fibre carrier providing the link between our Bristol and London locations stating the following.

"Please be aware that our provider has scheduled a maintenance window on one of your services.

Service ID: [REDACTED]

Time Window: 06:00 - 23:59

Date: 11/03/18

Impact: Full loss of service"

The last mile for all circuits terminating into our Bristol location are provided via Openreach instead of Zayo. Because of this we had to ask if this was maintenance to Openreach's network or Zayo's. As we have full resiliency on the Zayo end having 2 national routes back to London we wouldn't be affected if it just maintenance on Zayo's network.

After over 2 weeks of waiting for a response, today we've finally had confirmation that it is in fact maintenance on Openreach's network, so we will be affected by this after all.

The work being carried out is extensive, and we've been informed that Openreach will be performing a full rebuild on a fibre joint close by to our Bristol facility. 

As it stands they've opened a window which is roughly ~18 hours in duration from 06:00 AM GMT on Sunday morning till 23:59 PM GMT on the same day (11/03/2018). 

Because this could potentially be 18 hours where services are unavailable we're offering all our customers in Bristol an additional couple of days on their billing cycles to help make up for this. Please contact our billing support via a support ticket to have this applied to your order.

Whilst we're fairly sure services won't be unavailable for this entire duration we're still making everyone aware that it is still a possibility. 

Hello,

On the 19th of February we received a scheduled maintenance notification from our fibre carrier providing the link between our Bristol and London locations stating the following.

"Please be aware that our provider has scheduled a maintenance window on one of your services.

Service ID: [REDACTED]

Time Window: 06:00 - 23:59

Date: 11/03/18

Impact: Full loss of service"

The last mile for all circuits terminating into our Bristol location are provided via Openreach instead of Zayo. Because of this we had to ask if this was maintenance to Openreach's network or Zayo's. As we have full resiliency on the Zayo end having 2 national routes back to London we wouldn't be affected if it just maintenance on Zayo's network.

After over 2 weeks of waiting for a response, today we've finally had confirmation that it is in fact maintenance on Openreach's network, so we will be affected by this after all.

The work being carried out is extensive, and we've been informed that Openreach will be performing a full rebuild on a fibre joint close by to our Bristol facility. 

As it stands they've opened a window which is roughly ~18 hours in duration from 06:00 AM GMT on Sunday morning till 23:59 PM GMT on the same day (11/03/2018). 

Because this could potentially be 18 hours where services are unavailable we're offering all our customers in Bristol an additional couple of days on their billing cycles to help make up for this. Please contact our billing support via a support ticket to have this applied to your order.

Whilst we're fairly sure services won't be unavailable for this entire duration we're still making everyone aware that it is still a possibility. 

Hello,

On the 19th of February we received a scheduled maintenance notification from our fibre carrier providing the link between our Bristol and London locations stating the following.

"Please be aware that our provider has scheduled a maintenance window on one of your services.

Service ID: [REDACTED]

Time Window: 06:00 - 23:59

Date: 11/03/18

Impact: Full loss of service"

The last mile for all circuits terminating into our Bristol location are provided via Openreach instead of Zayo. Because of this we had to ask if this was maintenance to Openreach's network or Zayo's. As we have full resiliency on the Zayo end having 2 national routes back to London we wouldn't be affected if it just maintenance on Zayo's network.

After over 2 weeks of waiting for a response, today we've finally had confirmation that it is in fact maintenance on Openreach's network, so we will be affected by this after all.

The work being carried out is extensive, and we've been informed that Openreach will be performing a full rebuild on a fibre joint close by to our Bristol facility. 

As it stands they've opened a window which is roughly ~18 hours in duration from 06:00 AM GMT on Sunday morning till 23:59 PM GMT on the same day (11/03/2018). 

Because this could potentially be 18 hours where services are unavailable we're offering all our customers in Bristol an additional couple of days on their billing cycles to help make up for this. Please contact our billing support via a support ticket to have this applied to your order.

Whilst we're fairly sure services won't be unavailable for this entire duration we're still making everyone aware that it is still a possibility.