Network Issues - Resolved

Dear All,

All networking issues have now been resolved.

A supervisor control card failed in one of our two border routers at our Manchester data centre. Although these routers run independently, So Internet traffic requires both routers to be run for its traffic to route correctly.

Initial indications pointed to the fact that one of the line cards (the circuit boards that contain the actual network switch ports) had failed. The 10gb/s line card was replaced first, but the router would not remain stable. After completely stripping the chassis, our last option was the supervisor (the actual control for the router)

Upon replacing this, the router would boot and remain stable. The M247 team then replaced the other line cards and restored the configuration line by line.

Full service was restored around 14.10 today.

We will be further reviewing this outage and making adjustments to protect against re-occurance.

If you are still experiencing issues, please do contact our support team - 0161 615 1275

Routing issues - update 3

We have now been able to locate the faulty line card and as a result replaced the supervisor.

The M247 team are now restoring the config from backups and we should be running again within the next 10 - 15 minutes.

Routing issues - update 2

One of our main data centre routers is refusing to boot correctly. Although a large percentage of the data centre is running, the So Internet connectivity requires both routers to be running for it to remain stable.

We are currently working through all the different options to try and get this router to restart correctly and remain stable.

We will be conducting a full postmortem after the event to find other ways of avoiding this happening in further.

In the meantime, thank you for your patience.

Routing issues - update

We have located the faulty piece of equipment and engineers are currently replacing it.

We are aware that this may only affect a small number of customers at the moment and will update again shortly.

Networking routing issues - 12.30 - 12-11-10

Hi All,

We are currently seeing a few network routing issues with traffic getting to our Manchester network. Engineers are currently working on this issue and service should be restored shortly.

Email issues fixed

Dear all,

By around 2.30pm today we were able to restore full service after replacing a failed fan tray in one of our NAS servers.

Thank you for your patience while we corrected this issue.

Issues affecting MS SQL & Email

Hi,

Around 20 minutes ago, we seemed to have experienced a power surge to one of our racks.

These caused a number of our servers that handle email and MS SQL (basically those connected to our central storage devices) to reboot, or require reboots.

We are currently bring all services back online.

Email is currently offline and we are just correcting an issue with the gateway server caused by this reboot. We expect this work to be completed within the next 30 mins

I will update the ticket shortly once the issue is corrected.

Once again, thank you for your patience.