Current Network Status - Planned Maintenance

Network Backbone - No Known Issues


CCS Datacentre DC14 - No Known Issues


CCS Datacentre DC1 - No Known Issues


Scheduled Maintenance - Planned Maintenance 1st and 2nd June
Please see: Planned Engineering Works for further details



Customer Networks - No Known Issues


Leased Lines - No Known Issues


VoIP - No Known Issues


xDSL Platform - No Known Issues


MLPPP Bonded xDSL - No Known Issues


Mail Services - No Known Issues


DNS Services - No Known Issues


Web Hosting - No Known Issues

RESOLVED - IP Transit Issues

At 16:30:07 we became aware of a drop in traffic on our core network.

The cause was quickly established to be because Lonap, a public peering house, has shut down our connection in Telehouse East due to a supposed port security violation.

We confirmed there was no violation and they resumed the port at 16:40:03 when traffic returned to normal.

We have since raised the issue with Lonap (again, sadly this is not the first time this has happened) and they have yet to provide a reason for the port shutdown.

While awaiting a coherent answer for the issue, we have taken mitigating steps to ensure in the event of a port trip, that it cannot be reset causing the routing changes experienced today, and merely re-routing traffic over other carriers until the issue is addressed.

For information, LONAP is a neutral not-for-profit Internet Exchange Point over which ISPs (UK primarily) exchange traffic away from public routing providers, and as such much traffic to and from ourselves to other UK providers and ISPs would have been affected. Little or no effect would have been noted to more distant locations

I apologise for the inconvenience this would have caused and can ensure appropriate steps will be taken to prevent such a reoccurrence.

http://www.ccsleeds.co.uk/

Add to Technorati Favorites

RESOLVED - LLU ADSL Issues

Good morning.

At 02:43 we noted a drop in connected ADSL customers.

Upon further investigation, we have established that this is specifically Telefonica based LLU customers.

We have communicated with TF on a number of occasions about this and they have accepted an issue in the Birmingham area, which is central to their network, affected all wholesale customers.

Their troubled customer count is in the millions, so as I am sure you can appreciate, this is top priority for them.

*** Update 05:00 ****


The troubles are still being worked upon, but we expect an imminent update and I am advised return to service.


*** Update 06:34 ***

Approximately 40% of our customers are back in service, and TF have advised they are continuing to restore the core network.


*** RESOLVED 0800 ****

TF have advised the core network config has been totally restored and we are seeing normal connection counts. If you are still having troubles, please re-boot your router before contacting support if you would be so kind.

Apologies for the inconvenience this is causing.


http://www.ccsleeds.co.uk

RESOLVED - Virgin Provided Leased Line Issues.

At 17:23 we noted a loss of service to around 90% of our VirginMedia provided leased line services affecting many of our customers.

Service returned at 17:37 to almost all customers and then promptly failed at 17:57 once again.

Most customers remained without connectivity until 19:20 when we saw a 100% return to service.

During this interval Virgin have confirmed a national network outage and have noted that senior engineering staff and manufacturer representatives are both on hand to address the issue.

Sadly while typing this update, I note a loss of service to a small handful of customers once again.

We can only apologise for the inconvenience and please rest assured we are keeping a close eye on the situation.

Further updates will be posted as provided by Virgin.

*** RESOLVED 19:20 ***

I am pleased to advise that all services have returned and traffic is flowing as expected. Virgin have noted that a full incident report will be posted tomorrow and this will of course be dispatched to all affected customers.

.http://www.ccsleeds.co.uk/

Leeds DC7 - Planned Maintenance

Planned Maintenance Window Announcement

Window Start: 10th Jan 2012 - 22:00
Window End: 11th Jan 2012 - 06:00

In the above window, we will be performing a range of upgrade tasks on the London network, bringing on line further capacity and patching a vendor security bug at the same time.

We expect traffic to re-route on more than one occasion to Leeds DC7 and no other locations or services. There will also be two brief outages while cables are re-patched into new line cards and a router re-booted.

We would anticipate only a few very short minutes of outage in total, however the window of maintenance has been listed as 8 hours due to any unforeseen circumstances.

We apologise for the inconvenience in advance.

http://www.ccsleeds.co.uk/

RESOLVED - Leeds DC7 - Connectivity Issues

We have recently become aware of connectivity issues in Leeds DC7.

At present we are investigating the cause.

Apologies for the inconvenience.

*** RESOLVED 16:42 ***

The issue has now been resolved and a full incident report will be promptly dispatched to those affected. Once again, apologies for the inconvenience this afternoon.

.http://www.ccsleeds.co.uk/

RESOLVED - VoIP - Intermittent Call Loss

Good afternoon.

We have become aware of a loss of calls of approximately one in three on our Voip service at present.

This is being investigated as a Major Incident with the relevant provider and updates will follow as soon as possible.


*** RESOLVED 16:03 ***

We have been advised our PSTN providers' had an issue with some call routers in the London area where we break from VoIP to PSTN and that the issue has now been resolved.

Apologies for the inconvenience all.

.http://www.ccsleeds.co.uk/

Add to Technorati Favorites