7:25pm Back to normal
6:20pm Downtown Manhattan is experiencing outages at the major hubs. We are looking into this at the moment
11:50pm We’ve just fixed a problem at Soft Surplus that was causing slow speeds in Brooklyn since this afternoon.
2:45pm We are replacing a cable at Henry hub. This means there will be 3 minutes downtime while the router reboots. After the cable is replaced speeds from downtown hubs should increase significantly.
3:45pm: Fixed the problem, just a reboot needed. Back to normal now.
2:10pm: We have a problem at Henry hub affecting Rivington, Ave A, 7th St and Ave D hubs downtown Manhattan. Looking into this now. We should have a fix by 4:30pm.
12:10am: There was an 8 minute downtime due to an internet exchange problem. Back to normal now.
8:00pm: Things should be connected again and bandwidth should improve over the next few days.
12:00pm: We are having problems with our Soft Surplus hub node (1417) in Brooklyn since midnight. We are looking into this now. This is affecting the 9 connected nodes.
4:30pm: Normal service resumed downtown. This was a problem with integrating our new mesh configuration with some older installs.
4:25pm: Our Henry St hub is having problems and this is affecting many of our downtown nodes. We are working on fixing this now.
11:00am: We had a problem with our internet exchange for the past two hours. It appears to be back to normal now.
Five nodes connected to Ave A hub were down around 2pm today due to a configuration issue. They are back up as of 6:20pm
1pm: We rebooted the Rivington sector and it is working normally again
11am: One of the Rivington Sectors is having problems disconnecting 100Ava and about 5 connected nodes in the LES and East Village. We are working on this at the moment
Brooklyn is experiencing slow speeds especially around 8 to 9pm (“Netflix time”). We are working on fixing this.
A few people are having problems with LinkNYC kiosk using our new SXTsq VPN config. We are trying to solve this problem, but it might be better to switch back to the non-vpn config for now.
There will be a couple of short interuptions between 2pm and 3pm today as a cable is changed at Supernode 1
1:43pm maintenance completed
Due to maintenance at one of our main Brooklyn hubs there may be some temporary interruptions to between 12:30pm and 1pm today
6:00pm Power restored. All connections should be good now
5:45pm Electricians have arrived at the Saratoga building and will be working on this tonight until fixed.
9:24am There is a power outage at our Saratoga hub. This is causing problems in a large part of Brooklyn. We are sending someone out now to look at this.
DE-CIX New York, our internet exchange point is giving us new IPv4 addresses. This will cause a small amount of downtime as we switch addresses over the next two days.
Change period starts: on 2019-01-28 at 10:00 EST (15:00 UTC) End of change period: on 2019-01-30 at 07:00 EST (12:00 UTC)
Finished connecting Henry to SN1 using 60GHz gear, this upgrades Rivington connection to about 800Mbps.
Antenna alignment today at Ehud to fix ongoing bandwidth problem at Saratoga in Brooklyn. Bandwidth should improve in the afternoon after a short downtime around noon.
Rivington hub is still in process of upgrades. This is affecting 4 active nodes in the Lower East Side. We are planning to get full bandwidth there next week by connecting to Henry hub and SN1 using new gigabit 60Ghz gear.
Fixed node 1417 connection at 5:45pm
A storm took down our node 1417 and is affecting 3 nodes in the East Williamsburg area. We hope to have this fixed tomorrow.
Problem fixed! All ok.
A failed upgrade caused downtime at the major hub node in Brooklyn this morning. It was fixed at 4:15pm today.
You can also get updates in the #1340-alerts channel on Slack.
Recently our Supernode 2 had to be taken down for roof repair which forced us to make major changes to the area’s network. We are still working on improving the new connections. The good news is many new hub nodes are coming online, so there will be more choices to connect to in the coming weeks.