Support Board
Date/Time: Sun, 24 Nov 2024 02:14:13 +0000
[Locked] - US Equities/CFE Data Early Morning Weekday Carrier Maintenance Notification
View Count: 3680
| [2024-11-07 06:17:53] |
| Sierra_Chart Engineering - Posts: 17154 |
|
Our US equities and CFE data, is delivered, over a wavelength connection from Chicago. And we have been informed of three early-morning maintenance activities, from the carrier during November. We will be posting them here. We will ensure, the feeds are functioning properly after the maintenance. The maintenance, ranges anywhere from 12 AM to 7 AM, and can cause an interruption of 2 to 4 hours. We were not aware that this type of maintenance was going to be occurring, when we had ordered this circuit and we are working on a alternative connection, to compensate for this kind of issue. We are working on that as soon as possible. This is the first scheduled maintenance: *Location of Work: *Lansing, MI
Scheduled Start Date & Time Scheduled End Date & Time Time Zone 11/12/2024 1:00 AM 11/12/2024 7:00 AM Eastern 11/12/2024 12:00 AM 11/12/2024 6:00 AM Central 11/11/2024 11:00 PM 11/12/2024 5:00 AM Mountain 11/11/2024 10:00 PM 11/12/2024 4:00 AM Pacific 11/12/2024 6:00 AM 11/12/2024 12:00 PM GMT Alternate Start Date & Time Alternate End Date & Time Time Zone 11/14/2024 1:00 AM 11/14/2024 7:00 AM Eastern 11/14/2024 12:00 AM 11/14/2024 6:00 AM Central 11/13/2024 11:00 PM 11/14/2024 5:00 AM Mountain 11/13/2024 10:00 PM 11/14/2024 4:00 AM Pacific 11/14/2024 6:00 AM 11/14/2024 12:00 PM GMT *Expected Customer Impact:* Service Affecting *Expected Impact Duration:* Up to 4 Hours As we said, we are working as quickly as possible, for an alternative connection to keep the data feed up and running during the maintenance operations. But we do not expect any change in November. The soonest would be more like about two months from now. We have contacted the carrier about this issue, and we have not been given any solution yet to avoid an interruption. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-11-12 05:02:44] |
| Sierra_Chart Engineering - Posts: 17154 |
|
We are waiting for this maintenance to start. Nothing yet. And we hope it gets completed well before 4 AM. And we are taking steps to obtain another circuit, that we can use when this happens.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-11-12 14:13:48] |
| Sierra_Chart Engineering - Posts: 17154 |
|
The above maintenance described in post #1 did not result in any interruption to our data feeds.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-11-12 14:23:00] |
| Sierra_Chart Engineering - Posts: 17154 |
|
This is another maintenance we have been notified of: Location of Work: Lansing, MI Scheduled Start Date & Time Scheduled End Date & Time Time Zone 11/15/2024 1:00 AM 11/15/2024 7:00 AM Eastern 11/15/2024 12:00 AM 11/15/2024 6:00 AM Central 11/14/2024 11:00 PM 11/15/2024 5:00 AM Mountain 11/14/2024 10:00 PM 11/15/2024 4:00 AM Pacific 11/15/2024 6:00 AM 11/15/2024 12:00 PM GMT Expected Customer Impact: Service Affecting We do not know, the extent of the interruption if any. Maintenance done any time between approximately 8:30 PM, and 3:30 AM US Eastern time is not going to affect the US equities feeds because they are not active during that time range. Certainly our CFE data feed would be affected but there is not much activity during that time. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-11-15 05:34:46] |
| Sierra_Chart Engineering - Posts: 17154 |
|
The above maintenance, in post #4 is interrupting our CFE data feed at this time. We hope the maintenance is completed, before 3:30 AM US Eastern time.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-11-15 12:11:09] |
| Sierra_Chart Engineering - Posts: 17154 |
|
Our US equities and CFE and CBOE Global Indexes data feeds are down at this time due to this carrier maintenance. We really apologize for this. Sierra Chart market statistics are also affected and unavailable. We are adding another backup circuit as soon as possible. We will be patching the missing US equities data feeds over the weekend. We will have to see about patching the CFE data. Our wavelength circuit provider Crown Castle has informed us, that the maintenance today, is extending until 11 AM. Which means, that the US equities feed, CFE and CBOE data feeds will be down, beyond 9:30 AM. They tell us a partner carrier, is doing maintenance. And we insisted previous to this incident that any maintenance be done over the weekend but they said they have no control over that. We apologize for this and we are getting a backup circuit in as soon as possible but that does take time. It is going to be more than 30 days from now. We already have all of the infrastructure on our side, installed and ready to go for a backup circuit. Also, once the carrier circuit comes up, later this morning, NASDAQ TotalView historical data will be immediately filled in. That is because that type of connection we use to NASDAQ TotalView gives us the historical data immediately. The other exchanges we have to work on the historical data. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-11-15 13:57:17
|
| [2024-11-15 15:36:29] |
| Sierra_Chart Engineering - Posts: 17154 |
|
When we were told of this upcoming maintenance, we complained very forcefully and as much as we could, to the carrier and told them that this maintenance must only be done on a weekend. Their response was, it is being done by a partner carrier and they have no control over it. Our response to that is that we did our due diligence, and we asked the carrier before we ordered this service, do you control the entire length of fiber between Chicago to Southfield Michigan. The answer was yes. This was a question we were told to ask to ensure, issues like this would be avoided. We were lied to. And we were told a few days ago, that these carriers lie and they should never be trusted. We did not know that. Fiber carriers, can never be trusted and they are liars. We were told, that no matter what promises a carrier makes, they can never be trusted. They will say whatever they can to get the business, and they can reroute circuits, at a later point in time. So even if what they say is true in the beginning it can later change. We can see now, this is fact. This is something we were not aware of. We thought we were making an educated, decision and did our due diligence. Little did we know, we were being lied to. Regarding the prior paragraph, what we were specifically told, is that if a carrier, says that they can provide a backup circuit, that is a diverse route from, the primary circuit, do not believe them. Never trust them. It will not be true. Even if it is true in the beginning, it could and will later change and no longer be true. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-11-15 16:07:53
|
| [2024-11-15 17:07:44] |
| Sierra_Chart Engineering - Posts: 17154 |
|
From carrier: We sincerely apologize for the outage and have our Change Control Engineer actively working with our vendor partner to get your service restored as quickly as possible.
Our team is currently dispatching another tech to another location so that we can shot (troubleshoot?) the fibers to further isolate the problem. Again we apologize and thank you for your patience. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-11-15 17:07:58
|
| [2024-11-15 17:17:29] |
| Sierra_Chart Engineering - Posts: 17154 |
|
There is no other way to say this, other than the fact that, this whole situation has turned into a disaster. Some maintenance, which should have been completed early in the morning, has gone horribly wrong and disrupted our data feeds. We really do apologize for this. And this maintenance should have been done and must have been done on a weekend and we really forcefully complained about this but were just simply ignored. We are taking steps, to add a backup circuit, through a completely different carrier with a completely different diverse route. And there is other redundancy we are going to be adding, to another data center, in Illinois for our US equities and CFE feeds. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-11-15 17:17:37
|
| [2024-11-15 19:30:59] |
| Sierra_Chart Engineering - Posts: 17154 |
|
At this point there is no reason to believe the service is going to be restored, for the remainder of the trading day (4 PM US Eastern time). We apologize for this. We are waiting for an update from the fiber carrier. We had a similar problem, about a year ago with EUREX data where we had an outage for an entire day and we took steps to ensure that never happens again. And we are ready to be releasing that, we expect in December. A fully 100% redundant set up in all areas. We have been actively working on that. We actually did some work on our new EUREX set up yesterday. There is a lot of time and effort, an investment that. We always, do the best and make sure we provide the very best service. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-11-15 19:34:18
|
| [2024-11-15 20:03:03] |
| Sierra_Chart Engineering - Posts: 17154 |
|
This is too embarrassing to post but this is the latest we got from our fiber carrier: The team is still trouble shooting, so far they have found some flipped fibers. So is this the result, of the early morning maintenance that they did? They did not get the fiber connection splicing right? And once again we protested quite strongly, well in advance of all of this, that maintenance must only be done on a weekend. The response was "we'll see what we can do". Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-11-15 20:06:24
|
| [2024-11-15 20:17:00] |
| Sierra_Chart Engineering - Posts: 17154 |
|
Latest update from fiber carrier: The activities associated with this event have extended beyond our expected maintenance window. This maintenance has been extended until 7 PM Eastern Time. Our crews are still on site and actively working to complete this maintenance as soon as possible. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-11-15 20:17:17
|
| [2024-11-15 22:22:11] |
| Sierra_Chart Engineering - Posts: 17154 |
|
Update: For some context for the below message, Sierra Chart relocated servers permanently out of Chicago in September 2024. We utilize a 100 Gb wavelength connection for US equities, CFE and CBOE indices, to our new data center location in Michigan. We saw a post, about was why we do not have a backup source for these data feeds. We do not need an additional backup source for these data feeds because the service provider we get the data feeds from out of the Chicago data center, has a fully, highly redundant set up with direct connectivity to the exchanges. We already have more than enough backup and redundancy there. There simply is not a need for anything more. This would only technically complicate the service. The service provider we work with, has always delivered us 100% uptime, with these data feeds we get from them. They are very good with what they do. We know, because we have been in many technical discussions with them, and understand their network and architecture, and hardware, and it is all exceptional. The weak point, is a single 100Gb Wavelength between our data center location and the Chicago data center where the feeds are originating from. We know, this is a vulnerability, but it is not expected, based upon our understanding of a wavelength circuit, and promises made, that an issue like this would occur. Basically we were lied to. We did express concerns, in the beginning but they reassured us, that this is a reliable and stable circuit, which is going to maintain 99.9% uptime per month. There was never any disclosure, of maintenance. Another provider we checked with Cogent, has said that they have had zero maintenance or any other incidents over the last six months for this particular circuit/route if we got it from them. The decision to not use Cogent had to do with them not being ready to deliver the circuit to the particular data center we wanted. So as you can see, there has been an awful lot of research and consideration, but the there are always limitations and issues and we have to find what is the right balance. Cogent is now able to deliver a circuit like this, to the data center that we use. Another issue, that was not told to us, is FCC fees for an interstate connection. They lied to us on price. And other providers, were honest to to us about price, and included these fees, and it made it look as though they were overpriced compared to other providers. The FCC fee issue, is a non-issue though. Although interestingly, it was ruled, in a federal court in July or about then, that these fees are unconstitutional. There are different ways of solving this problem with the connectivity. We could keep servers within Chicago/Illinois area. But that is a very significant infrastructure set up, that will have its own vulnerabilities, unless it is fully redundant including redundant Internet connectivity. And this is exactly what we got away from. We got the hell out of Chicago and very very happy with that. There was an awful lot of, effort and work involved in that, and we did not want to start putting servers back into the Chicago or Illinois area. There is a very strong natural, avoidance to even consider that especially, with all kinds of restrictions being placed upon us with connectivity with who would be hosting the servers. It gets to be a point of, tremendous frustration. But nevertheless it is under active consideration. But it is a significant infrastructure set up to even accomplish with all kinds of technical complications. Another possibility is to use a lower bandwidth secondary connection, over to Chicago as a backup connection. The other possibility is to use, a normalized feed, through an API which has lower bandwidth requirements, and maintain 2 lower bandwidth connections over to Chicago CH2 and NY4. All of these things, have their own issues and complexities. There is no simple answer and this is something which is being gradually, worked out in the coming months . Unfortunately this incident has occurred before, a solution was determined. And taking our time, for the right solution, is very sensible. We want to get this right. We have been actively, looking at, connectivity redundancy for our US equities feeds and indices, and CFE but as of yet we have not come to a definite conclusion as the best arrangement. But we are gradually coming up with some choices that will work well. And we have been very actively looking at all of this including during the past two weeks. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-11-16 18:05:44
|
| [2024-11-15 22:54:14] |
| Sierra_Chart Engineering - Posts: 17154 |
|
We just talked with Cogent, we are going to get a 10 Gb connection, as a backup, over to the NY4 data center in New Jersey . This will be a diverse route through a different carrier. And will be a good backup for us. They say delivery time is 17 days. We are going to work hard to prioritize this and get it done. It does not mean the service is going to be down for 17 days. We expect Crown Castle, the current carrier to get this current outstanding issue resolved tonight. Latest: This has been classified as an outage and is being transitioned over to our Network Operation Center. Please know we still have people in the field and a NOC Engineer is now running with this. Unfortunately, it doesn't appear we have any re-route options at this time.
Thankyou for your patience. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-11-15 22:56:10
|
| [2024-11-16 00:00:24] |
| Sierra_Chart Engineering - Posts: 17154 |
|
Latest: Crown Castle Fiber's Tier III Engineer reviewed the standing alarms and concluded an outside plant fiber issue still exists. The partner carrier's field crews continue to troubleshoot this outside plant fiber issue. A Crown Castle Fiber Field Operations Technician is in transit to our local office to provide additional support to our partner carrier and their field crews. The Crown Castle Fiber Field Operations Technician expects to reach our local office by 9:00pm, ET. Additional updates will be shared as they become available.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-11-16 00:48:53] |
| Sierra_Chart Engineering - Posts: 17154 |
|
The missing data has been patched for US Equities Consolidated Tape symbols. Follow the instructions here: Historical Intraday Data (Missing Data / Download Errors / Viewing More Data): Retrying the Download When There is Data in the Chart Or you can use this menu command to update, any chart: Edit Menu: Edit >> Download Data from End of Chart We will be working on patching the other exchanges affected (CFE , NASDAQ TotalView, CBOE Indices). CBOE Global Indices, we need to get the data from the exchange. And for NASDAQ TotalView, we have asked the exchange for a packet capture file. Normally we should be able to recover that same day, but since this connectivity circuit is still down, we are close to losing the opportunity to do that. It would have to be up before 8 PM. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-11-16 01:25:08
|
| [2024-11-16 03:20:42] |
| Sierra_Chart Engineering - Posts: 17154 |
|
Latest: Crown Castle Fiber's NOC, Tier III Engineer and Field Operations Technician continue to troubleshoot this event with our partner carrier. All parties are conducting the troubleshooting in collaboratively via a conference bridge. We will share the next update by 12:00am, ET, or upon new information.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-11-16 05:33:49] |
| Sierra_Chart Engineering - Posts: 17154 |
|
Resolved: Crown Castle Fiber's Field Operations Technician isolated the impairment to faulty jumper cable on our partner carrier's network. A new jumper cable was installed and services restored. The Crown Castle Fiber NOC observed all alarms cleared and services restored. A formal RFO will be shared with all impacted customers. What is interesting about this, is this was the issue, with our EUREX feed as well. A faulty jumper cable. The problem is, in the case of EUREX how does a fiber-optic jumper cable suddenly become faulty in the middle of the night in a secure data center. No one ever took responsibility for what happened. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-11-16 05:51:18
|
| [2024-11-16 19:01:08] |
| Sierra_Chart Engineering - Posts: 17154 |
|
We want to provide some background information, and philosophy, regarding, all of our, hardware and software, and connectivity for our US Equities, CFE, and index data feeds. Sierra Chart has a very strong philosophy, of independence, and doing everything low-level, and doing things ourselves, and doing things right and using superior engineering. Even if we do not get things right in the very beginning, we work towards doing that. First, Sierra Chart, is a direct market data vendor, with NYSE, NASDAQ, and CBOE. We report to these exchanges, every month. And we are subject to audits. This puts us in a class, unlike, competitors. They are not direct vendors. We obtain, through, through a service provider, the raw data feeds, from these exchanges. In the case of US equities, these are called the CTA and UTP data feeds. Raw data feeds are always an advantage, because we can process these data feeds properly. For example, among other things, being able to deliver accurate Bid Volume and Ask Volume. So it is a technical advantage, to work with the raw data feeds. This puts as above and beyond competitors. When working with raw data feeds, we have a choice of many service providers providing data from exchanges, both in Chicago CH1/CH2 and the Secaucus New Jersey NY4 data centers. We do not have to rely on any one provider. We can decide to switch providers at any point in time (subject to contractual obligations though). Each of these providers, are very good and provide full redundancy. There is no need to work with two of them. One of them is enough. But we have the flexibility to move providers. Whereas if we used some API normalizing these data feeds, Then we are locked into that particular provider and it makes it difficult to transition. Whereas now we can just say we want to work with a different provider, and we simply get a cross connect to that provider in the data center using our existing connectivity circuits. It is not something that can be done in one day, but it is a transition that probably can be done within 30 days. We have all of our own networking equipment. We use two of these switches at our Michigan data center location: https://www.ipinfusion.com/wp-content/uploads/2020/09/EdgeCore-AS5912-54X.pdf (the particular switches we use though does not appear to be exactly what is shown in this PDF because the ones we use, use Intel Atom processors). Otherwise, the specifications are the same. This switch does have, deep packet buffers which we wanted to ensure that there is not packet loss. The operating system we use on the switches is: OcNOS from IP Infusion. These are very expensive and good quality network switches. We have looked at other network switches, and these are very good. For example, the operating temperatures of the optical transceivers, inserted into these switches is maintained at lower temperatures compared to others. By having our own Network Equipment, this gives us independence, and allows us to work with any fiber carrier we want to, and any service provider we decide to connect into. The other component, of what we need, is the physical connectivity, between our data center location, and the Chicago, or Secaucus New Jersey data centers. If we are located in Chicago or Secaucus New Jersey, then we would just use cross connects within the data center that would cost about $300 a month. The problem is, we do not want to operate our equipment out of those data centers. Instead, we chose Southfield Michigan. Which is strategically located between, Illinois, and New York. It is a good location. At our data center location, we have 4 Internet connections. (One of them is still pending from Lumen). We have a lot of capability there, and plenty of power, and very good power redundancy. And they use very good methods of cooling for the data center which would not result in the problem we saw, at Equinix in Chicago, at the beginning of this year. Or was it December 2023? We are not sure. But that data center had a cooling failure, and overheated and got over 120°. It was a complete and very serious disaster. So we have complete flexibility and freedom to make changes. So we have flexibility to choose, the service provider providing the raw data feeds, and the fiber carriers, and we can choose between 2 data centers (CH1/CH2, NY4) for connectivity. This is the philosophy behind what we are doing. We are not locking ourselves, into a particular API, a particular data center, a particular service provider. We have the flexibility to make changes and adapt. We had this outage, on Friday, and we are taking steps to remedy that it will not happen again. This was already in progress in the weeks before! We were and are actively working on that, in the weeks before because when we saw all of these maintenance notices from Crown Castle we were worried about a problem. And it did happen! And we complained very forcefully to Crown Castle ahead of time not to do this maintenance on a weekday. They ignored us, because there were partner carriers involved. And we were told to ask them ahead of time do they control the entire length of fiber between Chicago to Southfield Michigan. They told us they do. That was incorrect. If they control the entire length of fiber, then they could choose to do the maintenance, when they want like on a weekend. So this is a learning experience for us, that we have to be aware that fiber carriers will not disclose, relevant information and we have to be able to figure out the actual truth independently. And it is not just us, being affected by this. There is is a redundancy situation, at the CME Aurora data center, we became aware of about a year ago where a fiber cut which knocked out both Zayo and Cogent. It was not known that these two carriers, are working on the same fiber cable. None of us knew that including our service provider at the CME data center. And Cogent is not going to offer the information that in the background, that they in part are relying on Zayo in that data center. Our service provider, is taking steps to add redundancy. And we reminded them about that yesterday. The reason it is taking a long time, is the particular fiber carrier they are planning to use, is taking a long time, with the installation of the circuit. We have that same frustration with Lumen. They have taken a very long time with the installation of a new Internet connection, at our Southfield Michigan data center location. They had to, add additional capacity. One of the main issues here, is our inexperience working with fiber carriers providing ethernet point-to-point circuits and Wavelength circuits. We have since learned, now that we have seen all of these maintenance announcements from Crown Castle, and FCC interstate fees for these types of circuits, that fiber carriers cannot be trusted. This is something that we did not understand previously. We trusted what we were being told but none of it is true. We now know better. For example, our service provider providing the data feeds, as we understand has multiple circuits (possibly as many as 4), crossing the Atlantic between the United States and Europe. Since in the case of submarine cables, if there is a fault, that is going to be out until it can be repaired which obviously involves sending a ship out and specialized repair equipment and people. This could take months. So what we are going to do, is add another circuit over to NY4 for these data feeds. This is going to be a really good set up. We have a data center location in Michigan with connectivity over to Chicago and connectivity over to Secaucus New Jersey. These are completely diverse fiber paths. Both of these connections will be always on and active at the same time with distribution of feeds over both. If there is a problem with one, that the remaining feeds can be switched automatically over to the other. We have multiple servers, at our Michigan data center location, with 100 Gb connections into our Edge Core network switches. Currently we have 4 servers processing these data feeds, each with two 100 Gb connections. One into each of the Edge Core switches. One switch, is primary currently going to Chicago. The second switch, is on standby and we intend to connect it to NY4. So each of the servers has the ability to access the data both from CH1/CH2 and NY4 simultaneously. All of this, is the current set up. The one thing just missing is this additional connection to NY4. We have an abundance of servers and networking gear there, with easy access for maintenance. All of this, gives us, the freedom and flexibility, that we want. Here is some reference material about wavelength circuits: https://www.cogentco.com/en/products-and-services/transport/optical-wavelengths https://www.cogentco.com/files/docs/network/performance/global_sla.pdf (Cogent claims 100% uptime) When we queried Cogent in-depth, in the weeks before this incident because were planning on, using a circuit from Cogent, they say they have had zero faults or maintenance activities, on the route between Chicago and Southfield Michigan. So that is good. The question is whether it is true. Not sure how we could independently verify this. Although we do have experience, using Cogent connections between Illinois and Michigan, and the uptime has been really what seems to be 100%. The reason we chose a Wavelength type of circuit is because we wanted to ensure that there is no packet loss. As we understand these types of circuits are better suited to multicast type of data which we get from the exchanges. And the reason we went to 100 Gb rather than a lower speed is because with a wavelength the speeds are only 10 Gb and then 100 Gb. 10 Gb was not enough for all of the data feeds that we are pulling simultaneously. And we are being told that at this time, really with longer haul connections that the speed has to be at 100 Gb. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-11-17 18:04:56
|
To post a message in this thread, you need to log in with your Sierra Chart account: