Support Board
Date/Time: Wed, 27 Nov 2024 11:42:19 +0000
[Locked] - Glitch in feed at 10am Est
View Count: 710
| [2023-08-25 15:13:04] |
| User449884 - Posts: 155 |
|
Was there a glitch with the feed or platform around 10 am today as Powel spoke? Everything stopped on my DOMs for about 1 min
|
| [2023-08-25 15:21:00] |
| Sierra_Chart Engineering - Posts: 17191 |
|
No definitely there was not on the Sierra Chart side. That would not be possible.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2023-08-25 15:21:10
|
| [2023-08-25 15:33:53] |
| Rui S - Posts: 190 |
|
Was there a glitch with the feed or platform around 10 am today as Powel spoke? Everything stopped on my DOMs for about 1 min
Yes, there definitely were glitches and freezes! And definitely more than once. Date Time Of Last Edit: 2023-08-25 16:46:48
|
| [2023-08-25 15:37:19] |
| User412102 - Posts: 5 |
|
Yup. Me too - on Teton. Observed for several minutes before it settled at about 10:06.
|
| [2023-08-25 16:19:33] |
| allons_trading - Posts: 30 |
|
Saw the same thing today as well. I have two instances running on my computer. One instance stopped while the other continued without a hitch. I disconnected and reconnected the instance that stopped, and everything went back to normal after the reconnect.
|
| [2023-08-25 18:10:31] |
| User753428 - Posts: 163 |
|
absolutely there was a glitch and freeze. it lasted for around 40 seconds for me which is unacceptable. i was deliberately viewing the message log at the time for the detailed heartbeat logging but it totally skipped over the freeze, which is even more concerning. for example, if my chart update interval is set to 50ms and there is a datafeed freeze, the message log will show a heartbeat log with a TransmissionDelayInMilliseconds higher than 50ms and higher than usual NumberOfOutstandingSendBuffers to reflect the freeze; but in this case, it just didn't show it at all. Date Time Of Last Edit: 2023-08-25 18:11:43
|
| [2023-08-25 20:41:57] |
| user_xyz - Posts: 416 |
|
Same here. There is 100% something going on around news releases that appear to overload the buffer (maybe related to Depth of Market?) and fixed with disconnecting/reconnecting/restarting SC.
|
| [2023-08-25 23:57:05] |
| User449884 - Posts: 155 |
|
If soomeone at SC or Teton wants to see i have the actual screen recording. But someone needs to look into this. Happens often around big news or numbers but today was the worst
|
| [2023-08-26 10:52:56] |
| Rui S - Posts: 190 |
|
But someone needs to look into this. Happens often around big news or numbers but today was the worst
I second that. This is happening every single time there is a spike in volume along with fast price action (e.g. big news, important releasings and sometimes also RTH openings). I have already discarded all other possibilities on my side (I am always aware of that) and I'm absolutely sure the problem is not in my dual high speed internet, my powerful pc running bellow 5% or my SC configurations. Also, I have another data feed running side by side and it doesn't show any similar problem. I didn't complain earlier because the answer is always the same, as we can see in post #2. Hopefully this time will be different. |
| [2023-08-26 15:43:44] |
| Sierra_Chart Engineering - Posts: 17191 |
|
We apologize for the delay here. Now that there are more reports, we are looking at this again. We will follow-up later today. One thing that we can definitely say is that this is not an issue with the core CME data processing and distribution of that data. There is no data lagging or stopping with that. This has been clearly verified. And our answer, at post #2 is regarding that. With what we can tell so far, this does once again look like an Internet connectivity issue and side effects caused by that. Extensive work has been done, this year to provide more bandwidth, and handle bursts of market activity with very low latency and overall improved data feed performance. Throughput, on our real-time server process, has been tripled, at a minimum which is a huge throughput increase. This ensures, latency remains very low, in the microsecond to low millisecond range. However, when there are external network communication factors, this is not going to make a difference. We are disappointed to hear these reports. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2023-08-26 23:59:55
|
| [2023-08-26 23:16:35] |
| Sierra_Chart Engineering - Posts: 17191 |
|
It is clear the problem is network communication related and once there begins to be a problem it worsens. And what is affecting of a small number of users then begins to affect other users. We are going to be doing the following: Disabling interrupt moderation on the network adapters used for the connections. The one system that was not affected by the problem had interrupt moderation disabled. We will decide what is best either between low moderation or disabled. We have substantially increased the amount of send and receive buffers used by the network adapters. We will be adding an additional server, which we already have available. We may not be able to bring this out Sunday night but we will try. This is not relevant to the issue, but we are just adding another server for load-balancing purposes. We will add an additional gigabyte of Internet capacity which we already have in reserve. Most importantly, we have to limit the size of user send buffers. They are allowed to get as large as 40 MB and this is way out of line and this is what causes the problem to get out of control and affecting users who would not normally be affected. We will put a much tighter limit. Maybe about 1 MB. Or perhaps less. We will see. We might take this down to about 750 K and see if users run into unexpected disconnections related to this. We are going to increase the network connectivity to each server from 10 Gb to 20 Gb by using two connections, and use one for the incoming CME data, and another for outgoing data to users. This probably is not relevant, but it is easy enough for us to do. Will not hurt. Some other internal performance improvements will also be done. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2023-08-27 00:02:00
|
| [2023-08-26 23:42:01] |
| User449884 - Posts: 155 |
|
Thanks for addressing this so quickly SC engineering. Much appreciated |
| [2023-08-27 05:56:17] |
| fireworks11 - Posts: 63 |
|
I had the same issue and I am glad that SierraChart is working on this problem.
|
| [2023-08-27 11:20:31] |
| Rui S - Posts: 190 |
|
It is clear the problem is network communication related and once there begins to be a problem it worsens. And what is affecting of a small number of users then begins to affect other users.
Yes, this describes exactly what I have been feeling for quite a while now. As a scalper on those volatility times, I guess I am one of those that started to feel the problem first. And you are absolutely correct, once the problem started it gradually have gotten worse. We are going to be doing the following:
Thank you very much for addressing this issue so decidedly. I understand the complexity of what you are dealing with. What we all ask is you do the best you can! I'm sure you will find the best way to mitigate the problem, as always. |
To post a message in this thread, you need to log in with your Sierra Chart account: