Support Board
Date/Time: Wed, 27 Nov 2024 08:42:37 +0000
[Locked] - Denali Data Feed Upgrades
View Count: 382
| [2023-09-02 18:58:31] |
| Sierra_Chart Engineering - Posts: 17191 |
|
We have had a handful of reports recently of when the market is very active, that some users have noticed lagging data, and/or stopping data with the Denali Data Feed. We just want to explain the reason why the data feed can stop. This is not an unusual occurrence, for users connected via the Internet. The Denali data feed uses TCP connections. Every packet of data that is send out from the server requires an acknowledgment packet. If there are not acknowledgment packets for the packets being sent, the operating system will stop the flow of packets, and wait for the acknowledgments or a retransmission request. This is the reason why the data feed can stop. There can be bandwidth limitations, and congestion somewhere along the path of communication from our server to the users computer leading to this condition. We have mentioned in another thread, some work we are doing with the Denali Data Feed and we are restating that further down below. Additionally we are going to be doing the following: - Two of the four servers for the Denali data feed have room for additional CPU capacity. We are going to be upgrading, the CPUs with additional cores, doubling the cores, and this will increase the capacity on these servers. This is a major amount of CPU capacity being added. We are going to be adding 2 Intel Xeon Gold 28 core CPUs to each server. - We are increasing our Internet capacity and making a change for more efficient use of the bandwidth which is currently spread across 2 connections. Additionally we do have two additional gigabits of capacity used for other purposes and in reserve for redundancy and we are using that reserve now in the meantime. - We are going to offload but not all of the historical data serving to another facility. - We have increased the throughput, on our real-time servers significantly earlier this year, by at least three times. There are going to be some refinements to this improvement, to make it more memory efficient. Additionally there is going to be another performance improvement, done to the real-time server to ensure logging does not impact, the real-time data processing. Previously stated improvements: Disabling interrupt moderation on the network adapters used for the connections. The one system that was not affected by the problem had interrupt moderation disabled. We will decide what is best either between low moderation or disabled. We have substantially increased the amount of send and receive buffers used by the network adapters. We will be adding an additional server, which we already have available. We may not be able to bring this out Sunday night but we will try. This is not relevant to the issue, but we are just adding another server for load-balancing purposes. We will add an additional gigabyte of Internet capacity which we already have in reserve. Most importantly, we have to limit the size of user send buffers. They are allowed to get as large as 40 MB and this is way out of line and this is what causes the problem to get out of control and affecting users who would not normally be affected. We will put a much tighter limit. Maybe about 1 MB. Or perhaps less. We will see. We might take this down to about 750 K and see if users run into unexpected disconnections related to this. We are going to increase the network connectivity to each server from 10 Gb to 20 Gb by using two connections, and use one for the incoming CME data, and another for outgoing data to users. This probably is not relevant, but it is easy enough for us to do. Will not hurt. (This is still pending. We are actually doing the work for this, this weekend.) Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2023-09-02 18:59:47] |
| Sierra_Chart Engineering - Posts: 17191 |
|
We are also going to standardize the procedure, for investigating an issue with the data feed that is presented to us. We will allow the user to specify a particular connection address and port , and we will increase the heart beat frequency on that particular server, and then we will do close monitoring of that particular process and compare our results, to the users result at the time they have a problem. This will allow us, to quickly, identify the source of an issue assuming it is a recurring issue. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2023-09-02 19:00:24
|
To post a message in this thread, you need to log in with your Sierra Chart account: