Support Board
Date/Time: Thu, 28 Nov 2024 08:39:23 +0000
Post From: Technical Notice: US Consolidated Tape Packet Loss Issue
| [2023-05-07 16:10:10] |
| Sierra_Chart Engineering - Posts: 17200 |
|
This posting is about packet loss. Sierra Chart obtains, the NASDAQ, NYSE and AMEX data feeds, through services called UTP and CTA. These are controlled by NASDAQ and NYSE respectively. These services provide us what are called multicast data feeds. These are UDP data feeds. It is possible to lose packets somewhere in the process of the transmission of these feeds, or the processing of these feeds. This results, in missing data which will result in lower volumes that you will see for the symbols on these data feeds. The new consolidated tape is data feed for NYSE and AMEX was released May 1, 2023. We had no packet loss, for this data feed on that day. And the NASDAQ data was released in the weeks earlier and we only had very minimal packet loss with that feed. Minimal packet loss can be recovered through the use of the retransmission feed which we are currently working on. Since May 2, 2023 we have been experiencing significant packet loss with NYSE and AMEX. This ranges anywhere from .2% to 2%. The particular amount of loss, depends upon the particular symbol, and the particular multicast group that symbol is on. It is hard to isolate the source of a problem like this, but with our experience with packet loss, and having tried numerous things to resolve this, and with careful consideration and consultation with a service provider, giving us access to these feeds, it has become clear that the issue, most likely is microbursts of data transmission, which are exceeding the network connection capacity that we currently have. This was something that we thought would not be a problem because we were not seeing it with the NASDAQ feeds, and we did not see it May 1 and we have had no trouble with the equivalent data feeds being provided by Barchart which are also multicast feeds and providing data from NASDAQ, NYSE and AMEX. The difference though with the feeds from Barchart as compared direct from the exchanges, is that from the exchanges, they are providing us low latency feeds, from network switches, which have a higher capacity than 10 Gb. The connections we use are 10 Gb. We are going to have to upgrade our connections to 40 Gb. That may not happen this week. We have to rely upon a service provider for this. We hope it will be completed this week but if not this coming week, it will be completed the following week ending May 19. We are doing our best to get this done as quickly as possible. All the necessary hardware to support this, we already possess. Until we get this issue resolved we are not going to be offering delayed data for the US Consolidated Tape. --- Another thing to understand, that has confused some Sierra Chart users is the new real time US Consolidated Tape data feed, is different than the NASDAQ TotalView data feed. NASDAQ TotalView is documented here: Denali Exchange Data Feed: Nasdaq TotalView Data Feed Notes If you are using the NASDAQ TotalView symbols like: AAPL-NQTV You will see lower volumes. Subscribing to the US Equities Consolidated Tape data feed does not change this. You need to use, one of these two symbols to see the full volume: AAPL AAPL-USCT Use one symbol or the other but not both. Using both will lead to a conflict where you will only get data for one symbol. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2023-05-08 08:11:29
|