Support Board
Date/Time: Mon, 25 Nov 2024 18:26:37 +0000
Sub-instance vs standalone folder installation
View Count: 602
| [2024-02-14 20:06:20] |
| joshtrader - Posts: 488 |
|
I have a primary instance which uses Denali + Rithmic for data, and a subinstance which uses the primary instance's DTC server for data. I'd like to make the primary only use Rithmic and the other instance use Denali. Are there any advantages to using the subinstance method for this use case? Or, would it be simpler to just move the installation from its current subinstance folder into its own separate folder, since it will not be using DTC? I should note that the primary instance with Rithmic will only be DOMs and will not need historical data really. Only the Denali instance will use historical data. Date Time Of Last Edit: 2024-02-14 20:09:41
|
| [2024-02-14 20:20:30] |
| Rui S - Posts: 190 |
|
Hi joshtrader, To do what you want you will have to use two separate instances. If I can ask, what's the reason you want your DOM with Rithmic data feed? |
| [2024-02-14 20:37:39] |
| joshtrader - Posts: 488 |
|
If I can ask, what's the reason you want your DOM with Rithmic data feed?
My primary instance is DOMs only and I only have 3 open, but compared to a separate test instance that is Rithmic, the DOMs are much less responsive. I think this may be due to the fact that the DTC server is serving the subinstance which has about 15 charts. So, I want to try 1 instance for execution with Rithmic, and 1 for charts with Denali, with them being completely separate. Date Time Of Last Edit: 2024-02-14 20:38:04
|
| [2024-02-14 20:59:39] |
| Rui S - Posts: 190 |
|
My primary instance is DOMs only and I only have 3 open, but compared to a separate test instance that is Rithmic, the DOMs are much less responsive. I think this may be due to the fact that the DTC server is serving the subinstance which has about 15 charts. So, I want to try 1 instance for execution with Rithmic, and 1 for charts with Denali, with them being completely separate.
My primary instance is DOMs only and I only have 3 open, but compared to a separate test instance that is Rithmic, the DOMs are much less responsive. I think this may be due to the fact that the DTC server is serving the subinstance which has about 15 charts. So, I want to try 1 instance for execution with Rithmic, and 1 for charts with Denali, with them being completely separate. Thank you for the clarification. I don't think the difference you notice is due to the sub-instance over the DTC connection. If it was the inverse (DOMs on the sub-instance), it could be. What might be happening is the sub-instance "weighing" on the primary instance in terms of CPU, thus causing the primary instance to lag. If that's the case, separate instances will be definitely better. I know this is a controversial issue, but it works for me. See this thread here: 10x faster than 4 Instances !!! | Post: 327413 Please post your conclusions here, it's always good to keep up. Good luck. |
| [2024-02-14 22:03:44] |
| John - SC Support - Posts: 36260 |
|
As previously stated, the only way to get the data from Denali and Rithmic is to have two completely separate installations. Refer to the following for how to install a second Sierra Chart: Using Multiple Data and Trading Services at the Same Time: Step-By-Step Instructions to Install Multiple Copies of Sierra Chart Then, in the installation where you want to get the Rithmic data, you need to set the option for "Global Settings >> Data/Trade Service Settings >> Common Settings >> Allow Support For Sierra Chart Data Feeds" to "No". For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-02-15 20:24:35] |
| joshtrader - Posts: 488 |
|
Please post your conclusions here, it's always good to keep up.
After running scenarios for the first hour of trading today, my conclusions follow. Running one Rithmic instance with a single DOM vs another Denali instance shows the Rithmic feed has many more depth, and even price, changes. While this leads to a more "responsive" view of the market, unfortunately, due to so many depth and price changes, the result is that adding even one or two more symbols causes Sierra to lag in processing every update, leading to lags across all DOMs when market activity is high. The Denali feed will generally have less lag than Rithmic when using multiple symbols, but this is because it does not seem to have as many updates, or at least, Sierra does not seem to process as many. This is objective and easy to see. The Denali DOM updates inside market depth maybe 5-8 times per second max. A Rithmic DOM shows easily 15-20 updates per second. Again, the contrast is most easily observed when market activity is higher. It doesn't have to be VERY high, just not the midday lull when nothing's going on. In my tests, I saw Rithmic instance A lag compared to Rithmic instance B even with the identical windows open, when instance A had previously tracked NQ and YM. In other words, it's not necessarily the updating of the DOMs which is causing the lag, but simply processing the incoming data (using the "ST: ..." status in the main window to determine how many symbols are being tracked). Upon a disconnect and reconnect, the lag went away as both A and B instances were identical in symbols tracked and windows open (1 symbol tracked, 1 DOM open on both). So, my conclusion is that if you have a single instance with a single instrument, you can get lightning fast DOM views by setting the update interval to about 15ms (60 updates per second, faster than the monitor refresh most likely and close to the limits of what the human brain can perceive), and by adding no additional symbols for Sierra to track. Repeat: if you do add additional symbols, there is a *very* good chance that the data you see is delayed during times like news and market open/close, and I mean sometimes delayed by several seconds. This can be observed by putting the time in the DOM header and comparing it to a real time clock. Sometimes you will see that the last trade is several seconds behind a real time clock (you can even use sierra's real time clock study to see this easily). On the other hand, if you need to track multiple instruments in a single instance, you're better off using Denali, since Sierra is able to better process the data. I am considering three instances of Sierra: (1) Denali only, for charts; (2) Rithmic for my main ES DOM; (3) secondary login for Rithmic (extra $ per month) for YM and NQ DOMs. This spreads out the processing over 3 instances and will likely give the best result. Note that I have a gigabit, fiber internet connection with low latency and jitter. I say this to emphasize that the size of the pipe coming in is not an issue, and my gigabit NIC and cables are not either. The bottleneck seems to be the fact that a single Rithmic data feed with ES is handled well by Sierra, but a single Rithmic data feed with ES + NQ + YM is not. Next I will try the second Rithmic login to see whether ES on one and YM+NQ on the other will be suitable. |
| [2024-02-15 21:36:52] |
| User150671 - Posts: 69 |
|
I'll share my experience running Sierra with multiple installs: 1x Install Charts Denali CME top of Book 1x Install DOMS running Rithmic OR 1x install of Older Sierra running CQG. Tracking a fuck ton of symbols through Charts using Denali (754 symbols today, has been up to ~1500) My DOMS instances have 6x DOMS all set to either 50MS: ES, NQ, CL, GC and 75MS: ZN, 6E. I used to have ES and NQ set to 20ms but noticed no difference - so set them back, your mileage may vary and props to you if you can perceive the difference! I am moving away from Rithmic because it DOES lag significantly on a regular basis. I have seen over 20s delay with 1x ES DOM through Rithmic compared to a full suite of DOMS through Denali or CQG. I am not sure if this is an issue with their DTC server implementation, or just Rithmic being Rithmic. I know traders who use other platforms and have experienced similar lag with Rithmic, although not as severely as I have. I have not noticed any delays or lag with Denali from within USA when using Denali Full Depth for DOMS - although it has been a while since I used Denali for DOMS. Only times I have noticed Delays with CQG are either a fault with CQG, or because I move country and forget to have my broker set the correct localized server. You can add the Data Delay to the Trade DOM Header to show data delay easily (although it is kinda long at 13 characters 'DD: -00:00:00' and really only need the last 4 of those digits). Sierra is honestly really light weight in it's hardware requirements, my DOMS instance sits at about 160mb of RAM usage and I have never seen it take up much CPU usage at all (<1% right now after close, I assume it gets higher during volatile events, but I am looking at my DOMS not CPU usage then!). My Charts Instance is closer to 600MB with about 6% CPU. Other platforms I have used, which are written in Java/HTML I have seen use >500MB of RAM and over 15% CPU just to display some DOMS. Honestly - for everyones talk of needing powerful PC's to trade with, the data processing really is minimal compared to what modern computers are capable of. |
| [2024-02-15 22:06:42] |
| Rui S - Posts: 190 |
|
joshtrader, Thank you for your detailed description. After running scenarios for the first hour of trading today, my conclusions follow.
I feel very similar as you describe. I trade just the first 30 minutes after the open and also the volatile releasings. So, like you, I need fluidity too. My big advantage is that I only trade NQ, so I have just one instrument in my primary and secondary instances. Just for charting, I have a third connection with one primary instance and three sub-instances through DTC. I am considering three instances of Sierra: (1) Denali only, for charts; (2) Rithmic for my main ES DOM; (3) secondary login for Rithmic (extra $ per month) for YM and NQ DOMs. This spreads out the processing over 3 instances and will likely give the best result.
I believe you will be better this way. A suggestion though: before paying for an extra Rithmic connection, you could try one of the three Denali allowed connections just for one of the DOMs. Hopefully you will find the best solution for what you need. If you feel like it, please post here the outcome of your future tests. It's always good to know what are the best available options. Regards :) Date Time Of Last Edit: 2024-02-15 22:07:00
|
| [2024-02-15 22:08:58] |
| Rui S - Posts: 190 |
|
User150671, Thank you for sharing. |
| [2024-02-15 23:05:01] |
| joshtrader - Posts: 488 |
|
Rui, after giving it some thought, I've decided to just use Denali for both instances. The DOM responsiveness is great with Rithmic, but there are two problems: (1) having yet another (3rd) instance of SC to just host one DOM means I can't quickly recenter my DOMs which I do all day long. I have ES, NQ, and YM side by side (used to have others too) and being able to recenter all is a big part of my workflow. I guess an autohotkey script could do this but it's just one more piece that must be in place for core functionality and not something I want to have to worry about. (2) even with only ES and only one DOM in an instance, at the close today I noted about 3 seconds of delay. This is not a huge deal because I would never trade the close like this, but I know that the same thing could happen during NFP or CPI or FOMC and I just don't want to have to worry about even the possibility that my data feed is lagging by more than a second or so. To be clear, I think the issue here is that SC cannot process the huge flood of data from Rithmic. But when using Denali+Rithmic, my understanding is that the Rithmic connection is only used for execution, not data, so, when an order is placed with Rithmic in this case, it does not matter that SC would be behind processing Rithmic data, because there are no market data updates from Rithmic. I have not noticed any trade execution issues with my current Denali+Rithmic setup. Previously when using only Rithmic during volatility I would sometimes see that with the market at X, I would place a bid at X+4 ticks or so, but it would not fill because the market was already higher, yet the lagging data processing made it appear that the market was lower. So, even though Denali is not as snappy and does not update as frequently, at least it does not fall behind for me, which is most important. So, my plan now is currently to just use Denali on one instance for charts, and Denali+Rithmic on the other for execution, and it will have to be just good enough for now. One change I made (which would not be make sense given my hypothesis that it's the processing of the data, not the updating of the DOMs, which causes the delay) was on the NQ DOM which I may change back now that I'm going to Denali+Rithmic. Previously I aggregated NQ to 0.50 ticks to avoid showing the quarter ticks, and also aggregated depth levels to show a single depth per 0.50. I can imagine that with numerous depth updates in NQ, even with a smart algorithm, it's still at least a few hundred/thousand more instructions and can't help. So, I changed this back to the default 0.25. But since I'm going back to Denali, I think I'll make it how it's better for me trading, which is 0.50. I remember in past years, like, 10+ years ago, that the DOM was always responsive and snappy like it usually is with Rithmic data today. I think one difference today is likely that the number of market data messages has simply gone higher, so during the open/close/volatility, SC has so many more messages to process, and we get the lag. I have previously worked on low latency data feeds and from an engineering perspective, it is not easy to reduce the latency, and depending on the delivery mechanism (in this case the Rithmic API) it may be impossible to split the data in a way that makes it faster. The only thing I can think to try is a 3rd party data feed like IQFeed to see if the updates are snappier than Denali, but I suspect we'd run into the same issue, which is that if the updates are very frequent, SC simply won't be able to deliver the updates without lag. At any rate, I'm going to shift focus on good trading and try to leave the tech part behind for now :) Date Time Of Last Edit: 2024-02-15 23:22:33
|
| [2024-02-15 23:47:20] |
| joshtrader - Posts: 488 |
|
You can add the Data Delay to the Trade DOM Header to show data delay easily (although it is kinda long at 13 characters 'DD: -00:00:00' and really only need the last 4 of those digits).
Thank you User150671, did not know about this and have added it! |
| [2024-02-16 03:43:29] |
| Sierra_Chart Engineering - Posts: 17174 |
|
I think the issue here is that SC cannot process the huge flood of data from Rithmic. No this is definitely not the case. Sierra Chart is extremely efficient with market data processing. The amount of data that users process is nothing compared to the amount of market data processed on our servers by Sierra Chart, which is going to be thousands of times higher.For example, a single instance of Sierra Chart is processing the entire NASDAQ TotalView data feed which is an order by order data feed of every order going through NASDAQ. This is all done without any delay. The processing occurs on a single thread. Buffering of incoming data though occurs on another thread. The entire CME with market by order data, is also processed in a single instance of Sierra Chart, across multiple threads. Two threads per CME channel. Excluding CBOT, COMEX and NYMEX and options. So, even though Denali is not as snappy and does not update as frequently,
which would not be make sense given my hypothesis that it's the processing of the data,
You can definitely get very fast updates, refer to:Chart Trading and the Chart DOM: Improving Performance Of Chart / Trade DOM Also refer to this video as a point of reference: Severe lag on newer versions | Post: 364300 IQ Feed is definitely not going to help you. The data is converted to text by IQ Feed, we then have to convert it back to numbers, and it is processed on the primary thread. It is an antique. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-02-16 03:45:07
|
| [2024-02-16 03:46:56] |
| Sierra_Chart Engineering - Posts: 17174 |
|
The DTC implementation is a possible explanation, but not completely sure: I am moving away from Rithmic because it DOES lag significantly on a regular basis. I have seen over 20s delay with 1x ES DOM through Rithmic compared to a full suite of DOMS through Denali or CQG.
I am not sure if this is an issue with their DTC server implementation, Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-02-16 03:47:18
|
| [2024-02-16 15:48:59] |
| Rui S - Posts: 190 |
|
Hi Josh, Thank you again for your detailed explanation. So, even though Denali is not as snappy and does not update as frequently, at least it does not fall behind for me, which is most important. So, my plan now is currently to just use Denali on one instance for charts, and Denali+Rithmic on the other for execution, and it will have to be just good enough for now.
Yes, we have to compromise in order to get the best outcome possible. I think one difference today is likely that the number of market data messages has simply gone higher, so during the open/close/volatility, SC has so many more messages to process, and we get the lag.
Absolutely! Specially in the volatile times, the volume / number of trades is much higher nowadays. At any rate, I'm going to shift focus on good trading and try to leave the tech part behind for now
Yes, I'm trying to the same. Have a good trading :) |
To post a message in this thread, you need to log in with your Sierra Chart account: