Home >> Support Board

Support Board

Date/Time: Sun, 08 Sep 2024 00:40:22 +0000

How to Post a Message for Support

Market Replay vs. Live Simulation issue

View Count: 203

[2024-07-10 21:53:27] #1 [Go To First Post]
User414533 - Posts: 106
Market Replay has given fairly consistent results for me when compared to Live Trading. But Live Simulation/Demo is showing inconsistencies. Do you have any guidelines regarding setting up Live Simulation instances so max accuracy is obtained? I've spent a ton of time reading through the documentation, and all I've uncovered so far is changing Global Settings / Data/Trade Service Settings / Intraday Data Storage Time Unit, to 1 Tick. But I didn't know if that setting is just for replays, or if it actually affects Live Sim data also. I also have the Chart Update Interval = 10ms The problem I'm having is that limit orders are filling on live trades, as well as market replay trades, but on Live Sim, some are "missed". A little more backstory: I'm running many automated algos through separate instances on a dedicated server. I'm also wondering if there is some CPU lag causing trades to be missed. The server is fairly robust: Xeon E-2288G CPU @ 3.70GHz w/ 128 GB of ram. The CPU utilization rarely gets above 50%. If changing the data storage time unit to 1 tick is applicable in my situation, I can try that. Just wanting a little advice on anything else that may be causing these skipped orders. Also, I'm NOT running low time-frames.

[2024-07-10 22:23:38] #2 [Go To First Post]
John - SC Support - Posts: 34254
If we are understanding the situation correctly, then this should not be occurring. So we think we are missing something. We assume by "Live Simulation" you mean that you have "Simulation Mode On" enabled and you are NOT running a replay. You are just getting the data as it comes into the system. What are the exact orders in which they are not filling in this case? Are these Attached Limit orders to close a position, or is it a primary Limit order waiting to open a position? Are you using the built-in simulation mode or are you using the "Trading Evaluator" service? Please look back at your Trade Activity Log to a time when one of these situations arose, and take a look at the "Orders" around that time and see if there is any information in the Trade Activity Log as to why an order did not fill when expected. The "OrderActionSource" should have information if there is a reason why it did not fill. For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing

[2024-07-10 23:27:50] #3 [Go To First Post]
User414533 - Posts: 106
We assume by "Live Simulation" you mean that you have "Simulation Mode On" enabled and you are NOT running a replay. You are just getting the data as it comes into the system. This is correct. What are the exact orders in which they are not filling in this case? Are these Attached Limit orders to close a position, or is it a primary Limit order waiting to open a position? Primary limit to open. Are you using the built-in simulation mode or are you using the "Trading Evaluator" service? Built-in simulation mode. Attached are a couple screenshots illustrating the issue. The 1st screenshot entitled, "MarketReplay...", is the correct trade result. It's been verified manually on a chart and is what will occur on a market replay or live trading scenario. The 2nd screenshot, "LiveSim..." is the algo missing the trade (and taking the opposite as per the code logic) in a "Simulation Mode On" scenario. The 3rd screenshot (SignalFail) is tick data that appears to have been "missed" during live simulation which would cause the trade errors that I witnessed. Date Time Of Last Edit: 2024-07-10 23:29:10


Private File Private File

[2024-07-10 23:28:42] #4 [Go To First Post]
User414533 - Posts: 106
I was limited to only 2 files so I included it here.


Private File

[2024-07-11 13:39:48] #5 [Go To First Post]
John - SC Support - Posts: 34254
The 2nd screenshot, "LiveSim..." is the algo missing the trade (and taking the opposite as per the code logic) in a "Simulation Mode On" scenario. What do you mean by "and taking the opposite as per the code logic"? What is the particular order in the Trade Activity Log? The order at 13.02.30.776668 is accepted and then filled at 13:15:18.259360. So we are not seeing anything wrong from the perspective of the log. For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing

[2024-07-11 16:32:45] #6 [Go To First Post]
User414533 - Posts: 106
What do you mean by "and taking the opposite as per the code logic"? What is the particular order in the Trade Activity Log? The order at 13.02.30.776668 is accepted and then filled at 13:15:18.259360. So we are not seeing anything wrong from the perspective of the log. The SignalFail screenshot shows price breaking above that blue line. That "signal" in the logic should have eventually triggered a short on Live Simulation, like it did/does on Market Replay and Live Trading (shown on MarketReplay screenshot). There was some kind of lag happening and I don't believe price breaking above that blue line registered. Oh, and my server is in Aurora, so there's very little lag between my server and the exchange. Does changing Global Settings / Data/Trade Service Settings / Intraday Data Storage Time Unit, to 1 Tick affect live simulation responsiveness to tick data like it does in Market Replay?

[2024-07-11 17:25:18] #7 [Go To First Post]
John - SC Support - Posts: 34254
We are very confused at this point. How are your orders being generated? Is this not a working Limit order that was placed manually and then not filled? You state there is a "Lag". So is the order getting filled, just not as fast as you were expecting? For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing

[2024-07-11 18:01:12] #8 [Go To First Post]
User414533 - Posts: 106
No, this is not a manual order. It's automated. The algo has been extensively tested live and in market replay. I'm just trying to run it continually in Live Demo/Simulation for various reasons. I'd like to make sure I've got everything set up correctly in Sierra Chart to eliminate any potential data delays. There is no lag in market replay because the "market" is inside my own machine. VERY little lag in live trading as well (on another server). Maybe I could ask the question a different way. Is the Denali data feed for Simulation the exact same feed/quality for live trading? If it is, then I might be having a server issue... You state there is a "Lag". So is the order getting filled, just not as fast as you were expecting? In that screenshot, showing price breaking above the blue line. If that "fairly quick" price surge had been received by Sierra Chart, a limit order in the opposite direction would have been eventually sent to be filled. The limit order was NEVER sent. So I am assuming it was never seen/received by SC. And as you can see by the screenshot, this wasn't just a wick on a candle, it was several seconds, and hundreds of orders transacting above that blue line. So the break was clear. And just to provide further confirmation that the first signal was missed, the eventual limit order filled was a long. The ONLY way that could ever happen with the logic of my algo, is for the first signal (the break above the blue line) to be missed. Sorry for getting deep into my algo stuff. But I'm certain this wasn't my code. Either I have overlooked a setting on SC, or there's a different data feed for live vs simulated trading, or my "big fancy" new server is overloaded and I have to rethink my architecture; ie. spend more money :(.

[2024-07-11 21:31:36] #9 [Go To First Post]
John - SC Support - Posts: 34254
Maybe I could ask the question a different way. Is the Denali data feed for Simulation the exact same feed/quality for live trading? If it is, then I might be having a server issue... Yes, absolutely. There is no difference between when you are connected to your live account or in Simulation Mode. Let's step back from all of this. We need to look at the particular situation that occurred. Please answer these questions: - Was there an open order at the time that the algorithm should have fired when it crossed the blue line? - What kind of order should have been placed when the price hit the blue line? - Was an order generated of the correct type? - Do you know if your algorithm did what it was supposed to do? Or did it not fire at all? For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing

[2024-07-11 23:10:32] #10 [Go To First Post]
User414533 - Posts: 106
Yes, absolutely. There is no difference between when you are connected to your live account or in Simulation Mode. Thanks for this! Was there an open order at the time that the algorithm should have fired when it crossed the blue line? Not on this particular instance. And this is where I should provide more backstory: I'm running many dozens of sub-instances of SC across 3 separate installs. While there were no orders open on this specific sub-instance, there absolutely were other orders, maybe even around the same time, on other instances. I was going to start a new support post asking for guidelines on the best way to run many sub-instances in the most resource-efficient way possible. I'm currently reading the documentation on performance and high-CPU load. I don't have any studies running beyond my algo. And each instance has 3 charts. My algo has a "63" behind it, which if my memory serves, is a rating of how much time it takes to load? I've been talking to server administrators looking for a customized solution in order to run a bunch of SC instances in as efficient a manner as possible. There was questions about the majority of the processes running through a single core causing me to theoretically always have one instance waiting on another to "take the trade". I also just finished reading the performance/feature section of the documentation which plainly states that multiple instances should be taking advantage of multiple cores. The server I'm running utilizes a 2288G CPU w/ turbo boost (5GHz) permanently enabled. It has 8 cores and 16 threads. I also have 128GB of ram. I barely use 15GB's of memory, but I can run up to 50% of the CPU % during the day. What kind of order should have been placed when the price hit the blue line? No order. But because the threshold was hit, the algo should have been looking for a short if the next criteria is satisfied. And it was. Was an order generated of the correct type? No type of order was generated. But as demonstrated in live (on another server), and market replay (on my machine), the correct timing and order type occurs. Do you know if your algorithm did what it was supposed to do? Or did it not fire at all? It did exactly what it was supposed to do, if price never eclipsed the blue line, which we can see it absolutely did. Date Time Of Last Edit: 2024-07-11 23:13:06

[2024-07-12 14:22:17] #11 [Go To First Post]
John - SC Support - Posts: 34254
What kind of order should have been placed when the price hit the blue line? No order. But because the threshold was hit, the algo should have been looking for a short if the next criteria is satisfied. And it was. This has us confused again. When the price hit the blue line, what should your algorithm have done? For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing

[2024-07-12 17:28:41] #12 [Go To First Post]
User414533 - Posts: 106
This has us confused again. When the price hit the blue line, what should your algorithm have done? That particular algorithm should have noted the break above the blue line (Step 1), then taken a short when the next criteria is satisfied, which it was. We'll call that Step #2. Since the short wasn't taken at Step #2, it became Step #1. Then a long was taken at what would be Step #3. Everything got out of order on this particular algo because the break above the blue line was never recognized. And again, the algo works perfectly, every time, on a market replay over the same area of concern. And since these trades are occurring over higher timeframes, there really shouldn't be that much difference between market replay and live trading results, which my records show there hasn't been. I'm just hoping to glean some tips or tweaks, I may have missed on getting these little guys running as efficiently as possible. Maybe some tips on ideal server/CPU configurations for running Sierra in the way that I am would be helpful as well.

[2024-07-12 18:51:42] #13 [Go To First Post]
John - SC Support - Posts: 34254
In terms of best configurations, refer to the following: Chart Trading and the Chart DOM: Improving Performance Of Chart / Trade DOM High CPU Usage \| Inactive User Interface \| Poor Performance \| Long Time to Load Chart Data \| Charts Reloading Often: 30.48 - Basic Steps to Resolve Performance Issue in Sierra Chart -- But everything you are stating sounds like an issue with your algorithm. We understand what you are saying about it working with live trading, but there has to be something about it that would be giving this difference. You really need to analyze what is occurring at that moment and see why you are not getting what you expect. For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing

[2024-07-12 20:49:13] #14 [Go To First Post]
User414533 - Posts: 106
We're gonna add verbose logging back to a few of the algos on the server and hopefully catch another aberration in the act. Thanks for the support!

To post a message in this thread, you need to log in with your Sierra Chart account: