Support Board
Date/Time: Sun, 24 Nov 2024 04:28:43 +0000
Replay tick data changes week to week
View Count: 275
| [2024-06-03 17:43:05] |
| User414533 - Posts: 106 |
|
I've been monitoring this "issue" for a while in hopes that I could post a clear support request. At the end of the week, I run market replays on the - just completed - week's market data. When I run the exact same replays the following week, my algos sometimes take a trade they didn't the prior week, or won't take a trade they took the prior week. I am certain this isn't my algos because I'm running the replays with many algos and the exact same disappearing/appearing trade is occurring across dozens of repetitive backtests. A buddy of mine who's familiar with the data business said perhaps the data is getting "cleaned" and the replays I'm running the following week are with the cleaned data, negating some trades that were previously taken, or causing new trades to show up. I keep very good records of backtest results so when I run the EXACT same test a week later and a new trade shows up or disappears, it is glaringly obvious. I'm also a bit worried as to which is the "right" data. Once these changes to the data occurs, they don't occur again. Is the "2nd version" the correct one? Also, do you have any explanation as to what is happening? Here are a few dates I've recorded where changes in the data occurred from one week to the next: 3/20 - trade disappeared on following week's backtest 3/22 - trade disappeared on following week's backtest 3/25 - trade disappeared on following week's backtest 3/28 - new trade appeared on following week's backtest 4/02 - new trade appeared on following week's backtest 4/11 - trade disappeared on following week's backtest 4/12 - new trade appeared on following week's backtest 4/16 - trade disappeared on following week's backtest 4/26 - trade disappeared on following week's backtest Thanks! |
| [2024-06-03 22:30:00] |
| Sierra_Chart Engineering - Posts: 17154 |
|
This is impossible for us to answer. We can only refer you to: Automated Trading Management: Troubleshooting Automated Trading System Behavior The data will not change from week to week. Unless you have re-downloaded the data, and for some unknown reason there is a change with the data but that is not going to happen in the case of futures data or stock data. There can be slight variations with Forex and crypto data due to the fact that those are garbage feeds. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-06-05 21:49:15] |
| User414533 - Posts: 106 |
|
The data will not change from week to week. Unless you have re-downloaded the data, and for some unknown reason there is a change with the data but that is not going to happen in the case of futures data or stock data.
I am not "delete and redownloading" the data. I am running market replays on multiple strategies at the close of the trading week. Let's say I go back 6 months or so, then the next end of week (5PM Fri), I do the same thing, 6 months back. A few trades that would trigger the prior week do not now. And a few trades that didn't trigger in the prior week, do now. The important point to note, is that once a trade result has changed from the prior week, it does not change again. It appears very much to be a tick data "correction" of some kind that is likely occurring over the weekend with the market closed, for the prior week. Unfortunately, I can't wait until the end of the weekend to start my analysis because of scheduling constraints. All variables that could cause this have been accounted for on my end (famous last words I know). Same algo code run each week, many consecutive market replays showing exact same results ON THE SAME WEEK. But when the process is duplicated exactly the following week, there's always a trade or two that doesn't match. And this specific trade or two is confirmed through many repeated replay runs. I've noticed some of the time stamps have been around market close (15:59:xx etc). Anyway, I was just wanting some guidance if there is any. "Clean data" is a big thing in this business (as you know), and folks are always wary when results change and the only variable seems to be the data. One thing I haven't done yet is compare live trade data and live demo data with the market replay data over a two week period to gain more understanding of what is "real" and what is not. I will do that next at great expense to my hairline. :) Thanks! Date Time Of Last Edit: 2024-06-05 21:54:11
|
| [2024-06-06 22:03:53] |
| Sierra_Chart Engineering - Posts: 17154 |
|
One thing to be aware of is variable time frame chart bars can change when a chart is reloaded. And the reason for this is explained here: Working With Charts: Resolving Differences of Chart Bars Between Charts Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-06-24 20:58:46] |
| User414533 - Posts: 106 |
|
One thing to be aware of is variable time frame chart bars can change when a chart is reloaded. And the reason for this is explained here:
Working With Charts: Resolving Differences of Chart Bars Between Charts "One way to ensure consistency between chart bars between two different charts of the same symbol in the same installation of Sierra Chart, is to duplicate one of the charts. After duplicating one of the charts, reload the data in the original chart with Chart >> Reload and Recalculate. Both the original chart and the duplicated chart will now be identical." ---The occasional variances I am getting in trade results are on the exact same chart, instance, and algo. The ONLY variable is the time when the backtest is run, and it's one week later. After the trade results change, they don't change again. I can't see how this is anything other than changing tick data. "Ensure there is complete and consistent data for the charts. If you are comparing charts between different instances of Sierra Chart, check the Intraday Data Storage Time Unit setting. It needs to be set the same between instances. Re-download the data by going to the chart and selecting Edit >> Delete All Data and Download. If the charts are within the same instance of Sierra Chart, this only needs to be done for one of the charts. If the charts are in different instances of Sierra Chart, then this procedure needs to be followed for each instance. If you are using sub instances (File >> New Instance), perform this procedure first in the main/server instance of Sierra Chart and then in the sub instance." ---I have no problems with results consistency beyond the first week AFTER I run the backtests. I could run them a thousand times and the results would be identical. But when week "A1" passes, and I run a backtest at the end of week "A1" generating a trade history, then week "A2" passes and I run a backtest at the end of week "A2", the prior week's days get retested and that is where I will find an occasional descrepancy. From then on, the week in question's data is stable. So something is editing/changing the data used in market replays after the week in question's trading week closes on Friday. "The next step in order to make sure that you remove any data that has been loaded in the chart earlier than the Days to Load setting in Chart >> Chart Settings, which occurs over time, is to select Chart >> Reload and Recalculate for each of the charts being compared. This step will make a difference if the first Date-Time in the chart is different between the two charts you are comparing. You can see what this is by going to each chart and selecting Chart >> Goto Beginning of Chart and looking at the Date-Time of the first bar." ---All tests are being run off sub-instances and the discrepancy is not between instances, or between charts. It's on the SAME instance, on the SAME chart. "Go to one of the charts you are comparing, and examine all of the settings in Chart >> Chart Settings and compare them to the Chart Settings to the other chart being compared to see where the difference is with these settings. The settings on the following tabs can change chart bars: Symbol, Data Limiting, Bar Period, Session Times, Chart Data Also the following settings will cause differences and need to be the same between the charts: Days to Load New Bar at Session Start Combine Trades into Original Summary Trade" ---Everything is identical --- Same answer for 5, 6, 7, 8. --- If I were to run a backtest from, say... 1/1/23-6/1/24 a million times, I would get exactly the same results. But if I run a backtest from 1/1/23-6/28/24, at 5pm CST on 6/28/24, and then run that exact same backtest next week. I will see variations in the results over the week of 6/24. Any ideas on how I can get some consistency? I'm fully prepared to rent a server and run my algos live/demo to get correct, real-time tick data. Would you see any potential issues with doing that? Also, if you think the above idea would yield the results I'm looking for, could I connect to the Simulated Futures Trading Service with dozens of sub-instances in order to get even more realistic fills, or is there a limit to the # of connections a member can have? And/or is that service less reliable/robust than Denali? And finally, if you deem this appropriate for my request, is there an outside data service you would recommend that might cost a bit more, but would yield very, VERY clean data. This is an issue that has been hounding me for months now. I would be very grateful to put it to bed once and for all. Thanks! |
| [2024-07-03 13:50:22] |
| Sierra_Chart Engineering - Posts: 17154 |
|
The main question we have is whether the underlying bar data in the chart is actually different. We recommend exporting the data before the change is noticed: Edit Menu: Edit >> Export Bar and Study Data to Text File And then export it after you see the change. Use a program like this to see if there are differences between these text files: https://winmerge.org/ No issue: I'm fully prepared to rent a server and run my algos live/demo to get correct, real-time tick data. Would you see any potential issues with doing that?
Sub instances would not be directly connecting to the Simulated Futures Trading service: Also, if you think the above idea would yield the results I'm looking for, could I connect to the Simulated Futures Trading Service with dozens of sub-instances in order to get even more realistic fills, or is there a limit to the # of connections a member can have? And/or is that service less reliable/robust than Denali?
Only the main instance will. However, there are no limitations on the number of connections but there is a limitation on the number of connections for real time Data from exchanges. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-07-03 13:50:46
|
To post a message in this thread, you need to log in with your Sierra Chart account: