Support Board
Date/Time: Sun, 05 Jan 2025 23:37:00 +0000
Depth market data for backtesting
View Count: 201
| [2024-12-11 17:19:07] |
| User377189 - Posts: 198 |
|
What is best way to download and replay depth of market data for the entire year? I primarily use DOM and no charts so would love to replay the DOM for any day |
| [2024-12-11 19:35:52] |
| John - SC Support - Posts: 36980 |
|
The general information for how to replay Market Depth data is at the following link: Replaying Charts: Replaying of Market Depth Data But we only have historical Market Depth data for 6 months. So if you did not record the data before the most recent 6 months, then there is no data available for that prior time. Refer to the following for how to setup and download historical market depth data: Market Depth Historical Graph: Downloading Of Historical Market Depth Data Also, refer to the information for the following setting that will determine how much Market Depth data you can download: Sierra Chart Server Settings: Maximum Historical Market Depth Days to Download (Global Settings >> Sierra Chart Server Settings >> General >> General) And finally, if you are using a Continuous Futures Contract, then the Market Depth data does not get downloaded for anything other than the current front contract. To get data for previous contracts, you would need to open a chart specifically for that contract and download the historical Market Depth data through that chart. Then it will be available in the Continuous Futures contract. For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-12-11 20:10:06] |
| User377189 - Posts: 198 |
|
I have set all my settings as per above I only have DOM's so does that mean that market depth data is automatically downloaded? And i can access it going forward anytime. This is what i understood from the following line in the documentation "When the Market Depth Historical Graph is on a chart, the Market Depth data is automatically stored." If 6-months of market depth data is stored i can access the recent 6-months data today? Also what happens post 6-months does the data get replaced by new 6-months or can i store this data for myself so that i can have a years data i can retrieve it later? For continuous contract if i setup regular charts (no DOMs) then even for previous expired contract i am getting message that market depth data will be downloaded starting at 2024-12-09 and not earlier than that (see attached screenshot) |
Private File |
| [2024-12-11 21:24:20] |
| John - SC Support - Posts: 36980 |
|
I only have DOM's so does that mean that market depth data is automatically downloaded? And i can access it going forward anytime. This is what i understood from the following line in the documentation
"When the Market Depth Historical Graph is on a chart, the Market Depth data is automatically stored." Since you only are using DOMs you do not have the "Market Depth Historical Graph" study on the DOM, therefore the Market Depth data is not automatically stored. You would need to setup the symbols for your DOMs to record the Market Depth Data, or you would need to rely on downloading the historical market depth data we have available (refer to 3rd bullet in the same section you referenced). If 6-months of market depth data is stored i can access the recent 6-months data today? Also what happens post 6-months does the data get replaced by new 6-months or can i store this data for myself so that i can have a years data i can retrieve it later?
Yes, you can access the 6 months of data today. It is a rolling 6 months that is stored, so as a new day is added the oldest day is removed. So it is always 6 months of data going back from today. You will need to store the data yourself as noted above and in the section for replaying the Market Depth data. Once you have the older data, then you can replay 12 months with the Market Depth data. For continuous contract if i setup regular charts (no DOMs) then even for previous expired contract i am getting message that market depth data will be downloaded starting at 2024-12-09 and not earlier than that (see attached screenshot)
Refer to the last two paragraphs in Post #2 above. For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-12-12 01:57:09] |
| User377189 - Posts: 198 |
|
For backtesting DOM and Time and sales do i need the market depth historical chart or this is not required? I have set my symbols as per this part of the documentation ---> "Market Depth data can be forced to be stored as it is received by setting the Symbol Settings for a particular symbol to have the option for Record Market Depth Data set to Yes" If the symbol settings have been changed to record market depth data, does the data start recording from day it was set to yes and for prior months i will need to load chart with study market depth historical chart? Do i need to change symbol setting to record market depth data in both instances individually or does it carry forward to second instance? Date Time Of Last Edit: 2024-12-12 02:39:21
|
| [2024-12-12 15:10:17] |
| John - SC Support - Posts: 36980 |
|
For backtesting DOM and Time and sales do i need the market depth historical chart or this is not required?
You do not need the Market Depth Historical Graph for any of this. If the symbol settings have been changed to record market depth data, does the data start recording from day it was set to yes and for prior months i will need to load chart with study market depth historical chart?
The data starts getting recorded from the time that you made the change. In order to get prior data, you need to follow the instructions at Step #8 at the following link: Market Depth Historical Graph: Downloading Of Historical Market Depth Data You do not need the Market Depth Historical Graph to be on the chart in order to download the market depth data. Do i need to change symbol setting to record market depth data in both instances individually or does it carry forward to second instance?
If this is a separate installation, then you will need to update the symbol settings for it as well. If this is a sub-instance, started by selecting "File >> New Instance", then as long as the setting for "Global Settings >> General Settings >> General >> Use This Instance Symbol Settings In New Instances When Started" is set to "Yes", then the symbol settings will be automatically copied to the sub-instance when it is started. General Settings Window: Use Symbol Settings in New Instances when Started (Global Settings >> General Settings >> General >> New Instances) For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-12-12 22:30:13] |
| User377189 - Posts: 198 |
|
See attached i selected ES to backtest for date 2024-04-19. I see the market depth data is downloaded by when i replay i am not seeing the market depth it just shows 1 across bid and ask and also does not show my studies. studies not showing: - Volume profile is not visible - open price study - this draws a blue line at open price - opening range study - there are two lines that are drawn at 60 second after open to show the high and low formed in that time-frame and other study related to opening range high and low |
Private File |
| [2024-12-12 23:20:05] |
| John - SC Support - Posts: 36980 |
|
Your issue is that you do not have enough days loaded in the DOM to cover the time from when you want to start the backtesting/replay. You are starting on April 19, 2024 so you need to make sure you have enough days loaded in the DOM to go back to at least that far back. It would be easier to use the "Date Range" for the "Load Data Limiting Method" and then set the "Date Range From" to "April 18, 2024" to ensure you have at least the data going back far enough. Refer to the following: Replaying Charts: Replaying Trade DOM Windows For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-12-13 14:43:27] |
| User377189 - Posts: 198 |
|
Good catch. I created a second instance for sierra chart for back-testing. I am assuming all settings i.e. symbol settings for recording market depth etc. is carried forward. Will the market depth data be stored automatically or do i need to open the second instance everyday? Also while back-testing will the quote board also show data as they day it was on the simulated day? |
| [2024-12-13 14:57:50] |
| John - SC Support - Posts: 36980 |
|
I created a second instance for sierra chart for back-testing. I am assuming all settings i.e. symbol settings for recording market depth etc. is carried forward.
If this is a Sub-Instance (started with File >> New Instance) then all the settings as they were in the main instance are copied to the sub-instance at that time. From that point forward, the sub-instance is completely separate from the Main Instance, so changes you make in one will not affect the other. There are options to force the Global Settings and Symbol Settings to sub-instances from the main instance when the sub-instance is started. Refer to the following the setting below it: General Settings Window: Use Symbol Settings in New Instances when Started (Global Settings >> General Settings >> General >> New Instances) If this is a completely separate installation, then there is no connectivity between the installations. You would have to copy files manually if you want to keep them the same. Will the market depth data be stored automatically or do i need to open the second instance everyday?
You will have to open the sub-instance every day to store the Market Depth data, just like the main instance. Again, these instances are completely separate in terms of the stored data. Also while back-testing will the quote board also show data as they day it was on the simulated day?
The Quote Board does not replay. So no, it will not show the data as it was on that day. It only shows the current information. For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-12-14 03:43:03] |
| User377189 - Posts: 198 |
|
Even after loading sufficient days the DOM is not showing bid/ask properly, see attached
|
Private File |
| [2024-12-14 16:48:26] |
| Tr00pz - Posts: 12 |
|
Don't mean to hijack the post, just had a relevant question to storing historical market depth data that may also be useful to the original poster. Each trade service you download data from stores the data file separately, though they all are using Denali data. Teton CME Order Routing - NQZ24_FUT_CME.2024-12-13.depth SC Data - All Services - NQZ24-CME.2024-12-13.depth Rithmic Direct- DTC - NQZ4.CME.2024-12-13.depth I'm planning on sticking with SC for many years and would like to store depth data past the 6 month rolling limit so should everything be stored using Teton? It's unfortunate that Denali data can't have a unified symbol for storage purposes regardless of what trade service you are connected to when it's downloaded, I'm sure you would have done so if it was possible. Thank you for your time. |
| [2024-12-14 18:54:12] |
| Sierra_Chart Engineering - Posts: 17831 |
|
Regarding post #11, it is very difficult troubleshooting this issue on a Trading DOM. Do the replay, on a standard chart. And check the setting described in step 7 here: Market Depth Historical Graph: Market Depth Historical Graph Not Displaying Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-12-14 18:54:29
|
| [2024-12-14 18:55:29] |
| Sierra_Chart Engineering - Posts: 17831 |
|
Regarding post #12, you can use either or both of these services for historical market depth data: Teton CME Order Routing - NQZ24_FUT_CME.2024-12-13.depth SC Data - All Services - NQZ24-CME.2024-12-13.depth Definitively do not use Rithmic symbols. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-12-16 16:47:40] |
| User377189 - Posts: 198 |
|
I did the following but still do not see the chart replay "Regarding post #11, it is very difficult troubleshooting this issue on a Trading DOM. Do the replay, on a standard chart. And check the setting described in step 7 here: Market Depth Historical Graph: Market Depth Historical Graph Not Displaying" See attached |
Private File |
| [2024-12-16 19:23:30] |
| John - SC Support - Posts: 36980 |
|
You are going back in time to where there was nearly no trading for the symbol, so there may not be any Market Depth data to display from that time. Move forward to where more trading was occurring and see what shows.
For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
To post a message in this thread, you need to log in with your Sierra Chart account: