Support Board
Date/Time: Tue, 26 Nov 2024 11:32:39 +0000
Insufficient historical intraday data loading into TPO charts
View Count: 417
| [2023-12-01 02:38:35] |
| EverEdge - Posts: 15 |
|
Hello, I'm trying to load a year or longer of historical intraday data into TPO charts. I have done this successfully many times. In fact, the issue cropped up upon transferring the chartbook to a new system. I'm running 2564, 2 complete installations on the same machine, each with subinstances, if that matters. Trade service / data is set to Teton. I cannot seem to go back more than 30-45 days on TPO charts. I can select "400" for "days to load for intraday chart data type" under "data limiting", but it consistently loads about a month and change only. I've checked all the obvious things: - Bar period is set to intraday. Bar period value - have changed this everything from 1 min to 30 min - in my experience, on a TPO chart, this shouldn't matter anyway, but it doesn't make a difference. - Under Global Settings >> Data/Trade Service Settings >> Common Settings, made sure Maximum Historical Intraday Days To Download is set properly. - Re-downloaded all the data with Edit >> Delete All Data And Download, after disconnect/reconnect. Significantly, this behavior only seems to occur with non-futures symbols, e.g. it happens with SPY and QQQ but not with ES and NQ. The latter can load data going back years using the same global settings in the same instance. This leads me to suspect the data itself. 1. Is there some restriction that I'm not aware of where I cannot load equities data going back farther while connected to Teton? 2. I'm also subscribed to NASDAQ Total View and US Equities Consolidated Tape, but since to my knowledge, you cannot have 2 data feeds for the same SC installation and the current instance needs to trade futures, I'm using Teton connecting to an FCM. I did try SPY-NQTV as symbol instead of SPY. That does not go back any farther either. Also, is it really loading the NQTV feed when you call the SPY-NQTV symbol, although you're connected to Teton and not "SC Data - All services"? 3. I could run a version of this chartbook on a second machine but (correct me if I'm wrong) I'd have to pay the exchange fees all over again, even if I select "SC Data all Services" to access NQTV or US Equities Consolidated Tape rather than Teton. When I start SC on a second machine simultaneously, even if I connect to "SC Data - All Services", the data does not stream. It does connect (it connects to Teton as well), but the data does not update. I'm assuming that is because of the exchange restriction. 4. On a related note, can you 'synchronize' a chartbook between 2 machines by having software that detects changed files in the SC hierarchy and copies them across the network _if_ the software is smart enough to not mess with pointers on open files, maybe uses volume shadow copy in Windows, etc., since there will be a lot of data files open and being accessed/written to while an instance is running? In other words, are there any unique IDs or sessions or anything else associated with any unique SC instance or a chartbook and/or the data files themselves, or can they all be copied and synced in near real-time like ordinary files across computers, thereby maintaining a "standby" or "warm spare" version of SC so that if the first computer went down, you could connect and trade through the second (not withstanding what would happen to open orders and server-side brackets)? Thank you! |
| [2023-12-01 16:24:31] |
| John - SC Support - Posts: 36286 |
|
1. There is no restriction. You should be able to get at least 1 year of data. Refer to the following: Historical Intraday Data (Missing Data / Download Errors / Viewing More Data): Downloading/Viewing More Historical Intraday Data 2. It does not matter what service you are connected to. If you have the exchange activated from our Denali Exchange Data Feed, then you will get that data. To get the NASDAQ TotalView data, you use the suffix of -NQTV, so SPY-NQTV will give you the NASDAQ TotalView data. Using an equity symbol alone will get you the data from the "U.S. Equities Consolidated Tape", such as 'SPY'. And if you have both of these exchanges active, then you can use the suffix of '_MBO' to get the combined U.S. Equities Consolidated Tape with the Market Depth from the NASDAQ TotalView exchange. 3. If you use a second computer, then you would need to pay the exchange fees a second time in order to get the data to both computers at the same time. 4. We do not recommend using any kind of backup or sync'ing software to copy files. The Sierra Chart files are constantly being written to, so if a file is replaced on one system while Sierra Chart is running, it could create a problem and you could lose your data. You can copy the files, but you have to ensure that Sierra Chart is not running. For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2023-12-02 00:11:56] |
| EverEdge - Posts: 15 |
|
I figured it out. Because intraday storage time is set to 1 tick, it was using the 1-tick data setting under maximum historical days to download. I had set this to 30 days. Curiously though, the futures data was able to go back well over a year despite having this global setting in place. I had lowered that to 30 days following instructions at another link that has a laundry list of potential performance improvements, because I'm running 2 full installations on the same desktop, both with subinstances. And while I was able to bring the SC CPU usage down to the 1% range for each instance, there are other mysterious performance issues without any evidence of CPU, memory, or I/O problems. Running process analyzers and a packet sniffer. For example, opening chart studies (F6) is instant, but double-clicking on an individual study has a 5-6 second lag to pull up the study. Right-click menu lags similarly. Meanwhile, while that's happening, total CPU usage on the system is in the teens, memory under 40%. And the data comes in _very_ fast - I can set the chart update interval to as little as 40 ms and see a noticeable difference in DOM/heat map over 100 or 300 ms, etc., i.e. noticeably faster. Current price line 'flutters' the same way it does when you're connected directly to the CME at Aurora. OpenGL, distributing across more than one GPU/video card, etc. have all been tried and helped somewhat. But what has helped out of all proportion to anything else has been (1) switching some profile charts to blocks (from letters), (2) decreasing the # of days loaded into charts, (c) having fewer detached charts spread across fewer monitors, and (d) increasing VbP tick increment. Hope this helps others. Since the "number of days loaded" is on that list of most effective performance improvements, I don't necessarily want to take it to 186, 252 or 500 days and make things worse. But thank you for your help John. Every time I ask a question, I learn something else useful I didn't know about SC. |
To post a message in this thread, you need to log in with your Sierra Chart account: