Support Board
Date/Time: Sat, 23 Nov 2024 05:34:10 +0000
Post From: Daily/Intraday data reconciliation failure / ES data quality problem
| [2024-09-15 12:43:42] |
| User512353 - Posts: 42 |
|
SC2677 Symbol: ESZ24-CME Data/Trade service: SC Data Intraday Data Storage Time Unit: 1 Tick Preliminary note: I do use a different definition of "day", i.e. my cut-over point is 17:00.00 CST/CDT, whereas the default is 16:15.00 EST/EDT. However, this does not explain the data reconciliation problems below. ESZ24-CME data was freshly downloaded today, 2024/09/15, beginning around 10:52h UTC (file time: ESZ24-CME.dly=10:53:01 UTC, ESZ24-CME.scid=11:32:36 UTC). ESZ24-CME.dly starts 2021/06/04 ESZ24-CME.scid starts 2023/09/27 (all scid data as observed from the intraday candle chart (period:1sec)) The data for the day 2023/09/27 in dly and scid is consistent. Examples of inconsistent data: a) The next day after 2023/09/27 with nonzero volume data is: dly:2023/10/03 - scid:2023/10/25. 9 days with nonzero dly volume are not present in scid. b) 2023/11/03: dly:volume=116 - scid:volume=11. c) 2024/03/07: dly:volume=437 - scid:volume=13. d) 2024/05/24: dly:volume=590 - scid:volume=29. e) 2024/07/12: dly:volume=1107 - scid:volume=467. f) 2024/08/05: dly:volume=16392 - scid:volume=1908. g) 2024/09/12: dly:volume=264608 - scid:volume=290546. These are just examples. Of course I did not check all the data. But spontaneously I could not find ANY day with matching volume data. So, where do these discrepancies come from?? I also experimented with setting the intraday session time to 00:00.00-23:59.59. For example g) above this results in a 2024/09/11 volume=58674, 2024/09/12 volume=116096, and 2024/09/13 volume=286858. This does not reconcile either. Is there any explanation for this? What volume data can we trust here??? |