Support Board
Date/Time: Mon, 25 Nov 2024 20:37:41 +0000
Post From: Strange behaviour when feeding SCID using Intraday Storage Time Unit greater than 1 tick
| [2024-02-01 19:24:15] |
| weav - Posts: 30 |
|
Hi I am feeding scid files via a python program. The Intraday storage time unit is 1 minute on my instance of SC. I write the first record after the minute barrier has passed and then I update that same record with subsequent ticks until the next minute barrier when I write a new record again. On the updates I change only the high (if required) the low (if required), the last, the number of trades, and the volume. I don't touch the open or timestamp, the remain they same. When I export the underlying data it is normal for 1 minute storage time unit, e.g. 2024/2/1, 08:38:00, 14.13, 14.13, 14.12, 14.12, 6, 6, 0, 0 2024/2/1, 08:39:00, 14.11, 14.11, 14.11, 14.11, 6, 6, 0, 0 2024/2/1, 08:40:00, 14.11, 14.11, 14.10, 14.10, 7, 7, 0, 0 2024/2/1, 08:41:00, 14.10, 14.12, 14.10, 14.12, 7, 7, 0, 0 2024/2/1, 08:42:00, 14.12, 14.12, 14.11, 14.11, 8, 8, 0, 0 2024/2/1, 08:43:00, 14.11, 14.11, 14.11, 14.11, 5, 5, 0, 0 2024/2/1, 08:44:00, 14.11, 14.11, 14.10, 14.10, 5, 5, 0, 0 2024/2/1, 08:45:00, 14.10, 14.10, 14.09, 14.10, 8, 8, 0, 0 2024/2/1, 08:46:00, 14.10, 14.10, 14.09, 14.10, 4, 4, 0, 0 However if export BAR data to a text file I get this, note the extra 1 minute bar being added with prior timestamp after each new bar: 2024/2/1, 08:40:00, 14.11, 14.11, 14.10, 14.10, 7, 7, 0, 0 2024/1/31, 21:59:00, 14.35, 14.35, 14.35, 14.35, 0, 1, 0, 0 2024/2/1, 08:41:00, 14.10, 14.12, 14.10, 14.12, 7, 7, 0, 0 2024/1/31, 21:59:00, 14.35, 14.35, 14.35, 14.35, 0, 1, 0, 0 2024/2/1, 08:42:00, 14.12, 14.12, 14.11, 14.11, 8, 8, 0, 0 2024/1/31, 21:59:00, 14.35, 14.35, 14.35, 14.35, 0, 1, 0, 0 2024/2/1, 08:43:00, 14.11, 14.11, 14.11, 14.11, 5, 5, 0, 0 2024/1/31, 21:59:00, 14.35, 14.35, 14.35, 14.35, 0, 1, 0, 0 2024/2/1, 08:44:00, 14.11, 14.11, 14.10, 14.10, 5, 5, 0, 0 2024/1/31, 21:59:00, 14.35, 14.35, 14.35, 14.35, 0, 1, 0, 0 2024/2/1, 08:45:00, 14.10, 14.10, 14.09, 14.10, 8, 8, 0, 0 2024/1/31, 21:59:00, 14.35, 14.35, 14.35, 14.35, 0, 1, 0, 0 And this is how my bars print on the chart. One good 1 minute bar, which is immediately followed by another "out of order timestamp" 21:59:00 bar from the day before. But that is not how my intraday data is being stored in scid. My python program definitely isn't adding the 21:59:00 bar as you can see from the first intraday data text export. If I disconnect/reconnect the data feed then the spurious extra 1 minute bar with prior timestamp that gets added after every new bar has the timestamp and price of the last bar before reconnection. i.e. if I disconnected/reconnected at 08:30, instead of the extra 1 minute bar having a timestamp of 21:59:00 it will now be 08:30:00 and the last price will be the price at 08:30:00. Then this 08:30:00 bar will be added after every new bar that is added by my program. Its like SC is storing the last known price prior to connection and then not allowing that to be overwritten by my program. Important to note: If I take the 1 minute scid file, and my 1 minute bar writer python program, and move it to another install of SC which uses Intraday storage time unit of 1 Tick, then there is no problem, no extra bars with prior timestamp. Chart is normal and 1 minute bars are correctly displayed. (as per my intraday text file export) So, it's something to do with Intraday Storage Time Unit greater than 1 tick that is causing an extra bar (with timestamp / price of last bar on chart at time of data feed connection) to be added after every new bar written by my program. Somehow (I think) SC is storing that last 1 minute bar at time of connection, and then adding it on the chart after every new 1 min bar is added to scid by my python program. Also something else to note: If i reload the chart all the extra bars disappear. They only occur when SC is updating the bar chart in real time. Does this ring any bells with any of the SC engineering team? I would love to know what causes it, and to be able to use 1 minute storage time unit setting in SC with my externally written scid files. Maybe for intraday time storage unit of greater than 1 tick this will only work if I feed SC via DTC? I've attached a chart to show you how this looks on a chart |
 2024-02-01_19-14-16.png / V - Attached On 2024-02-01 19:19:19 UTC - Size: 47.74 KB - 63 views |