Support Board
Date/Time: Wed, 05 Mar 2025 04:24:21 +0000
Post From: Issues with DTC Historical Data Requests
| [2022-01-07 19:16:11] |
| User41727 - Posts: 124 |
|
When using the historical data server via DTC with Protobuf encoding in Python, requests can terminate unexpectedly and erroneously. This applies to version 2339 as well as earlier ones. Steps to reproduce: 1) Go through the DTC connection procedure. 2) Request (daily) historical data for any symbol, e.g. SPY for the last 10 years (HISTORICAL_PRICE_DATA_REQUEST) 3) Bars are being returned via HISTORICAL_PRICE_DATA_RECORD_RESPONSE until a point is reached where the DTC server sends an incomplete payload, i.e. the header type is correct, but when reading the data from the socket, fewer bytes have been sent than indicated by the header, resulting in an inability to parse the packet. The aforementioned issue is reproducible every time, but only seems to occur after more than 1000 bars have already been sent. In most cases it took between 1300 and 1400 bars independent of the starting point. The issue can be avoided by not requesting more than 1000 bars per request. However, there appears to be a second issue where the "EndDateTime" setting is being ignored (MaxDaysToReturn = 0), so the server is always reaching the error point if StartDateTime lies too far in the past. The only solution for this is to disconnect the client before that happens and then start new requests for any remaining data in chunks of 1000 bars. |