Support Board
Date/Time: Thu, 28 Nov 2024 00:36:58 +0000
download more than 365 days of historical data from IBKR
View Count: 500
| [2023-05-25 21:38:48] |
| Jeff Li - Posts: 58 |
|
hello, I am trying to use IBKR as the data for the cboe cash indices. I have the maximum historical intraday days to download limit set to 1500.. I want data from the last couple of years. But I can only get up to 365 days of historical data. Intraday I am currently getting around 2 years worth. Any tips>? |
| [2023-05-25 21:38:53] |
| Jeff Li - Posts: 58 |
|
Software version: 2504 64-bit | 2023-05-25 17:36:01.409 Primary Thread ID: 44160 | 2023-05-25 17:36:01.409 Usage end date: 2023-06-08 | 2023-05-25 17:36:01.409 Enabled for: Advanced Features 2. | 2023-05-25 17:36:01.409 Enabled for: Sierra Chart Historical Data Service. | 2023-05-25 17:36:01.409 Enabled for: Denali Real-Time Exchange Data Feed. | 2023-05-25 17:36:01.409 Enabled for: Delayed Denali Exchange Data Feed. | 2023-05-25 17:36:01.409 Enabled for exchange2: CME (Trading Account Required) | 2023-05-25 17:36:01.409 Enabled for exchange2: CBOT (Trading Account Required) | 2023-05-25 17:36:01.409 Enabled for exchange2: COMEX (Trading Account Required) | 2023-05-25 17:36:01.409 Enabled for exchange2: NYMEX (Trading Account Required) | 2023-05-25 17:36:01.409 Enabled for exchange2: fairx_exchange.data | 2023-05-25 17:36:01.409 Enabled for exchange2: US Equities Consolidated Tape | 2023-05-25 17:36:01.409 Enabled for exchange2: NASDAQ Global Indexes | 2023-05-25 17:36:01.409 Allow Support for Sierra Chart Data Feeds is enabled. | 2023-05-25 17:36:01.409 Current selected Data/Trading service: Interactive Brokers | 2023-05-25 17:36:01.409 Custom symbol settings values: enabled | 2023-05-25 17:36:01.409 Chart Update Interval: 250 | 2023-05-25 17:36:01.409 Intraday Data Storage Time Unit: 0 | 2023-05-25 17:36:01.409 Time Zone: -04:00:00 (EST-05EDT+01,M3.2.0/02:00,M11.1.0/02:00) | 2023-05-25 17:36:01.409 2023-05-25 21:36:01 Local computer time in UTC | 2023-05-25 17:36:01.409 2023-05-25 17:36:01 Local computer time in SC Time Zone | 2023-05-25 17:36:01.409 2023-05-25 21:10:07 Server time in UTC | 2023-05-25 17:36:01.409 Local computer UTC time and Server UTC time difference: 0 seconds. | 2023-05-25 17:36:01.409 Program path: C:\SierraChart2\ | 2023-05-25 17:36:01.409 Data Files path: C:\SierraChart2\Data\ | 2023-05-25 17:36:01.409 OS Version Number: 10.0 | 2023-05-25 17:36:01.409 Locale Setting: C | 2023-05-25 17:36:01.409 DLLs: SmashCalcs_64.dll, UserContributedStudies_64.dll | 2023-05-25 17:36:01.409 Allowed protected custom studies: | 2023-05-25 17:36:01.409 Crash reporter started: true | 2023-05-25 17:36:01.409 Denali Data Feed | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=2, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=40, ServerSendBufferSizeInBytes=0, ActualMessageDelay=0.3 seconds, DataCompRatio=1.41, UncompressedBytes=1015, CompressionTime=0.000070(secs), NumCompressions=12, Source=10.10.20.141 | 2023-05-25 17:36:02.128 Denali Data Feed | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=4, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=40, ServerSendBufferSizeInBytes=0, ActualMessageDelay=0.3 seconds, DataCompRatio=1.50, UncompressedBytes=3010, CompressionTime=0.000239(secs), NumCompressions=55, Source=10.10.20.141 | 2023-05-25 17:36:02.128 SC DTC Data with Market Depth | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=13, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=40, ServerSendBufferSizeInBytes=0, ActualMessageDelay=0.0 seconds, DataCompRatio=1.15, UncompressedBytes=48178, CompressionTime=0.003890(secs), NumCompressions=1112, Source=10.10.20.81 | 2023-05-25 17:36:09.914 Interactive Brokers | Using primary service for historical data for SPX-IND-CBOE-USD | 2023-05-25 17:36:13.478 Added historical Daily data request for SPX-IND-CBOE-USD to the queue. | 2023-05-25 17:36:13.478 Triggering next historical data download in queue. | 2023-05-25 17:36:13.478 HD Request # 44 | Downloading Historical Daily chart data for SPX-IND-CBOE-USD. Starting date: 1973-05-25. Service: interactive_brokers.trading | 2023-05-25 17:36:13.478 HD Request # 44 | Requesting 365 days of historical Daily data for SPX-IND-CBOE-USD with end date 2023-05-25 | 2023-05-25 17:36:13.478 HD Request # 44 | Processing historical data message from TWS | 2023-05-25 17:36:13.709 HD Request # 44 | Receiving historical Daily data for SPX-IND-CBOE-USD starting at 2022-05-25. | 2023-05-25 17:36:13.709 HD Request # 44 | Writing historical Daily data to the file SPX-IND-CBOE-USD.dly. | 2023-05-25 17:36:13.709 HD Request # 44 | Received 251 records and wrote 251 records from 2022-05-25 00:00:00 to 2023-05-24 00:00:00. | 2023-05-25 17:36:13.710 HD Request # 44 | Received 251 Daily data records from 2022-05-25 00:00:00 to 2023-05-24 00:00:00 (365.0 days) and wrote 251 records for SPX-IND-CBOE-USD | 2023-05-25 17:36:13.710 HD Request # 44 | Completion time: 0s. | 2023-05-25 17:36:13.711 HD Request # 44 | Daily data download complete for SPX-IND-CBOE-USD. Unique request ID: 40 | 2023-05-25 17:36:13.711 Removing historical data download ID 40. | 2023-05-25 17:36:13.711 Triggering next historical data download in queue. | 2023-05-25 17:36:13.711 No download requests in the queue to start downloads for. | 2023-05-25 17:36:13.711 SPX-IND-CBOE-USD Daily #17 | Reloading chart. | 2023-05-25 17:36:13.880 Chart #17 has tagged chart #10 for full recalculation. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:13.894 Chart #17 has tagged chart #2 for full recalculation. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:13.894 Chart #17 has tagged chart #46 for full recalculation. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:13.894 Chart #17 has tagged chart #6 for full recalculation. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:13.894 Chart #17 has tagged chart #8 for full recalculation. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:13.894 SPX-IND-CBOE-USD 5 Min #8 | Performing a full recalculation because it has been tagged. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:13.909 SPX-IND-CBOE-USD Weekly #46 | Reloading chart. | 2023-05-25 17:36:13.944 SPX-IND-CBOE-USD 60 Min #2 | Performing a full recalculation because it has been tagged. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:14.091 Chart #2 has tagged chart #10 for full recalculation. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:14.096 Chart #2 has tagged chart #8 for full recalculation. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:14.096 SPX-IND-CBOE-USD 30 Min #10 | Performing a full recalculation because it has been tagged. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:14.113 SPX-IND-CBOE-USD 15 Min #6 | Performing a full recalculation because it has been tagged. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:14.127 Chart #6 has tagged chart #10 for full recalculation. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:14.130 Chart #6 has tagged chart #46 for full recalculation. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:14.130 Chart #6 has tagged chart #8 for full recalculation. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:14.130 SPX-IND-CBOE-USD 5 Min #8 | Performing a full recalculation because it has been tagged. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:14.143 SPX-IND-CBOE-USD Weekly #46 | Performing a full recalculation because it has been tagged. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:14.194 SPX-IND-CBOE-USD 30 Min #10 | Performing a full recalculation because it has been tagged. Chartbook: SPX_resized_V5.Cht | 2023-05-25 17:36:14.368 Denali Data Feed | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=4, NumberOfOutstandingSendBuffers=1, TransmissionDelayInMilliseconds=40, ServerSendBufferSizeInBytes=28, ActualMessageDelay=0.3 seconds, DataCompRatio=1.61, UncompressedBytes=4478, CompressionTime=0.000342(secs), NumCompressions=76, Source=10.10.20.141 | 2023-05-25 17:36:22.143 Denali Data Feed | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=2, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=40, ServerSendBufferSizeInBytes=0, ActualMessageDelay=0.3 seconds, DataCompRatio=1.41, UncompressedBytes=1077, CompressionTime=0.000076(secs), NumCompressions=13, Source=10.10.20.141 | 2023-05-25 17:36:22.143 Message from IB: Market data farm connection is broken:usfuture. IB Error Code: 2103. Request ID: -1. | 2023-05-25 17:36:29.545 SC DTC Data with Market Depth | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=13, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=40, ServerSendBufferSizeInBytes=27, ActualMessageDelay=-0.0 seconds, DataCompRatio=1.15, UncompressedBytes=53748, CompressionTime=0.004322(secs), NumCompressions=1244, Source=10.10.20.81 | 2023-05-25 17:36:29.858 Message from IB: Market data farm connection is OK:usfuture. IB Error Code: 2104. Request ID: -1. | 2023-05-25 17:36:30.115 Denali Data Feed | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=3, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=40, ServerSendBufferSizeInBytes=0, ActualMessageDelay=0.3 seconds, DataCompRatio=1.82, UncompressedBytes=6270, CompressionTime=0.000425(secs), NumCompressions=90, Source=10.10.20.141 | 2023-05-25 17:36:42.123 Denali Data Feed | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=2, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=40, ServerSendBufferSizeInBytes=0, ActualMessageDelay=0.3 seconds, DataCompRatio=1.41, UncompressedBytes=1139, CompressionTime=0.000081(secs), NumCompressions=14, Source=10.10.20.141 | 2023-05-25 17:36:42.123 Saved Chartbook: SPX_resized_V5.Cht. Time: 0.0563 seconds. | 2023-05-25 17:36:42.456 |
| [2023-05-26 21:47:11] |
| Sierra_Chart Engineering - Posts: 17199 |
|
There is a limit with the number of days downloaded from Interactive Brokers for historical daily data. This is a limit set on the Interactive Brokers side. Maybe it has changed.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2023-05-26 22:08:53] |
| Jeff Li - Posts: 58 |
|
Directly from their website: "Note: 1. At this time Historical Data Limitations for barSize = "1 mins" and greater have been lifted. However, please use caution when requesting large amounts of historical data or sending historical data requests too frequently. Though IB has lifted the "hard" limit, we still implement a "soft" slow to load-balance client requests vs. server response. Requesting too much historical data can lead to throttling and eventual disconnect of the API client. If a request requires more than several minutes to return data, it would be best to cancel the request using the IBApi.EClient.cancelHistoricalData function." https://interactivebrokers.github.io/tws-api/historical_limitations.html |
| [2023-05-30 13:30:46] |
| Sierra_Chart Engineering - Posts: 17199 |
|
Ok we will try an increase but we make no guarantees. And it may create a problem.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2023-05-30 13:30:53
|
| [2023-05-30 14:51:45] |
| Jeff Li - Posts: 58 |
|
I am no coder but I think we might be requesting too much data at once (from 1973). Perhaps we might be better off requesting in blocks of 365 days? With multiple unique download request ids? Looking forward to your reply. Thx in advance. |
| [2023-06-14 15:05:27] |
| Jeff Li - Posts: 58 |
|
hey, wonder if you've looked into this yet
|
| [2023-06-15 13:03:52] |
| Sierra_Chart Engineering - Posts: 17199 |
|
We are not going to do this: Perhaps we might be better off requesting in blocks of 365 days? With multiple unique download request ids? Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
To post a message in this thread, you need to log in with your Sierra Chart account: