Home >> DTC Protocol Discussion Forum

DTC Protocol Discussion Forum

Date/Time: Fri, 29 Nov 2024 14:39:44 +0000

How to Post a Message for Support

Does SC historical connection support Google Protocol Buffers?

View Count: 9750

[2017-05-05 22:02:05] #1 [Go To First Post]
Queeq - Posts: 42
Hi, I am testing DTC protocol with Sierra Chart (version 1549). SC server encoding is set to automatic. Intraday logon sequence goes fine, SC acknowledges the use of Protobuf, and the following requests are flowing successfully. Here's the Encoding Response: b'\x07\x00\x00\x00\x04\x00\x00\x00DTC\x00' ^^^^^^^^ However, when connecting to Historical data port, Encoding Response indicates binary: b'\x07\x00\x00\x00\x00\x00\x00\x00DTC\x00' ^^^^^^^^ Consequently, the login sequence fails. SC itself reports the following in the log: HD Server \| Unexpected DTC encoding request: 4 \| 2017-05-05 21:41:07 The client program is written in Python and protobuf classes are generated from proto v2 file. Have I missed something? Date Time Of Last Edit: 2017-05-05 22:03:04

[2017-05-12 09:50:05] #2 [Go To First Post]
Queeq - Posts: 42
Answering my own question: there is no support for anything else than binary (Data and Trading Communications (DTC) Protocol Server: Historical Price Data Server ): Only the DTC Binary Encoding is currently supported. For other people looking for answers, there's much longer discussion on Sierra Chart forum: DTC Protocol Dear SC team, I need to understand how to move forward. Any plans or better ETA for other encodings support? Date Time Of Last Edit: 2017-05-12 09:50:29

[2017-05-14 04:18:21] #3 [Go To First Post]
DTC Engineering - Posts: 320
We have to look into this. This may not be difficult for us to add support for the other encodings for the historical data server within Sierra Chart.

[2017-05-14 04:43:24] #4 [Go To First Post]
DTC Engineering - Posts: 320
We will try to get all encodings supported with the DTC Protocol historical data server in Sierra Chart version 1558. This should be out this coming week. Only binary encoding and binary encoding with variable length strings will support compression. The others will not. Date Time Of Last Edit: 2017-05-15 05:56:57

[2017-05-17 15:45:25] #5 [Go To First Post]
DTC Engineering - Posts: 320
Version 1558 of Sierra Chart has been released and it supports all of the DTC Protocol encodings including Google protocol buffers for historical data. Please test and let us know how that works.

[2017-06-09 05:45:57] #6 [Go To First Post]
Queeq - Posts: 42
Tested with historical data connection (the whole sequence from encoding request to data reception). So far so good. I will report if there are any issues. Thanks a lot!

[2017-06-09 20:44:58] #7 [Go To First Post]
DTC Engineering - Posts: 320
This is good.

[2017-12-28 08:34:15] #8 [Go To First Post]
EricH - Posts: 18
Queeq, Can you share with us the python with google protobuf code? I am stuck even at the login message... Thanks!

[2017-12-28 08:38:12] #9 [Go To First Post]
DTC Engineering - Posts: 320
Make sure you read the information here about using Google protocol buffers: DTC Protocol: Google Protocol Buffers (GPB)

[2017-12-28 08:43:37] #10 [Go To First Post]
Queeq - Posts: 42
Hi EricH, I was going to open-source the code I've written for myself, but it requires quite some cleanup (mostly getting rid of multithreading as it turned out to be slower than single thread), "librarization" and packaging, so it's easily pluggable to any project. This hasn't been a priority for me, though, but I'll do it sooner or later. Date Time Of Last Edit: 2017-12-28 08:45:34

[2017-12-28 08:51:08] #11 [Go To First Post]
EricH - Posts: 18
Dear both, the last URL of 'header/msg length' note does the tricks. And your code did exactly that! I thought it was specifically for encoding request! Still, look forward to your github :)

[2018-01-02 02:29:06] #12 [Go To First Post]
EricH - Posts: 18
To continue the Python/google protobuf and DTC experiments. Today's issue is on 'how do you specify the 'start/end date' for a historical data request. From the spec, both start/end time should of unix epoch time in sfixed64. So my question is what's the equivalent of sfixed64 in python? I tried the following: >>> message = Dtc.HistoricalPriceDataRequest() message.RequestID = 99 message.Symbol = "CRM" message.RecordInterval = Dtc.INTERVAL_1_MINUTE message.StartDateTime = 0 message.EndDateTime = long(1512000000) message.MaxDaysToReturn =int(30) message.UseZLibCompression = 0 >>> and send the request to SC. The EndDateTime, 1512000000 is 11/30/2017. SC log shows HD Server \| Received login. Requesting authorization. \| Username: EricHW. \| 2018-01-02 10:15:14 HD Server \| Login has been authorized. \| Username: EricHW. \| 2018-01-02 10:15:14 HD Server \| HD Request #99 \| Socket: 6986 \| Symbol: CRM \| ServiceCode: \| RecordInterval: 60 \| MaximumDaysRequested: 30 \| Start date-time: Since beginning \| ClientRequestID: 99 \| Username: EricHW. \| 2018-01-02 10:15:14 HD Server \| HD Request # 99 \| Socket: 6986 \| Symbol: CRM \| ServiceCode: \| RecordInterval: 60 \| Records Served: 17651 \| ClientRequestID: 99 \| Username: EricHW. \| 2018-01-02 10:15:14 HD Server \| Timed out waiting for client to close socket. \| Username: EricHW. \| 2018-01-02 10:16:14 From python debugger I can see roughly 60 days of records, 1st and last's timestamp decoded as time.struct_time(tm_year=2017, tm_mon=10, tm_mday=31, tm_hour=11, tm_min=27, tm_sec=5, tm_wday=1, tm_yday=304, tm_isdst=0) time.struct_time(tm_year=2017, tm_mon=12, tm_mday=29, tm_hour=22, tm_min=29, tm_sec=45, tm_wday=4, tm_yday=363, tm_isdst=0) So my questions are: 1. how do you specify the EndDate in this case? I've tried in python casting to int and long, or no casting, all with the same results. 2. From the log it seems SC does not properly decode the 'enddate' since there's no log message about it. 3. Why is 30-day request not respected? Date Time Of Last Edit: 2018-01-02 15:07:44

[2018-01-03 09:30:06] #13 [Go To First Post]
DTC Engineering - Posts: 320
1, 2, 3. These particular message fields are not currently supported but we will get these supported in the next release of Sierra Chart.

[2018-01-03 09:57:44] #14 [Go To First Post]
Queeq - Posts: 42
EricH, I'm going to release what I have this week. I've already extracted it from my project, but now need to test it as a separate library. It has several limitations (e.g. only historical data), but could still save you quite some time.

[2018-01-04 09:52:23] #16 [Go To First Post]
Queeq - Posts: 42
https://github.com/Queeq/pydtc Although working, it's at a very basic alpha stage. Feel free to post bugs and pull requests on Github or privately adjust for your own needs. Date Time Of Last Edit: 2018-01-04 09:53:15

[2018-01-04 15:31:27] #17 [Go To First Post]
EricH - Posts: 18
Queeq Thanks a lot!!

[2018-01-04 21:01:29] #18 [Go To First Post]
DTC Engineering - Posts: 320
In regards to: HISTORICAL_PRICE_DATA_REQUEST::EndDateTime HISTORICAL_PRICE_DATA_REQUEST::MaxDaysToReturn Refer to the paragraphs after the steps in this section here: Data and Trading Communications (DTC) Protocol Server: Historical Price Data Server And use Sierra Chart prerelease version 1682 for the latest support for these.

[2018-01-15 04:12:25] #19 [Go To First Post]
EricH - Posts: 18
I can confirm 1682's endDateTime/MaxDaysToReturn as documented above is working fine. More questions on historical quotes: Can SC/DTC server accept multiple HISTORICAL_PRICE_DATA_REQUEST within the same 'login session'? I.e., I tried login, then his_data_request, and can get the correct records back from SC, but when I issue the next his_data_request, and then wait (indefinitely) on getting the SC to respond to this request. Also SC log did not show the 2nd HD Request...

[2018-01-15 05:00:44] #20 [Go To First Post]
DTC Engineering - Posts: 320
Can SC/DTC server accept multiple HISTORICAL_PRICE_DATA_REQUEST within the same 'login session'? No, this is not supported. Date Time Of Last Edit: 2018-01-15 05:00:51

To post a message in this thread, you need to log in with your Sierra Chart account: