Support Board
Date/Time: Fri, 29 Nov 2024 18:24:39 +0000
[Locked] - DTC Protocol
View Count: 85230
| [2015-08-26 07:47:13] |
| |
|
We do not know what is meant by "custom live market data feeds". In any case, it does not sound like something we would provide an example of. In any case it should be obvious how to work with the protocol. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2015-08-26 08:24:14] |
| Dema - Posts: 42 |
|
Fair enough - if you're able to provide an example for market data as opposed to order data that would be sufficient.
|
| [2015-10-10 04:02:29] |
| |
|
In this video, an AT&T representative back in 1985 talks about the benefits and coming changes from information technology and the information age: https://www.youtube.com/watch?v=qWsC9KE-PHY We believe this is kind of like the effort we are making now with the DTC Protocol. While it may take time to catch on, we have the future vision of something which is inevitable. Update: It was at roughly this point in the video: https://youtu.be/qWsC9KE-PHY?t=909 Where the discussion reminded us of the DTC Protocol effort and how this can bring about great change and how it makes so much sense. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2015-10-10 05:16:46
|
| [2015-10-10 04:38:33] |
| |
|
From this section here: http://dtcprotocol.org/#WhyDTC Here is a quotation: DTC is completely neutral. What this means is that it does not favor the Client or the Server.
Think if a web browser had to be designed to work with a different protocol from every single website out there. Would that make sense? The obvious answer is that this does not make sense and this is why establishing a common open specification protocol is needed. The DTC protocol is the solution. Standards are very common in electronic data communications and physical electronic networks. The Internet would not even be possible without standards. There is the TCP/IP standard. Hardware layers are highly standardized. There is a standard for Ethernet. There is no way that physical networks can interoperate without standardization. Where would the Internet be without communication standards? The explosive growth seen on the Internet which began in the 1990s, throughout the first decade of the 21st century, and continues to this day, would not have taken place without standards. Think about personal computers before USB? Think about all the trouble we had to get a piece of hardware hooked up to a computer before USB. In the old days, we had to deal with setting interrupt request numbers for a piece of hardware. We hope people will begin to see the light here! We know many of our users do, but it is Trading services and Data services, which need to wake up and realize that eventually they are going to be isolated if they do not use standardized protocols. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2015-10-10 20:54:01
|
| [2015-10-10 05:22:54] |
| |
|
Regarding this: and another one for a Python wrapper :) This is now supported with the DTC Protocol because the DTC Protocol now supports Google Protocol Buffering encoding which supports Python. Have a look at: https://developers.google.com/protocol-buffers/ Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2015-10-12 01:38:53] |
| ejtrader - Posts: 688 |
|
SC Team - Did you had a chance to work on post #24 ( providing a working example ). Really looking forward to a working example on DTC client/server. thanks |
| [2015-10-12 02:40:34] |
| |
|
Not yet. This is the next most significant item after the DTC Protocol Server is complete within Sierra Chart. That should be all done by the end of this week. So probably next week we will start on the DTC client and server example code. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2015-10-14 17:50:00] |
| Guilherme - Posts: 66 |
|
Is there a FIX 4.X to and from DTC protocol bridge already available?
|
| [2015-10-14 18:49:07] |
| |
|
No, this does not exist. There is not a need for this because Sierra Chart already directly supports FIX. Regarding Post #33, what is the purpose of using a FIX to DTC bridge? Maybe we can help you if we have a better understanding of the need for it. One difficulty though of bridging FIX to DTC is with order IDs. In FIX there are the following order IDs: OrderID , ClOrdID, OrigClOrdID. These are not directly mappable to DTC so there has to be some order ID management within the bridge program which does make it more complicated. DTC just uses the ClientOrderID and ServerOrderID for simplicity. There really is not a need for more than this. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2015-10-15 00:52:33
|
| [2015-10-15 00:52:46] |
| |
|
The prior post has been updated.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2015-11-20 18:38:22] |
| ejtrader - Posts: 688 |
|
Not yet. This is the next most significant item after the DTC Protocol Server is complete within Sierra Chart. That should be all done by the end of this week.
So probably next week we will start on the DTC client and server example code. SC Team - post #31 - Still very much looking forward to these examples - even if they are in "raw" form :) Hopefully would have a chance to explore these during the holiday season. thanks Date Time Of Last Edit: 2015-11-20 18:39:27
|
| [2015-11-21 02:02:53] |
| |
|
We are behind on this, and it seems unlikely we will have this done before the end of the year. But we would not think it would be longer than the end of January.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2016-01-19 11:12:18] |
| paska.houso - Posts: 7 |
|
Dear SierraChart Team, I had a look at your application and had a project in my backlog to implement a DTC Server in Python. With the advent of 2016 I have started that and rather than hand-coding everything and risking falling behind potential changes, I have written a small C++ Header Parser (specifically suited for the DTCProtocol.h). The syntax allows that and with an "almost" automatic translation to Python. I have only come to 3 corner cases, 2 of which you may possibly and easily iron out. Here my feedback (C++ Fixed Length Strings) 1. Some of the "structs" define a "Clear" method which is invoked from the constructor. This should either be the standard for ALL structs or not be present in anyone. Consistency is key. The "Clear" method does not only clear standard "operating" values of the "struct" but also things like the "Size" and "Type" fields, which can be seen as an integral part of each instance. I would personally opt for having a Clear Method for each and every struct. 2. Two of the structs feature a "SetToUnsetValues" method which is doing more or less (it doesn't touch Size/Type) than Clear. The most annoying thing about this method is that unlike "Clear" ... it is not defined in the header file but rather in the "cpp" part. If "Clear" breaks consistency, this "SetToUnsetValues" goes a step beyond. In this case you could put the functionality of "SetToUnsetValues" into a "Clear" method in the header and then have the "SetToUnsetValues" call "Clear" (for backwards compatibility) Structs with this method: s_MarketDataSnapshot (call to it missing in the constructor) and s_MarketDataSnapshot_Int 3. Nested "struct" by having 2 arrays of s_DepthEntry structs as members of s_MarketDepthFullUpdate20 and s_MarketDepthFullUpdate10. Although for a regular API this is no issue, for an over-the-wire transmission it is not. It is not because these two do not resist the comparison with all other structs which are made up of simple types. Encoding/Decoding over-the-wire packets may be as simple as doing a "memcpy" in C/C++, but this adds an unneeded burden on Dynamic Languages in which the exact representation of double and floats (for example Python) is not simply a set of bytes, but rather and object. Of course it can be overcome and given these 2 structs are already well defined, there is no easy way out. But let me suggest that future structures should avoid declaring composite types. In any case and as soon as I am reasonable satisfied with my development, it will be available on GitHub. Best regards P.S. Having your files and documents on GitHub would also be a good idea to allow people to propose changes/corrections/improvements via Pull Requests (GitHub is just a suggestion, many other sites fare also as good) |
| [2016-01-19 12:37:33] |
| Guilherme - Posts: 66 |
|
Have you considered using Protocol Buffer encoding? I'm using it in my Java/Scala implementation for DTC Protocol. It really made my life a lot easier.
|
| [2016-01-19 13:25:19] |
| paska.houso - Posts: 7 |
|
It is my aim to implement all the Encodings if possible (under Python JSON Encoding would be the most obvious choice) but wanted to start with the lowest common denominator which should be Binary. In any case my points 1 and 2 are Encoding-Independent. |
| [2016-01-20 08:11:08] |
| |
|
1. We are working on implementing the Clear function for all message structures in the header file. 2. We will replace the SetToUnsetValues functions with a Clear function in the header file. 3. Noted but the structures cannot be changed at this time. However, it really is recommended that you use: s_MarketDepthSnapshotLevel instead The 10 and 20 levels market depth structures are not really meant to be part of the official protocol. They are there for ease-of-use. 4. Here is the public repository: https://github.com/DTC-protocol Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2016-01-20 09:26:26] |
| paska.houso - Posts: 7 |
|
1. Thanks (I already tweaked my rudimentary parser to implement the Clear method even if missing in the sources) 2. Thanks too (I also tweaked my parser to 1st read the c++ implementation and scan any SetToUnsetValues and make a "Clear" out of it) 3. Thanks for the tip. I will stick to it. 4. Thanks again. Somehow and even after searching in the docs for it I was unable to find a link Best regards |
| [2016-02-28 00:25:04] |
| ejtrader - Posts: 688 |
|
SC Team - Wondering if you would have a chance to work on the example code you have referred to in Post #31 and #36. Thanks Not yet. This is the next most significant item after the DTC Protocol Server is complete within Sierra Chart. That should be all done by the end of this week.
So probably next week we will start on the DTC client and server example code. We are behind on this, and it seems unlikely we will have this done before the end of the year. But we would not think it would be longer than the end of January.
|
| [2016-02-28 22:09:29] |
| |
|
Not yet. We are working hard over the next three months to catch up on overdue tasks. But we are not sure when we can get to this. We should have a better idea in about a month. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2016-02-28 22:09:40
|
| [2016-03-08 00:30:18] |
| User716386 - Posts: 4 |
|
Hi, Unfortunately, As I understand DTC/FIX bridge example not in Your priority list. Please let me know how I can connect new FIX based brokers to Sierra. I would like connect accounts from: - PrimeXM FIX; - CFH Clearing API v 2.5 (FIX 4.4 based) I already use them from my own strategies, but I'll be happy integrate it to Sierra. Best Regards |
| [2016-03-08 04:04:40] |
| |
|
We do not ever remember saying that we would provide an example of a DTC to FIX bridge. We only said that an example of the DTC client and server would be provided, but that would only be helpful for those who have little programming experience/knowledge. There is no reason if you know what you are doing that you should need these examples. You should be completely fine without them especially if you have experience with FIX and network connections. Please let me know how I can connect new FIX based brokers to Sierra. Want kind of answer are you looking for? This is far too involved for us to answer. You have to write a DTC server and use the DTC client in Sierra Chart: http://www.sierrachart.com/index.php?page=doc/doc_DTC_TestClient.php Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2016-03-08 04:05:12
|
| [2016-03-23 21:41:03] |
| User783015 - Posts: 7 |
|
I am trying to write a DTC client Python class which my trading software will use to connect to Sierra Chart DTC server. I'd like to use the JSON encoding for obvious reasons. However the EncodingRequest message must be sent in a binary format and that's quite a problem. I got a crazy idea to write a small C++ application which would construct this message and store it to a file. This would allow me to take that string, hardcode it to my Python script and send it over the socket to force the server to JSON encoding. But there must be an easier way to do this, right? Looking into it a bit more, the s_EncodingRequest structure is much more complex than I originally thought. I now see all the methods it is implementing and I assume that when you serialize the structure to send it over the wire, you're including also all the method code in the binary message. I was thinking about using the protocol buffers but there's the same problem again - first you need to send the EncodingRequest message in the binary format to request the encoding change. The whole point of using JSON or protocol buffers is to enable interoperability with languages which may use different binary representation of data than C++ and converting the data to this binary format may be tedious, right? Forcing the application to send the EncodingRequest message in the binary format imho defeats that point. If I'm writing a DTC client which may connect to different unknown DTC servers, I'm pretty much forced to write the client in C++. Or am I missing anything? Date Time Of Last Edit: 2016-03-23 22:32:50
|
| [2016-03-24 00:23:04] |
| vbmithr - Posts: 204 |
|
I got a crazy idea to write a small C++ application which would construct this message and store it to a file.
You don't have to do this. Indeed you need to support the handshake procedure using the binary encoding but anything you can do in C++ you can do it in Python too. you're including also all the method code in the binary message.
Yes you are missing the point that the binary encoding is just a serialization format for sending a message over the wire. The fact that they provide a C++ header file is irrelevant, I coded www.bitsouk.com in OCaml, never needed a line of C++. The header file (DTCProtocol.h) defines C++ structures. You need to understand how those datastructures are represented in memory and send a message of this exact same format. The representation in memory is simple (fields are just added in order), but sometimes the compiler do padding. I recommand using protocol buffers because you benefit from the provided protocol description and don't have to follow protocol upgrades. You just have to handle logon request and encoding request/response from the binary protocol (I think). |
| [2016-03-24 01:11:25] |
| |
|
The Encoding Request is a very simple data structure consisting of 16 bytes. How numbers and text are encoded is standardized. The only variation is some hardware platforms use what is called big endian byte ordering instead little endian. But the overwhelming amount of systems out there use little endian. This only applies to integer and floating-point numbers. Anyway what we will do in the DTC Server in Sierra Chart is we will add an option to force a particular encoding so you do not have to bother with the encoding request and response. We will have this out next week. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2016-03-24 01:24:03
|
| [2016-03-24 19:59:39] |
| User783015 - Posts: 7 |
|
Thanks a lot for your replies! I understand that the binary encoding is just a serialization format for sending the message over the wire. The challenge is how do I create a data structure in Python which will result in a binary data stream identical to the one produced by the C++ structure defined in the DTC protocol - in other words the binary data stream which the DTC server is expecting. Python is a dynamically typed scripting language - you do not define a data type when you declare/define a variable. Consider the following python code: import sys
var = 5 print( sys.getsizeof( var ) ) The output I get is "24" (on a 64-bit Win7 machine). So my tiny integer number is stored in a 24 byte big int memory space. My point is that every language may use totally different binary encoding of its data and high level scripting languages like Perl or Python handle this in the background and don't even "bother" the user with these internals. Trying to make two different languages talk to each other in a binary format may be problematic. After some googling around I found https://docs.python.org/3.0/library/struct.html which should probably do what I'm looking for. But the question is how many other languages have something like that. I would suggest to reconsider using the binary format as the default encoding for DTC. While you can construct a JSON file using any language you want - even as primitive as a batch file - constructing a binary message may be significantly more difficult. I guess that's also why the RESTful APIs became so popular - they allow very simple and limitless interoperability between different languages and even platforms. Using such a platform independent (yet less efficient, I agree) encoding by default, you may significantly increase the chances of wider adoption of the DTC protocol. And thanks, SC Support, for the offer to add the encoding selection to the Sierra Chart DTC Server! I'll play with the Python struct module above but I will gladly use that new feature. Happy Easter! Petr |
| [2016-03-24 20:15:26] |
| |
|
The DTC protocol does not have a default encoding. The binary encoding was the first encoding but it should not be considered the default. However, the encoding request is a binary data structure. It is part of the protocol that both the Client and Server can agree upon a particular encoding to use and completely bypass the encoding request and response. We have already added the option to specify the default encoding in the DTC Server in Sierra Chart and it will be out later today. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2016-03-24 21:42:24
|
To post a message in this thread, you need to log in with your Sierra Chart account: