Support Board
Date/Time: Sat, 23 Nov 2024 15:30:49 +0000
[User Discussion] - Python for Sierra Chart
View Count: 38862
| [2022-01-16 14:32:33] |
| user2837 - Posts: 76 |
|
Thanks Kiwi, lots of great info here.
|
| [2022-01-17 21:20:23] |
| User155017 - Posts: 41 |
|
THANK YOU!! Using the google buffer python code would be slower? The c++ client code is obtained from google buffer? I see it supports maxquantity, so it must be a recent version. The client does not implement it, but i could be added. |
| [2022-01-17 21:59:33] |
| Kiwi - Posts: 375 |
|
Although Python is slower than C it's likely that it doesn't matter in the time scales involved here. That's why its so popular in so many people facing applications - unless you're processing arrays of numbers as fast as you can the result is the same. The funny thing I discovered the other day is that the benchmarks game rules preclude language modules in what seems like an arbitrary fashion and as a result you can't use numpy, numba or cython in the python tests. If you do, the disadvantage largely drops away. So, if you're a bit of a greeny like me these days then go for C to reduce your carbon impact (less cpu cycles == less wasted heat == less problems for the grandkids) but otherwise Python is probably just fine. |
| [2022-01-18 02:39:21] |
| User155017 - Posts: 41 |
|
After many efforts I managed to compile in linux, modifying the make file. The built of dtcpy is successful, but the only think which resembles it is dtcpy.cpython-38-x86_64-linux-gnu.so, I placed in the dtc.py directory, but it is not imported , it throws the error "ibdtcclient.so: undefined symbol: _ZN6google21kLogSiteUninitializedE". I think it is problem in linking glog library (in fact I had to modify the make file ). How can the make file be fixed tp avoid this error? I copy below the make file which works under linux cmake_minimum_required(VERSION 3.13) project(dtcclient) set(CMAKE_CXX_STANDARD 17) find_package(PkgConfig REQUIRED) pkg_check_modules(glog REQUIRED libglog) option(WITH_GFLAGS "Use gflags" OFF) #find_package (glog 0.6.0 REQUIRED) find_package(Boost COMPONENTS system) set(THREADS_PREFER_PTHREAD_FLAG ON) find_package(Threads REQUIRED) set(srcs_link ${Boost_LIBRARIES} Threads::Threads #glog::glog stdc++fs ) include_directories( src src/dtc ) set(srcs src/dtc/DTCProtocol.cpp src/dtc_session.cc src/dtc_message_logging.cc src/symbol_data.cc src/dtc_trading.cc src/dtc_historical_data.cc src/dtc_market_data.cc src/util.cc) add_library(dtcclient SHARED ${srcs} #glog::glog ${glog_INCLUDE_DIRS} ${glog_INCLUDE_DIRS} ) target_link_libraries(dtcclient ${srcs_link} ${glog_INCLUDE_DIRS}) target_include_directories(dtcclient PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/src ${CMAKE_CURRENT_SOURCE_DIR}/src/dtc ${glog_INCLUDE_DIRS} ) target_compile_definitions(dtcclient PRIVATE VERSION_INFO=${EXAMPLE_VERSION_INFO}) add_library(dtcclient::dtcclient ALIAS dtcclient) add_subdirectory(pybind11) pybind11_add_module(dtcpy src/dtcpy.cpp) #PYBIND11_MODULE(dtcpy src/dtcpy.cpp) target_compile_definitions(dtcpy PRIVATE VERSION_INFO=${EXAMPLE_VERSION_INFO}) target_link_libraries(dtcpy PRIVATE dtcclient) |
| [2022-02-04 13:12:56] |
| user2837 - Posts: 76 |
|
Kiwi and others, thanks for sharing. I have not experimented with your python codes yet. But I have two questions: 1. My impression is that SC stores time and sales data in scid. Is that correct? And does your script extract that information. 2. As data streams in, I would expect the scid files to change. How long does it take for streaming data to be recorded in *.scid and can your python script handle scid files that are changing? When SC is running, perhaps the scid files will be locked. |
| [2022-02-04 22:31:41] |
| Kiwi - Posts: 375 |
|
1. Yes. 2. Instant (very fast in human time) and yes, I use a loop to update with the scid but you could set up other mechanisms. I haven't noticed an issue with locking but you could simply use try except to catch an error if raised and loop again. |
| [2022-06-02 05:28:57] |
| User409050 - Posts: 22 |
|
hi all, after loading the intraday files into python and doing some processing, I noticed that the intraday files value do not match with daily file values for OHLC. for example on 5/31 OHLC in the intraday file for that date is O H L C 2022-05-31 00:00:00-04:00 4164.50 4169.50 4102.75 4148.25 But on the SC daily: its H is 4202.25, Last is 4131.25 does anyone know why there are differences between the 2 files |
| [2022-06-09 02:56:56] |
| Kiwi - Posts: 375 |
|
Make sure the times of day are the same for daily and your intraday simulation of daily. I generally close with the globex close time.
|
| [2022-06-09 03:29:39] |
| User409050 - Posts: 22 |
|
@Kiwi I have double checked so many times, but still couldn't find why there's the difference between those two. the problem is, in the intraday file, there were no trades at that price. may be at the end of a day, CME exchange gives a final update for the day and then at that time, some trades hadn't combined into the file now written in the file. I'm not sure Date Time Of Last Edit: 2022-06-09 03:32:22
|
| [2022-06-09 04:26:26] |
| onnb - Posts: 662 |
|
Regarding the Last price - the Last price on the daily is most likely the ES settlement price for that day. The last price in the intraday files will be the last traded price. These two are not going to be the same. If you look on the cme website for the settlement price for that day, you should see that it matches the daily OHLC last price. The link below shows the last and the settlement https://www.cmegroup.com/markets/equities/sp/e-mini-sandp500.settlements.html You can see that todays settlement is 4114.00 (same in SC daily as the cme website) That said, I would expect the Open/High/Low of the daily to match the Open/High/Low of the intraday. |
| [2022-06-09 05:06:30] |
| User409050 - Posts: 22 |
|
Hi onnb, thank for your response, I'm not comparing SC daily data or intraday data with data on the CME website, I'm comparing daily data SC with intraday (tick data) of SC itself. so when I look at a daily bar, and if I have the data of all trades for the day, supposedly I can go back and forth between them without any discrepancies. But it's not the case and I dont know why, the closes might be different because the closing price for a day is last 30s VWAP. but the High and Low must match. Date Time Of Last Edit: 2022-06-09 05:08:17
|
| [2022-06-09 07:26:40] |
| User431178 - Posts: 540 |
|
@user409050 There are two problems here. 1. Monday 30th May was Memorial Day holiday, so daily chart bar for 31st May will also include this trade data (you will see Monday is missed from the Daily chart). 2. If you are reading directly from the scid file, bear in mind that the data is time-stamped UTC time zone. Also to compare with daily data you need to read the time range corresponding to the trading session, which, using the ES as your example, would mean that the trading day starts at 22:00 UTC and ends the following day at 21:00 UTC. |
| [2022-06-09 15:01:27] |
| onnb - Posts: 662 |
|
I'm comparing daily data SC with intraday (tick data) of SC itself
For the daily data you are looking at the dly files? The historical daily bars so to speak?
|
| [2022-06-09 16:41:50] |
| User409050 - Posts: 22 |
|
For the daily data you are looking at the dly files? The historical daily bars so to speak?
here how I did,I read the data in scid file into python and built TPO chart and number bar in python because some of the stats that I need are not in SC default study. and building a study in SC using C++ is painful, so I do the prototype first in python and if it works well then I will deploy in SC later. So I built the TPO chart, I noticed that my chart was different than the ones in SC then I double checked and to see where the problem came from, and I saw that the High and Low of a couple daily bars didn't match. For timezone: I do convert UTC to my local timezone. but I don't think that's the problem because I not only checked the data in python with SC daily bar data, I also used the data in SC (tick data) to compare with daily bar in SC as I mentioned above. its easy to do that just use any 5-15... or any time resolution other than daily and compare the high and low within a day with daily bar OHLC of that day. theoretically you should see the trades at High and Low take place in higher time resolution. but it's not the case here. For example the date '2022-05-13' wednesday. the High and Low are 4050.5 and 3923.75 but if u go over 5 min bar or tick bar U won't see the trade at Low |
| [2022-06-09 17:22:44] |
| onnb - Posts: 662 |
|
Yes, I would expect the high and Low to match You would not expect the Last to match See here: Historical Chart Prices: 12.7 - Displaying Last Trade Price Instead of Official Settlement Price |
| [2022-06-09 18:14:47] |
| User431178 - Posts: 540 |
|
For example the date '2022-05-13' wednesday.
You mean 2022-05-11, being as 13th was a Friday.the High and Low are 4050.5 and 3923.75 but if u go over 5 min bar or tick bar U won't see the trade at Low
You can see it just fine, even on a daily chart built using intraday data....If your data/bars don't match, there must be a time mismatch between the datasets you are comparing. |
 comparing bars.png / V - Attached On 2022-06-09 18:14:33 UTC - Size: 77.82 KB - 235 views |
| [2022-06-09 19:36:33] |
| User183724 - Posts: 183 |
|
i was working on getting a python program up and running a while back and tried using the "Write bar and study data to file" instead of the scid. I also was working on a ASIL study to port the data to a USB port instead of a text file. My thinking was to avoid file sharing problems between SC and Python on the same computer accessing the same text file. If Python was running on a Raspberry PI and received the data from a usb/serial port, it has plenty of computing power to calculate trades and could even enter those trades though a keyboard interface. Kind of lost interest in it but have been thinking about it again. Have any of you considered a hardware solution? |
| [2022-06-09 23:09:37] |
| User409050 - Posts: 22 |
|
See here: 12.7 - Displaying Last Trade Price Instead of Official Settlement Price
hi onnb, thank for the linkI have figured out why. it turns out in order for them to match, I have to combine tick data from 18h pm of the previous day. it was stupid of me =.= |
| [2022-07-05 06:54:33] |
| User441394 - Posts: 1 |
|
Hi everybody. I would like to know what do the .dly files that are created by SC along with SCID files contain ? I would be grateful if somebody can provide the answer
|
| [2022-07-05 09:03:11] |
| |
|
Refer to: Text/CSV Data Format And: Intraday Data File Format Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2022-09-27 18:54:44] |
| User17018123 - Posts: 5 |
|
There was git hub project of a dtc server managed by a SC user. It was based on a c++ client and a python wrapper, but it was removed. The DTC limitation on CME data is a big one, since it makes practically impossible to use DTC to access SC data, but trading is still possible via DTC, so an external data feed could be used. Is there any workaround this? In 2022 there is still no open source way to trade directly from Python via DTC, and no open source example of an alternative solution based on ACSIL and write/read file to communicate? |
| [2022-12-10 20:58:35] |
| User719512 - Posts: 262 |
|
I found 1 bug and 1 nit in the excellent code posted by Kiwi in post #67 (Python for Sierra Chart | Post: 269962). Edge case with a time very close to midnight where the rounding code rounds the value up to 86400 which then causes helper() to use a factor of 1 that causes the hour to come back as 24 (valid range is 0-23). Since the code discards milliseconds, an easy fix is to detect this and coerce the time to 86399 (23:59:59). In deserialize(), change code to: def deserialize(excelDateAndTime): """ Returns excel date time as Python datetime object """ date_tok = int(excelDateAndTime) time_tok = round(86400*(excelDateAndTime - date_tok)) # account for rounding error for excelDateAndTime like 44899.99999549769 if time_tok == 86400: time_tok = 86399 d = date(1899, 12, 30) + timedelta(date_tok) t = time(*helper(time_tok, (24, 60, 60, 1000000))) return datetime.combine(d, t) On Windows, the csv will have extra newlines between each line. A fix to that is to change the open() call to pass a `newline` argument: ftsv = open(fileoutput, 'w', newline='') Thanks again Kiwi for the sample code. Not only did that help with my issue to parse scid files, I learned a bit more about Python as well in an area I had not gained much experience yet. Date Time Of Last Edit: 2022-12-10 23:13:30
|
To post a message in this thread, you need to log in with your Sierra Chart account: