Support Board

*** This probably won't work anymore with the date change in SC 2150.

This program is also an example of a working SCID reader. If
Version 2 comes out, this program will likely need updating. Using
this program on another version will probably corrupt that version.

Building and running: This program uses specific sized data types and should
compile cleanly on 32bit and 64bit systems.

Linux:
g++ -O3 --static -o SC_CompressDataUnitSize SC_CompressDataUnitSize.cpp
Copy SC_CompressDataUnitSize to the SC Data directory.
Example: ./SC_CompressDataUnitSize -c4 -tm -u1 GBPUSD.scid.old GBPUSD.scid

Windows:
Change directory to C:\SierraChart\CPPCompiler\bin
Copy the SC_CompressDataUnitSize.cpp source file here.
Open a DOS window.
g++.exe -O3 --static -o SC_CompressDataUnitSize.exe SC_CompressDataUnitSize.cpp
Copy SC_CompressDataUnitSize.exe to the SC Data directory.
If you haven't already, fully exit SC to avoid causing data corruption.
While in the Data directory, rename what ever files you want to compress with
a ".old" extension. In this example GBPUSD.scid gets renamed to GBPUSD.scid.old.
If there is a problem, delete the bad SCID file (GBPUSD.scid in this example)
and rename GBPUSD.scid.old back to GBPUSD.scid.
Open a DOS window and run from the Data directory:
SC_CompressDataUnitSize.exe -c4 -tm -u1 GBPUSD.scid.old GBPUSD.scid

After this program finishes, use SC to export the data to CSV if necessary.
This program includes CSV exports (mainly for debugging), but SC will probably
export cleaner and more usable files.

This program aligns the output bars to the beginning of each time block.
This keeps bars aligned in non-market graphing programs and spreadsheets.

My ChartBook Load Speed Test:
Original 1 Tick SCID Size: 6g (around 4-5min to load)
smashed by SC = 80sec (49megs).
1bar = 80sec (48megs). This is essentially similar to smashed by SC.
2bar = ?sec (95megs). Hangs
4bar = 14sec (191megs).

Usage: SC_CompressDataUnitSize.exe -opts InFile.scid OutFile.scid
Program to compress Sierra Chart Version 1 SCID tick by tick files down
in size with different methods while preserving up and down volume counts.
This version is 10-20x faster than the built in SC function. The "-c4"
option also offers a 6x run time speed up than the traditional compression.

Options (-opts) start with the dash (-) character followed by a letter and
a number (replaces the #) with some options.
-c#: Bar Consolidation Type. 1 is similar to SC's built in function. 2 hangs SC
for some reason. It will produce an up and down bar for each time unit. 4 will
give the 6x speed up. It produces 4 bars for Open, High, Low, and Close ticks.
-t#: Time Prefix. Options are s for seconds, m for minutes, h for hours, and d
for days.
-u#: Time Units. The number of units for -t. "-tm -u1" would give 1min bars.
Note SCID intraday files can have a maximum bar length of 1 day. Anything over
that will be rounded down to 1 day. Use the daily CSV text format for anything
higher than a day. Watch out for uneven dividing of the time units into 1
trading day. This program does not convert time zones. Be careful with larger
time units.
-x#: Cut days older than # back. This is used for trimming down SCID files.
-y#: Do not process (just pass them through) ticks from the last # days.
Days back are CALENDAR days, not trading days. Watch out for weekends and
holidays. Usually give 3-4 extra days to account for that.
-r: Write out CSV file from the SCID input. Watch out for file size.
-R: Same as -r except more human readable for debugging.
-w: Write out CSV file from the SCID output. Watch out for file size.
-W: Same as -w except more human readable for debugging.
-d: Enable debugging mode. More output is given.
-B: Batch mode. Doesn't display the warning. Use with caution.

Time Options Examples: 1sec bars: -ts -u1. 30sec bars: -ts -u30.
1min bars: -ts -u60 or -tm -u1. 10min bars: -tm -u10.
45min bars: -tm -u45. 1hr bars: -tm -u60 or -th -u1.
4hr bars: -th -u4. 6hr bars: -th -u6. 1day bars: -td -u1.

Convert forex EUR/USD tick by tick data to fast 1min bars discarding anything
older than 30 days and not converting the past 7 days with debug CSV files:
SC_CompressDataUnitSize.exe -c4 -tm -u1 -x30 -y7 -R -W EURUSD.scid.old EURUSD.scid

Version 0.9 2016-02-03 GPL'd and Open Sourced by Brett Johnson