Intraday Data File Format

Introduction and General Information

This page explains the format for Intraday data files used in Sierra Chart. Intraday data files are the files that contain data for Intraday charts. They simply contain only the actual price/value and volume data, nothing else. There are two purposes for this documentation. The first is so that you can read the data in other programs and the second is so that you can feed Sierra Chart data from your own source.

An Intraday chart data file first begins with a single header of type s_IntradayHeader followed by multiple records of type s_IntradayRecord for each time unit of price/value data for a symbol. The file extension is .scid. When connected to a streaming data feed, Intraday data files are continuously updated. Sierra Chart opens the file in shared mode when it reads or writes to the file.

If you are creating the Intraday data files yourself and writing data to them, when the file is opened as a chart in Sierra Chart, the chart will read the data in the file beginning at the Date which is calculated from the last Date in the file minus the Days to Load setting in Chart >> Chart Settings .

New data appended to the file will be read from the file at the rate set by the Chart Update Interval setting in Global Settings >> General Settings.

Data in a Sierra Chart Intraday Data file (File extension: .scid) is a non-text format. It can be exported to a text format. To do this, refer to Exporting and Importing Intraday Data Files.

When Sierra Chart is writing received streaming data to Intraday chart data files, it does not write to them every trade or bid/ask update, it will write to an internal buffer and then this data is then written to the file every 5 seconds. The flush time can be changed through Global Settings >> Data/Trade Service Settings >> Advanced >> Intraday File Flush Time in Milliseconds. If set to 0, then a default flush time is used.

Data Structures Used

The Intraday data file header, s_IntradayHeader, is 56 bytes in size.

struct s_IntradayHeader
{
    char FileTypeUniqueHeaderID[4];  // Set to the text string: "SCID"
    u_int32 HeaderSize; // Set to the header size in bytes.

    u_int32 RecordSize; // Set to the record size in bytes.
    u_int16 Version; // Automatically set to the current version. Currently 1.
    u_int16 Unused1; // Not used.

    u_int32 UTCStartIndex;  // This should be 0.

    char Reserve[36]; // Not used.

    s_IntradayHeader()
    {
        strncpy (FileTypeUniqueHeaderID, "SCID", 4);
        HeaderSize = sizeof(s_IntradayHeader);
        RecordSize = sizeof(s_Record);
        Version = 1;
        Unused1 = 0;
        UTCStartIndex = 0;
        memset(Reserve, 0, sizeof(Reserve));
    }
};
    

The Intraday data record structure, s_IntradayRecord, is 40 bytes in size.

struct s_IntradayRecord
{
    static const float SINGLE_TRADE_WITH_BID_ASK;
    static const float FIRST_SUB_TRADE_OF_UNBUNDLED_TRADE;
    static const float LAST_SUB_TRADE_OF_UNBUNDLED_TRADE;

    SCDateTimeMS DateTime;

    float Open;
    float High;
    float Low;
    float Close;

    u_int32 NumTrades;
    u_int32 TotalVolume;
    u_int32 BidVolume;
    u_int32 AskVolume;


    s_IntradayRecord()// Constructor
    {
        DateTime;
        Open = 0.0;
        High = 0.0;
        Low = 0.0;
        Close = 0.0;
        NumTrades = 0;
        TotalVolume = 0;
        BidVolume = 0;
        AskVolume = 0;
    }
};

//Definitions for the above constants in s_IntradayRecord.
const float s_IntradayRecord::SINGLE_TRADE_WITH_BID_ASK = 0.0F;
const float s_IntradayRecord::FIRST_SUB_TRADE_OF_UNBUNDLED_TRADE = -1.99900095e+37F;
const float s_IntradayRecord::LAST_SUB_TRADE_OF_UNBUNDLED_TRADE = -1.99900197e+37F;

    

s_IntradayRecord Structure Member Descriptions

How to Feed Sierra Chart Data Through Intraday Data Files

This section explains in general how to feed Sierra Chart data from an external source by creating and updating Intraday data files in the format documented here.

A more modern and up-to-date method of feeding Sierra Chart data is through the DTC Protocol which Sierra Chart fully supports.

Although updating Intraday chart data files directly is a reasonable method. The data received by the Data and Trading services that Sierra Chart is integrated with, through the Sierra Chart program, also write to Intraday chart data files.

1. First, create a file with the following file name format: [symbol].scid . This file must be located in the Sierra Chart Data Files Folder which is specified through Global Settings >> General Settings.
   
   If the Intraday data file you are creating is not located in this folder and you try to open it from another location with File >> New/Open Intraday Chart , Sierra Chart will automatically change the path to the Data Files Folder and create a new file. It will then be reading an empty data file.
2. The file must be opened with file share read and file share write access. This is essential.
3. Write an Intraday file header of type s_IntradayHeader to the beginning of the file. Once this is done, it does not need to be done again. For an understanding of the different fields of the header, see the structure definition above and the comments.
4. Write Intraday data records to the file. The structure type of these records is s_IntradayRecord . These records are written immediately after the header and are in sequence. The first record represents the first and earliest time and the last record represents the last and latest time. An Intraday data record, of type s_IntradayRecord , can represent 1 tick or any specific period of time such as 1 second, 10 seconds, 1 minute or any other period up to 1 day. For daily data which represents one day of trading, we use an Text data format.
5. You will write Intraday data records to the file as data is received in real time from your data feed or to fill the file with historical data.
   
   If a data record represents one single tick/trade, then the Intraday data record Open, High, Low, and Close can be set to the same value and you will write one record for every tick/trade.
   
   If you want to record the Bid and Ask price for the trade, then set the Open to 0, the High to the Ask price, the Low to the Bid price, and the Close to the trade price.
   
   If a data record represents a specific period of time, such as 10 seconds, then the Open, High, Low, and Close and Volume fields are for that 10 second period of time. You will write one record for every 10 seconds of time there is trading activity. If there is no trading activity for a particular period of time, then no record needs to be written.
6. Example: When you are writing data in real-time and the record represents a specific period of time other than one tick/trade, and your Intraday record length is 1 minute, and 10 trades have occurred, then what you will do is add a new record on the first trade, and update that record 9 additional times when those trades occur. When you initially add the record, the Open, High, Low, and Close will all be the same value. When you update it 9 additional times, first read the last record written, increase the High if necessary, decrease the Low if necessary, update the Close, increase all of the Volume fields, and write the updated record back to the same location in the file.