Home >> Support Board

Support Board

Date/Time: Sat, 04 Jan 2025 07:22:57 +0000

How to Post a Message for Support

Possible rare bug in AutoTrade Backtest Exit, True but no Fill

View Count: 3271

[2016-05-10 13:58:49] #1 [Go To First Post]
User791263 - Posts: 151
I've worked long hours daily with S.C. for a year,6 months with autotrade via spreadsheets & studied Autotrade, Sim & fills documentation, including https://www.sierrachart.com/index.php?l=doc/doc_TradeActivityLog.php#QuestionsAboutTradeActivityLogData For 3 months, I've studied backtest slow-motion fills conditions in my scalping intrabar-trades system. My point is: I understand SC autotrade well, and as an ex-programmer & ex-TradeStation programmer, I think I can recognize a possible rare bug that would only be found by exhaustive audit like I've done. I always use the Trade Activity Fills Window to track entries-exits because intrabar trades often don't keep the True state, so I rely on T.A. Fills. but then... I wasn't looking for non-fills. I thought all True's filled because 99.9% do. Normally, fills are always instant-simultaneous in back-testing. The first time I saw this, I thought it was just a 1 in a million odd data & bar-alignment issue not worth worrying about. The 2nd time, I wanted to still believe that. Since watching for it more, I see it could be a $$ problem. Seems only on Exits (as I recall). Note THIS one is subsequent Bar, but seems other two exits were 2-4 bars after entry. Likely this occurs only with intrabar, maybe related to how positions are recorded. Attached, see picture with arrows and notes, comparing Trade Window to Spreadsheet. Clearly there's a short entry in previous bar and a True in next bar N, but no exit Fill, ever, during that entire bar-row. The Trade Service Log has no message in re this last row. It's hard to discover because intrabar True's often disappear. Here it held. My settings don't limit exits (pretty sure): Reset contions on New Bar - Yes Multiple entries same Direction - Yes Signal only once/bar - No Cancel orders on Entries - yes (used both ways, same) Allow Entries with working orders - yes Signal on Bar Close - no SUpport Reversals - No Allow Opposite Entry with Opposing Position - Yes - - - - I previously sent a spreadsheet where you found & fixed a Circular Reference SC Shutdown issue. The example spreadsheet here is no longer in the state, data.


Private File

[2016-05-10 14:28:00] #2 [Go To First Post]
User791263 - Posts: 151
Note: The prior occurances of this issue were under a couple of different settings: Cancel orders on Entries - no Allow entries with working orders - no . So, at least some settings may not be involved.

[2016-05-10 14:51:55] #3 [Go To First Post]
User791263 - Posts: 151
Exits are Market orders. If you need J-Column picture, let me know. Working Orders 0 Position w Working Orders -1 Position Qty -1 Position Qty w Working Exit ORders -1 Prior Position Qty 0 Position Qty -1 Position avg Price 2052.5 Bid 2052.75 Bid Size 1 Ask size 1 - - - - - I created a Forced Exit and stepped through 4 more bars, still no Exit. (M disappeared when long term chart fields updated at end of 2nd bar after (that's not uncommon; usually never affects positions nor exits)) See Attached Pic showing 4 bars later Position still -1 - - - Finally exited ON NEXT BAR change, still Forcing Trues (simplified to "=1" in N) SO: seems Exits sometimes only execute on Bar Close even if settings not set to that. Date Time Of Last Edit: 2016-05-10 16:33:01


Private File

[2016-05-10 17:16:51] #4 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
Are you sure there was no reason given in the Trade >> Trade Service Log about why the Sell Exit order was ignored? Also, even if you manually set N3 to =1, because the state does not go back to false and then to true, it will not change anything. We need definite and simple steps to duplicate this. If you can consistently duplicate this during the replay, then you should be able to give us the steps, and Spreadsheet to reproduce. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2016-05-10 23:37:54

[2016-05-10 22:56:03] #5 [Go To First Post]
User791263 - Posts: 151
I studied the Trade Service Log I keep open under Trade Activity Log-Fills-- I watch it. Nothing in it referred to an ignored Sell Exit. (which I see often at positions limit,etc) With my setting, Reset Condition on New Bar on Yes, N=1 should give a signal every new bar, No? I've studied and you explained intrabar conditions changing True Off and On. Each off-then-on True gives a New order (ie up to positions limit). This problem behaves oppositely to such 99.5%+ normal exits. There's nothing different in the condition trigger, which works fine most of the time. I've studied http://www.sierrachart.com/index.php?page=doc/doc_SystemsAlerts.php#DisappearingArrows and http://www.sierrachart.com/index.php?page=doc/doc_SystemsAlerts.php#IgnoredSignalsDelayedTrueResult Because 30+ charts & a few spreadsheets feed this spreadsheet, alone by itself it won't reflect actual data-conditions (it won't have data from feed charts for 5+ minutes). ie: When cold-started, the S.Sheet won't have the same data it does after running that long. This error was near mid-day after running non-stop from 8:23AM Central. Hence, I used simpler conditions (E3> E10,N =1) that don't rely on other data to test (per your instructions) & show you that the exit appears independent of other data or conditions. Since SC shows the -1 position, and all else works and agrees (including TradeService Log), it seems any short position creator could do this, rarely. Watching SC in .1 & .4 speed reveals at least one other odd issue: Sometimes OHLC Overlayed 1-second data will go blank (zeros) for an instant at beginning of target bar, even set to "Fill empty bars with last data"- Rarely also. This might be similar. Here, it seems the logic rarely doesn't see the Evaluate at end of bar Setting is OFF, for Exits.. sometimes. A glance at Exits' code for something re Beginning or End of Bar, if it checks settings in J-column-- might find it? Maybe a brief short Exit True within the Sell Entry row (prior to Entry) upsets the enabling of function of subsequent exits (though appearing).

[2016-05-10 23:46:01] #6 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
With my setting, Reset Condition on New Bar on Yes, N=1 should give a signal every new bar, No? Yes this has been verified now. Sometimes OHLC Overlayed 1-second data will go blank (zeros) for an instant at beginning of target bar, even set to "Fill empty bars with last data"- Rarely also. This might be similar. If you have set the Fill Blanks with Last Value Input to Yes, we would not normally expect this but unless we are given a simple example we can test we cannot comment on this. You need to provide a Chartbook, Spreadsheet and the steps to reproduce and then we can look at to scenario you originally described. We have no further comment until we get these and no further information has any meaning to us. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2016-05-10 23:47:24

[2016-05-25 21:24:35] #7 [Go To First Post]
User791263 - Posts: 151
We put in months & years of effort, study, tests to try to make S.C. & ourselves-- successful. I've spent more HOURS on this problem. It would be Fair to give me 5 minutes more before I try to build a test chartbook for you, within a week. Recall it's an Intra-bar orders problem- order not issuing on TRUE L,N & No ignore message. Over 2 more weeks, I've tested in slow-mo this sometimes-failure of SC to issue Exit orders intrabar - WITHOUT IGNORE MESSAGES. This time I used completely different formulas, different Column (N this time). It appears not unique to particular formulas. PLEASE LOOK at this, more EVIDENCE. It shows there IS a problem. I "stepped" at .2 speed the Message Log, Activity Log, Time&Sales, Spreadsheet through 12 seconds WITHIN a bar to show the N-col go TRUE, FALSE & back TRUE, but there was no "ignored because"-Log-Message & no Exit Order. (position -2) I made sure the signal goes TRUE, to OFF to TRUE again within the bar 2-Ways: 1) using an H-col. flip-flop, 2) with/without flipflop, where a signal goes True/off/True by itself within a bar. You first said an L or N might not be OFF before back TRUE, but admitted in #6 that could not be it under my Settings of Reset on New Bar. (My signals often go ON/OFF 2 or 3 times in a bar & 99.7% Issue normally). If had not been in Slo-Mo audit mode, I'd have never seen this. PLEASE look at the new Step-Pictures & answer ONE Question for all of us: It should not matter WHAT turns N or L true... does it? ... so when a Short Position exists, an Exit Market Order should issue on N-TRUE turning on in the bar, Correct ? If that is always true, then conclude you DO have a bug, and I'll try to help. Otherwise, CORRECT my Misunderstanding- Tell us what else bears, so I can see if I've overlooked weighting some other possibility. ie: What "always true" Basic fact can we rely upon?


Private File Private File Private File

[2016-05-26 09:20:39] #8 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
We have looked this over. Update Sierra Chart to the latest prerelease following these instructions: https://www.sierrachart.com/index.php?page=doc/download.php#FastUpdate You will need version 1419. In the Input settings for the Spreadsheet System for Trading study, set the Enhanced Auto Trade Signal Related Logging Input to Yes. Use this only when necessary because it will create a higher CPU load and a very large amount of messages. With this setting, you will know exactly why the signal is getting ignored. Also be aware that unless you see the corresponding Subgraph on the chart show an arrow for the bar that has a value of 1 in the corresponding cell, then the signal is getting ignored because there has not yet been at chart update picking up that new value. Same thing with 0. So the flip-flopping as you call it is not necessarily getting picked up by the chart. Do you have Global Settings >> Spreadsheet Settings >> Recalculate the Chart When a Formula Column Is Changed enabled? It is not necessary. We are just wondering how it is set. It can create more of a CPU load unnecessarily as you are working on the Spreadsheet. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2016-05-26 09:22:09

[2016-05-26 16:25:44] #9 [Go To First Post]
User791263 - Posts: 151
Thanks! You guys are Wonderful. Yes, Global Settings >> Spreadsheet Settings >> Recalculate the Chart When a Formula Column Is Changed was / is ON. (Necessary for fast intrabar accuracy for spreadsheet signals? or fast chart arrows?) What do you mean "unnecessarily" in "It can create more of a CPU load unnecessarily as you are working on the Spreadsheet"? Do you mean the Chart may not need to redraw because the spreadsheet recalculates on formula change regardless? (ie: Set Recalc Chart When Formula Change only if chart-redraw is needed when manually trading off the Chart?) But, set OFF might conceal Arrows, no? Arrows help to monitor Auto-Trade CPU load is important to us scalpers. Last night, I saw an approx. 1-second+ delay in order creation-issuance from a TRUE (Not col. N), which agrees with your explaination. I'll watch for that, for arrows, and follow your instructions w/ V-1419. So,duration of the TRUE is a possible factor. (BUT see below) This enhanced verbose Trade Signal Logging debugger looks GREAT! Just what we need! (A setting for Backtest "Pause" on TRUE or Order Issue would save TONS of time (you added a .01 (typed-in) speed, but it's slower than .01 yet still sometimes fails to show a very-brief True which issues order)). Some extreme-short-duration TRUE's which INSTANTLY issue orders (some may be on bar change where some fields are zeroed for next bar- I've seen)-- don't seem to follow your "getting ignored because there has not yet been at chart update picking up that new value". Those are the most-perplexing Orders and TRUE's that I can't find even on .01 (.003 actual) speed. I use Detail view all the time. A backtest "Pause on Turn TRUE" would help see data-conditions, such as an extreme brief combination such as fields initialize between bars or a precedence or Overlay data-timing issue (which we have ways to overcome IF we know what it is). (Yes, Fill Zeroes with Last Value was/is ON). A "Pause on Order Issue" would show the delay exactly (or a setting to pause on either). Now we're Getting somewhere! I'll report observations in a couple of days.

[2016-05-28 09:07:19] #10 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
For documentation for Global Settings >> Spreadsheet Settings >> Recalculate the Chart When a Formula Column Is Changed , refer to: http://www.sierrachart.com/index.php?page=doc/doc_GlobalSettingsMenu.html#SpreadsheetSettings Do you mean the Chart may not need to redraw because the spreadsheet recalculates on formula change regardless? Yes this is correct. We will see about adding an automatic replay pause. But we need to know under what condition specifically. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2016-05-28 09:07:41

[2016-05-30 15:13:52] #11 [Go To First Post]
User791263 - Posts: 151
Specifically, a setting to PAUSE on REPLAY when EITHER: 1) any K L M N goes TRUE or 2) on Order Issuing after True and/or (maybe) 3) Both of above-going TRUE & Order Issue & possibly on (not as important) 4) Order fill (simulation seems to sometimes have realistic slight delay even on market orders) Note: with Intrabar orders settings & 1-second data storage, current position may not be seen by some of the logic, and as you know, True's may go away before fill, & often by end-of-bar. SC not-seeing position within the bar is especially visible when "use attached orders" for Exits & Stops on Trade Window is checked, and TRUE goes ON/OFF/ON/OFF repeatedly-- gets that many orders (6-10+). (I would use auto AND attached Exits together if I could). Consistently, "Maximum Positions allowed" is exceeded when intrabar auto-trade K,M Entry TRUE's flicker on and off several times, then Orders & fills run up to 8+ when Max Positions is set on 3 (for example). Please keep a lookout for that, while looking at creating Pauses. (ie: "kill 2 birds with one stone"). On subject of Pause, you probably know the new Enhanced Message Logging (I'm trying to use) runs fast ALL the time, when Disconnected and Replay is Paused (ie: when not serving trade data). By the time I turn that setting off, the sought-condition is about 50 messages back-hard to find. While in there,please see if Enhanced Mssgs could pause on paused Replay or no records served, like normal messages do. Even if these improvements take weeks (we know programming is difficult & takes time), they'll be useful to both SC Engineers & traders. Date Time Of Last Edit: 2016-05-30 16:21:13

[2016-05-31 02:48:01] #12 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
Consistently, "Maximum Positions allowed" is exceeded when intrabar auto-trade K,M Entry TRUE's flicker on and off several times, then Orders & fills run up to 8+ when Max Positions is set on 3 (for example). Due to some recent changes with this, we would not expect this but apparently there are conditions where the Position Quantity can exceed the setting if that is what you are saying. It is not something we would look into or spend any time with, unless we are given a very simple and definite scenario where this can occur. After reviewing your request about pausing under certain conditions we decided we will not do this. It does not make sense because there are too many different and vague conditions when it should happen and we realize there is a better way. Instead we will add the ability to step through data record by data record. While in there,please see if Enhanced Mssgs could pause on paused Replay or no records served, like normal messages do. We would not do this. And normally we would not expect any messages when a chart is paused. If messages are being added, then they need to occur but we do not know under what condition that is happening. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2016-05-31 02:50:38

To post a message in this thread, you need to log in with your Sierra Chart account: