Support Board
Date/Time: Sun, 19 Jan 2025 09:12:34 +0000
sim (becktest) vs. live real time
View Count: 2701
| [2018-03-24 10:02:16] |
| User30743 - Posts: 365 |
|
i wanted to ask, how it is possible that there are such significant differences between real-time live automatic trading based on ACSIL and sim bactest results of the same AOS? for example on Friday, a simple script based on price action was set to do real-time trading (written in ACSIL) and it did not take a trade that it was supposed to take according to the condition written in the code. i did a sim backtest of the same day with the same AOS and it did take the particular trade. the question is - how it is possible that in backtest it takes a trade and in realtime live it does not - when it is completely the same code?? if it works like this, then all the coding and bactesting of an aos strategy is a complete waste of time. i understand that there might be some slippage or that a simulation environment is slightly difeerent ets.. something like this, but i dont understand why it does not take a trade when the condition is met.. and in sim beckest mod it takes it with no problem can you explain please? Date Time Of Last Edit: 2018-03-24 10:08:44
|
 live mode.png / V - Attached On 2018-03-24 10:01:55 UTC - Size: 39.57 KB - 465 views sim mode.png / V - Attached On 2018-03-24 10:01:59 UTC - Size: 53.93 KB - 425 views live mode result.png / V - Attached On 2018-03-24 10:02:03 UTC - Size: 33.91 KB - 372 views sim mode result.png / V - Attached On 2018-03-24 10:02:05 UTC - Size: 35.99 KB - 432 views |
| [2018-03-24 11:24:07] |
| |
|
Why are you asking us this question? It is necessary for you to explain this to us. This is your trading system. Not ours. So what is the answer? Tell us what the problem is. Refer to: Automated Trading Management: Troubleshooting Automated Trading System Behavior Auto Trade System Back Testing: Differences Between Back Testing and Real-Time Auto Trade System Evaluation Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2018-03-24 11:27:36
|
| [2018-03-24 17:37:43] |
| User30743 - Posts: 365 |
|
well, ok i understand digging into someones AOS is beyond your scope of support on the other hand i am not asking anything so strange or super detailed to be explained. my question is simple and very general: why an acsil DLL behaves differently in realtime live and on sim backtest mode? when i have the same code, the same setting, in the same market with same timeframe, i expect it to do the same thing on live as well as on sim mode. (except some slight differences in fills due to slippage etc.) these two pics show the difference: on sim becktest it takes a trade on realtime it does not (without sending anything to service log) - it just skips it Date Time Of Last Edit: 2018-03-24 17:40:36
|
| sim mode (1).png / V - Attached On 2018-03-24 17:38:50 UTC - Size: 53.93 KB - 423 views live mode.png / V - Attached On 2018-03-24 17:39:07 UTC - Size: 39.57 KB - 363 views |
| [2018-03-24 20:49:49] |
| |
|
Refer to: Auto Trade System Back Testing: Differences Between Back Testing and Real-Time Auto Trade System Evaluation Automated Trading Management: Differences Between Live Automated Trading and Simulated Automated Trading Automated Trading Management: Troubleshooting Automated Trading System Behavior Read through each of these sections in their entirety. This is a question that only you can answer: my question is simple and very general: why an acsil DLL behaves differently in realtime live and on sim backtest mode? If you fail to recognize that, it only shows that you should not be creating your own automated trading system and it is beyond your abilities. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2018-03-24 21:11:47
|
| [2018-03-25 02:41:42] |
| |
|
We added some additional items of information to this section: Automated Trading Management: Troubleshooting Automated Trading System Behavior Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2018-03-25 02:52:45
|
| [2018-03-25 21:55:25] |
| User30743 - Posts: 365 |
|
ok, thanx. of course i have read these topics before and as they did not tell much about this problem, that was why i sent a request to the support. there are couple of other things which you might not have any idea about and which does not make any sense at all: for example 1) the same dll with the same setting when being becktested on sim1, sim2, sim3, sim4... etc. at the same copy of sierra opens trades at different timestamps for each sim.. 2) the same dll with the same setting when being bectested on different systems (laptop vs. VPN) returns different results. 3) etc.. Date Time Of Last Edit: 2018-03-25 21:57:52
|
| [2018-03-26 02:24:31] |
| |
|
You still are failing to simply examine your own trading system. You must examine your own code. The problem is on your side. That is abundantly clear. 1. If this is true, then you would notice differences with repeated back tests on the same simulation account. What this only shows is that your trading system is unstable in its behavior. Why your system is giving such inconsistencies is something you need to resolve. Refer to: Auto Trade System Back Testing: Consistency Between Back Tests We are not sure what type of back test you are performing as there are several supported. If you are using a replay back test with Accurate Trading System Back Test Mode, there is the potential for some slight inconsistencies when it comes to the number of times the study function is called. We know this is going to be very small and we did an experiment to see the extent of the problem during the day trading session for 3 days: Replay 120X: YMM18 1 Min #1 | Back test processing: NumberOfTimesCalculateCalled (Cumulative) = 237879 | 2018-03-25 22:14:08 Replay 120X: YMM18 1 Min #1 | Back test processing: NumberOfTimesCalculateCalled (Cumulative) = 237880 | 2018-03-25 22:14:22 Replay 120X: YMM18 1 Min #1 | Back test processing: NumberOfTimesCalculateCalled (Cumulative) = 237880 | 2018-03-25 22:14:38 Replay 120X: YMM18 1 Min #1 | Back test processing: NumberOfTimesCalculateCalled (Cumulative) = 237879 | 2018-03-25 22:14:48 Replay 120X: YMM18 1 Min #1 | Back test processing: NumberOfTimesCalculateCalled (Cumulative) = 237879 | 2018-03-25 22:14:59 Replay 120X: YMM18 1 Min #1 | Back test processing: NumberOfTimesCalculateCalled (Cumulative) = 237880 | 2018-03-25 22:15:14 Replay 120X: YMM18 1 Min #1 | Back test processing: NumberOfTimesCalculateCalled (Cumulative) = 237880 | 2018-03-25 22:15:26 Replay 120X: YMM18 1 Min #1 | Back test processing: NumberOfTimesCalculateCalled (Cumulative) = 237880 | 2018-03-25 22:15:44 Replay 120X: YMM18 1 Min #1 | Back test processing: NumberOfTimesCalculateCalled (Cumulative) = 237879 | 2018-03-25 22:15:55 So as you can see, there is a difference of 1 between these back tests in regards to the number of times the study function is called. This is tiny and insignificant and essentially irrelevant. The basic point that the documentation makes is that if your trading system is so affected by this, then it shows it is unstable and not well designed. And while it may seem better to get a consistent result it is completely meaningless when it comes to the fact that during real-time updating you are going to get yet another result. opens trades at different timestamps OK, so tell us why this is the case. It is trading system doing this! You are failing to understand this.2. This is likely because there is some variation with the data between systems. That is the more likely explanation. And once again what this is really revealing is your trading system does not have a stable design to it. And also, we do not even know how exactly how you are doing the back test and whether the back test is started at the exact same Date-Time in the chart. If we are mistaken and there is some inconsistency on the Sierra Chart side, you can determine precisely what it is and let us know. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2018-03-26 04:03:27
|
| [2018-03-26 12:44:07] |
| User30743 - Posts: 365 |
|
it is not my code, it is your code that i tried to auto-backtest on different systems and on different sim account within the same copy of sierra to see how it behaves. what i found out was that it behaves slightly differently each time. if your code does it, then mine will do it too for sure. why? because: (1) i am not a fulltime expert c++ senior engineer (i wish i were) and (2) i often use some parts of your code given as examples to build upon. yes, you might say that the differences are slight and insignificant. i agree (sort of). when an entry timestamps differs in couple of dozens of milliseconds it is "insignificat" for some people, for others it is not, because different timestamp of an entry also means different price of an entry and it implies different price of and exit which may completely mess up all the rest. again, i understand it behaves differently in live and sim. but i expected sim1, sim2, sim3 to be exactly the same for the same dll with the same setting (and it is your dll, not mine). dont get me wrong -i am not accusing you of anything. i mean i really admire and appreciate what you did. i understand it must be extremely difficult to create something like a trading platform of this kind, with all its features, functions, support etc. - you are all heroes to me, no joke. but please don´t make a fool of me telling me that i should investigate my code and find out a why it is ustable - when it is your code given as an example Date Time Of Last Edit: 2018-03-26 12:49:06
|
| [2018-03-27 04:31:39] |
| |
|
We have never noticed a back testing inconsistency with any of the trading systems which are referenced here: Automated Trading From an Advanced Custom Study: Example Trading Systems and Code We just tested one of them earlier today and there was a consistency between two backs started with Trade >> Auto Trade System Replay BackTest. What particular trading system in Analysis >> Studies >> Add Custom Study >> Sierra Chart Custom Studies and Examples are you using? Provide a chart containing that trading system, following the instructions here: Support Board Posting Information: Providing Chartbook with Only a Single Chart Before attaching the chart, set that chart to use a specific non-global time zone: Time Zone Setting: Using Different Time Zone in Individual Charts We will then test and see why there is a problem. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2018-03-27 04:32:43
|
| [2018-03-28 13:50:08] |
| User30743 - Posts: 365 |
|
i used trading example with hardcoded attached orders (scsf_TradingExampleWithAttachedOrdersDirectlyDefined), but the same happens with all other studies. add this study to a clean chart on sim1, run a bectest, then switch to sim2, run the same bectest, switch to sim3 run the same becktest... in all cases u will see the timestamp entry differs (couple of dozens miliseconds - not a big deal but still, it is there. to me as a non-pro software designer this is a mystery why it can happen). on different systems it differs for a different ammount of time. sometimes 10 ms, sometimes 20, sometimes 30... when i restart siera and run the test "for the first time", the timestamp is the same as it was on the previous sim1. so it seems like if siera holded some latency somewhere inside after each test that influence other tests. but i have really no clue. not a big inconsistence, that is true. but when i take into account that this is really a very very simple trading example which u designed, i am asking to myself what kind of inconsistency i should expect from a trading system with several loops using tick data, market depth, itterating through stl containers and other much more complex stuff (á ala hft algo), which will be designed by me, an amateur.. Date Time Of Last Edit: 2018-03-28 19:31:58
|
 rty for sc support.Cht - Attached On 2018-03-28 13:44:35 UTC - Size: 31.02 KB - 344 views |
| [2018-03-29 02:51:08] |
| |
|
In post 10, you are apparently only pointing out inconsistency with timestamps. This is meaningless. And it has to do with the fact that there is other data in the Trade Activity Log, which is accumulating as you are changing the trade account, and timestamps need to be unique so this is why the milliseconds are incremented to avoid conflicts. There are no profit/loss discrepancies in our testing. None at all. So after this entire thread, what have you proven here? We have not identified any fault here in Sierra Chart. If there is, will take care of it. Therefore you can understand, why we are very assertive to users at times. When we know there is not a problem we are going to be very clear about that. And there is no problem. We simply cannot identify any issue. Obviously the original issue is an inconsistency with the submitted orders, and Profit/Loss. Millisecond timestamps differences are meaningless. So once again as we have said time and time again check your system if it is giving different results. Our example trading systems are not. We do not know why you are having an inconsistency with yours assuming you are with back tests. Now while looking into this, we did make a small performance improvement in 1726, and made absolutely certain there is 0.000000% potential for any variation. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2018-03-29 05:02:24
|
| [2018-04-03 13:39:45] |
| User30743 - Posts: 365 |
|
oooh, sorry. i did not mean to make u feel upset or angry because of that..
Date Time Of Last Edit: 2018-04-03 13:47:06
|
To post a message in this thread, you need to log in with your Sierra Chart account: