Support Board
Date/Time: Thu, 21 Nov 2024 21:34:12 +0000
Backtest positions does not correlate with trade result and sc.GetTradePosition
View Count: 557
| [2024-08-24 16:01:24] |
| PS - Posts: 40 |
|
Hello, I have a strategy I am backtesting whose orders (as per the order markings on the chart and the trade activity window) are not working correctly at all. To debug this, I did the following: 1) made my strategy draw arrows on the chart when buy and sell signals are generated 2) printed the trade result of every trade when it is placed 3) printed the current position using sc.GetTradePosition every 5 bars These 3 measures above all show trades being taken at the correct times. That is, arrows are drawn when and where they are supposed to, every single trade result printed = 1 and is at the correct price and time, and the sc.GetTradePosition data printed reflects what the strategy should be doing. However, despite these 3 checks being accurate, the actual positions shown by the Trade Activity Log are completely wrong and seemingly uncorrelated to my strategy. Any idea why this is occurring? Normally, I would think this is merely a problem on my end with my code, but since the 3 checks I mentioned above are working "right", I'm conflicted. |
| [2024-08-26 14:52:02] |
| PS - Posts: 40 |
|
@Sierra_Chart Engineering, do you have any idea as to why this may be happening?
|
| [2024-08-26 19:33:31] |
| John - SC Support - Posts: 36198 |
|
We are not understanding the particular issue. Are you stating that the trades are correct in real-time, but are not working properly in back-testing? If so, then this could be due to the back-testing method you are using? How are you performing the back-testing? Refer to the following: Auto Trade System Back Testing: Differences Between Back Testing and Real-Time Auto Trade System Evaluation If it is something else that you are trying to point out, then we need more information on what exactly it is. For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-08-26 20:46:08] |
| PS - Posts: 40 |
|
I apologize for the lack of clarity. This strategy is only being used in a backtest. No live trading is occurring. In the backtest, I am using 3 methods to verify that orders are being placed correctly (again, in the backtest). Those three methods are: 1) made my strategy draw arrows on the chart when buy and sell signals are generated 2) printed the trade result of every trade when it is placed 3) printed the current position using sc.GetTradePosition every 5 bars These three methods above show that orders (in the backtest) are being entered correctly. HOWEVER, my trade activity log, and the trades that show up on the chart when you enable the Trade/Show Order Fills setting are entirely incorrect. Once again, sorry about the confusion. Is the problem clear now? |
| [2024-08-26 21:24:36] |
| John - SC Support - Posts: 36198 |
|
We need to see images of what you are seeing. This simply is not making sense that the order fills in the Trade Activity Log are different from where the arrows are drawing for the order fills on the chart.
For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-08-27 15:41:15] |
| PS - Posts: 40 |
|
I apologize, I think there is still some confusion. There are a total of 5 "sources" I'm using to validate the backtest trades. 1) literal arrow drawings I'm creating via the chart drawing capabilities in the backtest 2) printed trade result (trade result = 1) of each trade 3) Printed current position using sc.GetTradePosition 4) buy / sell orders shown on chart via the Show Order Fills setting 5) trade activity log Sources 1-3 all show the "correct" trades being taken. That is, the backtest working as it should. Sources 4 and 5 show entirely different, incorrect orders. Let me share an example with pictures. The first trade in the backtest should be taken on 8/5 at 16:30 (15m chart). I've attached 3 images. My message log, the picture of the chart, and the trade activity log. In the message log, you'll see that a message printed saying buy conditions were met, and the trade result printed as 1 (successful). The current position 5 minutes later also shows long 1 contract. In the chart image, you'll see a light blue arrow at the time the trade was SUPPOSED to be taken. However, you'll also see the green triangle much later on at a different price displayed by the show orders / fills setting. Lastly, in the trade activity log, you will see no trade taken on 8/5 at 16:30. |
 sc trade activity log.jpg / V - Attached On 2024-08-27 15:40:28 UTC - Size: 283.64 KB - 46 views sc message log.jpg / V - Attached On 2024-08-27 15:40:44 UTC - Size: 239.85 KB - 47 views sc arrow.jpg / V - Attached On 2024-08-27 15:40:52 UTC - Size: 99.41 KB - 47 views |
| [2024-08-27 17:00:29] |
| John - SC Support - Posts: 36198 |
|
Have you run a basic replay (not a backtest) and verified that the point at which you are expecting the order to be placed is where it is being placed? We do not know if the issue you are seeing has anything to do with your study or with backtesting. We would need to know that your study is correctly placing the trades in the replay situation, otherwise you need to look into your code to determine what is not working correctly. For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-08-30 17:17:09] |
| PS - Posts: 40 |
|
Apologies for the delayed reply. I ran the replay (although I'm not quite sure what the difference is between a replay and replay based backtest), and I got the same issue. That being said, how is it an error in my strategy code if the Trade Result messages getting printed and the sc.CurrentPosition is working as it "should"? Thank you, PS |
| [2024-08-30 18:20:37] |
| John - SC Support - Posts: 36198 |
|
Depending on the backtest method you are using, you can get results that are not exactly what you expect. We needed to know that you are seeing the same affect with a standard replay, as this tells us that the issue is with your code. You are simply doing something wrong in the way you are entering the trade. We do not know what it is or why, but this is NOT a Sierra Chart specific problem. Note that we do not give programming help. You will need to debug your code to figure out what is occurring. For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-09-04 01:07:52] |
| PS - Posts: 40 |
|
Hello. I have done some further digging, and am running into even more curious problems. I tried running a strategy of mine in a backtest that I know previously worked as I had expected. I was running into similar issues now, of the trades listed being totally inaccurate. So, I then tried using one of Sierra's templated strategies, since those surely should work correctly. And, in doing a Replay BackTest, I once again ran into the same problem. I've attached a picture of a TSLA chart with Sierra's template moving average crossover strategy run on it. Please note that the green / red arrows displayed by the strategy itself do not match the order markers. I will try uninstalling and reinstalling Sierra, to see if that helps. Will report back what happens. |
| Sierra TSLA inaccuracy.jpg / V - Attached On 2024-09-04 01:07:42 UTC - Size: 304.26 KB - 44 views |
| [2024-09-05 12:07:54] |
| PS - Posts: 40 |
|
Hello again. Uninstalling and reinstalling didn't work. I'm quite confused, as Sierra's own template strategies don't seem to be working as expected on my machine. I'm certain the Sierra template strategies were written correctly, so perhaps I am doing something incorrectly in running the backtest that is causing these errors. And those errors may also be what's causing troubles with my own strategies. Please advise. Thank you, PS |
| [2024-09-05 14:18:39] |
| Sierra_Chart Engineering - Posts: 17138 |
|
You definitely do not want to uninstall Sierra Chart. That is a completely unnecessary procedure.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-09-05 17:02:01] |
| PS - Posts: 40 |
|
Well, I didn't have many presets and I sent myself all my code files, etc. So it ended up just being a waste of about a half hour or so. Do you have any ideas as to what might be going on and how I can resolve it? |
| [2024-09-05 19:43:47] |
| Sierra_Chart Engineering - Posts: 17138 |
|
Regarding post #10 the arrows from a trading system are not necessarily going to correspond to the order fills you receive. An order could be ignored, and the reason for that would be given in the Trade >> Trade Service Log. However, what we see does not look right to us. We need to do some testing and get back to you on this. Review the information here because it does affect, how the position quantity and open/close state is determined, for order fills: Trade Activity Log: Understanding and Setting the Start Date-Time for a Trades List There is this new setting in Chart Settings, and for back testing we would recommend keeping it at 0: Chart Settings: Order Fills Number of Days to Load (Chart >> Chart Settings >> Trading menu) Most likely this is the issue. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-09-05 19:46:02
|
| [2024-09-06 11:53:10] |
| PS - Posts: 40 |
|
Hello, I tried adjusting the "Order Fills Number of Days to Load" setting to 0, and this did not resolve the issues. And, in regards to the arrows not matching the orders because of potential order ignoring, I checked the Trade Service Log and it was empty. So no trades were being ignored. Plus, according to the backtest, orders are occurring in spots where an arrow was never even there, which is outside the scope of an order being ignored. I've attached a picture of the results from another Replay Backtest using Sierra's MA Crossover strategy on a 15m TSLA chart. Similar inaccuracies remain. Please do let me know what you discover in your testing -- I look forward to resolving this issue and getting back to some development using Sierra's incredible software! Thanks, PS |
| TSLA Sierra MA Backtest.jpg / V - Attached On 2024-09-06 11:49:06 UTC - Size: 580.65 KB - 41 views |
| [2024-09-09 12:20:29] |
| PS - Posts: 40 |
|
Good morning, Sierra Engineering. Hope you had a good weekend. Were you able to reproduce this issue / do some testing? Thanks, PS |
| [2024-09-09 13:12:20] |
| Sierra_Chart Engineering - Posts: 17138 |
|
We did not do any testing yet until now. We need to know precisely the name of the study you are using for this: Replay Backtest using Sierra's MA Crossover strategy
We tested just now: Trading Example: Moving Average Crossover with Attached Orders Here is what we see at a portion of the chart: https://www.sierrachart.com/image.php?Image=1725887452256.png We did not observe any issues looking at some of the order fills. One thing to understand, is that the exit order which is done by an Attached Order is not going to have any arrow on the chart. You just see an order fill on its own for it. We apologize for the colors. They are not the best. One thing that we did though is we paused the back test in the current state. Maybe there is an issue, when the back test completes and then later if there is a full recalculation of the study the arrow placement is different. Then we need to examine why there is a difference in that case. We also think we are going to add a back testing speed setting. So that you can vary the speed during a back test. And make it slower to observe what is happening. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-09-09 14:06:56
|
| [2024-09-09 13:15:34] |
| Sierra_Chart Engineering - Posts: 17138 |
|
Just now we did a Chart >> Recalculate and everything looks the same: https://www.sierrachart.com/image.php?Image=1725887715205.png So we do not observe any issue with the recalculation either. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2024-09-09 13:15:46
|
| [2024-09-09 14:25:35] |
| PS - Posts: 40 |
|
Thank you for sharing. I am a bit busy with trading at the moment -- I will share all the requested details later today, and will try to even screen record my Sierra Chart as I perform the backtest, so you can see exactly what I am doing and perhaps point out a mistake. Thanks again, PS |
| [2024-09-11 20:31:10] |
| PS - Posts: 40 |
|
Apologies for the delay. Here's a link to a Loom video about 3m long, showing me running the backtest. The incorrect orders are evident on the screen. Please let me know what I may be doing wrong or how to resolve this. Thanks. https://www.loom.com/share/ee6b451c8bea4aa8a6afc4e48bfd3eb4?sid=6e1f1e99-34da-47a0-b2b2-2190e8703159 Date Time Of Last Edit: 2024-09-11 20:31:48
|
| [2024-09-18 16:53:52] |
| PS - Posts: 40 |
|
Apologies for the double message, although I think this may have been missed. Please do watch the short video when you get a chance and let me know what I may be doing wrong in running the backtest, if anything. To reiterate, the strategy being run is Sierra's own moving average cross strategy (as shown in the video), and the backtest is showing clearly incorrect trades. Thanks, PS |
| [2024-09-18 17:17:22] |
| User431178 - Posts: 537 |
|
Please do watch the short video when you get a chance and let me know what I may be doing wrong in running the backtest, if anything.
Read the docs: Replaying Charts: Replay Mode Don't use Standard Replay for testing automated systems, instead use Accurate Trading System Back Test Mode If you are performing a replay based back test of an automated trading system when using the Spreadsheet System for Trading study or an Advanced Custom Study that uses the ACSIL Trading functions, then you need to set the Replay Mode to Accurate Trading System Back Test Mode.
|
| [2024-10-03 14:10:43] |
| Sierra_Chart Engineering - Posts: 17138 |
|
Yes this was overlooked. If there is no longer an issue that is good. When providing a video, attach it here. Do not link to an external website.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2024-11-05 16:56:11] |
| PS - Posts: 40 |
|
Hello. Apologies for the late reply. I tried following as mentioned, switching to Accurate Mode, and orders are still incorrect. Attaching picture here. Orders should simply be made on the moving average cross, as per Sierra's Trading Example MA Crossover strategy code.
|
| Sierra Accurate Mode.jpg / V - Attached On 2024-11-05 16:55:23 UTC - Size: 612.15 KB - 13 views |
| [2024-11-05 18:00:32] |
| User431178 - Posts: 537 |
|
Attaching picture here. Orders should simply be made on the moving average cross, as per Sierra's Trading Example MA Crossover strategy code. Picture looks normal and correct. That system enters on close of bar. Where do you see errors? |
To post a message in this thread, you need to log in with your Sierra Chart account: