Forex strategy optimization is a must before using it in real trading. How exactly to do the optimization?
A standard practice is back testing on the historical market data. As each strategy has a number of parameters, we need to repeat tests for different parameter values and find out the values that make the most profit.
Parameters are independent, so the number of available combinations can be substantial. Consequently, solving such a problem of finding a multi-dimensional maximum can take a considerable time. Especially if tests are run on a long data interval.
To complete tests in reasonable time, it makes sence to test first those parameters combinations, which are believed to have the most influence on the risk level. Using market patterns, e.g. trending or rangebound market, is another way to reduce the computing time.
Another problem is – how much historical data should we use? What is the minimum data interval to test on to be confident in our strategy? There is no correct answer here.
In our example, our automated trading strategy was optimized on one month of data for Asian trading sessions. Asian trading is the least volatile and easier to trade. Still, for the next month the same parameters resulted in losses.
Back testing can’t guarantee success – Forex market is unpredictable. Does it make sense to test at all?
Our approach is to monitor the market in real time, and adapt the strategy to current market conditions on-the-fly.
Forex Strategy Optimization In Real Time
Strategy optimization with back testing can be called static, because the outcome of the tests are some fixed set of parameter values that is used unchanged in future.
Our approach we can call “dynamic” optimization. Contrary to static approach, here the tests are run in real-time on the latest market data. and so the test data always changes. We are working on an algorithm for adjusting the strategy parameters on-the-fly.
Dynamic optimization is an important part of our strategy development toolkit, which already includes a real-time pivot point detecting algorithm. Which generates trade signals at market tops and bottoms. As you understand, optimizing the detection process in real time will have an enormous profit potential.
By Forex simulation we usually understand back testing. While “real-time Forex simulation” sounds more like a demo-account functionality.
The advantages of testing on historical data are evident. You can practice at any time, and don’t have to wait for the market to open. And you don’t have to depend on the market action, either.
So does it make sense to try simulation in real-time? The answer, in our opinion, depends on the tools.
With a regular demo-account, the only benefit is practicing on live – i.e. unpredicatble – data. However, with a specialized tools, the real-time simulation has more advantages.
One of the main benefits is in so called “time shift”. This is what is supported in many modern DVR devices, where you can put your live TV broadcast on pause while answering a phone call.
The DVR continues saving the signal so that you can resume watching later from the moment you paused – or jump to the current moment.
The main advantage of this is many cases you can very quickly catch up with the current broadcast without losing too much – e.g. by fast forwarding slower scenes of completely skipping commertials .
Same idea can be fully implemented in Forex simulation. You can resume practicing with delay, and then gradually catch up by fast forwarding the slow moving markets.
Our Forex Simulator supports the real-time mode starting from the release 1.1. You can select of 10 main currency pairs.
Expert Advisors will be supported as a means to define Forex trading strategies under test in the next major release of the Smart Forex Tester. We are already working on that important new feature.
To give you some background, Smart Forex Tester is natively working with trading strategies defined as text-based state machines. We have developed a simple price action strategy example that illustrates how it is working.
But for some users, expert advisors are more familiar tool so it makes sense to add such option to the Smart Forex Tester. How can we do that?
First of all, let’s note that we can’t use the Expert Advisors in binary format (i.e. .ex4 – files). This is virtually prohibited by the Metatrader’s license. Without doing reverse engineering (which is out of question, of course), we can only test binary EAs if we have access to a Metatrader server. Which is just too costly for this purpose.
Fortunately, this is not a show stopper. If we define our own strategy that we want tested, we must have access to the source code. This is doable.
Expert Advisors Put Into The Sandbox
We will be using Expert Advisors normally in their “native habitat” – Metatrader. But we will replace all function calls that manipulate quotes and orders with our own functions. Other functions can work with Metatrader normally.
How Expert Advisors can be used with Smart Forex Tester
So we can see that Expert Advisors will be sand-boxed. After that, you can run the trading strategy tests like you normally do.
How to prepare the Expert Advisor for the test? You can open it in any text editor and made a context search-replace. We might also provide a simple tool for doing that.
Subscribe to be updated on the current development status.
You are probably aware of the sharp drop in the EURUSD on October, 21. Following the ECB announcement, the Euro fell over 2% against the Dollar.
It happened so that on that day we were testing the latest version of Smart Forex Tester. This upcoming release of our software will support forward testing – i.e. test the automated trading strategies on live data feed. For that, we have developed a special Expert Advisor that sends tick data from Metatrader.
So, we were running the stability test. We were using our automated trading strategy (comes bundled with with Smart Forex Tester). We left the tester running on its own. When we checked it the next day, we were stunned. Not by the fact that the tester was still up and running (this was expected).
Smart Forex Tester runs automated trading strategy in real time (click to enlarge)
What came as a surprise was that our automated strategy absolutely crashed it! It did 27 trades, out of which 60% were winners. And it raked 132 pips profit!
Automated trading is the way to go!
This test convinced us again that automated trading can be made profitable. We understand, that for this simple strategy that we were using, we had a bit of luck. The reason being that this strategy doesn’t yet have a trend following algorithm. We are only developing it.
Our line of reasoning is simple. First, to be successful, a Forex trader must have a clearly formalized strategy. Second, this strategy has to be strictly followed. No room for emotions. Now, if we have done the former – guess who will be more successful in the latter – a person or a computer? The answer is obvious.
If you are interested – stay tuned! Download Smart Forex Tester and wait for the announcement of the latest release.
In addition to back testing, Smart Forex Tester can also test trading strategies on live market data – in real-time!
For this functionality to work, you need to download the latest version of Forex Data Recorder, which receives real-time quotes from Metatrader via a special Expert Advisor (included in the download bundle).
Smart Forex Tester Reads Ticks From Metatrader
The figure below schematically shows how forward testing works.
Smart Forex Tester Supports Forward Testing
The EA box inside the Metatrader client terminal is a specially developed Expert Advisor.
Forex Data Recorder receives the ticks from the Expert Advisor and re-translates them to the Smart Forex Tester. All communication is implemented with the help of Windows messaging. This is fast and reliable method of data exchange within one machine.
Forex Data Recorder can save the received data to the file for later use for back testing. This is done in parallel with casting. In addition, the software has inbuilt test data generator which is useful for debugging purposes during the time when Forex markets are closed.
Stop loss orders are essential part of leveraged Forex trading, where the risk of margin call is significant. Manual trading without stops is of course possible, but it requires constant monitoring. For automated trading, stop loss orders are evidently a must.
Here we will discuss some ideas how stop loss orders can be optimized in an automated Forex trading algorithm. We will suggest market adaptation of the stop loss orders position based on our pivot point detection algorithm.
Our trading algorithm sets stop loss orders the next tick after positions opening, and then moves these stops in a trailing manner, securing the profit.
The screenshot below shows the very beginning of the test for the April, 24, 2105. In this baseline testing example, take profit value was set to 8 pips. Also stop loss orders were placed at a 8 pips distance from the trade, and were trailing market in the same increments.
Smart Forex Tester – Regular Stop Loss Orders
The first position opened was a short. We see that this sell order was closed profitably (leftmost green line). Few ticks after, the algorithm placed another sell order, which was a loser (leftmost right line). The stop loss oder was triggered almost immediately as the market continued higher.
The question is whether we would be better off setting the stop threshold looser? At least we could have the first position stay open longer, as we see that eventually market peaked and went down. And then we wouldn’t have the second loser trade at all.
On the other hand, with tighter stop, we had losses on the second trade, but then a new short position was opened very soon. And it turned out a success. You can see the stop loss order was trailed 5 times before it was triggered. The position yielded about 30 pips of profit.
Let’s see the test results for the looser stops.
Stop Loss Orders Set Further From Market
Looser stops might be justified for trades with low leverage or without it. To simulate it, we increased the threshold for triggering stop loss orders. In the example above, we set it to 20 pips instead of 8.
Smart Forex Tester – Stop Loss Orders Set Looser
As expected, the first position was closed much later than for the tighter stop value. And we avoided placing the second (loser) sell order.
However, this didn’t increase our profit. As seen on the screenshot above, the looser stop loss order was also triggered later (green line) so the profit potential was used only slightly.
This simple example brings us to the conclusion that the algorithm must use some logic to modify stop loss orders and also adjust to the market situation.
Stop Loss Orders Modification Algorithm
First of all, it appears we should differentiate between the cases of initial placement of a stop loss order and the case of its trailing. If looser stops are acceptable as such (e.g. no leverage), then it might make sense to wait longer for testing the market entry decision.
But when the position becomes profitable, the stop loss order should be moved closer to the market. And even closer for each subsequent move. This will secure more profit when the market reverses.
Moreover, while moving the stops, we can make the distance between the stop and the market proportional to the size of the preceding market’s advance. In other words, the faster-moving is the market, the slower we can afford to approach it.
If a stop loss order is triggered during a counter-trend movement, it’s the trading algorithm’s task to re-enter the market.
On the screenshot below you see the same market data with detected peaks (red dots) and bottoms (blue dots).
The accuracy of the detection might not look ideal, but keep in mind that it was performed in real time! Meaning that for each detected extreme, only the market data that was available prior to that time, was used. And also note that the market data plotted is smoothened for better readability. Whereas the detection has to use real-time quotes that might fluctuate quite a lot.
Smart Forex Tester – Pivot Points Detection in Real-time
However imperfect, these signals are good enough for our purposes of modifying the stop loss orders. So, e.g. for the looser stop, if we used the bottom signals, we could have 2 of them around the very bottom and could have fixed almost maximum possible profit. See the 2 blue dots to the right of the black vertical cursor line in the middle.
This post is a comprehensive example of Forex strategy testing with our Smart Forex Tester software. We used our example automated Forex day trading strategy.
Our test plan was to run the strategy on all Asian sessions for one month. The strategy is very simple and it doesn’t adjust its parameters to the market conditions. So we wanted to keep all algorithm’s parameters the same during the testing.
Asian sessions were selected as test data because their volatility is known to be low. It is important, because our simple strategy doesn’t have a trend following algorithm.
We tested on EURUSD. We used tick-by-tick market data provided by TrueFX. We took a whole month data file for April 2015 and prepared our test data with the help of our Forex Data Manager software (download).
Overall test results were positive. Total profit exceeded 280 pips. Out of 17 trading sessions, 12 were profitable.
Day Trading Strategy Test Results
The best result we got for the April, 24. We can see that our algorithm managed to enter the market at tops and bottoms quite well and stop orders’ trailing worked properly as well.
Forex Day Trading Strategy Testing – The Most Profitable Asian Session
As the algorithm doesn’t have any trend following functionality, we can only explain this fact by the slow enough market movement.
The worst session was on April, 21. This can be attributed to visibly higher jitter in quotes, so our algorithm had a lot of false signals.
Forex Day Trading Strategy Testing – Asian Session (Worst Case)
Interestingly, we could easily turn this worst trading session into a profitable one by tweaking the Pivot Points detector parameters. But we didn’t do that as this was against our test plan.
However, this observation is one of the most important results of the testing, which gave us strategy development ideas.
If you want to repeat our testing, here are our parameters.
Our testing results show that automated day trading can be profitable.
Our strategy is very simple but still it could win 70% of all trading sessions!
It is clear that priority needs to be assigned to implementing algorithm parameters adjustments on the fly.
An obvious idea to try first would be adjusting the detector parameters. We can optimize them on each tick (or, say, every 10-100 ticks) using the data from beginning of the test. This way, the jitter observed on April, 21 would have been easily eliminated.
We show how we prepare tick by tick data for our Forex strategy testing. Test data quality is one reason why we don’t use Metatrader Strategy Tester – it is generating ticks by extrapolating M1 bars data. Such artificially smooth quotes are not reliable test data. So we only use real market data.
We use the tick by tick data provided by TrueFX. It is real tradable quotes data – and it is free. History market data is stored as a separate *.csv data file for each month. The only problem is that the size of a monthly file is substantial – for a typical month it exceeds 200 MB, which makes it too big for MS Excel to open.
Tick By Tick Data Editing By Forex Data Manager Software
However, working with big data files is not convenient and is also not always reasonable. E.g. we don’t test on the data intervals spanning over weekends.
To get the most of the real market data, we have developed Forex Data Manager. This software reads a month long tick by tick data file, checks data integrity and parses it into trading days. Then we can select smaller chunks of the data in the GUI and save them into their own files. We can save one or multiple trading days or even less than a day – e.g. a single trading session.
Forex Data Manager is included into Smart Forex Tester bundle (Download). After you unzip the archive, start the software by double clicking on fdm.exe.
In the example below we show how we parsed the EURUSD data file for April 2015 and extracted Asian sessions as separate test data files.
In the GUI, click Browse and select a full month tick by tick data file that you downloaded from TrueFX. After parsing is complete, you will see the following:
Tick-by-tick Data Editor Software GUI
By default, all trading days are checked. To remove selection, click on Toggle All. Then you can check one or multiple days and click Save.
We will be extracting the data for Asian session. It is more complicated as it spans over 2 days: between 23.00-08.00 GMT. For that, we click on Options and check the box Specific hours and then type in the From and To values as shown. Note that we typed 07 – which means that all ticks until 07.59.59:999 will be included.
Tick-by-tick data editor: selecting data by hours
Now, we click OK and we select two adjacent days, e.g. April 23rd and April 24th. This means that we will be saving a data segment spanning from 23.00 of April 23rd until 08.00 of April 24th.
Tick-by-tick data editor software – saving selected data
You give a file name and click Save – the process is completed. We included this saved session file as an example in the Smart Forex Tester download bundle.
Why are we interested in Asian sessions?
Because there is less volatility there and usually there are no breaking news and other market moving events like e.g. FOMC press-releases. It is often recommented for newbee traders to start from Asian session.
We used this data for testing our example Forex day trading stratedy. This algorithm is very simple and doesn’t have the trend following functionality, so this setup suits perfectly.
Historical data is a prerequisite for Forex trading strategy back testing. Its quality directly influence the reliability of the testing results.
So, well-known MetaTrader strategy tester generates tick data by interpolating it from M1 data. The quotes obtained this way are artificial and can lead to incorrect testing results.
The best option is using real market tick data. We use historical Forex data provided by TrueFX. They share Interbank high quality quotes for 15 major currency pairs. You can download archive data since 2009.
In addition, you can get streaming quotes via HTTP API. So you can very conveniently do forward testing as well.
And the best part is that TrueFX provides this quality data absolutely free!
The data is arranged in .csv files each containing 1 month of tick quotes. So the files are over 200 MB in size, which is a lot. MS Excel fails to open files of such size.
Our Smart Forex Tester software supports the TrueFX quotes and it can load a monthly file without problems. The Strategy Tester also supports cutting fragments and saving them as separate files.
However, it is convenient to split big files in smaller. For example, a week is a logical portion – from the start of trading on Sunday in Tokyo to the end on Friday in New York. We split for even smaller chunks – trading sessions or other strategy test scenarios.
To make splitting the historical market data files easier, we developed Forex Data Manager software. It is included in our free Forex Simulator.
Historical Forex Data Editing Software
Data manager reads a .csv file and shows each trading day it found there. During download, data integrity is checked and if errors are found, status string shows that and error log shows which data is wrong. Then you need to fix the problem manually.
If data parsing is successful, you can select the data subset you want to save by using day checkboxes and start/stop time controls. They are shown if you click “Options“-button. Once selection is done, click “Save” and specify a file name.
You can do one save at a time. If multiple days are selected, they will be merged.
Start/stop times are specified in 24H format. Times are GMT. If the stop time is earlier than start time, you need to select 2 adjacent days, and then a data segment will be written that stretches over midnight.
Historical Forex Data Editing – Asian Session
For example, if you want to write the Asian session, you set start time to 23 and the stop time to 7. The latter means that all ticks with timestamp up to 07.59:999 will be written.
