Forex Historical Data For Strategy BackTesting

Forex strategy testing – as any other testing – is impossible without proper tools and methodics.

Take a look at the results of our poll – it shows that many think that in testing Forex, historical data quality is a top priority. We can’t agree more.

The Most Important for Forex Strategy Testing is ...

View Results

Loading ... Loading ...

But what is the quality market data? First of all, it must be real quotes. For this reason, we can’t be satisfied in MetaTrader approach, where tick data is not real but is instead interpolated from M1 data. This makes the quotes artificially smooth so it can result in incorrect testing results.

We only use real interbank tick quotes for testing. Where to download such data?

Excellent source of Forex historical data is TrueFX. They provide high quality quotes archive since 2009 and they also provide streaming quotes for 15 currency pairs.

Even better news is that the market data is free! You only need to register with them. We use their data and recommend you download your copy.

The latest release of our Smart Forex Tester works with the TrueFX quotes.

Each data file has one month of tick data. You can open it in the Smart Strategy Tester, and then split it and save as smaller files (e.g. a week each).

 

Simple Forex Tester With Trading Strategy Builder

How to develop a simple Forex tester?

First of all, it is essential to make testing tools simple, as Forex strategy testing is a very complex activity. A lot of factors need to be taken into account and the importance of the correct results is high. Testing errors will be punished in real trading immediately.

It goes without saying, that simplicity should not be achieved at the expence of important functionality.

We think that one of the main factors the simplicity is about is how the software defines the a trading strategy. This is always a challenge. In the majority of the testing software, the Forex trading strategy is defined as a program. E.g. to test with MetaTrader strategy tester, you need to master mq4 coding.

But it is clear that not everybody interested in Forex testing is even familiar with coding, to say nothing about comfortable with it to write programs quickly and reliably.

So to make a tool useful for much wider audience, we desided that to be simple, a Forex tester must not require users to have coding skills to be able to test their trading strategies.

Simple Forex Tester = Simple Trading Strategy Definition

Our software only needs a user to make a simple textual definition of the trading strategy.

On the figure below, we explained how we make it simple: Forex tester software itself includes all standardized parts and only requires the user to define her trading strategy  as a text-based state machine.

Simple Forex Tester Design
Simple Forex Tester Design

All the rest is handled by our software framework.

TO BE CONTINUED!

Download Smart Forex Tester

NEW! Day Trading Strategy building with Smart Forex Tester (example) 

The Best Forex Strategy Ever?

What is the best Forex strategy? Or should we fisrt ask whether such thing as the best strategy exists? These are literally “million dollar questions”.

But before even thinking of the best strategy, let’s try to understand what a good strategy is. How can we define it?

The obvious answer is that good should be a synonym to winning. This is true of course, but we are afraid there is just no such thing as “always profitable Forex strategy”.

So better answer would probably be that a good strategy  is winning with 50%+ probability in some predictable market conditions. If we can’t formalize them, the use of the strategy would be questionable. An example of what we mean can be market behavior aroung a major news release. There are known patterns, e.g. price might shoot in one direction, then turn on a dime and go even further in the opposite direction.

Next, we need to find reliable numerical criteria of evaluating a strategy and so be able to compare strategies to each other and rank them.

These are the problems that we are solving on this site, and we’ve also started a specialized site devoted to trading strategies to discuss in more detail how to build strategies.

But back to the question, how can we find the best Forex strategy? To do so we need at least a broad selection of them to test and compare. How can we do it?

Breeding The Best Forex Strategy

The idea that we are going to try is to reuse the principle of evolution. Imagine that each trading strategy is a creature, and they are all exposed to hostile environment (which means we put them to the difficult tests). We can then make selection based on the test results and implement cross-mutations between the selected algorithms.

The Best Forex Strategy Selection
The Best Forex Strategy Selection By Genetical Approach

The whole process must definitely be automated. Manually we won’t be able to do much. Can we do that?

First of all, need the adequate software tools that will make it possible.

  • the strategy development framework that can efficiently generate strategies, make changes in them and merge them automatically (cross-mutation)
  • strategy testing environment that makes automated testing and selection.

We shared some more ideas here: www.robotsforforextrading.com

Trend Following Trading Strategy

The very same ideas and algorithm for market entry are fully applicable for exiting the market or taking profits. Here we won’t discuss the case of exiting the market with a stop-loss. And the most important case where this method is needed is trend following.

Standard practice of taking profits is placing a takeprofit order with trailing. Trailing order follows the current market value, being behind it for configured number op pips. At the first glance, the idea is excellent, but the problem is again the market volatility. Trailing order only goes in one direction, so if placed too close to the market, it can be triggered by a pullback. This happens a lot, e.g. because many market operators like to do “stop hunting”.

We suggest using our algorithm to make trailing logic more flexible. To avoid excessive or false triggering, we set the stop level far from the current market level. Basically just over the break-even level. And next we are detecting the local extremes by our algorithm. Once they are identified, we move the stop order close to the extreme. If the market continues moving to the same direction, and we are lucky and the order is not triggered by a pullback, we just repeat the procedure until the stop is triggered.

Trend Following Is Benefitial Only With Accurate Signals

Once the stop is triggered, we are waiting for the next significant extreme in the opposite direction and decide whether we want to re-enter the market in the same direction. This can be lucrative if the trend resumes. However, we can never predict that, so at some point the last re-entry will most probably be a loser.

There is also idea to profit from pullbacks. Which is often called “catching a falling knife”. Which means that having fixed our profit, we open a position in the opposite direction.

This is extremely risky trade, but on strong trends it can be very profitable.

For trend following trading strategy, we use our pivot point analyzer as a trend indicator. As the figure below shows, once we have a trend, the output of the detector is mostly the peaks in the same direction (maximums for the uptrend seen on the screenshot). This, along with other simple geometrical measures makes trend identification quite reliable.

Trend Following Forex Trading Strategy
Trend Following Forex Trading Strategy: Trend Confirmation By Pivot Points Analyzer

 

 

 

Trading Strategies Tips – How To Enter Forex Market

Forex trading strategies are using multiple time frames. However, even if you are not a day trader but just want to open a long-term position, you have to enter and exit the market in the real time. Period.

This means we have to deal with daily volatility, which can sometimes reach 2% and even more. This is a significant number for leveraged trading. Even if the trading strategies are to stay in the position for long time in anticipation of bigger gains. Good entry timing not only gives you more profit, but also lets you place better stops.

Indicators are of no help in real-time here because they are lagging.  E.g. popular oscillator indicators like RSI can show extreme values for days. Market may really be topping or bottoming, but you won’t be able to find the best moment to enter the market.

Markets don’t move in a straight line.  Even powerful trends always have pullbacks. The only reliable observation we can make is that we always see zigzags. If we see a local top, this will always be followed by a turn and then a local bottom.

In addition, markets have a fractal nature. It means that the pattern we see on a short time frame will become a smaller part of the same picture on the longer time frame.

We used this observation in developing price pattern market entry algorithm.  We use it as a building block of all our trading strategies.

Practical Implementation For Trading Strategies

We implemented the algorithm as a part of our Smart Forex Strategy Tester software.

Here is couple of examples how the algorithm works. Both refer to the same sample of market data covering only couple of hours. The detected pivot points are shown as black dots on the graph.

Market Signals Generation For Forex Day Trading Strategies
Example #1. Price action algorithm based market signals for Forex day trading strategies

You can see that the volatility for these 2 hours is over 1%. The graph is an average of bid and ask, slightly filtered for easier viewing.  Note that the pivot points are calculated in real time. I.e. each point is only evaluated based on the price data to the left of it.

In the example #1, the algorithm sensitivity is set higher (note short horizontal sliders) and it captures more pivot points.  As you can see, during  these 2 hours the algorithm generated almost 30 signals. Which are definitely can be used for profitable day trading strategies.

Market Signals For Forex Trading Strategies
Example #2. Price action algorithm adjusted for long-term Forex trading strategies

In the example #2, we lowered the sensitivity of the algorithm (see longer values of horizontal sliders) and it only detected 3 pivot points within same 2 hours.  You can see that all 3 are profitable market entries. Moreover, the algorithm managed to capture both the market top and bottom for given interval.

If you are interested, you can download Smart Forex Strategy Tester and try for yourself.

 

 

Forex Trading Simulation: A Nice Idea To Avoid

Forex Trading is often learned by using trading simulation. The easiest way to do that is using demo accounts. There is a also special simulation software. Compared to demo account, it is more advanced. It can accelerate of slow down the market,  or easily start from scratch. And computer simulation is essentially the same as “paper trading” that was used before the PC became affordable to masses.

At the first glance, the idea is clever. Simulators are used virtually everywhere, even in pilots’ training. Pilots are responsible for peoples’ lives – so what can be wrong with the idea of learning forex trading by simulators?

There are principal differences, though. Flight simulators are useful, because airplanes have a lot of different controls that a pilot needs to learn. Doing that in simulator is a way to avoid the risks of their misuse.

Now, let’s take Forex. On the contrary, Forex trading is technically very simple. You either buy or sell. Of course, there are multiple positions handling, stop losses, risk management, etc.  But these all are technicalities. Learning them is important but on itself does not train you in the main component of Forex trading.

Forex Trading Is About Psychology

The problem with learning Forex trading in simulators is that it is too comfortable. But real Forex trading, when there are your own money at stake, is very stressful. And the simulator doesn’t principally teach you how to manage your stress. You might think that you learn some skills that can be then reused in real trading. But this might only work for some, but definitely not for all.

Our Smart Forex Strategy Tester framework can easily be modified to support simulation mode. We are planning to do that, but we don’t see it as a first priority. For majority of us simulators will only do bad service.

Renowned traders don’t recommend paper trading, either. E.g. Raghee Horner (“ForeX Trading for Maximum Profit: The Best Kept Secret Off Wall Street”) and Courtney D. Smith (“How to make a living trading foreign exchange: a guaranteed income for life” our review).

The best way to learn Forex trading is to trade for real. Micro lots are excellent way to start.

Forex Tester Tips

Are you a Forex tester? If you trade Forex market or want to – you have to be one! Trading with an untested strategy is a way to draining your account fast.

What Forex testing means in practice? How exactly do you do it? Let’s assume you have all the software tools and market data you want. What next?

Obvious answer is – with the help of testing software, run your strategy against the market data and see whether it wins or loses.  This is correct, but is only the beginning.

Here we will try to give some tips on what exactly should be done and how. We hope that a forex tester will find them useful. Your feedback is welcome!

Forex Tester Challenges

It is known that to properly define the problem means to half solve it. So, what are the problems that Forex testing must address?

Market data is huge. This problem is the most obvious one. Many years of quotes are available for many currency pairs. Even if you only decide to test on main pairs, still you can literally spend days and days to run your strategy against all data.

Too many moving parts. Also quite an obvious problem, at least for those who tried.

Regardless of how long you spend on running tests, this time will obviously be well spent if eventually you can confirm your strategy is consistently winning. But what if it is not?

Based on tests run, you might get many ideas how to improve your strategy, implement the changes and repeat the tests. But as you know (or will know),  this is a never-ending process. We have too many combinations. Couple them with vastness of market data, and your testing might already take weeks to complete.

Exit criteria and quality evaluation. We bet this is least obvious, and not every Forex tester even thinks in this direction. How much testing is enough? In other words, how can we decide that we can stop testing our strategy and start trading it? And after all the testing that has been done, what is the minimum quality level we can guarantee?

How To Apply Standard Testing Practices To Forex Testing

Testing of man-made products is a well-studied area. Properly done, testing is stricly formalized and meticulously planned activity. Any test project is limited in time, so test planning revolves around the questions like:

  • what exactly to test,
  • to what level of quality,
  • with what resources, etc.

Answering which makes it possible to get predictable level of product quality within desired timeframe and budget.

Does the same approach work for a forex tester? Yes. in the second part we will discuss how.

Forex Tester Tips (part 2)

The Most Important for Forex Strategy Testing is ...

View Results

Loading ... Loading ...