E-mail:    

TRADING SYSTEMS

Grammatical evolution

automated generation of trading rules

trading systems automated trading rules

In recent years we have witnessed a steep increase in the application of natural computing techniques in finance.

In this article we will discuss the application of Grammatical Evolution, in our view one of the most promising and flexible evolutionary computing approaches to FOREX financial time series. We will build a trading model using Grammatical Evolution on the 60 minute time frame series of GBPUSD, EURUSD and USDCHF over 88 trading weeks and present the results. We will also propose the set up of a number of portfolios to exploit the potential of the rules emerged via Grammatical Evolution. We will not focus on programming techniques as our aim is to show the logic that lies under this new and interesting approach and its potential.

Background

If we assume that all the information available for a specific market is captured by the equilibrium market price arising from the interaction of rational agents, there would be no point in trying to identify price dynamics or patterns because there would be no predictability. But we don’t believe in the Efficient Market Hypothesis in its strict formulation and, starting from this premise, we do our research.

In finance the adoption of techniques to evolve or extract trading rules from a time series finds its theoretical foundation in the weakness of the premises of the Efficient Market Hypothesis: many researchers question the rationality of market agents and the fact that all the information that concur to price equilibrium are disposable and perceived by all the agents in the same way. Other hypothesis that involve relative efficiency of markets, like The Adaptive Markets Hypothesis (AMH), where market efficiency emerges as an evolutionary process through competition between agents justify the adoption of evolutionary expectational models and represent, in our view, a better framework in approaching financial markets.

Natural computing can be seen as a way of developing programs and algorithms, taking inspiration from real world phenomena. Grammatical evolution specifically is a particular Genetic Programming approach to generate computer programs (in our case trading rules) in an arbitrary language defined by the user. We refer to Genetic Programming as to an evolutionary computation technique where a population search based approach is used to evolve a solution to a certain problem. To make things clearer to non specialists we will describe the genetic process as a way to build a ‘valuable’ trading rule for a specific time series over a period of time. We will present some examples in the following part of the article that should help in understanding how the subject works.

We can say that Genetic programming mimics the Darwinian principle of evolutionto evolve a solution to a specific problem. Real world phenomena are characterized by uncertainty and noisiness in dynamic environments and are modified by the interaction of multiple agents. For those very reasons a Natural Computing approach seems well suited for financial where similar conditions are met.

Hands on grammar

After a brief theoretical introduction we will describe the process of setting up a Grammatical Evolution-based trading system. As we briefly mentioned before, Grammatical Evolution is a form of genetic programming that allows the user to evolve computer programs, rule sets or more generally, sentences in any language. It is a methodology that allows us to evolve the solution to a specific problem (as happens in genetic programming) with a plus: we can define the grammar, i.e. the shape in which the solution is to be expressed.

Our goal is to produce a trading rule for a specific time series over a time period. We don’t know how this rule will look like, we just know which are the building blocks that will compose it. In our case we chose some well know technical indicators and a proprietary one (see table 1)

Technical indicators

ADX (arg)

Average Directional Movement

ROC (arg)

Rate of Change indicator.

RSI (arg)

Relative Strength Index indicator.

Stdev (arg)

Standard Deviation indicator.

ATR (arg)

Average true range indicaror.

SMA (arg)

Simple Moving Average indicaror.

MIN (arg) MAX(arg)

Minimum and maximum.

HT (arg)

HT Trendline indicaror.

LTS (arg1,arg2,arg3)

Proprietary trend following–mean reverting

Table 1

We tried to use as few indicators as reasonable in respect to the principle of parsimony though representing volatility, directionality, momentum and breakout. Each indicator can take one or more arguments accordingly to its structure, for example SMA takes one argument which is the length of the moving average considered.

In grammatical evolution we  define a ‘Grammar’ that determines the shape of every bit of the solution that has to be evolved, we will not  focus on  an in-depth analysis of the grammar writing process because this falls out of the scope of this article but, we would still like to give an idea of the logic laying beneath it.

 For instance, we define a <Trading Condition> as a combination of the mentioned indicators with <arg>, the function argument, being an integer number between 1 and 100, < operator > being ‘> greater than’ or ‘< lesser than’ and < num > being a floating number between 0 and 100. The grammar would look a bit like this:

trading systems automated trading rules

Here are some examples of <Trading Condition> generated by the algorithm following the previous grammar:

code-1 trading systems automated trading rules

Those trading conditions can be combined together by the algorithm in ‘and’ and ‘or’ logic to build sentences like:

code-1 trading systems automated trading rules

Last step is now to define the grammar of the solution, our trading system. We want to generate a trading rule, so we imagine two possible outcomes in terms of grammar: a simple buy and sell rule that reverses the position at every new signal, and a more complex rule that has a customized exit rule for the buy and the sell signal:

code-1 trading systems automated trading rules

The computer can now evolve a solution combining trading rules accordingly to the grammars.

A set of rules that combines the two bits of grammar described above would look like this* :

*(please note that the following rule is meaningless and for demonstration purpose only):

code-1 trading systems automated trading rules

Well this is a trading system!

The evolutionary process

How does the genetic programming algorithm evolve the best rules? In the beginning it generates a ‘population’ of trading rules randomly. Every single rule, like the one we showed in the example, is called ‘individual’. A population can consist of a number of individuals defined a priori by the user. Every single individual is then evaluated against a ‘fitness function’. The fitness function returns a value that quantifies how that individual is doing well in relation to a specific problem. In our case the fitness function can be the profit generated by a trading rule on a specific time series or a risk adjusted measure of profit. Specifically we use a proprietary combination of profit, draw-downs, max draw-downs and cumulative profit curve shape. Those individuals (rules) of the population that have the highest values of fitness function become the parents of a new generation of individuals. In that way winning bits of rules are mixed together to generate new rules. Some random variability is added so that there is a mutation in each new generation that will be not only made of the brute mix of parents’ genetic inheritance. It is very obvious that this evolving process mimics the Darwinian evolutionary process. It  turns out that this is an extremely powerful and efficient  way to explore a space of solutions that cannot be searched by a brute force combination of parameters because the dimension of the space of solutions would be huge and  the exploration of it all would take a significant amount of time.

Using this approach at the end of each optimization we have a number of individuals with a high fitness value. That means we have a set of good trading rules optimized for the time series we were using during the evolution.
 As every trading system expert knows, the tricky part is yet to come. The fact that we have found good rules for a time series in a specific time period, no matter how sophisticated our approach is, doesn’t mean we have found trading rules that are able to generalize well on unseen data.

The data

Three years of data have been used, from December 2005 to November 2008. We analyzed the 60 minute time frame series of GBPUSD, EURUSD and USDCHF. The data has been collected by our automated trading engine and outliers have been cleaned. Slippage for each cross is the average value of the real slippage we had in our real money automated trading experience and is as follows:

0.6 pips on EURUSD

1.1 pips on USDCHF

1.6 pips on GBPUSD

We have chosen the 60 minutes time frame because we wanted to have a high frequency approach and, at the same time, reduce the impact of a potential widening in market spreads. The limited number of trades generated by a 60 minutes model, compared with a 5 or 10 minutes one gives us more confidence about the stability of results against a potential worsening of market conditions.

Training and Testing methodology

One of the most complex issues in financial time series analysis is the choice of the training and testing time span. Financial time series may exhibit persistence in price dynamics for enough time to exploit them with profit but are also subject to sudden changes in volatility and price dynamics because of a wide variety of factors that range from financial to economical to social or technological. In our test we use a proprietary methodology where we train the time series on a variable time span that ranges between 3 and 12 months according to the persistence of some time series volatility indicators. The rules extracted in each time span are then applied for the following 4 weeks. So basically we used a sliding window approach where at time T we extract the trading rules (in our example using Grammatical Evolution) on a time span ranging from T-x to T where x={12 weeks,24 weeks,36 weeks,48 weeks} and trade them from time T+1 to time T+4. We repeat the process till the end of the series. This way, considering that a bit more than one year of time series is used for the first train, we have 88 weeks left for testing purpose. That is, in our case, 22 periods of 4 weeks to test the real market performance of the system.

At every evolution we kept the best 50 trading rules and traded them applying the average real market slippage we had with our real-money automated trading system in the last three years. The results will be presented for each Cross and split into 4 groups: in the first group we average the returns generated by the average of all the 50 rules, in the second group we average the first 25 rules, in the third the first 10 rules and in the last we consider the returns generated by the best rule only. Rules are ranked according to their training fitness value.

Results

Please refer to chart 1, 2 and 3 for the cumulative returns on each cross for each group. Results are summarized in table 2.

trading systems automated trading rules eur usd

trading systems automated trading rules gbp usd

 trading systems automated trading rules usd chf

Cross/Individuals

Return %

EURUSD ind 1

27.78

EURUSD ind 10

5.40

EURUSD ind 25

7.00

EURUSD ind 50

3.44

GBPUSD ind 1

20.07

GBPUSD ind 10

14.54

GBPUSD ind 25

14.63

GBPUSD ind 50

11.88

USDCHF ind 1

40.68

USDCHF ind 10

7.65

USDCHF ind 25

8.47

USDCHF ind 50

5.27

Table 2

Results already include slippage and, they are in leverage 1:1 and must be considered in that perspective. We notice that every cross has a positive cumulative profit at the end of the testing period. It is also interesting  that the best individual is always the best performer and is producing significantly higher returns. The 25 individuals group slightly outperforms the 10 group on each Cross considered.  The 50 individuals group is always the worst performer. This suggests that the algorithm, combined with the fitness function and the training methodology, are well suited to exploit potential trading rules, as the performance on unseen data is increasing as the training fitness value increases.

A small portfolio experiment

Even if the best individual has such a better performance, it is always risky to allocate the whole capital on a single rule, even if each Cross has its own best rule on each time span, so we decided to simulate the outcome of two to test the impact of using multiple rules at a time: a first portfolio named “Average portfolio” where 25% of the capital is allocated on each of the group mentioned above and split between the three Cross, a second portfolio named “Best Portfolio” where capital is divided only between the best individuals of each cross.  (please see table 3 and 4 and chart 4).

AVER PORTFOLIO

EURUSD

GBPUSD

USDCHF

best individual

33% of 25% capital

33% of 25% capital

33% of 25% capital

best 10 individuals

33% of 25% capital

33% of 25% capital

33% of 25% capital

best 25 individuals

33% of 25% capital

33% of 25% capital

33% of 25% capital

best 50 individuals

33% of 25% capital

33% of 25% capital

33% of 25% capital

 

 

 

 

BEST PORTFOLIO

EURUSD

GBPUSD

GBPUSD

best individual

33% of capital

33% of capital

33% of capital

Table 3

4  trading systems automated trading rules best average

PORTFOLIO

RETURN %

CALMAR RATIO

AVERAGE

13.90

4.77

BEST

29.51

7.74

Table 4

Results favor the ‘best’ portfolio in terms of absolute profit and Calmar Ratio. Anyway, the differentiation induced by using a combination of 50 individuals, even if the best ranked ones are over-weighted, is worth in our view the loss in performance, should a drastic change in price behavior occur. Facing an unknown change with a portfolio of tens of individuals, rather than 3, is reassuring and must be considered as an option.

Conclusions

Evolutionary  algorithms are becoming more and more popular in the financial community thanks to their flexibility and efficiency in exploring huge solution spaces and deliver interesting results, the theoretical premises lying behind this approach are intuitively compatible with a dynamic and complex phenomenon like the price discovery process represented by a financial time series. A point that deserves more attention is the quick adaptation to a dynamic environment and the ability to detect abrupt changes in price behavioural patterns and some work has yet to be done in this direction. The approach used in this article of a rolling train-test window is quite native and has some drawbacks but, was mainly intended to illustrate the potential and the logic of genetic programming, focusing on the flexibility of grammatical evolution.

Luca Taroni