Momentum Strategies: Comparison of Programming Language Performance

Abstract

Given the increase in the popularity of algorithmic trading resulting from an increase in market participants, more considerations are now required to prototype a profitable trading strategy. Trading strategies, which require optimization of parameters based on linear or nonlinear relationships, cause an increase in complexity, which in turn increases computational run time. We find that C provides the best performance for prototyping quantitative trading strategies; however, it is the most time-consuming to implement. Among the languages that allow for faster development times, the difference between Cython and Julia is relatively small, so choice between them comes down to user preference and other factors. We find Julia to be the standout programming language due to its simplicity and high performance.

G iven the increase in popularity of algorithmic trading resulting from an increase in market participants, more considerations are now required to prototype a profitable trading strategy. Trading strategies, which require optimization of parameters based on linear or nonlinear relationships, cause an increase in complexity, which in turn increases computational run time. For strategy prototyping, the choice of programming language has become more critical than ever. The goal is to have a programming language that has a simple syntax that is computationally fast and robust.

The history of academic publications of comparisons of programming languages is extensive. Prechelt [2000] produced a comprehensive comparison of C, C++, Java, Perl, and Python. The author measured productivity as a function of properties such as run time, memory consumption, program structure, reliability, and the amount of effort needed of the programmer. Prechelt showed that scripting languages such as Perl and Python are more productive than conventional languages such as C++, C, and Java.

There has not been much literature on comparisons of programming languages for investment finance purposes. However, Aruoba and Fernández-Villaverde [2014] compared programming languages applied to problems in economics. The authors had a strong emphasis for speed and compared the run time for solving an economic model called the Stochastic Neoclassical Growth Model using C++, FORTRAN, Java, Julia, Python, MATLAB, Mathematica, and R.

No literature exists on programming comparisons for investment strategy prototyping and testing, and that is where our article answers the following question: Which programming languages are most appropriate for strategy backtesting and prototyping? This question was motivated by our practical applications, for which we require our trading tools to be fast yet easy to understand. In this article, we test the run time of basic momentum strategies on EUR/USD data using Python, Cython, Julia, and C. We chose each language for a precise reason: C is known to be one of the fastest programming languages, used to implement numerical-heavy code; Python is widely used by researchers and traders; Cython promises C-like speed with the ability to be called by Python programs; and Julia is a new language that was specifically designed for scientific computing. Java and C# are very popular; however, our objective is to compare scripting languages relative to C. We find Julia to be a strong candidate because of its simplicity and high performance.

The remainder of the article is organized as follows: We outline our empirical methodology, present our results, and then offer our conclusions.

EMPIRICAL METHODS AND ASSUMPTIONS

To test each programming language, we implemented strategies based on the most popular technical analysis signals: simple moving average crossover (SMA), moving average convergence divergence (MACD), relative strength index (RSI), and Bollinger Bands (BB). Exhibit 1 contains a table with all the indicators and the corresponding default parameter values used in the backtest.

Exhibit 1

Parameters Used in the Indicator Benchmarks

[Figure omitted. See PDF]

The implementation in each language all follow the same structure: We started by loading the data in memory, calculated the indicators, and ran the backtest. We also ran a rolling optimization of the SMA indicator to simulate a more computationally heavy task. The optimization was a simple exhaustive search algorithm with a one-week rolling window to find the parameters that give the highest Sharpe ratio. In this way we could test how the language behaved when switching between computing the signal and backtesting. Each optimization step looked at around 20 parameters for SMA. We started from the most commonly used values (in the case of SMA, default values of 8 and 18 for fast- and slow-moving averages, respectively; see Exhibit 1 for default values for each indicator) and tested a range of 20 parameters around these values. We believe this is a good number of parameters to use because in practice we would run multiple simulations in parallel, each with a relatively small number of parameters as in our test.

The dataset comprised of intraday bid-ask quotes on EUR/USD for 2014, which was then loaded into memory. All implementations loaded the prices from an SQLite database. Since this step is dependent on the database library used, it was not included when considering the language performance.

To have a meaningful overview of the speed of the languages, we ran each indicator, backtest, and optimization multiple times, and we collected a set of summary statistics related to run time. We note that to provide a good benchmark, it is necessary to avoid compiler optimizations, such as dead code elimination; hence, we made sure to use the results of each computation.

In the remainder of this section, we describe implementation details that are language-specific. These details should give traders and researchers a good understanding of what kind of considerations are necessary to write high-performance programs in each language. To keep the comparison fair, each implementation follows best practices as much as possible, but no advanced optimization is performed.

The Python implementation used the NumPy library to handle vector operations. NumPy is a library that implements vector operations in efficient C code. For this reason, the best way to obtain good performance out of Python code is to have NumPy do as much of the heavy lifting as possible. This means that, unlike the other languages considered in this article, trying to vectorize the algorithms resulted in better performance. In our experience, using NumPy functions over pure Python functions can lead to speedups of 100 to 1,000 times, depending on the complexity of the algorithm. Simpler functions that translate to fewer NumPy calls perform better than bigger and more complex functions. As later shown in our results, the Python indicators implemented only using NumPy ran at almost the same speed as the C reference implementation while being considerably shorter to write.

Programming in Cython is not substantially different from writing code in pure Python. All the signal-generating functions are implemented in a Cython module and then called from the main Python program. This is done to simulate real-world usage of Cython, where computationally expensive functions are called from the main Python program. Since we were interested in getting the best performance out of our implementation, we disabled bounds check, wraparound indexing, and Python division. These changes require the trader/researcher to be more careful about accessing arrays and doing divisions, but result in fewer operations done at run time. Another source of degradation in performance is the interaction between C and Python code. Luckily, Cython provides a way to annotate the source code to highlight these interactions. Finally, a note on Cython support for OpenMP: Although it is easy to enable, doing so does not result in a performance gain. This should not come as a surprise to anyone who has experience in parallel programming, since parallelizing algorithms requires changes to the code structure to take advantage of the newly gained parallel program flow.

Julia has a syntax that is familiar to MATLAB users but, unlike MATLAB, writing nonvectorized code results in better performance. Julia allows the trader to use for loops without penalizing performance. In most cases, writing for loops is a more familiar way to express operations on vectors, resulting in code that is easy to write and runs in a reasonable amount of time. Additionally, it is possible to change the original program to run faster by following the recommendations in the documentation. The excellent tools that come in the default Julia installation allow users to write the optimized version of the program in a short amount of time.

A key performance tip in Julia is to avoid using abstract types, especially for containers, since this requires Julia to allocate an array of pointers instead of allocating concrete types, which results in worse performance. Julia provides the @code_warntype macro to highlight these abstract types, making this kind of optimization easy to perform. One last optimization is based on the use of performance annotations to disable bounds checking or to enable single instruction, multiple data (SIMD) features. This is done on a function-by-function basis, depending on which way yields better results in microbenchmarks.

Exhibit 2 shows the approximate time in hours it took to implement the strategies from scratch in each programming language. The C implementation required by far the most time to implement. This is because, although the language allows the trader great flexibility, it also requires careful control over memory allocation and access. We can also see that Julia was the easiest to develop relative to the other languages. To better simulate real-world usage of C, we followed the standard convention of allocating memory inside the indicator functions and letting users free the memory after they have finished using it. Furthermore, we tested a number of different compiler flags to see which one obtained the best performance. The use of advanced optimization techniques, such as Intel Intrinsics or OpenMP, is outside the scope of this article since they would require a considerable amount of work in addition to simply implementing the indicators. Unlike the other languages in this comparison, the resulting C program does not require further work to obtain acceptable performance.

Exhibit 2

Approximate Time Taken to Create Our Strategy across Different Programming Languages (lower is better)

[Figure omitted. See PDF]

RESULTS

Exhibit 3 presents the results for each benchmark, including the minimum, maximum, mean, and standard deviation of the running times for each language and function.

Exhibit 3

Running Time in Seconds (lower is better)

[Figure omitted. See PDF]

As far as Python is concerned, good performance was achieved by delegating most of the work to NumPy. In this way, all operations were executed by the library C code. As soon as the algorithm was implemented using a mix of NumPy and pure Python, performance degraded to unacceptable levels.

Cython run times for SMA and MACD were comparable to those of Python, but Cython does much better in the other trading indicators that are not vectorized. This does not come as a surprise because we delegated most of the computation to the generated C library by using Cython.

Julia's performance was very good, especially considering how close the implementation was to pseudo code. A result that stands out is the bad performance in the BB test. Julia was slow in calculating the BB because of problems caused when computing the rolling standard deviation. A number of attempts have been made to fix the problem, but none has solved this issue. C had the best performance out of the languages considered in this article; for this reason it will be considered as the baseline reference for comparing languages run times.

A very interesting result shown in Exhibit 3 is the standard deviation of each language's running times. Python times have low standard deviation when using NumPy functions, whereas for pure Python functions the standard deviation is larger. Julia has relatively high standard deviation, which can be explained by the presence of a garbage collector. This difference is not caused by the just-in-time (JIT) compiler, since the first function call is not included in the benchmark. Cython and C both have very low standard deviation in running times, which can be explained by a more predictable memory model for both languages.

Exhibit 4 shows the relative performance of each language compared to C and confirms the results highlighted in the previous paragraphs. In the optimization benchmark, the gap between Python and the remaining languages was smaller than in the backtest benchmark. This can be explained by the constant context switching between generating the signal and backtesting, which worsens the performance of vectorized languages more than that of Python. One additional note: Both Cython and Julia had one instance in which they performed much worse relative to C. Moreover, all languages performed badly in the backtest benchmark because the test did not involve any numerically heavy code.

Exhibit 4

Relative Running Time Compared to C (lower is better)

[Figure omitted. See PDF]

In addition to performance, other issues we considered were connectivity and the programming languages' accessibility and ease of use. All of the programming languages considered in our backtest have connectivity with SQLite. Connectivity of the strategy for live trading usually falls under the scope of execution traders/developers (through either application program interfaces [APIs] or financial information exchange [FIX] protocol). Most institutional brokers support a variety of languages for connectivity, such as Excel (VBA), C, Java, and any language that supports hypertext transfer protocol (HTTP). Moreover, as long as a broker has a C/C++ API, a C wrapper can be used to provide connectivity with Cython, Python, and Julia. Another solution would be to use a message queue to communicate between our strategy and an execution handler for live trading.

In terms of active communities, according to Python.org, there are 401 Python user groups in 37 countries, with more than 127,100 members. Additionally, at the time of writing, Stack Overflow, one of the most popular language-independent collaboratively edited question-and-answer sites for programmers, had more than 524,151 questions related to Python. In comparison, Cython presented only 6,286, and Julia had only 3,429. The RedMonk Programming Language Rankings, which provide a general indication of use of languages observable in both GitHub and Stack Overflow, usually includes C and Python in the top 10 languages. Julia, mentioned among the notable languages to watch, has been progressing steadily though the rankings, achieving position 56 in 2015.

The huge following and available resources make Python very accessible to programmers switching from other languages. However, Julia.org provides a wealth of resources that allow programmers of any skill level to master the language easily, as the entry barriers for new users of this language are much lower.

Additionally, a note on stability: C and Python were the most mature languages in our test, and if we pay attention in following the standard (in the case of C) or interpreter version (in the case of Python), we can run strategies on different operating systems and compiler and interpreter versions. Julia is the newest language and has only been around since 2012. Julia's most recent version is 0.4.3, which means the language can still be subject to changes and break compatibility with older versions. Despite this, all compatibility changes to the language are well-documented in the release notes and do not pose a problem in practice.

CONCLUSION

In this article, we discussed four popular languages used by quantitative researchers and traders to program their models, considering them from a performance point of view while still considering how easy it is to write programs that run in an acceptable amount of time.

When comparing programming languages, performance should not be the only factor to consider. Another factor can be how easy a language is to learn. We find that languages like Python and Julia are faster to pick up than C, which requires years of experience to master.

A programming language community and ecosystem is also an important factor to consider when choosing a language. For example, Python has seen an increase in popularity in the last few years and now has many libraries to interact with data sources and brokers. These advantages of Python are also transferable to Cython. The youth of Julia, on the other hand, means that the libraries are not as mature as the corresponding libraries in other languages that have been around longer.

In conclusion, C provided the best performance for prototyping trading strategies; however, it is the most time-consuming to implement. Among the languages that allow faster development times, the difference between Cython and Julia was relatively small, so the choice comes down to user preference and other factors. We find Julia to be the standout programming language because of its sheer simplicity and high performance.

To order reprints of this article, please contact Dewey Palmieri at [email protected] or 212-224-3675.

References

Aruoba, S.B., and J. Fernández-Villaverde. "A Comparison of Programming Languages in Economics." Working paper, National Bureau of Economic Research, 2014.

Prechelt, L. "An Empirical Comparison of C, C++, Java, Perl, Python, Rexx and Tcl."IEEE Computer , Vol. 33, No. 10 (2000) pp. 23-29.