Zorro can analyze the performance of its own strategy scripts, as well as of trade lists that are generated by other software and exported as .csv files. For the latter case an example can be found in the Simulate script.
A click on [Result] produces a chart and a performance report with portfolio analysis for the previous [Test]. Chart and report are stored in the Log folder and displayed in Zorro's image viewer and text editor. The chart is a plain .png image file; its properties, colors, and curves can be defined in the script. The report is a simple .txt file. Both chart and report can be easily posted to websites or imported in documents. Additionally, charts and other results from test or training can be exported and further evaluated with third party software, f.i. an R data analysis package.
In [Trade] mode the performance report is part of the .htm status report. It can be displayed in a web browser and is updated every minute. It also contains a list of all open and pending trades, as well as any specific information that is printed by the script.
A chart example:
By default, the chart shows equity and drawdown profiles, the price curve, and the trades of the asset selected with the [Asset] scrollbox. Elements can be removed from the chart by setting their corresponding Color parameters to 0. Trade entry and exit points are marked with green or red dots for winning or losing trades. The blue equity profile is summed up from the whole portfolio and is averaged over all oversampling cycles. If Capital is invested, it adds to the equity curve dependent on PlotMode settings. The red spikes below are the drawdown profile (the "underwater equity"). On a detail scale the price curve is displayed with candles. The size and scale of the chart can be set up with plot parameters (click [Test] again when the script was changed). The price curve with the displayed trades can be selected with the [Assets] scrollbox before clicking [Result]. The equity curve is always from the sum of all trades of the portfolio. The number of bars per chart can be limited with the PlotBars variable.
The performance is calculated either from the trade sequence and equity curve of the simulation, or from a distribution of many sequences and equity curves using Monte Carlo analysis. This is an example of a performance report:
Walk-Forward Test Z4 portfolio Simulated account AssetsFix.csv (NFA) Bar Period 4 hours (avg 266 min) Simulation period 06.06.2004-02.05.2012 (9483 bars) Test period 10.06.2007-02.05.2012 (6259 bars) WFO test cycles 11 x 569 bars (137 days) WFO training cycles 12 x 3224 bars (111 weeks) Lookback period 700 bars (169 days) Monte Carlo cycles 200 Fill mode Realistic (slippage 5.0 sec) Avg bar 5.1 ticks 470.5 pips range Spread 0.7 pips (roll -0.70/-0.31) Contracts per lot 1.0 Capital invested 5000$ Gross win/loss 60452$ / -43672$ (+16252 pips) Virtual win/loss 60024$ / -44112$ Average profit 4309$/year, 359$/month, 17$/day Max drawdown -2025$ 12% (MAE -2287$ 14%) Total down time 78% (TAE 93%) Max down time 119 days from May 2009 Max open margin 656$ Max open risk 1863$ Trade volume $10164943 (2610626$/year) Transaction costs -637$ spr, -365$ slp, -906$ rol, -1102$ com Capital required $2681 Number of trades 1370 (351/year) Percent winning 44% Max win/loss 810$ / -407$ Avg trade profit 12$ 10.4p (+99.0p / -56.5p) Avg trade slippage -0.27$ 0.2p (+1.1p / -1.4p) Avg trade bars 16 (+23 / -10) Max trade bars 141 (34 days) Time in market 355% Max open trades 14 Max loss streak 17 (uncorrelated 13) Annual return 161% Profit factor 1.38 (PRR 1.28) Sharpe ratio 1.94 Kelly criterion 0.63 Ulcer index 12% Cycle performance 1.39 1.40 1.31 1.33 1.37 1.38 Confidence level AR DDMax Capital 10% 236% 1440$ 1390$ 20% 227% 1550$ 1470$ 30% 218% 1680$ 1570$ 40% 209% 1830$ 1680$ 50% 202% 1940$ 1760$ 60% 193% 2140$ 1900$ 70% 186% 2340$ 2040$ 80% 174% 2730$ 2320$ 90% 166% 3080$ 2580$ 95% 146% 4010$ 3580$ 100% 104% 5640$ 4710$ Portfolio analysis OptF ProF Win/Loss Wgt% Cycles AUD/USD avg .014 1.36 281/475 11.4 XXXXXXXXXX/ EUR/USD avg .006 1.41 91/115 6.2 XXXXXX\XXXX GER30 avg .030 1.33 34/45 2.5 X/X/\.\XXXX SPX500 avg .077 1.75 96/162 16.4 XXXXXXXXXXX USD/JPY avg .006 1.69 130/175 8.7 XXXXXXXXXX/ XAG/USD avg .008 1.35 113/152 5.3 \\XXXXXXXX/ BB avg .010 1.29 82/111 2.6 X/\XXX/\XXX CT avg .019 1.38 106/126 7.6 XXXX\XXXXXX CY avg .017 1.83 60/66 5.6 /XXX\/XXX/X ES avg .025 1.34 163/275 7.4 XXXXX/XXXXX HP avg .017 1.34 332/440 19.2 XXXXXXXXXXX LP avg .009 1.54 106/219 11.3 \XXXXXXXXXX VO avg .014 1.71 217/288 24.2 /XXXXXXXXXX AUD/USD:ES .036 1.14 45/87 0.1 \\/\\//\\// AUD/USD:ES:L .036 1.14 45/87 0.1 \\/\\//\\// AUD/USD:ES:S .000 ---- 0/0 0.0 ........... AUD/USD:HP .024 1.18 76/102 2.2 X/\XX\/XX\/ AUD/USD:HP:L .024 1.13 47/68 1.2 //./\\///\/ AUD/USD:HP:S .043 1.35 29/34 1.0 \/\\/./\\\/ AUD/USD:LP .029 1.66 75/149 7.5 \/X///X\/X/ AUD/USD:LP:L .029 1.80 42/71 6.8 \//////\/\/ AUD/USD:LP:S .058 1.23 33/78 0.6 \/\///\\/// EUR/USD:CT .009 1.11 22/33 0.5 X/\X\X\X/XX EUR/USD:CT:L .027 1.22 10/19 0.7 \/\/\\\\.// EUR/USD:CT:S .000 0.91 12/14 -0.2 /.\\\/\//\\ EUR/USD:HP .036 1.32 45/59 2.4 \///\/\\\/\ EUR/USD:HP:L .036 1.32 45/59 2.4 \///\/\\\/\ EUR/USD:HP:S .000 ---- 0/0 0.0 ........... EUR/USD:VO .027 2.20 24/23 3.3 .X.//.\X/XX EUR/USD:VO:L .027 1.58 12/11 0.9 ././...//\\ EUR/USD:VO:S .032 2.90 12/12 2.5 .\../.\\.// GER30:BB .038 1.03 2/4 0.0 /......\\/\ GER30:BB:L .038 1.03 2/4 0.0 /......\\/\ GER30:BB:S .000 ---- 0/0 0.0 ........... GER30:CT .256 1.92 3/2 0.6 \/./...\/.. GER30:CT:L .256 1.92 3/2 0.6 \/./...\/.. GER30:CT:S .000 ---- 0/0 0.0 ........... GER30:ES .267 1.82 2/3 0.2 ..\/\../..\ GER30:ES:L .000 ---- 0/0 0.0 ........... GER30:ES:S .267 1.82 2/3 0.2 ..\/\../..\ GER30:VO .051 1.31 27/36 1.7 ////\.\\/\/ GER30:VO:L .051 1.31 27/36 1.7 ////\.\\/\/ GER30:VO:S .000 ---- 0/0 0.0 ........... SPX500:ES .110 1.44 17/21 1.3 //\/\///\\\ SPX500:ES:L .110 1.44 17/21 1.3 //\/\///\\\ SPX500:ES:S .000 ---- 0/0 0.0 ........... SPX500:LP .006 1.04 17/55 0.2 \\/\\\\\//\ SPX500:LP:L .006 1.04 17/55 0.2 \\/\\\\\//\ SPX500:LP:S .000 ---- 0/0 0.0 ........... USD/JPY:BB .057 1.51 24/36 0.6 \/\\/\/.\./ USD/JPY:BB:L .000 ---- 0/0 0.0 ........... USD/JPY:BB:S .057 1.51 24/36 0.6 \/\\/\/.\./ USD/JPY:CT .016 1.46 16/19 2.0 \./..\/.\// USD/JPY:CT:L .016 1.46 16/19 2.0 \./..\/.\// USD/JPY:CT:S .000 ---- 0/0 0.0 ........... USD/JPY:HP .024 2.24 34/29 6.4 .XX/\/\/.\/ USD/JPY:HP:L .024 1.22 9/13 0.3 .\//\/\..\/ USD/JPY:HP:S .028 2.65 25/16 6.0 ./\/.../.\/ XAG/USD:CT .038 1.87 13/13 0.6 ....\///\\/ XAG/USD:CT:L .000 ---- 0/0 0.0 ........... XAG/USD:CT:S .038 1.87 13/13 0.6 ....\///\\/ XAG/USD:HP .018 1.43 44/46 1.1 \\/X/\//X// XAG/USD:HP:L .105 4.16 6/3 0.3 .\./....\/. XAG/USD:HP:S .014 1.33 38/43 0.8 \\/\/\///// XAG/USD:VO .011 1.39 40/62 3.0 .\/\//\\\// XAG/USD:VO:L .000 ---- 0/0 0.0 ........... XAG/USD:VO:S .011 1.39 40/62 3.0 .\/\//\\\//
The following table shows the meaning of the values. Most of the calculated values are only valid for a profitable report (Gross Win > Gross Loss); they are meaningless when the profit is negative. Indicated values with a "$" suffix are in units of the account currency (not necessarily US-$), indicated values with a "p" or "pips" suffix are in pips.
Performance report | |
Bar period | Bar period in seconds, minutes, or hours, and average bar duration in minutes. Variations in bar duration are caused by weekends, holidays, or special bars. |
Simulation period | Time of the WFO run (usually 4..5 years) and the number of bars, without the preceding lookback period. |
Test period | Time and bar number of the test; simulation period without training and lookback. |
WFO test cycles | Number and length of the WFO test cycles (this is also the recommended re-training period). |
WFO train cycles | Number and length of the WFO training cycles. |
Lookback period | Amount of data to be collected before test or training begins. |
Monte Carlo cycles (Confidence) | Number of Monte Carlo simulation cycles, and selected confidence level (if any) for the following performance figures. |
Fill mode | Simulated fill mode (Naive or Realistic) and Slippage. |
Avg bar | Average number of price ticks and average high-low difference per bar in the simulation; for single asset strategies. |
Spread | Spread in pips, and long/short rollover fee in account currency units; for single asset strategies. |
Commission | Roundturn commission in account currency units; for single asset strategies. |
Contracts per lot | Lot size; for single asset strategies. |
Gross win (loss) | Sums of all wins and all losses in currency units, and the overall volume-neutral result in pips. Note that there can be a negative pip result and a positive gross win (or vice versa) due to different assets pip costs and different trade volumes. |
Virtual win (loss) | Sums of all wins and all losses of phantom trades in currency units, for virtual hedging. Normally worse than the gross win / loss due to higher transaction costs. |
Average profit | Annual, monthly, and daily profit (workdays only), calculated from the difference between start and end balance. |
Max drawdown (MAE) |
Maximum drawdown during the test period, in account currency units and in percent from the gross profit. The drawdown is the difference between a balance peak and the lowest subsequent equity valley (balance = account value, equity = balance plus value of the all open trades). MAE (maximum adverse excursion) is the difference between an equity peak and the lowest subsequent equity valley. Drawdown is dangerous, MAE is normally not (see remarks below). |
Total down time (TAE) | Percentage of time when the current equity is below a preceding balance peak. TAE (time in adverse excursion) is the time when trades are open and the current equity is below a preceding equity peak. Strategies often have up to 90% down time, but can be still profitable. |
Max down time | Longest drawdown duration, i.e. maximum time from a balance peak to the lowest subsequent equity valley. |
Max open margin | Maximum total margin allocated during the backtest period. |
Max open risk | Maximum loss when all open trades hit their initial Stop at the worst possible moment. Dependent on the stop distances and the likeliness of such an event, the Max open risk can far exceed the Capital required. |
Trade volume | Total and annualized value of all assets bought and sold, in units of the account currency (see remarks). |
Transaction costs | Total costs of spread (Spr), slippage (Slp), swap/rollover (Rol) and commission (Com) for all trades. Slippage and rollover can increase the profit in some cases; costs are then positive. In test mode the simulated costs are displayed, in trade mode the real costs. Slippage cost in trade mode is calculated from the difference of price at order time and fill price. |
Capital required | Required initial capital for trading the strategy. For strategies that reinvest profits (Capital variable nonzero), the displayed value is the minimum initial capital for avoiding a margin call in the test period. Otherwise, it's the maximum margin on non-leveraged accounts, or the sum of maximum margin and normalized maximum drawdown (or normalized MAE, when the MAECAPITAL flag is set) on leveraged accounts. This amount would be required when the strategy is entered at the worst possible moment of the simulation, i.e. directly at the balance peak preceding the largest drawdown. |
Number of trades | Number of trades in the backtest period. Only real trades are counted, not phantom or pending trades. |
Percent winning | Percentage of winning trades. |
Max win (loss) | Maximum win and loss of all trades. |
Avg trade profit | Average return of a trade in account currency units and in volume-neutral pips; separately listed for winning (+) and losing (-) trades. Robust strategies should return a multiple of the spread. Avoid systems that generate either many trades with small average returns, or few trades with very large average returns. |
Avg trade slippage | Average slippage cost of a trade in account currency units and in pips; separately listed for positive (+) and negative (-) slippage. In test mode it's the simulated slippage, in trade mode the real slippage. |
Avg trade bars | Average number of bars of a trade; separately for winning (+) and losing (-) trades. |
Max trade bars | Maximum time a trade was open. |
Time in market | Total time of all trades compared to the backtest time. This can be more than 100% when several trades are open at the same time. The smaller the time in market, the less exposed is the capital to the market risk. |
Max open trades | Maximum number of simultaneously open trades. |
Max loss streak (uncorrelated) |
Maximum number of consecutive losses during the test, and the theoretical number under the assumption of uncorrelated returns, i.e. equally distributed wins and losses. If the real number is noticeably higher, wins and losses tend to cluster with this strategy, and an equity curve trading algorithm could improve the performance. |
Annual return | Annualized profit divided by the required initial capital. As this value depends on drawdown, it can be subject to random fluctuations (see remarks). |
Annual growth rate | Compound annual growth rate (CAGR) of the investment; the nth root of the total equity growth, where n is the number of years in the test period. Only displayed for strategies that re-invest profits. |
Profit factor (PRR) | Gross win divided by gross loss. The pessimistic return ratio (PRR) is the profit factor adjusted by the number of trades; it gives a worse result when the number of trades is low. |
Sharpe ratio | Annualized ratio of per-bar profit mean and standard deviation, calculated from the equity curve. The Sharpe ratio is a good performance gauge (see remarks) and should be > 1 for tradeable strategies. |
Kelly criterion | Ratio of bar profit mean and variance; optimal investment factor for a single-asset, single-algo strategy to maximize the profit. |
R2 coefficient | Coefficient of determination; the similarity of the equity curve with a straight line ending up at the same profit. The closer R2 is to 1, the steadier are the profits and the better they will be possibly reproduced in real trading (see remarks). |
Ulcer index | Mean drawdown percentage; a measure of length and depth of drawdowns (see remarks). The higher the ulcer index, the stronger your stomach must be for trading the script. The ulcer index should be < 10% for preventing ulcer. |
Cycle performance | Separate profit factors of the oversampling cycles. High profit differences between cycles are a sign of an unstable strategy. |
Monte Carlo analysis | Performance analysis (see Monte Carlo Method) by evaluating many possible equity curves with different distributions of trades and returns. A strong serial correlation of trade returns can cause Monte Carlo results higher than the result from the real equity curve. |
Confidence level | Confidence level of the following performance parameters. F.i. at 95% confidence level, 95% of all simulations generated the same or better results, and 5% generated worse results. |
AR | Annual return at the given confidence level. |
DDMax | Maximum drawdown (not normalized) at the given confidence level. |
Capital | Capital requirement at the given confidence level. |
Portfolio analysis | Performance analysis per asset, per algorithm, and per component. Only components with trades are listed. The figures are taken from the last oversampling cycle; when the ALLCYCLES flag is set, they are taken from all cycles. |
OptF | OptimalF factors for portfolio strategies (see Money Management). When the factor is 0, the component was unprofitable in the test. |
ProF | Profit factor (gross win divided by gross loss, including phantom trades). A '++++' in the column indicates that there were only winners, '----' indicates that there were no winners. |
Win/Loss | Number of winning and losing trades, including phantom trades. |
Wgt% | Weight of the component in percent; component profit divided by total profit. Indicates the contribution of the component to the whole strategy. The weight can be negative, f.i. with a losing component and a positive overall result. |
Result | Current profit or loss of the component in live trading. |
Cycles | Profit separated by WFO cycles. '/' is a winning cycle, '\' a losing cycle, 'X' is a cycle with both winning and losing components, and '.' is a cycle without trades. |
Additionally, the performance can be evaluated by user criteria or stored for further evaluation with the user-supplied evaluate function.
Keep in mind that that all those performance figures are derived from historical data (even when it's out-of-sample data). The future is unknown, so there is no guarantee to achieve the same performance in live trading. Many figures - f.i. Sharpe ratio, Monte Carlo analysis, drawdown extrapolation, R2 coefficient - are based on mathematical models that assume a Gaussian distribution of returns. However there is no guarantee that real returns always follow a Gaussian distribution. For those reasons, don't interpret too much into performance figures. Even a system with excellent theoretical performance can cause real loss of money.
* See Malik Magdon-Ismail / Amir Atiya, "Maximum Drawdown", 2004.