Detrending and Price Curve Manipulation

When developing, testing, or training systems, it is often very useful to modify price curves, randomize price data, or remove trend from trades, indicators, or trained parameters. This allows training or testing strategies under defined conditions, such as with no trend, with artificial trend, or with artificial price peaks or drops. Randomized price curves can be used for Monte Carlo evaluations or reality checks. Backtesting with detrended or inverted price curves is a quick check to verify the validity and robustness of a strategy. 


          Artificial price drop for testing the behavior of a grid trading system


          Shuffled price curves for a reality check algorithm

One variable does it all:

Detrend

Accepts the flags below for detrending or other manipulations of the price curves or of trades, indicators, or trained parameters. The flags can be set with Detrend = Flag; or setf(Detrend,Flag);. Flags can be combined with '+' or '|':

Range:


TRADES Detrend trade results. The trade return is corrected by the average price slope of the current WFO cycle. This removes trend bias from a WFO test while keeping the properties of the price curve.
PRICES Detrend trade results and price f functions, as well as indicators derived from them. A corrrection factor as above is added to the returned prices. This ensures that any WFO cycle starts and ends at the same price, and detrends indicators and signals based on series generated with price() calls.
CURVE Detrend historical price data on loading. The curve is tilted so that the end and the start of the historical data are at the same level. This also affects the displayed price curve in the chart.
RECIPROCAL ReReplace historical prices with their reciprocal values. This it can be used for creating an artifical asset as the reciprocal of an existing forex or crypto pair (f.i. convert BTC/USD to USD/BTC).
INVERT Similar to RECIPROCAL, but keep the price range while inverting the price curve. This reverses all trends in the curve, and can be used for a reality check of a system that is symmetric in long and short positions.
SHAPE Bend the price curve to a predefined shape for testing strategy behavior under certain conditions such as sudden price drops or trend reverals, even when they don't appear in the original curve.The desired shape can be set up with the Shape array (see below).
SHUFFLE Randomize the price curve by shuffling the price ticks without replacement. Keeps its overall trend from start to end, but removes any short-term trends and correlations between the prices. Used for reality checks.
SHUFFLE+PEAK
SHUFFLE+VALLEY
Randomize the price curve as above, but generate a curve that does not exceed the highest peak or/and the lowest valley of the original price curve. Use this for keeping the original price range. Can be slow on curves with large price ranges.
BOOTSTRAP Randomize the price curve by shuffling the price ticks with replacement. Keeps its overall trend from start to end, but removes any short-term trends and correlations between the prices. Used for reality checks.
BOOTSTRAP+PEAK
BOOTSTRAP+VALLEY
Randomize the price curve as above, but generate a curve that does not exceed twice the maximum of the orginal price curve, or/and does not fall below zero. Can be slow on curves with large price ranges.
RANDOMWALK Generate a random walk price curve by moving the price in random steps that depend on original volatility. Removes any market inefficiency out of the price curve.
RANDOMWALK+PEAK
RANDOMWALK+VALLEY
Generate a random walk price curve that does not exceed twice the maximum of the orginal price curve, or/and does not fall below zero. Can be slow on curves with large volatility.
NOPRICE Do not do not detect gaps or fix outliers on loading. For data files that contain no prices, but other types of data.

Type:

int
 

Shape

Pointer to a 0-terminated array of price values for applying a shape to the price curve when Detrend = SHAPE is set. All prices of the array are placed in equal distances, and the price curve is bent so that it touches any of them. The more prices in the array, the closer is the curve adapted to the given shape. The last array element must be 0 for indicating the end (see example).

Type:

var*
   

Remarks:

Example (see also Detrend.c):

// detrend the price curve for training only
if(is(TRAINMODE)) Detrend = CURVE; 

// apply a rectangular shape to the curve
var SuddenDrop[50] = { 1.5, 1.5, 1.0, 1.0, 1.0, 1.0, 1.0, 1.5, 1.5, 0 };
Shape = SuddenDrop;
Detrend = SHAPE;

See also:

mode, WFO, DataSlope, randomize, setf

 

► latest version online