Mariner Backtesting - Submitting a Backtest

So you have written a script, how do you backtest it?

First press the New Test button at the top right of the screen (or press CTRL+B).

picture
Backtest Submission Form
picture

From here you can change the settings used in the back-tester.

TOP SECTION : Backtest Name and Script Selector

The first box is optional and you can use it to give a name to this specific backtest.

Next to it is a "drop down" menu to select the script you want to backtest.

Underneath that drop down box is the currently selected script, this defaults to the script you have just been editing, so unless you want to give a name to the backtest, most of the time you will need to change nothing in this section.

Parameter Passing

If you are passing parameters to your backest then there will be an extra box here, input your parameters in the form of a dictionary ie {"shares":100,"maxatr":2.5,"minrvol":2.0} Parameter Passing Documentation.

If you are having trouble passing parameters, try this site to validate your JSON data.

LEFT SECTION : Dates, Times and Symbols

Dates

The date range for your backtest. Keep this very short when testing, once you have your model working close to how you want you can expand this range. We have data back to January of 2011.

Script active

CORE HOURS ONLY This is the time period during the day that you want to run your script. Normal hours are 9:30 to 16:00 with shorter hours on those market days where trading is limited to half a day. md.market_open_time and md.market_close_time reflect these changes.

CORE + EXTENDED HOURS 00:00 to 23:59

CUSTOM (New York Time) Enter your custom hours. All times MUST be HH:MM. You can press the plus button to sleep during the day, so you can set it to run 9:20 to 9:45 and then pause and restart 15:40 to 16:05

Symbols

USE SPECIFIC SYMBOL You will normally select your symbol universe in is_symbol_qualified but you can check this box and use this section to manually define a symbol universe (comma separated), this can be useful if you just want to quickly test using a couple of symbols.

OTHER SYMBOLS TO LOAD This is equivalent to backtesting_extras_symbols, these are symbols that are loaded but not traded. Often used for reference symbols such as SPY, QQQ or sector ETFs.

CENTER SECTION : SIMULATOR BEHAVIOUR

These settings alter the way the back-tester simulates data flow, order handling as well as parallelization.

RUN AS SINGLE MULTIDAY JOB

Some models enter and exit the same day, this allows us to parallelize backtests, run multiple dates at once, but some models hold positions for a number of days and in order to backtest these models they must be run one at a time in order.

As examples, let's take a one year, 260 day backtest

One version CAN run in parallel. If it was allocated 65 cores at a time and if its backtest time is 60 seconds then it takes 4 minutes to run the year. The second version CANNOT run in parallel as it must run as a single multiday job, so it has to run each day in turn (260 x 60 secs) 260 minutes.. over 4 hours.

ENABLE FAST SIMULATION

ALLOW PARTIAL FILLS

ENABLE PROFILING

Latencies

These millisecond values allow you to set custom latencies for steps in your models interaction with outside agencies. Latency is only taken into account for MARKET orders.

ACK EXCHANGE FILL MARKET DATA

RIGHT SECTION : POST BACKTEST REPORTING

You can ask the backtesting system to generate the report of your choice once the backtest is completed. Don't worry if you forget as the reports can also be requested on the results tab after the backtest has completed. The time taken to generate a report depends on the complexity of the report and, more importantly, the size of your backtest.