I stumbled on the ttrTests R package as mentioned in my post ttrTests Experimentation. I did not recognize its potential until I spent much more time absorbing the basis of the package—David St. John’s thesis Technical Analysis Based on Moving Average Convergence and Divergence. Since the title specifically addresses MACD, which I have had little luck implementing, I dismissed much of the content. However, the power of the thesis extends well beyond MACD to all systematic methods and describes tests to ensure luck is not the source of a system’s returns. In the package documentation, there is a summary of the 5 main tests:
“Contains five major tests supported by other functions: Did the TTR strategy outperform a benchmark in the past data? Is the excess return significant, using bootstrapping to construct a confidence interval? Is the excess return explained by data snooping? Is the ’good’ choice of parameters robust across sub-samples? Is this robustness significant, using bootstrapping to construct a confidence interval?”
The tests expose luck, data snooping, trading costs, and parameter persistence across both degrees of freedom and subperiods. I look forward to documenting its power in my blog and also potentially working with the author to include in other R packages such as quantstrat.
Since I am running out of time, I first want to apply each of the tests to MACD in the same style as the package documentation and the thesis paper, but this time on a xts DJI object gathered through getSymbols rather than the spData provided with the package.
The output from the tests is very cumbersome, but I hope this set of examples will help provide a flavor for the package and its powerful tests. In my next couple of posts, I will run each test in much further detail on my basic custom CUD indicator and try to get the cumbersome output in a far more digestible and graphical format.
R code (click to download from Google Docs):
require(ttrTests)
require(quantmod) #get Dow Jones Industrials from Yahoo! Finance
getSymbols("^DJI",from="1896-01-01",to=Sys.Date())
#convert closing price to vector format which works best with ttrTests
DJI.vector <- as.vector(DJI[,4]) #using the defaults as mentioned in the thesis paper on MACD
#show each of the tests in order of their mention #quotes are from ttrTests package documentation
#"compares the performance of the TTR with some benchmark"
returnStats(DJI.vector) #"constructs a confidence interval for this performance"
#"and gives a p-value for the excess return observed in (1)."
nullModel(DJI.vector) #"constructs a p-value for the ’best’ choice"
#"of parameters within a given domain"
dataSnoop(DJI.vector,bSamples=3,test="RC")
dataSnoop(DJI.vector,bSamples=3,test="SPA") #"asks whether or not good choices of parameters"
#"were robust across different time periods"
#chose 8 since data is from 1928 will approximate by decade
subperiods(DJI.vector, periods=8) #and my favorite of all
#"tests if the persistence measure from subperiods()"
#"is statistically significant"
#this takes the longest (about 10 minutes on my i7 laptop)
paramPersist(DJI.vector)
I shall be very interested in this series of posts. I too would like to use the ttrTests package but I have a problem at the moment in that all my system input functions are written in C++ .oct functions for use in Octave. To rewrite these as R functions would be difficult and time consuming and it will be instructional to see how you intend to use your custom CUD indicator with the ttrTests package.
ReplyDeleteSince the purpose of any user supplied ttr function input is simply to produce a position vector consisting of 1,0 or -1 for the tests in the package, I wrote to the R_sig_finance list to ask if the tests would simply accept such a position vector as an input argument. Unfortunately I got no response to this request. You may find you come up against the same issue.
In my opinion perhaps rewriting or hacking the tests to accept such a position vector as an alternative input argument would be beneficial, and as your familiarity with R seems to be way beyond mine you might find it relatively easy to accomplish this. If so, I would be extremely interested to see how you do this.
@Dekalog: take a look at the documentation for ttrTests. The author explains how you can write a custom function to use with it -- this is probably what you are looking for. You'd simply write a function that spits out your position vector, calculated the way you want -- you could, for instance, parse it from your Octave outputs.
ReplyDelete