It has been a while since I discussed testing for overfitting in backtests. Since then, Marcos López de Prado and coauthors have done some very thoughtful work (see the bottom), and they even started a blog. Their newest paper builds on discoveries they made in their earlier work, and is an absolute must-read.
Bailey, David H. and Borwein, Jonathan M. and Lopez de Prado, Marcos and Zhu, Qiji Jim
Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance (October 7, 2013)
Available at SSRN: http://ssrn.com/abstract=2308659
Translating scientific papers into code is not always easy, but I spent some time implementing some of the concepts in R, so that I can understand this more fully. Just as a word of encouragement to others out there, I am no math genius nor have any advanced math education, so please don’t be intimidated by formulas. Below you will see a slidify/rCharts discussion demonstrating these first steps. I plan to research this much more thoroughly. As always, I blog to interact, so please let me know what you are thinking.
