I thought this was a very good and fun discussion http://quant.stackexchange.com/questions/1427/risk-factors-in-analysing-strategies. Please comment negative or positive. Unfortunately, I do not spend much time talking about all the failed methods that I have discovered but everyone should know that my failure rate for mathematically intense strategies is far higher than the failure rate for the simple. Now, I would much rather find more data than learn more math and stats.
One of my main philosophies is that based on joint probabilities the more decisions I make the less likely they will be correct.
If I have not stated enough over the last 7 months of blogging, simple methods tested with multi-country and sector datasets extending well beyond 1980 and hopefully well beyond 1950 provide me the confidence I need to risk my own money and maintain confidence in myself in mass chaos like 2008-2009. Waiting for mass chaos to test your methods and ask questions most likely guarantees a very unfavorable result.
I have seen the smartest people blow up fairly reliably over my career (see quant funds 2007-2008, LTCM 1998, Victor Niederhoffer, and on and on). In all my experience I have found it is very easy to lose money equally well with both sophisticated and basic. However, making money reliably seems to favor the simple. As long as skeptical questions like this are asked and money flows to shorter and shorter High Frequency products, I feel comfortable that I can pursue my infantile methods for profit in the markets. However, I am also an open and humble voracious reader who readily accepts any methods that pass my very stringent tests.
Here are some very good sources for additional reading:
Paul Wilmott and Emanuel Derman (easily two of the greats in quantitative finance) http://www.ederman.com/new/docs/fmm.pdf
Ned Davis http://www.amazon.com/Being-Right-Making-Money-Davis/dp/0970265107 (look at those used prices wow! must be something of value in there")
IPE Quant: All in the numbers http://ipe.com/magazine/all-in-the-numbers_40749.php?issue=June%202011
Bryan Urstadt The Blow-up http://bryanturstadt.com/uploads/Nov07FeatureQuants-2.pdf
New York Times “In Modeling Risk, the Human Factor Was Left Out” http://www.nytimes.com/2008/11/05/business/05risk.html
One of the very few quantitative success stories is Simons at Renaissance Cracks Code, Doubling Assets (Update1).