Friday, August 31, 2012

Russell 2000 Softail Fat Boy

If the Russell 2000 were a motorcycle, maybe it should be a Harley-Davidson Softail Fat Boy.  I have explored the exceptional case of the Russell 2000 in quite a few posts

but I still am not sure I have done a good job of clearly and simply explaining some of the unique characteristics of the Russell 2000 over the last  25 years.  The Russell 2000 has been extremely difficult to beat because the upside has been so good (fat boy) and the downside has been the same or better (softtail).

From TimelyPortfolio
From TimelyPortfolio
From TimelyPortfolio

R code from GIST:

Thursday, August 30, 2012

Another Great Google Summer of Code 2012 R Project

Tradeblotter announced the very nice features that will be added to the PerformanceAnalytics package as a result of the Google Summer of Code (GSOC) 2012 project:

“…Matthieu commenced to produce dozens of new functions, extend several more existing ones, and add more than 40 pages of additional documentation (complete with formulae and examples) to PerformanceAnalytics. He’s included Bacon’s small data set and several new table.* functions for testing and demonstrating that the functions match the published results. All on plan, I would add.

He also wrote a very nice overview of the functions developed from Bacon (2008) that are included in PerformanceAnalytics, which should be helpful to readers or teachers of Bacon’s work. Matthieu’s summary document will also be distributed as a vignette in the package, accessible using vignette('PA-Bacon'). Additional detail is included in the documentation for each function, as well. I’ll highlight some of those functions in later posts…”

Although the Bacon vignette is very helpful, I highly recommend reading the book also.  For those of you on O’Reilly’s Safari, you can add the book to your bookshelf.


or buy it from Amazon.

Since I saw the announcement (been patiently waiting) when I was also reading this very good article “Tomatoes and the Low Vol Effect” by Research Affiliates, I was already working on some R to explore the Sharpe and Information Ratio between a buy/hold on the S&P 500 and a 10 month moving average strategy.  When you look at the code, you might enjoy some of the other charts and methods, but  I decided to change directions with my research and integrate a new PerformanceAnalytics function and continue to push the limits with another fine Google Summer of Code 2012 addition xtsExtra plot.xts.  I was not real selective and opted for ProspectRatio, since it is a formula I do not often hear discussed.  On page 20, the Bacon vignette describes ProspectRatio as


I basically tried to include absolutely everything possible (scattered, smothered, covered, chunked, and diced) from xtsExtra plot.xts in this chart.  The chart includes cumulative growth of buy/hold and 10 month moving average on the S&P 500 in the top panel, shows drawdown in the middle, and then adds a horizon plot of Prospect Ratio for the ma strategy – Prospect Ratio for the buy/hold.  Since plot.xts also allows for blocks, I thought it would be helpful to also shade those periods when the 36 month rolling Sharpe Ratio of the ma strategy outperforms buy/hold.  While this is probably a bit much for a single chart, I think it demonstrates clearly how powerful these tools are.

From TimelyPortfolio

R code from GIST (do raw for copy/paste):

Monday, August 27, 2012

Horizon on ggplot2

SocialDataBlog’s kind reference in post Horizon plots with ggplot (not) motivated me to finish what the post started.  I knew that ggplot2 would be a little more difficult to use for the purpose of a horizon plot, but I felt compelled to provide at least one example of a horizon plot for each of the major R graphing packages.  I achieved a good result but the code is not as elegant or as flexible as I would like.  Readers more comfortable with ggplot2, please bash, fork, and improve.

From TimelyPortfolio

R code in GIST (do raw for copy/paste):

Thursday, August 23, 2012

Bonds Much Sharpe -r Than Buffett

Mebane Faber’s post Buffett’s Alpha points out Warren Buffett’s 0.76 Sharpe Ratio discussed in the similarly title paper Buffet’s Alpha.  I of course immediately think about the 8th Wonder of the World – the US Bond Market, whose Sharpe Ratio has trounced Buffett’s for the last 30 years.  What I like even better are all the tactical systems that employ US bonds in their backtests and make no adjustment for substantially different returns going forward.  For those of you who do not know, bonds with absolute certainty cannot achieve >8% annualized returns with max drawdown < 5% for the next 30 years with a starting yield to worst at 1.86% (Barclays Agg 8/23/2012). 

If anyone can show me where to get 8% annualized returns with max drawdown of 5% for the next 30 years, please let me know, and I will buy that with leverage and enjoy life. I’ll be happy to share my gains with whomever has the answer.

From TimelyPortfolio

In addition to an unbelievable Sharpe Ratio, bonds have exhibited a low/negative correlation with stocks during stocks’ bear market, which is also historically very anomalous.

From TimelyPortfolio

Just because I love horizon plots.

From TimelyPortfolio

For a longer perspective, here is rolling correlation since 1900 from the CSFB 2011 Yearbook.


So the experience can be dramatically different than what we have been fortunate enough to experience recently.


R code from GIST (install xtsExtra):

Monday, August 20, 2012

plot.xts with Moving Average Panel

(for all plot.xts posts, see

As another example of all that we can do with the new plot.xts, let’s try to do a price plot with a moving average overlays.  We will use the ETFs shown by Mebane Faber at  With the panel functionality, it is very easy to specify a panel to draw the price line and then add the calculated moving average.  Notice how in all the examples, the recession block appears easily and very nicely.

From TimelyPortfolio

Also, if you wanted to specify some funky layouts, we have that option.  For this case, I do not think it makes much sense, but in the future I will demonstrate some more appropriate uses.

From TimelyPortfolio

Or as one more example, let’s change it up just slightly.

From TimelyPortfolio

R code in GIST (do raw for copy/paste and also install xtsExtra from r-forge):

Friday, August 17, 2012

GARCH Panel in plot.xts

I’m clearly out of my realm of competence with most of the rugarch functions, but I thought it might be nice to provide an example combining plot.xts and uGARCHroll.


R code from GIST:

Horizon Plots with plot.xts

Anyone who has read

  1. 48 Industries (Dendrogram Ordered) Over 50 Years
  2. 48 Industries Since 1963
  3. “Trend is Not Your Friend” Applied to 48 Industries
  4. Horizon Plots in Base Graphics
  5. More on Horizon Charts
  6. Application of Horizon Plots
  7. Horizon Plot Already Available
  8. Cubism Horizon Charts in R

should already know that I really like horizon charts.  Also, you would probably guess that I would be very excited if the new plot.xts could produce horizon charts.  So I am excited, since with its panel functionality, plot.xts very capably creates horizon charts.

From TimelyPortfolio

R code from GIST (also, please install xtsExtra from r-forge):

Thursday, August 16, 2012

plot.xts is wonderful

As mentioned in FOSS Trading post A New plot.xts yesterday

“The Google Summer of Code (2012) project to extend xts has produced a very promising new plot.xts function. Michael Weylandt, the project's student, wrote R-SIG-Finance to request impressions, feedback, and bug reports. The function is housed in the xtsExtra package of the xts project on R-Forge.

Please try xtsExtra::plot.xts and let us know what you think. A sample of the eye-candy produced by the code in Michael's email is below. Granted, this isn't a one-liner, but it's certainly impressive! Great work Michael!”

and as announced by author Michael Weylandt in

“As the community which makes the most heavy use of xts, I would like to draw your attention to a new set of plotting functions for xts objects available as part of Google Summer of Code 2012. This work represents a major overhaul of previously existing plot.xts and should provide you with the most comprehensive and flexible time series plotting available in R. Features include:

  • "automagic" layout construction and axis alignment
  • smart argument recycling
  • panel function abilities
  • more attractive candle and bar plots for OHLC objects
  • scatterplots to view the co-evolution of multiple series
  • event markers
  • regime highlighting
  • time-oriented barplots via barplot.xts [based on code by Peter Carl]
  • interoperability with all known R time series classes using the xts try/reclass paradigm

while retaining the same smart axis formatting and gridlines that plot.xts provided. We have made every effort to maintain complete compatibility with documented usages of the old plot.xts and to be 95% compatible with plot.zoo. My goal has been to craft a design which uses smart defaults to put attractive and informative graphics ever at your fingertips, while remaining flexible enough for "power-users" to craft every detail as they desire…”

Michael has done a fantastic job with this Google Summer of Code (GSOC) project, and I look forward to incorporating all the new features of plot.xts.  As a quick example very different from that shown in the post and mailing list announcement, I thought it would be fun to show how we can use plot.xts to replace the veteran chart.TimeSeries and charts.PerformanceSummary plots from the PerformanceAnalytics package.

For the chart.TimeSeries, I will almost exactly replicate the chart given in the documentation.

From TimelyPortfolio

Not really as a useful application but for the sake of demonstration, we can produce something like this to separate styles and also avoid event label collision (just implemented by Michael very late last night).

From TimelyPortfolio

plot.xts even allows plot.zoo and lattice panel type functionality which allows us to do very nice things like in this charts.PerformanceSummary style chart.

From TimelyPortfolio

I very much appreciate this very powerful contribution to R.  Thanks to everyone involved.

R code in GIST (note: install xtsExtra from R-forge) :

Thursday, August 9, 2012

48 Industries (Dendrogram Ordered) Over 50 Years

Thanks to reader AHWest for the comment on post 48 Industries Since 1963.

“I think it would be interesting to see the industries ordered by some sort of similarity of returns.”

I think this is a great suggestion, and I would like to see it also.  I tried the dendrogram plot technique from Inspirational Stack Overflow Dendrogram Applied to Currencies, but then I spotted the dendrogramGrob in the latticeExtra documentation.  This was much easier, and in a couple of lines, we are able to order and connect the 48 industries.

From TimelyPortfolio
R code from GIST (do raw for copy/paste):

Wednesday, August 8, 2012

“Trend is Not Your Friend” Applied to 48 Industries

Please see previous post Crazy RUT in Academic Context Why Trend is Not Your Friend.

I’ll repeat the intro to the post mentioned above, so we can all get caught back up.

In response to Where are the Fat Tails?, reader vonjd very helpfully referred me to this paper The Trend is Not Your Friend! Why Empirical Timing Success is Determined by the Underlying’s Price Characteristics and Market Efficiency is Irrelevant by Peter Scholz and Ursula Walther. The authors conclude

“Our study on the basis of real data clearly confirms the hypothesis that the asset price characteristics of the underlying price process have a crucial impact on timing results. This allows us to forecast the timing success depending on the market's parameters. An OLS
regression analysis supports our predictions and verifies our assumption that the drift has the
strongest influence on timing success. By contrast, the higher moments (skewness, kurtosis)
seem not to have any significant impact on the timing result in the empirical sample. As we
presumed, the level of market development, and hence the degree of efficiency, does not play
any role. Trading worked coincidentally rather well in the developed world and quite poorly in
the emerging markets. The driving factor for the timing success is the parametric environment the trading system stumbles on…

Our study contributes to the discussion by providing a structured analysis of the relevance of the most important price process parameters. As a result, the traditional explanations for timing success can be abandoned: we find that it is very likely for the SMA trading rule to generate excess returns over its benchmark if the underlying price path exhibits negative drifts, high serial autocorrelation, low volatilities of returns, and highly clustered volatilities. Drift and autocorrelation of the underlying asset seem to have the largest impact, though.”

One of my initial ideas for extending the research was to incorporate a much larger set of indexes over a longer period of time.  As I was working on 48 Industries Since 1963, I decided 50 years of data on 48 different indexes would be a great dataset to apply the ideas and methods presented in the paper.

Using R and all its wonderful packages, it is surprisingly easy to accomplish.  Let’s see if we can test “drift and autocorrelation…have the largest impact” on excess returns with industries also.  I’ll try not to get too statistical.

In terms of drift or annualized return, we can see a linear inverse relationship between return and out(under)performance of the 200 day moving average system, so the better the performance of the industry, the less likely a moving average system is to outperform.

From TimelyPortfolio

In terms of the GARCH model effects on excess returns, a parallel coordinate chart will best start our exploration.  Lines are colored by the excess return of a moving average system on each industry.

From TimelyPortfolio

Mu and alpha1 seem to most heavily influence the ability of a moving average system to outperform.  Let’s isolate our chart to mu and alpha1 and add ar1 based on the authors’ findings.  I am very tempted to try to explain GARCH here, but for the sake of brevity, I’ll refrain.  This paper and this Portfolio Probe post offer a good introduction to GARCH.

From TimelyPortfolio

Now let’s look at a scatterplot of excess return versus each of the GARCH stats, starting with those with the most influence.

From TimelyPortfolio
From TimelyPortfolio
From TimelyPortfolio
From TimelyPortfolio
From TimelyPortfolio
From TimelyPortfolio

In conclusion, it seems the same effects observed by the authors also apply to US industry indexes.  In future posts, I’ll add a little more statistical rigor to the analysis and apply to other indexes.

Now, I just cannot resist using a horizon plot to evaluate the rolling 250 day excess returns of a moving average system over buy and hold.  As you can see, a bull market favors buy and hold.  The 70s and 2008-2009 were very kind to a moving average approach.

From TimelyPortfolio

R code in GIST (do raw for copy/paste):

Tuesday, August 7, 2012

48 Industries Since 1963

Please see for all horizon plot posts.

Once more thanks to Ken French for his data, we can accomplish something I think is fairly amazing.  In 640x800, we can see 250 day rollling returns for 48 U.S. industries since 1963.

From TimelyPortfolio

R code in GIST (do raw for copy/paste):

Friday, August 3, 2012

Horizon Plots in Base Graphics

for background please see prior posts More on Horizon Charts, Application of Horizon Plots, Horizon Plot Already Available, and Cubism Horizon Charts in R

There are three primary graphics routes in R (base graphics, lattice, and ggplot2), and each have their zealots.  Last time in More on Horizon Charts, I used lattice and latticeExtra. This time we will build horizon plots in base graphics, and I was pleased with the result. Unfortunately, there is one small issue in that the points of change from positive to negative overlap.  Please let me know if you have a solution. Thanks to helpful readers for their comments and code changes, this is now fixed.

From TimelyPortfolio
From TimelyPortfolio
From TimelyPortfolio
From TimelyPortfolio
From TimelyPortfolio
From TimelyPortfolio
From TimelyPortfolio

Now, let’s implement a for loop and mirror the negative values.

From TimelyPortfolio

With horizon chart functionality in base graphics, hopefully we can add this type now to other packages.  Here is a potential example using quantmod.

From TimelyPortfolio


R code in GIST (do raw for copy/paste)

Thursday, August 2, 2012

How do you say “We Will Do Whatever It Takes” in Thai?

As the market has already started to poke holes in Draghi’s promise, I thought it would be good to continue the series of posts that I began with the British version “We Will Do Whatever it Takes” with my favorite article written during the Asia Pacific crisis in 1997.

The Economist, “The baht spills over”: May 22, 1997

Too soon to crow

“Even if Thailand's troubles are in a regional class of their own, its neighbours still have reason to fear the knock-on effect of a successful onslaught on the baht. For now, Thailand has been crowing about a victory over the speculators. Mr Amnuay has even promised to bring interest rates, which have been kept high to protect the baht, down by two percentage points this year. It is as if a little guy, having survived one round with the big bullies, was dancing around with his fists up, saying, go on, hit me again.

They almost certainly will. The defence of the baht was achieved by a series of extraordinary measures that will be hard to sustain…”

A little graphical perspective might help us remember when this article was written.


For the entire set of Economist articles on Thailand during this period, go to  Also, please read the incredibly thorough account of the 1997 crisis at The Nation blog.

If throughout financial market history, the central banks and politicians ever defeated the markets (greedy speculators), please let me know.  I am absolutely amazed at the confidence in central banks and politicians, especially in the United States.


R code:

#plot Thai baht
DEXTHUS <- 1/DEXTHUS["1995-01-01::1999-12-31",]
       panel = function (x,y,...) {
         panel.abline(v=index(DEXTHUS)[which(index(DEXTHUS)=="1997-05-22")],col="black", lty=3)
         panel.text(x=index(DEXTHUS)[which(index(DEXTHUS)=="1997-05-22")], y = 0.025, labels="Economist article The Baht Spills Over", col="indianred4", srt=90, pos=3)
        main = "Thai Baht Before and After Crisis (1995-1999) \n source : Federal Reserve Bank of St. Louis")

Wednesday, August 1, 2012

More on Horizon Charts

for background please see prior posts Application of Horizon Plots, Horizon Plot Already Available, and Cubism Horizon Charts in R

Some feedback has led me to think that I might have been a little ambitious with my last post on horizon charts. I thought it might be helpful to quickly provide an overview on horizon charts. Since I am not a visualization expert, I will try to compile the best examples and tutorials that I have found. At the end, I will do a step by step walkthrough of the construction of a horizon chart in R.

Below is a great talk in its entirety by Mike Bostock.  For the discussion on horizon charts, skip to the 11:20 mark.

Mike Bostock @ Square talks about Time Series Visualization from Librato on Vimeo.

Fortunately, Mike also provides the interactive example from the video demonstrating the construction of a horizon chart from an area chart.  I have embedded it below (see for the full example from the original source).

This quick YouTube clip from Panopticon all the way back in 2009 also does a very nice job explaining construction.  If you learn better from reading, a short paper from Hannes Reijner of Panopticon Software covers the same material.

Jeffrey Heer of Stanford University with Nicholas Kong and Maneesh Agrawala from University of California, Berkeley, investigate the effectiveness of horizon plots and make some recommendations for their use.  The chart below also offers another explanation of the steps in building a horizon plot.


They recommend and conclude

Layered Bands Are Beneficial As Chart Size Decreases
We found that dividing a chart into layered bands reliably increased estimation time and increased estimation error at constant chart heights. However, we also found that 2-band mirrored charts led to better estimation accuracies for chart heights less than 24 pixels (6.8 mm on our displays). For larger chart sizes, we advise scaling 1-band mirrored charts.  For smaller sizes, we advise adding layered bands.

Extending the horizon into healthcare, here is an interesting project using horizon graphs for visualization of diabetes care.


Interactive Visual Exploration of Multivariate Medical Measurements in Diabetes Care


Lead / Contact
Wolfgang Aigner

Wolfgang Aigner, Vienna University of Technology
Michael Atanasov, HTBL Krems
Alexander Rind, Vienna University of Technology
Philipp Schindler, HTBL Krems
Reinhardt Wenzina, HTBL Krems

Vienna University of Technology, Institute of Software Technology & Interactive Systems
HTBL Krems, Department of Information Technology


Now For My Own Attempt at Explaining

If we use yesterday’s example of a 200 day moving average system where you enter when above the moving average and exit when below, we might like to see a standard time series plot like this one.

From TimelyPortfolio

In a simple world with only one asset or stock, this might be sufficient.  However, we probably will have multiple instruments that we would like to monitor, and dedicating this much height per instrument will require lots of space.  Ideally, we could condense each of these plots, so that we could see many at a time.

In the first step toward condensing, we could extract just the information that is most meaningful, which I consider to be the percent above or below the moving average.

From TimelyPortfolio

We might then try an area chart to better depict above or below 0.

From TimelyPortfolio

I’m sure you are wondering though when we will start reducing height and saving space.  We could start by changing all the negative values to positive values, and add color to represent positive or negative.  This means we cut our chart height by about 1/2.

From TimelyPortfolio

However, we need much more efficiency, so let’s separate the chart into bands.

From TimelyPortfolio

Is there any potential way of separating each of these bands and then recombining them to save space?  Let’s look at each band separately.

Band 1 (0 to 10%)

From TimelyPortfolio

Band 2 (10% to 20%)

From TimelyPortfolio

Band 3 (20% to 30%)

From TimelyPortfolio

Band 4 (30% to 40%) exceeding 3 bands is not recommended

From TimelyPortfolio

Notice that similar to a heat map, the band colors increase in intensity and opaqueness as their values increase. If we layer each band on top of each other, we get a horizon plot, and we can reduce the original chart by 1/6 or even more without significant loss of information.

From TimelyPortfolio

I hope this helps explain why we might use horizon plots and how to make them.  If nothing else, maybe you will have learned some lattice techniques in R.


R code in GIST (do raw for copy/paste):