rCharts version of d3 horizon

I love horizon plots.  My love shows up throughout my blog, and I have plotted horizon charts in base graphics, xtsExtra, lattice, and ggplot2.  Now with rCharts, we can implement Jason Davies d3.js horizon chart plugin to plot R data in html/javascript.  I put together a tutorial going into great detail on rCharts and convertin custom charts.  For the tutorial, go here, or click on the screenshot below.

Even More JGB Yield Charts with R lattice

See the last post for all the details. I just could not help creating a couple more.

Variations on Favorite Plot - Time Series Line of JGB Yields by Maturity

p2 <- xyplot(value ~ date | indexname, data = jgb.melt, 
    type = "l", layout = c(length(unique(jgb.melt$indexname)), 
        1), panel = function(x, y, ...) {
        panel.abline(h = c(min(y), max(y)))
        panel.xyplot(x = x, y = y, ...)
        panel.text(x = x[length(x)/2], y = max(y), 
            labels = levels(jgb.melt$indexname)[panel.number()], 
            cex = 0.7, pos = 3)
    }, scales = list(x = list(tck = c(1, 0), alternating = 1), 
        y = list(tck = c(1, 0), lwd = c(0, 1))), strip = FALSE, 
    par.settings = list(axis.line = list(col = 0)), 
    xlab = NULL, ylab = "Yield", main = "JGB Yields by Maturity Since Jan 2012")
p2 <- p2 + layer(panel.abline(h = pretty(jgb.melt$value), 
    lty = 3))
p2

From TimelyPortfolio

jgb.xts.diff <- jgb.xts["2012::", ] - matrix(rep(jgb.xts["2012::", 
    ][1, ], NROW(jgb.xts["2012::", ])), ncol = NCOL(jgb.xts), 
    byrow = TRUE)
jgb.diff.melt <- xtsMelt(jgb.xts.diff)
jgb.diff.melt$date <- as.Date(jgb.diff.melt$date)
jgb.diff.melt$value <- as.numeric(jgb.diff.melt$value)
jgb.diff.melt$indexname <- factor(jgb.diff.melt$indexname, 
    levels = colnames(jgb.xts))

p4 <- xyplot(value ~ date | indexname, data = jgb.diff.melt, 
    type = "h")

update(p2, ylim = c(min(jgb.diff.melt$value), max(jgb.melt$value) + 
    0.5)) + p4

From TimelyPortfolio

update(p2, ylim = c(min(jgb.diff.melt$value), max(jgb.melt$value) + 
    0.5), par.settings = list(axis.line = list(col = "gray70"))) + 
    update(p4, panel = function(x, y, col, ...) {
        # do color scale from red(negative) to
        # blue(positive)
        cc.palette <- colorRampPalette(c(brewer.pal("Reds", 
            n = 9)[7], "white", brewer.pal("Blues", 
            n = 9)[7]))
        cc.levpalette <- cc.palette(20)
        cc.levels <- level.colors(y, at = do.breaks(c(-0.3, 
            0.3), 20), col.regions = cc.levpalette)
        panel.xyplot(x = x, y = y, col = cc.levels, 
            ...)
    })

From TimelyPortfolio

p5 <- horizonplot(value ~ date | indexname, data = jgb.diff.melt, 
    layout = c(1, length(unique(jgb.diff.melt$indexname))), 
    scales = list(x = list(tck = c(1, 0))), xlab = NULL, 
    ylab = NULL)

p5

From TimelyPortfolio

update(p2, ylim = c(0, max(jgb.melt$value) + 0.5), 
    panel = panel.xyplot) + p5 + update(p2, ylim = c(0, 
    max(jgb.melt$value)))

From TimelyPortfolio

Variations on Yield Curve Evolution with Opacity Color Scale

# add alpha to colors
addalpha <- function(alpha = 180, cols) {
    rgbcomp <- col2rgb(cols)
    rgbcomp[4] <- alpha
    return(rgb(rgbcomp[1], rgbcomp[2], rgbcomp[3], 
        rgbcomp[4], maxColorValue = 255))
}

p3 <- xyplot(value ~ indexname, group = date, data = jgb.melt, 
    type = "l", lwd = 2, col = sapply(400/(as.numeric(Sys.Date() - 
        jgb.melt$date) + 1), FUN = addalpha, cols = brewer.pal("Blues", 
        n = 9)[7]), main = "JGB Yield Curve Evolution Since Jan 2012")

p3 <- update(asTheEconomist(p3), scales = list(x = list(cex = 0.7))) + 
    layer(panel.text(x = length(levels(jgb.melt$indexname)), 
        y = 0.15, label = "source: Japanese Ministry of Finance", 
        col = "gray70", font = 3, cex = 0.8, adj = 1))

# make point rather than line
update(p3, type = "p")

From TimelyPortfolio

# make point with just most current curve as line
update(p3, type = "p") + xyplot(value ~ indexname, 
    data = jgb.melt[which(jgb.melt$date == max(jgb.melt$date)), 
        ], type = "l", col = brewer.pal("Blues", n = 9)[7])

From TimelyPortfolio

Replicate Me with code at Gist

Slightly Different Measure of Valuation

I grow tired of the tried and true standard measures of valuation, and from time to time I try to think of alternate methods.  One thought was to analyze Ken French’s Market(ME) to Book(BE) Breakpoints by percentile.  We can see by year at what level is a stock considered cheap relative to the universe.  As these breakpoints move higher, the market is willing to pay a higher price.  In reverse, as these breakpoints move lower, stocks fetch a lower price or can be considered cheaper.  Since there are 20 fifth percentiles, a horizon plot can provide a good overall look at this measure of valuation.

Here is a horizon plot of absolute ME/BE valuation by fifth percentile since 1926.

From TimelyPortfolio

For a more representative look, let's plot a horizon chart of the ME-BE / historical mean - 1.

From TimelyPortfolio

For one more non-horizon look, we can use an xyplot.

From TimelyPortfolio

In theory, I think this could provide yet another gauge of the cheapness of stocks, but of course, there is lots of research to be done.

R Code from Gist:

Horizon on ggplot2

SocialDataBlog’s kind reference in post Horizon plots with ggplot (not) motivated me to finish what the post started.  I knew that ggplot2 would be a little more difficult to use for the purpose of a horizon plot, but I felt compelled to provide at least one example of a horizon plot for each of the major R graphing packages.  I achieved a good result but the code is not as elegant or as flexible as I would like.  Readers more comfortable with ggplot2, please bash, fork, and improve.

From TimelyPortfolio

R code in GIST (do raw for copy/paste):

Bonds Much Sharpe -r Than Buffett

Mebane Faber’s post Buffett’s Alpha points out Warren Buffett’s 0.76 Sharpe Ratio discussed in the similarly title paper Buffet’s Alpha.  I of course immediately think about the 8th Wonder of the World – the US Bond Market, whose Sharpe Ratio has trounced Buffett’s for the last 30 years.  What I like even better are all the tactical systems that employ US bonds in their backtests and make no adjustment for substantially different returns going forward.  For those of you who do not know, bonds with absolute certainty cannot achieve >8% annualized returns with max drawdown < 5% for the next 30 years with a starting yield to worst at 1.86% (Barclays Agg 8/23/2012). 

If anyone can show me where to get 8% annualized returns with max drawdown of 5% for the next 30 years, please let me know, and I will buy that with leverage and enjoy life. I’ll be happy to share my gains with whomever has the answer.

From TimelyPortfolio

In addition to an unbelievable Sharpe Ratio, bonds have exhibited a low/negative correlation with stocks during stocks’ bear market, which is also historically very anomalous.

From TimelyPortfolio

Just because I love horizon plots.

From TimelyPortfolio

For a longer perspective, here is rolling correlation since 1900 from the CSFB 2011 Yearbook.

So the experience can be dramatically different than what we have been fortunate enough to experience recently.

R code from GIST (install xtsExtra):

Horizon Plots with plot.xts

Anyone who has read

1. 48 Industries (Dendrogram Ordered) Over 50 Years
2. 48 Industries Since 1963
3. “Trend is Not Your Friend” Applied to 48 Industries
4. Horizon Plots in Base Graphics
5. More on Horizon Charts
6. Application of Horizon Plots
7. Horizon Plot Already Available
8. Cubism Horizon Charts in R

should already know that I really like horizon charts.  Also, you would probably guess that I would be very excited if the new plot.xts could produce horizon charts.  So I am excited, since with its panel functionality, plot.xts very capably creates horizon charts.

From TimelyPortfolio

R code from GIST (also, please install xtsExtra from r-forge):

48 Industries (Dendrogram Ordered) Over 50 Years

Thanks to reader AHWest for the comment on post 48 Industries Since 1963.

“I think it would be interesting to see the industries ordered by some sort of similarity of returns.”

I think this is a great suggestion, and I would like to see it also.  I tried the dendrogram plot technique from Inspirational Stack Overflow Dendrogram Applied to Currencies, but then I spotted the dendrogramGrob in the latticeExtra documentation.  This was much easier, and in a couple of lines, we are able to order and connect the 48 industries.

From TimelyPortfolio

R code from GIST (do raw for copy/paste):

“Trend is Not Your Friend” Applied to 48 Industries

Please see previous post Crazy RUT in Academic Context Why Trend is Not Your Friend.

I’ll repeat the intro to the post mentioned above, so we can all get caught back up.

In response to Where are the Fat Tails?, reader vonjd very helpfully referred me to this paper The Trend is Not Your Friend! Why Empirical Timing Success is Determined by the Underlying’s Price Characteristics and Market Efficiency is Irrelevant by Peter Scholz and Ursula Walther. The authors conclude

“Our study on the basis of real data clearly confirms the hypothesis that the asset price characteristics of the underlying price process have a crucial impact on timing results. This allows us to forecast the timing success depending on the market's parameters. An OLS
regression analysis supports our predictions and verifies our assumption that the drift has the
strongest influence on timing success. By contrast, the higher moments (skewness, kurtosis)
seem not to have any significant impact on the timing result in the empirical sample. As we
presumed, the level of market development, and hence the degree of efficiency, does not play
any role. Trading worked coincidentally rather well in the developed world and quite poorly in
the emerging markets. The driving factor for the timing success is the parametric environment the trading system stumbles on…

Our study contributes to the discussion by providing a structured analysis of the relevance of the most important price process parameters. As a result, the traditional explanations for timing success can be abandoned: we find that it is very likely for the SMA trading rule to generate excess returns over its benchmark if the underlying price path exhibits negative drifts, high serial autocorrelation, low volatilities of returns, and highly clustered volatilities. Drift and autocorrelation of the underlying asset seem to have the largest impact, though.”

One of my initial ideas for extending the research was to incorporate a much larger set of indexes over a longer period of time.  As I was working on 48 Industries Since 1963, I decided 50 years of data on 48 different indexes would be a great dataset to apply the ideas and methods presented in the paper.

Using R and all its wonderful packages, it is surprisingly easy to accomplish.  Let’s see if we can test “drift and autocorrelation…have the largest impact” on excess returns with industries also.  I’ll try not to get too statistical.

In terms of drift or annualized return, we can see a linear inverse relationship between return and out(under)performance of the 200 day moving average system, so the better the performance of the industry, the less likely a moving average system is to outperform.

From TimelyPortfolio

In terms of the GARCH model effects on excess returns, a parallel coordinate chart will best start our exploration.  Lines are colored by the excess return of a moving average system on each industry.

From TimelyPortfolio

Mu and alpha1 seem to most heavily influence the ability of a moving average system to outperform.  Let’s isolate our chart to mu and alpha1 and add ar1 based on the authors’ findings.  I am very tempted to try to explain GARCH here, but for the sake of brevity, I’ll refrain.  This paper and this Portfolio Probe post offer a good introduction to GARCH.

From TimelyPortfolio

Now let’s look at a scatterplot of excess return versus each of the GARCH stats, starting with those with the most influence.

From TimelyPortfolio

From TimelyPortfolio

From TimelyPortfolio

From TimelyPortfolio

From TimelyPortfolio

From TimelyPortfolio

In conclusion, it seems the same effects observed by the authors also apply to US industry indexes.  In future posts, I’ll add a little more statistical rigor to the analysis and apply to other indexes.

Now, I just cannot resist using a horizon plot to evaluate the rolling 250 day excess returns of a moving average system over buy and hold.  As you can see, a bull market favors buy and hold.  The 70s and 2008-2009 were very kind to a moving average approach.

From TimelyPortfolio

R code in GIST (do raw for copy/paste):

48 Industries Since 1963

Please see http://timelyportfolio.blogspot.com/search/label/horizonplot for all horizon plot posts.

Once more thanks to Ken French for his data, we can accomplish something I think is fairly amazing.  In 640x800, we can see 250 day rollling returns for 48 U.S. industries since 1963.

From TimelyPortfolio

R code in GIST (do raw for copy/paste):

Horizon Plots in Base Graphics

for background please see prior posts More on Horizon Charts, Application of Horizon Plots, Horizon Plot Already Available, and Cubism Horizon Charts in R

There are three primary graphics routes in R (base graphics, lattice, and ggplot2), and each have their zealots.  Last time in More on Horizon Charts, I used lattice and latticeExtra. This time we will build horizon plots in base graphics, and I was pleased with the result. Unfortunately, there is one small issue in that the points of change from positive to negative overlap.  Please let me know if you have a solution. Thanks to helpful readers for their comments and code changes, this is now fixed.

From TimelyPortfolio

From TimelyPortfolio

From TimelyPortfolio

From TimelyPortfolio

From TimelyPortfolio

From TimelyPortfolio

From TimelyPortfolio

Now, let’s implement a for loop and mirror the negative values.

From TimelyPortfolio

With horizon chart functionality in base graphics, hopefully we can add this type now to other packages.  Here is a potential example using quantmod.

From TimelyPortfolio

 

R code in GIST (do raw for copy/paste)

More on Horizon Charts

for background please see prior posts Application of Horizon Plots, Horizon Plot Already Available, and Cubism Horizon Charts in R

Some feedback has led me to think that I might have been a little ambitious with my last post on horizon charts. I thought it might be helpful to quickly provide an overview on horizon charts. Since I am not a visualization expert, I will try to compile the best examples and tutorials that I have found. At the end, I will do a step by step walkthrough of the construction of a horizon chart in R.

Below is a great talk in its entirety by Mike Bostock.  For the discussion on horizon charts, skip to the 11:20 mark.

Mike Bostock @ Square talks about Time Series Visualization from Librato on Vimeo.

Fortunately, Mike also provides the interactive example from the video demonstrating the construction of a horizon chart from an area chart.  I have embedded it below (see https://bl.ocks.org/1483226 for the full example from the original source).

This quick YouTube clip from Panopticon all the way back in 2009 also does a very nice job explaining construction.  If you learn better from reading, a short paper from Hannes Reijner of Panopticon Software covers the same material.

Jeffrey Heer of Stanford University with Nicholas Kong and Maneesh Agrawala from University of California, Berkeley, investigate the effectiveness of horizon plots and make some recommendations for their use.  The chart below also offers another explanation of the steps in building a horizon plot.

They recommend and conclude

Layered Bands Are Beneficial As Chart Size Decreases
We found that dividing a chart into layered bands reliably increased estimation time and increased estimation error at constant chart heights. However, we also found that 2-band mirrored charts led to better estimation accuracies for chart heights less than 24 pixels (6.8 mm on our displays). For larger chart sizes, we advise scaling 1-band mirrored charts.  For smaller sizes, we advise adding layered bands.

Extending the horizon into healthcare, here is an interesting project using horizon graphs for visualization of diabetes care.

HorizonVis

Interactive Visual Exploration of Multivariate Medical Measurements in Diabetes Care

Lead / Contact
Wolfgang Aigner

Team
Wolfgang Aigner, Vienna University of Technology
Michael Atanasov, HTBL Krems
Alexander Rind, Vienna University of Technology
Philipp Schindler, HTBL Krems
Reinhardt Wenzina, HTBL Krems

Partners
Vienna University of Technology, Institute of Software Technology & Interactive Systems
HTBL Krems, Department of Information Technology

 

Now For My Own Attempt at Explaining

If we use yesterday’s example of a 200 day moving average system where you enter when above the moving average and exit when below, we might like to see a standard time series plot like this one.

From TimelyPortfolio

In a simple world with only one asset or stock, this might be sufficient.  However, we probably will have multiple instruments that we would like to monitor, and dedicating this much height per instrument will require lots of space.  Ideally, we could condense each of these plots, so that we could see many at a time.

In the first step toward condensing, we could extract just the information that is most meaningful, which I consider to be the percent above or below the moving average.

From TimelyPortfolio

We might then try an area chart to better depict above or below 0.

From TimelyPortfolio

I’m sure you are wondering though when we will start reducing height and saving space.  We could start by changing all the negative values to positive values, and add color to represent positive or negative.  This means we cut our chart height by about 1/2.

From TimelyPortfolio

However, we need much more efficiency, so let’s separate the chart into bands.

From TimelyPortfolio

Is there any potential way of separating each of these bands and then recombining them to save space?  Let’s look at each band separately.

Band 1 (0 to 10%)

From TimelyPortfolio

Band 2 (10% to 20%)

From TimelyPortfolio

Band 3 (20% to 30%)

From TimelyPortfolio

Band 4 (30% to 40%) exceeding 3 bands is not recommended

From TimelyPortfolio

Notice that similar to a heat map, the band colors increase in intensity and opaqueness as their values increase. If we layer each band on top of each other, we get a horizon plot, and we can reduce the original chart by 1/6 or even more without significant loss of information.

From TimelyPortfolio

I hope this helps explain why we might use horizon plots and how to make them.  If nothing else, maybe you will have learned some lattice techniques in R.

 

R code in GIST (do raw for copy/paste):