Wednesday, July 31, 2013

Slidify Did That… and That… and…

In my exuberance for rCharts, I have not expressed my equal love for its older sibling slidify.  I adopted slidify a little more slowly than other R bloggers

because I failed to realize that slidify does way more than just slideshows.  It acts as an almost universal presentation layer for R.   I am guessing that you have already seen some presentations created by slidify.  To make sure that slidify gets the credit in my work, I wanted to provide a list of examples that slidify created straight from R.  In these slidify gives me the power of Bootstrap, Minimal, and Bootplus.

All of these are github repos, so you can see the source and reproduce.

Slidify is easy and intuitive, and it makes HTML/CSS/javascript all accessible to the average R user.  It has become part of my workflow, and I now start with .Rmd more than than I start with .R.

Friday, July 26, 2013

ggplot2 with Noam Ross theme

When I first saw Noam Ross' blog post "The null model for age effects with overdispersed infection", I immediately liked the look of his ggplot2 graphs. I was even more delighted when I discovered that he has made his theme available on github. Even though I am all into rCharts, I still love a beautiful publication quality R graphic.

Below is some code to combine Noam Ross' theme with autoplot.zoo which makes plotting xts/zoo objects with ggplot2 easy.


# get closing values for S&P 500 from Yahoo! Finance
sp500 <- getSymbols("^GSPC", auto.assign = FALSE)[, 4]

sp500$ma <- runMean(sp500, n = 200)

colnames(sp500) <- c("S&P500", "200d Mov Avg")

# get noam ross ggplot2 theme from github
# for some reason on my computer panel.background still shows up gray this
# fixes it but might not be necessary for others
theme_nr <- theme_nr() + theme(panel.background = element_rect(fill = "white")) +
theme(plot.title = element_text(size = rel(2), hjust = -0.05, vjust = -0.3)) +
theme(plot.margin = unit(c(1, 1, 2, 1), "cm"))

autoplot(sp500, facets = NULL) + theme_nr + theme(legend.position = "none") +
scale_colour_manual(values = brewer.pal("Blues", n = 9)[c(6, 3)]) + geom_line(size = 1.15) +
xlab(NULL) + ggtitle("S&P 500 (ggplot2 theme by Noam Ross)")

It isn’t perfect, but I think it offers a very nice starting point.  Using ggplot2 directly would have allowed us more control over the bothersome details.


Wednesday, July 17, 2013

All My Roads Lead Back to Finance–PIMCO Sankey

Even though the route might be circuitous, my seemingly random journeys all seem to lead back to finance.  My fun with rCharts sankey diagrams (Exploring Networks with Sankey) has led me into an exploration of the PIMCO network.  Although PIMCO is best known for its fixed income products, PIMCO has broadened its product offerings beyond bonds.  This shift away from bonds goes back much farther than the highly publicized moves in 2009-2010 ( “Interview: Bill Gross, PIMCO”) to the 2002 launches of the PIMCO CommodityRealReturn Strategy (“Pimco Dreams Up Funds to Overcome The Nightmare of Inflation”) and PIMCO All Asset Fund ( “PIMCO, Arnott team to offer an 'unusual' TAA mutual fund”, “Investing Without Blinders” and “PIMCO All Asset Shies From Stocks”).
The PIMCO All Asset funds are managed by Research Affiliates Robert Arnott and invest only in other PIMCO mutual funds rather than individual securities, so in many ways the fund’s success is dependent on the quality and breadth of other PIMCO funds.  Since it sounds slightly incestuous, I thought it would be a good subject for my new tools—sankey diagrams and basic network analysis.

Excel to Edgelist with Old-school VBA

Since PIMCO (and nobody else for that matter) doesn’t provide holdings in an igraph readable edgelist, I decided to revive some fond memories and use VBA to convert the spreadsheet into a 3 column edgelist (source, target, value).  Even though I am embarrassed by my VBA code, I will show it below.

Sub getedgelist()
    Dim clInfo As Range, clWrite As Range, tempcl As Range
    Dim offsetamt As Integer
    'this will be used throughout to indicate columns to offset for desired date
    'watch out for hidden rows
    offsetamt = 15
    'clInfo will be the cell with our reference information
    Set clInfo = ThisWorkbook.ActiveSheet.Range("a9")
    'clWrite will be the cell where we write the info that we gather
    Set clWrite = ThisWorkbook.ActiveSheet.Range("u9")
    Do Until clInfo = "Total Gross Asset Allocation"
        'looking at formatting is probably the easiest way to determine if group or fund
        'I chose indent to determine if group or fund
        'IndentLevel = 0 for groups and 1 for funds
        If clInfo.IndentLevel = 0 Then
            'if we are on a group write in clWrite the group and each fund within the group
            'with group as source and fund as target
            'we will loop until we are at the next group
            Set tempcl = clInfo
            'write mutual fund as source, group as target, and weight
            clWrite.Value = ThisWorkbook.ActiveSheet.Range("e6").Value
            clWrite.Offset(0, 1).Value = clInfo.Value
            clWrite.Offset(0, 2).Value = clInfo.Offset(0, offsetamt).Value
            Set clWrite = clWrite.Offset(1, 0)
            Do Until tempcl.Offset(1, 0).IndentLevel = 0
                'now loop through each fund in group
                'write group as source, held fund as target, and weight
                clWrite.Value = clInfo.Value
                clWrite.Offset(0, 1).Value = tempcl.Offset(1, 0).Value
                clWrite.Offset(0, 2).Value = tempcl.Offset(1, offsetamt).Value
                'next cell down
                Set tempcl = tempcl.Offset(1, 0)
                Set clWrite = clWrite.Offset(1, 0)
            Set clInfo = clInfo.Offset(1, 0)
        End If
        'if we are on a group write in clWrite the group and each fund within the group
        'with group as source and fund as target
        Debug.Print (clInfo)
        Set clInfo = clInfo.Offset(1, 0)
End Sub

Now we have everything we need to do the sankey diagram in R.

#sankey of PIMCO All Asset All Authority holdings
#data source


#originally read the data from clipboard of Excel copy
#for those interested here is how to do it
#read.delim(file = "clipboard")

holdings = read.delim("", skip = 3, header = FALSE, stringsAsFactors = FALSE)
colnames(holdings) <- c("source","target","value")

#get rid of holdings with 0 weight or since copy/paste from Excel -
holdings <- holdings[-which(holdings$value == "-"),]
holdings$value <- as.numeric(holdings$value)

#now we finally have the data in the form we need
sankeyPlot <- rCharts$new()

  data = holdings,
  nodeWidth = 15,
  nodePadding = 10,
  layout = 32,
  width = 750,
  height = 500,
  labelFormat = ".1%"


Blogger makes it hard to incorporate d3 directly into this post, so click here or on the screenshot below, to engage and interact with the sankey.


Just for good measure, here is the default plot from igraph.


Of course we could also plot the data a little more traditionally with a ggplot2 bar chart.


Any way we look at it, we can see that PIMCO now has way more than just bonds and the fund All Asset All Authority uses almost everything.

Tuesday, July 16, 2013

Exploring Networks with Sankey

Motivated by a tweet from Tony Hirst (, I started experimenting with an rCharts implementation of the d3 sankey plugin.  While I was putting together examples, I found lots of gaps in my knowledge of sankeys and network analysis.  I did not want to let good learning go to waste, so I put together this quick tutorial documenting what I learned.  Click here or on the screenshot below to see it.

For those wondering how this might be useful in finance, sankeys could be employed well in Portfolio Appraisal/Holdings type reports where nodes represent household, client accounts, asset classes, and security types.  Edge weight would be $ or % in each.


Wednesday, July 10, 2013

rCharts version of d3 horizon

I love horizon plots.  My love shows up throughout my blog, and I have plotted horizon charts in base graphics, xtsExtra, lattice, and ggplot2.  Now with rCharts, we can implement Jason Davies d3.js horizon chart plugin to plot R data in html/javascript.  I put together a tutorial going into great detail on rCharts and convertin custom charts.  For the tutorial, go here, or click on the screenshot below.