```{r setup, include=FALSE} knitr::opts_chunk$set(echo = TRUE) ``` The `dplyr` package acts on data frames, `filter`ing rows and `select`ing columns as one would in a database. The `purrr` package acts on lists, including data frames. The cool thing is that the lists may contain rather complicated objects. There is now a [purrr cheatsheat](https://github.com/rstudio/cheatsheets/raw/master/purrr.pdf) Below is a made-up example to illustrate some features. ```{r message=FALSE} library(purrr) library(dplyr) library(tidyr) ``` Let's create some fake data. Here `x` is random and `y` equals column number + `x` + noise. ```{r} nc <- 5 nr <- 10 ``` ```{r} x <- as.list(seq_len(nc)) ``` ```{r} x <- as.data.frame(matrix(rnorm(nr * nc), nr, nc)) x <- as.data.frame(x) ``` ```{r} y <- t(seq_len(nc) + t(x) * seq_len(nc)) + matrix(rnorm(nr * nc,, 0.001), nr, nc) y <- as.data.frame(y) ``` Now we want to run a regress of each column of `x` on each column of `y` and report out the coefficients. This could be done in a loop, but let's instead imagine what we would do with a single column. ```{r} lm_coef <- function(x1, y1) { dat <- data.frame(x = x1, y = y1) coef(lm(y ~ x, dat)) } ``` ```{r} lm_coef(x[,1], y[,1]) ``` We want to combine `x` and `y` into one list. We first make a list of lists, then transpoe it. ```{r} xy <- list(x = as.list(x), y = as.list(y)) str(xy) ``` ```{r} xy <- xy %>% transpose str(xy) ``` ```{r} xy %>% map(function(dat) coef(lm(y~x, dat))) ``` This can also be done in two steps. At the end, we organize data a bit. ```{r} xy %>% map(function(dat) lm(y~x, dat)) %>% map(coef) %>% as.data.frame %>% t ``` ## Redo with dplyr Of course all the above can be done readily with `dplyr` and `tidyr` using `group_by` and `do`, as shown below. However, two things are useful to consider: * While `summarize` works for single value operations, you must use `do` for multiple value operations. It is challenging to get `do` correct, as it must return a one-row data frame. * The `purrr` verbs `map` and `transpose` do not require the lists to be the same length or configuration. Thus, they can be used in a variety of settings where `dplyr`, working on data frames, cannot. ```{r} xx <- x %>% gather(var, xval) yy <- y %>% gather(var, yval) xx$yval <- yy$yval ``` ```{r} xx %>% group_by(var) %>% do( as.data.frame( t( coef( lm(yval ~ xval, .))))) ``` ## A more complicated example This example can be found in the [hotspot.R](https://github.com/byandell/qtl2shiny/blob/master/R/hotspot.R) file of package [qtl2shiny](https://github.com/byandell/qtl2shiny). We start with two objects -- `chr_pos` has positions within each chromosome, `peaks` has information about what traits have peaks where on each chromosome -- that are lists by chromosome. [The `peaks` object is turned into a list by use of `split`.] We transpose a "list of lists" so that `out_chr` is also a list by chromosome, having elements from the two objects. ```{r eval = FALSE} out_chr <- purrr::transpose(list(pos = chr_pos, peaks = split(peaks, peaks$chr))) ``` Now we create a new object `out_peaks` by counting peaks at each position along each chromosome. The `map` function acts across the list indexed by chromosome. Notice that the call to `peaks_type`, done for each chromosome, uses elements `x$pos` and `x$peaks` from each element of `out_chr`. Recall the `pos` and `peaks` objects have rather different shapes. ```{r eval = FALSE} out_peaks <- purrr::map(out_chr, function(x, peak_window) peaks_type(x$pos, x$peaks, peak_window), peak_window) ``` The routine `peaks_type` first `split`s the `peaks` by `pheno_type`, then `map`s a function `outer_window` (not shown here) for each element of `peaks_by_type`. ```{r eval=FALSE} peaks_by_type <- split(peaks, peaks$pheno_type) out <- purrr::map(peaks_by_type, outer_window, posi, peak_window) ``` Later on, `peaks_type` uses `map` again for another summary. The point here is not to follow the details, but to notice how `map` can be used to isolate thinking about a particular task. Then you can concentrate on solving that task without worrying about a bunch of indexing and looping.