Intro to R Adapted to dplyr
Modern data analysis with R and Rstudio tends to use a growing set of tools developed by Hadley Wickham and the Rstudio crew. This so-called Hadleyverse ⚠️ reimagines many of the basic concepts, nouns and verbs of R. Fundamental to this is dplyr. This companion to the Aggregating and analyzing data with dplyr lesson looks at some of the concepts from the Data and data frames lesson using dplyr.
Learning Objectives
- examine difference between a data frame and a tibble
- use
dplyrpackage to explore the structure and content of a tibble - know how to access any element of a
data.frame
Setup
We are going to use the readr and dplyr packages. If you do not have them, please use install.packages to install.
install.packages(readr,dplyr)
Now attach these packages as libraries for use.
library(readr)
library(dplyr)
Survey Data
We return to the species and weight of animals caught in plots in a study area in Arizona over time. The dataset is stored as a CSV file: each row holds information for a single animal, and the columns represent:
| Column | Description |
|---|---|
| record_id | Unique id for the observation |
| month | month of observation |
| day | day of observation |
| year | year of observation |
| plot_id | ID of a particular plot |
| species_id | 2-letter code |
| sex | sex of animal (“M”, “F”) |
| hindfoot_length | length of the hindfoot in mm |
| weight | weight of the animal in grams |
| genus | genus of animal |
| species | species of animal |
| taxa | e.g. Rodent, Reptile, Bird, Rabbit |
| plot_type | type of plot |
The data are available at http://kbroman.org/datacarp/portal_data_joined.csv.
We can read that data straight from the web. Here we use read_csv from the readr package. It is redesigned from read.csv. Note that read_csv treats blanks “” and “NA” both as missing, for instance.
surveys <- read_csv("http://kbroman.org/datacarp/portal_data_joined.csv")
Data frames and Tibbles
The readr and dplyr packages use a reimaging of a data.frame to a tibble, or augmented table. [There is a tibble package, but we won’t discuss it.] Tibbles have column names but do not have row names; hence rows and columns are treated somewhat differently, more like for tables in an SQL database, for instance.
Tibbles have many slick features, one being that you don’t need to use head to see the beginning of the table. All the inspection tools described before still apply, but sometimes there are nicer ways.
surveys
## # A tibble: 34,786 × 13
## record_id month day year plot_id species_id sex hindfoot_length weight
## <dbl> <dbl> <dbl> <dbl> <dbl> <chr> <chr> <dbl> <dbl>
## 1 1 7 16 1977 2 NL M 32 NA
## 2 72 8 19 1977 2 NL M 31 NA
## 3 224 9 13 1977 2 NL <NA> NA NA
## 4 266 10 16 1977 2 NL <NA> NA NA
## 5 349 11 12 1977 2 NL <NA> NA NA
## 6 363 11 12 1977 2 NL <NA> NA NA
## 7 435 12 10 1977 2 NL <NA> NA NA
## 8 506 1 8 1978 2 NL <NA> NA NA
## 9 588 2 18 1978 2 NL M NA 218
## 10 661 3 11 1978 2 NL <NA> NA NA
## # ℹ 34,776 more rows
## # ℹ 4 more variables: genus <chr>, species <chr>, taxa <chr>, plot_type <chr>
Challenge
Based on the tibble of surveys,
- How many rows and columns are in
surveys? - What are the classes of the columns for
sexandyear?
Indexing, Sequences, and Subsetting
We can pull out parts of a data frame using filter and select. We need to provide two values: filter row and select column.
For example, to get the element in the 1st row, 1st column, we can use the verbs filter and select, along with row_number, to return a 1 by 1 tibble:
surveys %>%
filter(row_number() == 1) %>%
select(1)
## # A tibble: 1 × 1
## record_id
## <dbl>
## 1 1
Notice that we can select a column by its number (or name – see below), but we filter rows by logical expressions, in this case the row number must be 1.
To get the element in the 2nd row, 7th column:
surveys %>%
filter(row_number() ==2) %>%
select(7)
## # A tibble: 1 × 1
## sex
## <chr>
## 1 M
To select the entire 2nd row:
surveys %>%
filter(row_number() ==2)
## # A tibble: 1 × 13
## record_id month day year plot_id species_id sex hindfoot_length weight
## <dbl> <dbl> <dbl> <dbl> <dbl> <chr> <chr> <dbl> <dbl>
## 1 72 8 19 1977 2 NL M 31 NA
## # ℹ 4 more variables: genus <chr>, species <chr>, taxa <chr>, plot_type <chr>
And to filter to the entire 7th column:
surveys %>%
select(7)
## # A tibble: 34,786 × 1
## sex
## <chr>
## 1 M
## 2 M
## 3 <NA>
## 4 <NA>
## 5 <NA>
## 6 <NA>
## 7 <NA>
## 8 <NA>
## 9 M
## 10 <NA>
## # ℹ 34,776 more rows
You can also refer to columns by name, in multiple ways. However, the class of the resulting object may differ
class(surveys %>% select(sex))
## [1] "tbl_df" "tbl" "data.frame"
class(surveys[,"sex"])
## [1] "tbl_df" "tbl" "data.frame"
class(surveys$sex)
## [1] "character"
class(surveys[["sex"]])
## [1] "character"
When we select a single column using dplyr tools, the result is still a tibble (or tbl_df); that is, it is a table with one column. When we select a column as if it were a data.frame, the result is a character vector.
Challenge
How would you get the first element of the sex column as a character if you started with a tibble representation such as surveys %>% select(sex)?
Slices
To get slices of our tibble, we can include a vector for the row or column indexes (or both). Notice the use of %in% to filter row numbers in a range.
surveys %>%
filter(row_number() %in% 1:3) %>%
select(7) # first three elements in the 7th column
## # A tibble: 3 × 1
## sex
## <chr>
## 1 M
## 2 M
## 3 <NA>
surveys %>%
filter(row_number() == 1) %>%
select(1:3) # first three columns in the first row
## # A tibble: 1 × 3
## record_id month day
## <dbl> <dbl> <dbl>
## 1 1 7 16
surveys %>%
filter(row_number() %in% 2:4) %>%
select(6:7) # rows 2-4, columns 6-7
## # A tibble: 3 × 2
## species_id sex
## <chr> <chr>
## 1 NL M
## 2 NL <NA>
## 3 NL <NA>
Note again that usually operations on a tibble result in a tibble, keeping column headers and table structure.
Challenge
Use row_number(), in conjuction with either seq() or the modulo (mod) operator %%, to create a new data.frame called surveys_by_10 that includes every 10th row of the survey data frame starting at row 10 (10, 20, 30, …).
[Hint for mod: what does seq(20) %% 5 yield?]
More tally stuff
The dplyr intro uses tally several times. If you look on the help page for tally you will find count. You will also find reference to n and sum. Here are several ways to do the same task. Why are there several ways? Who knows? But sometimes you want to do other things in tandem, and one of these might work best.
surveys %>%
group_by(sex) %>%
tally
## # A tibble: 3 × 2
## sex n
## <chr> <int>
## 1 F 15690
## 2 M 17348
## 3 <NA> 1748
surveys %>%
count(sex)
## # A tibble: 3 × 2
## sex n
## <chr> <int>
## 1 F 15690
## 2 M 17348
## 3 <NA> 1748
surveys %>%
group_by(sex) %>%
summarize(count = n())
## # A tibble: 3 × 2
## sex count
## <chr> <int>
## 1 F 15690
## 2 M 17348
## 3 <NA> 1748
surveys %>%
group_by(sex) %>%
summarize(count = sum(!is.na(year)))
## # A tibble: 3 × 2
## sex count
## <chr> <int>
## 1 F 15690
## 2 M 17348
## 3 <NA> 1748