---
title: "How to summarize data"
subtitle: "summarize(), group_by(), slice_(), across()"
toc-title: "Table of Contents"
toc: true
theme: flatly
format:
  html: 
    code-overflow: wrap
    toc: true
    toc-title: "Table of Contents"
    code-fold: false
include-in-header:
  - text: |
      <style>
      .cell-output-stdout code {
        word-break: break-wor !important;
        white-space: pre-wrap !important;
      }
      </style>
execute:
  warning: false
  message: false
---

Creating summaries of data is a very common practice in data science.  One way to get a summary is to use the base R `summary()`

```{r}
mtcars |> summary()
```
A more useful tool is in the `tidyverse`.  We begin with 

## group_by()


The main way to summarize data begins with [group_by()](https://r4ds.hadley.nz/data-transform#groups).

It shuffles a dataset into groups, so that subsequent analysis is done "by group".

```{r}
# important: group_by does not change how the data looks
library(tidyverse)
library(nycflights13)
flights |> 
  group_by(year,month,day) |> 
  summarize(mx = max(arr_delay,na.rm=TRUE)) |> 
  arrange((mx)) |> 
  print(n=30) 
```

`group_by()` does not remove or add data, it just adds a layer of grouping that is used by subsequent functions

```{r}
# important: group_by does not change how the data looks
diamonds |> group_by(cut)
```
### Ungrouping & [`.by()`](https://r4ds.hadley.nz/data-transform#by)

Occasionally you may want to remove grouping.  Use `ungroup()` to do this.  You may wish to ungroup because you want to do a summary of the entire dataset, not by group.  You may also wish to ungroup because you want to do a summary of a different grouping.

To remove grouping use `ungroup()`, and to do "in-line" grouping on a per-operation basis you can use `.by()`

```{r}
library(palmerpenguins)
penguins |> summarize(
  mx_bd = max(bill_depth_mm), 
  n = n(), 
  .by = species)
```




## using `summarize()` with `group_by()`

group_by is often used in tandem with summarize


[summarize()](https://r4ds.hadley.nz/data-transform#sec-summarize)  transforms your data table into one or more rows, where each row constitutes a summary of the corresponding data.

### Find the max of a single variable.
```{r}
library(tidyverse)
library(palmerpenguins)
penguins |> summarize(mx_bd = max(bill_depth_mm)) 
```

We notice ``NA` values in the `bill_depth_mm` variable.  We can remove them with `na.rm = TRUE`.

### Find the max of single variable *within each group*.
```{r}
penguins |> group_by(species) |>
  summarize(mx_bd = max(bill_depth_mm, na.rm = TRUE))
```

### Improve your summary with `n = n()`.  Every summary should include this.
```{r}
penguins |> group_by(species) |>
  summarize(mx_bd = max(bill_depth_mm, na.rm = TRUE), n = n())
```

###  Exercise:  What's wrong with this code?
```{r}
penguins |> group_by(species) |>
  summarize(mx_bd = max(bill_depth_mm, na.rm = TRUE, n = n()))
```

###  Find the number of observations (rows) within each group.
```{r}
penguins |> group_by(species) |> summarize(n = n())
```

##  cut()

Here we show how to use `cut()` to categorize continuous data into groups.  This is often useful for summarizing data.

1. Explore the diamonds dataset by summarizing additional values.
2. Arrange the output by cut.
3. Interpret the output.  Does anything surprise you?

```{r}
  diamonds |> group_by(carat) |>
    summarize(
      mean_price = mean(price),
    )
```

### Grouping numerical values & summarizing

#### Create a new variable in the `diamonds` data using `cut()` that
gives the relative size, eg., "small", "medium", etc.  The `quantile()` function may be helpful:

```{r}
quantile(diamonds$carat)
```

#### Group the data on this newly created variable and create a summary of the price (mean, max, min, etc.) for each group. 


```{r, echo=FALSE, eval=FALSE}
# diamonds |>   
#   mutate(
#     size = cut(carat,
#              breaks = quantile(diamonds$carat),
#              labels = c("tiny","small","medium",'large')),
#     .before = 1) |>
#   group_by(size) |>
#     summarize(mean_price = mean(price))
```

## `across()`

Apply a function to several variables in a data frame.

```{r}
# change character variables to factors
mpg |> mutate(across(where(is.character),
                     as.factor)) |> glimpse()
```

You can also use `across` inside of summarise.

```{r}
# compute the grouped-mean of all numeric variables
diamonds |> group_by(cut) |>
  summarise(across(where(is.numeric),
            mean))
```


```{r}
# compute the mean of several variable
mpg |> group_by(manufacturer) |>
  summarize(
    across(where(is.numeric) & contains("y") & !contains("year"),
           mean))
```

Here's another use of across

```{r}
mtcars |> mutate(across(c(1,5), round)) |> head()
```

## `slice` 

The `slice()` functions are used to select rows from a data frame.  They are often used in conjunction with `group_by()` and `summarize()`.  The slice functions are shown below.


The [slice_](https://r4ds.hadley.nz/data-transform#the-slice_-functions) functions

1. `df |> slice_head(n = 1)` takes the first row from each group.
2. `df |> slice_tail(n = 1)` takes the last row in each group.
3. `df |> slice_min(x, n = 1)` takes the row with the smallest value of column x.
4. `df |> slice_max(x, n = 1)` takes the row with the largest value of column x.
5. `df |> slice_sample(n = 1)` takes one random row.


### [bind_rows](https://dplyr.tidyverse.org/reference/bind_rows.html)

This has nothing to do with grouping, but it is an operation on a table.  

Glue two tables together using `bind_rows`.  Experiment to learn how different variables are handled.