---
title: "How to summarize data"
subtitle: "and other operations on data tables"
toc-title: "group_by()"
toc: true
theme: flatly
format:
  html: 
    code-overflow: wrap
    toc: true
    toc-title: "Table of Contents"
    code-fold: false
include-in-header:
  - text: |
      <style>
      .cell-output-stdout code {
        word-break: break-wor !important;
        white-space: pre-wrap !important;
      }
      </style>
execute:
  warning: false
  message: false
---

Creating summaries of data is a very common practice in data science.  One way to get a summary is to use the base R `summary()`

```{r}
mtcars |> summary()
```
A more useful tool is in the `tidyverse`.  We begin with 

### group_by()


The main way to summarize data begins with [group_by()](https://r4ds.hadley.nz/data-transform#groups).

It shuffles a dataset into groups, so that subsequent analysis is done "by group".

```{r}
# important: group_by does not change how the data looks
library(tidyverse)
library(nycflights13)
flights |> 
  group_by(year,month,day) |> 
  summarize(mx = max(arr_delay,na.rm=TRUE)) |> 
  arrange((mx)) |> 
  print(n=30) 
```

`group_by()` does not remove or add data, it just adds a layer of grouping that is used by subsequent functions

```{r}
# important: group_by does not change how the data looks
diamonds |> group_by(cut)
```

#### group_by is often used in tandem with summarize


[summarize()](https://r4ds.hadley.nz/data-transform#sec-summarize)  transforms your data table into one or more rows, where each row constitutes a summary of the corresponding data.

1. Find the max of a single variable.
```{r}
library(tidyverse)
library(palmerpenguins)
penguins |> summarize(mx_bd = max(bill_depth_mm)) 
```

2. Find the max of single variable *within each group*.
```{r}
penguins |> group_by(species) |>
  summarize(mx_bd = max(bill_depth_mm, na.rm = TRUE))
```
3. Improve your summary with `n = n()`.  Every summary should include this.
```{r}
penguins |> group_by(species) |>
  summarize(mx_bd = max(bill_depth_mm, na.rm = TRUE), n = n())
```

4.  Exercise:  What's wrong with this code?
```{r}
penguins |> group_by(species) |>
  summarize(mx_bd = max(bill_depth_mm, na.rm = TRUE, n = n()))
```

5. Find the number of observations (rows) within each group.
```{r}
penguins |> group_by(species) |> summarize(n = n())
```

##  grouping & cut on the diamonds data

1. Explore the diamonds dataset by summarizing additional values.
2. Arrange the output by cut.
3. Interpret the output.  Does anything surprise you?

```{r}
  diamonds |> group_by(carat) |>
    summarize(
      mean_price = mean(price),
    )
```

### Grouping numerical values & summarizing

1. Create a new variable in the `diamonds` data using `cut()` that
gives the relative size, eg., "small", "medium", etc.  The `quantile()` function may be helpful:

```{r}
quantile(diamonds$carat)
```

2. Group the data on this newly created variable and create a summary of the price (mean, max, min, etc.) for each group. 


```{r, echo=FALSE, eval=FALSE}
# diamonds |>   
#   mutate(
#     size = cut(carat,
#              breaks = quantile(diamonds$carat),
#              labels = c("tiny","small","medium",'large')),
#     .before = 1) |>
#   group_by(size) |>
#     summarize(mean_price = mean(price))
```

## `across()`

Apply a function to several variables in a data frame.

```{r}
# change character variables to factors
mpg |> mutate(across(where(is.character),
                     as.factor))
```

You can also use `across` inside of summarise.

```{r}
# compute the grouped-mean of all numeric variables
diamonds |> group_by(cut) |>
  summarise(across(where(is.numeric),
            mean))
```


```{r}
# compute the mean of several variable
mpg |> group_by(manufacturer) |>
  summarize(
    across(where(is.numeric) & contains("y") & !contains("year"),
           mean))
```

Here's another use of across

```{r}
mtcars |> mutate(across(c(1,5), round))
```

## More operations on tables

### The [slice_](https://r4ds.hadley.nz/data-transform#the-slice_-functions) functions

1. `df |> slice_head(n = 1)` takes the first row from each group.
2. `df |> slice_tail(n = 1)` takes the last row in each group.
3. `df |> slice_min(x, n = 1)` takes the row with the smallest value of column x.
4. `df |> slice_max(x, n = 1)` takes the row with the largest value of column x.
5. `df |> slice_sample(n = 1)` takes one random row.

### Ungrouping & [`.by()`](https://r4ds.hadley.nz/data-transform#by)

To remove grouping use `ungroup()`, and to do "in-line" grouping on a per-operation basis you can use `.by()`

```{r}
penguins |> summarize(
  mx_bd = max(bill_depth_mm), 
  n = n(), 
  .by = species)
```


### [bind_rows](https://dplyr.tidyverse.org/reference/bind_rows.html)

This has nothing to do with grouping, but it is an operation on a table.  

Glue two tables together using `bind_rows`.  Experiment to learn how different variables are handled.