---
title: "regex"
subtitle: "Ch. 15"
format: 
  html:
    code-fold: TRUE
toc: TRUE
theme: flatly
warning: FALSE
message: FALSE
---

### Regular Expressions

**key concepts: ** (escaping, anchoring, character classes, shorthand classes, quantifiers, precedence, and grouping)

### Data

```{r, echo = FALSE}
library(tidyverse)
library(babynames)
```

1. `fruit` contains the names of 80 fruits.
2. `words` contains 980 common English words.
3. `sentences` contains 720 short sentences.


### str_view()

When a second parameter is supplied, str_view() will show only the elements of the string vector that match, surrounding each match with <>, and, where possible, highlighting the match in blue.

```{r, eval=FALSE}
# syntax
str_view(data, "string")
```

### Basic Rules [link](https://r4ds.hadley.nz/regexps#sec-reg-basics)

1. **letters and numbers** match exactly and are called **literal** characters. 
2. *., +, *, *[, ]*, and *?* have special meanings and are called **metacharacters**.
3. Special Meanings
  i) `.` matches any character so `j.` will match any string that contains `j` followed by another character. 
  ii) `?` makes a pattern optional (i.e. it matches 0 or 1 times)
  iii) `+` lets a pattern repeat (i.e. it matches at least once)
  iv) `*` lets a pattern be optional or repeat (i.e. it matches any number of times, including 0).
  v) `^` signals the beginning of a the pattern Note: no surrounding  `[ ]` 
  vi) `$` signals the end
  vii) *Character classes* are defined by `[ ]` and let you match a set of characters, e.g., `[abcd]` matches “a”, “b”, “c”, or “d”. 
  viii) `[^abcd]` matches anything except “a”, “b”, “c”, or “d”. 
  ix) *alternation* `|` acts as "or"
  
### Examples

```{r}
# search for a SINGLE DIGIT number
str_detect(c("a","a1","a123","a1234"),"[0-9]")
str_view(c("a","a1","a123","a1234"),"[0-9]")
str_replace(c("a","a1","a123","a1234"),"[0-9]","x")
```
```{r}
# search for any n-DIGIT number
str_detect(c("a","a1","a123","a1234"),"[0-9]+")
str_view(c("a","a1","a123","a1234"),"[0-9]+")
str_replace(c("a","a1","a123","a1234"),"[0-9]+","x")
```
Now for a TWO digit number
```{r}
# search for any n-DIGIT number
str_detect(c("a","a1","a123","a1234"),"[0-9]{2}")
str_view(c("a","a1","a123","a1234"),"[0-9]{2}")
str_replace(c("a","a1","a123","a1234"),"[0-9]{2}","x")
```

Now match some date formatting.  Work with the `flights` data.

```{r}
library(nycflights13) 
# grab the unique values
df <- flights |> distinct(year,month,day)

# following the principle that one should avoid typing the same thing 3 times we loop:
for (cn in names(df)){
    print(cn)
    df <- mutate(df,{{cn}} := as.character(get(cn))) |> glimpse()
}

# now we join into a date format
  df <- df |> mutate(thedate = str_c(day,"_",month,"_",year))
  
# but that doesn't look very good, let's make everything 2 or 4 digits
# below:
# ^ look at the start of the word
# [0-9] look for any digit
# _ then look for an _
# . this is a metacharacter for any-single-character
# + this repeats the previous token as many times till the end of the string
  df <- df |> mutate(nicedate = str_replace(
    thedate,"^[0-9]_.+",str_c("0",thedate)))

# this works for some of them but not all:
  df |> arrange(df,nicedate)
  
```

#### Exercises

1. Fix the rest of the dates in nicedate
```{r echo=FALSE}
# the problem ones
str_view(df$nicedate,"\\d{2}_\\d_")

# work on the bad dates only
  badix <- str_detect(df$nicedate,"_\\d_")
  
  bdf <- filter(df,badix)
  bdf <- separate_wider_delim(bdf,nicedate,
                       delim = "_", 
                       names = c("newday","newmonth","newyear"))
  
  bdf <- bdf |> mutate(
    newmonth = str_c("0",newmonth),
    nicedate = str_c(newday,newmonth,newyear,sep="_"))
  
  # join back
  df <- left_join(bdf,df)
#
```

2. Replace the date that matches your birthday with the string: `It's my birthday!`.


### Detect Matches
`str_detect()` returns a logical vector that is TRUE if the pattern matches an element of the character vector and FALSE otherwise:

```{r}
str_detect(c("a", "b", "c"), "[aeiou]")
```

Since `str_detect()` returns a logical vector of the same length as the initial vector, it pairs well with `filter()`. For example, this code finds all the most popular names containing a lower-case “x”:

```{r}
babynames |> 
  filter(str_detect(name, "x")) |> 
  count(name, wt = n, sort = TRUE)
```

We can also use str_detect() with summarize() by pairing it with sum() or mean():

```{r}
babynames |> 
  group_by(year) |> 
  summarize(prop_x = mean(str_detect(name, "x"))) |> 
  ggplot(aes(x = year, y = prop_x)) + 
  geom_line()
```

There are two functions that are closely related to `str_detect()`: `str_subset()` and `str_which()`. `str_subset()` returns a character vector containing only the strings that match. `str_which()` returns an integer vector giving the positions of the strings that match.

### Show all strings that do not contain an e

The following [stack exchange](https://stackoverflow.com/questions/406230/regular-expression-to-match-a-line-that-doesnt-contain-a-word) suggests this solution:

```{r}
# find all fruits that don't have an e
str_subset(fruit,"^((?!e).)*$")
```


![advanced regex](advanced_regex.png)


### Count Matches
 
 `str_count()`: rather than a true or false, it tells you how many matches there are in each string.
 
```{r}
x <- c("apple", "banana", "pear")
str_count(x, "p")
```
 
 It’s natural to use str_count() with mutate(). The following example uses str_count() with character classes to count the number of vowels and consonants in each name.

```{r}
babynames |> 
  count(name) |> 
  mutate(
    vowels = str_count(name, "[aeiou]"),
    consonants = str_count(name, "[^aeiou]")
  )
```

**3 fixes to the issue above**

1. Add the upper case vowels to the character class: str_count(name, "[aeiouAEIOU]").

2. Tell the regular expression to ignore case: str_count(name, regex("[aeiou]", ignore_case = TRUE)).

3. Use str_to_lower() to convert the names to lower case: str_count(str_to_lower(name), "[aeiou]").

Choose one and resolove the issue above. 


### Replace Matches
As well as detecting and counting matches, we can also modify them with str_replace() and str_replace_all(). str_replace() replaces the first match, and as the name suggests, str_replace_all() replaces all matches.

```{r}
x <- c("apple", "pear", "banana")
str_replace_all(x, "[aeiou]", "-")
```


### Extract Matches

```{r}
df <- tribble(
  ~str,
  "<Sheryl>-F_34",
  "<Kisha>-F_45", 
  "<Brandon>-N_33",
  "<Sharon>-F_38", 
  "<Penny>-F_58",
  "<Justin>-M_41", 
  "<Patricia>-F_84", 
)
```

To extract this data using separate_wider_regex() we just need to construct a sequence of regular expressions that match each piece. If we want the contents of that piece to appear in the output, we give it a name:

```{r}
df |> 
  separate_wider_regex(
    str,
    patterns = c(
      "<", 
      name = "[A-Za-z]+", 
      ">-", 
      gender = ".",
      "_",
      age = "[0-9]+"
    )
  )
```

### Exercises

Practice [from the text](https://r4ds.hadley.nz/regexps#exercises)


### Grouping and Capturing [link](https://r4ds.hadley.nz/regexps#grouping-and-capturing)

The following pattern finds all fruits that have a repeated pair of letters:

```{r}
str_view(fruit, "(..)\\1")
```


And this one finds all words that start and end with the same pair of letters:

```{r}
str_view(words, "^(..).*\\1$")
```

### Regex Tidbits

1. Raw strings: 

```{r}
# more intuitive way to define a weird string
s <- r"{"'\?}"
```

### More Exercises

[15.4.7](https://r4ds.hadley.nz/regexps#exercises-1)

[Practice](https://r4ds.hadley.nz/regexps#practice)
