#Rstats mystery
library(tidyverse)
b <- tibble(pre=c(8,8,5,3), post=c(8, 8, 5, NA))
# This works
b %>% mutate(post = if_else(is.na(post), mean(post, na.rm=TRUE), post))
# But what if that expression is too hairy for the student? In other languages,
# you can separate out and name the parts. You can do that in R by using the
# completely different $ syntax:
mean_of_others <- mean(b$post, na.rm=TRUE)
revised <- if_else(is.na(b$post), mean_of_others, b$post)
b %>% mutate(post = revised)
# Why doesn't this work? Do mean and if_else not work with the pipe unless they
# are inside something like mutate?
mean_of_others <- b %>% mean(post, na.rm=TRUE) # Error!
revised <- b %>% if_else(is.na(post), mean_of_others, post) # Error!
b %>% mutate(post = revised)
@peterdrake Hey, the `b %>% mutate(post = revised)` part does work, at least in R 4.1.3.
The reason for why the other examples do not though is because of how the pipe works (`%>%`, or `|>`). Essentially what it does is that it takes whatever is evaluated on the left and inserts it as the first argument to the function on the right. So `a %>% b(c)` is equivalent to `b(a, c)`.
In your example the code simplifies to: `revised <- if_else(b, is.na(post), mean_of_others, post)`.
b %>% mutate(post=revised) works in the second section, but the next two lines after the "Why doesn't this work?" comment give errors.
I understand that in
b %>% mutate(m=mean(post, na.rm=TRUE))
mutate is taking an implicit first argument b, so this is equivalent to:
mutate(b, m=mean(post, na.rm=TRUE))
What confuses me here is mean. Is it taking an implicit first argument b?
If so, why doesn't
mean(b, post, na.rn=TRUE)
work?
If not, how does mean know where to find post?
It appears that mean is not a normal function, but some special thing that only works in the context of something like mutate.
I still don't understand how, in
b %>% mutate(m=mean(post, na.rm=TRUE))
mean figures out that "post" is really "b$post". It's as if mutate has made a substitution here.
@peterdrake Yep, that's pretty much what's happening, mutate uses the data argument to let you specify the names of b inside of mean.
I think the term the tidyverse team uses is tidy evaluation.
@matt1126 Okay, I can wrap my head around that.
If I use a name that's NOT a column name in b, mutate DOESN'T make this substitution:
x <- c(1, 2, 3, 4)
b %>% mutate(m=mean(x, na.rm=TRUE))
This context sensitivity still feels wrong, but at least now I have a workable mental model of what's going on. Thanks!
@peterdrake yeah, I think it uses the same rules of namespaces as R does, so if you have a column with that name in the tibble then it will use that first and only if it doesn't exist in that 'namespace' then it will look in the global environment. Which is where your object `x` is.
@matt1126 ... but I can't just refer to post at the top level, while I can refer to x. "post" looks like a variable name, but it's only available inside the argument list of certain functions.
Maybe I can think of the pipe as doing TWO things:
- feeding an implicit first argument into the next function, and
- locally defining all columns of the tibble as variables
@peterdrake I think the second part is more pertinent to attribute to the function you pipe into, rather than the pipe itself, otherwise b %>% mean(post) would have worked.
So mutate makes the namespace of b available to other functions you call within mutate.
I thought the reason b %>% mean(post) didn't work is that mean doesn't expect a tibble as its first argument, but the pipe gives it one.
@peterdrake yeah, exactly, so if mean had the ability to make the data names available to itself (like mutate does) then it would have worked. But since it just works with vectors directly it doesn't know what to do with a tibble.
Mutate takes that tibble and makes the content available in other functions called from itself but that is by design in the mutate function itself.
Hopefully that makes sense, though I'm starting to confuse myself a bit at this rate 😅
@matt1126 Ah, good point! It is mutate, not the pipe, that is doing this. That's why this works:
mutate(b, m=mean(post, na.rm=TRUE))
@peterdrake
> What confuses me here is mean. Is it taking an implicit first argument b?
`mean` does not take a first argument for the data, that is more of a tidyverse convention and `mean` is part of base-R (which I will grant you can be inconsistent at times), but mean isn't a special function.
> If not, how does mean know where to find post?
`mean` doesn't care where the numeric vector is, only that it's being given it. Typically through direct reference, so `mean(b$post)`