Why does tidy evaluation throw an error unless an argument is printed or forced?

Question

Code

## 1. Get Hours into Tibble  
## ---------------------
  hrsThreshold <- function(input_data, group1, group2, group2_String) {

    # print ({{group2}})   # for some ridiculous reason, code doesn't work without this print statement in!
    
  # Hours
    hrs <-
    # thresholds
      input_data %>% 
      mutate( score = rowSums(select(., starts_with("less_than")), na.rm = TRUE)) %>%

    # scores & summary
      group_by({{ group1 }}, {{ group2 }}, score) %>%
      summarise(n_points = n()) %>%
      mutate(percent = n_points / sum(n_points)) %>%
      mutate(score  = as_factor(score)) %>%

    # add percentage labels.
      group_by({{ group1 }}, {{ group2 }}) %>%

      mutate(ymax = cumsum(percent),
             ymin = c(0, head(cumsum(percent), n = -1))) %>%

      mutate(label_pos = (ymax+ymin) / 2,
             label = ifelse(percent < 0.01, "", paste0(round(percent*100, 0), "%"))) %>%

      ungroup()
    
    hrs
    

  # Hours All
    hrs_all <-
      hrs %>%
      drop_na( {{ group2 }} ) %>%
      select(-label) %>%

    # Sum n_points
      group_by({{ group1 }}, score) %>%
      summarise_at(
        vars(n_points),
        .funs = sum) %>%

      mutate(percent = n_points / sum(n_points)) %>%

    # redo geometry of rectangles.
      group_by({{ group1 }}) %>%
      mutate(ymax = cumsum(percent),
             ymin = c(0, head(cumsum(percent), n = -1))) %>%
      mutate(label_pos = (ymax+ymin) / 2,
             label = ifelse(percent < 0.01, "", paste0(round(percent*100, 0), "%"))) %>%

    # Change room_type = All
      mutate({{ group2 }} := "all")


    # Join All with Variable
      hrs_by_room <- full_join(hrs_all, hrs)
      return(hrs_by_room)
  }



  # Debug Code
    analysis_vars <- c("room_type", "tenure")
    analysis_index <- 2
    
  # Dynamic Variable
    mytempVar      <- analysis_vars[analysis_index]

    mytempVarTitle <- mytempVar
    mytempVarTitle <- str_replace(mytempVarTitle, "[_]", " ")
    mytempVarTitle <- str_to_title(mytempVarTitle)
    
    mytempVar_     <- sym(mytempVar)


  # run command
  hrs_by_room   <- hrsThreshold(risk_assigned, hhi, mytempVar_,   c({{mytempVar_}}))

Description of Issue

The code above only works if the line :

print ({{group2}})

is uncommented!!! This command, on the face of it, does NOTHING. But without it on my machine, the following error comes:

Error: Must group by variables found in `.data`.
* Column `mytempVar_` is not found.
Run `rlang::last_error()` to see where the error occurred.
> rlang::last_error()
<error/rlang_error>
Must group by variables found in `.data`.
* Column `mytempVar_` is not found.
Backtrace:
  1. global::hrsThreshold(...)
 11. dplyr:::group_by.data.frame(...)
 12. dplyr::group_by_prepare(.data, ..., .add = .add, caller_env = caller_env())
Run `rlang::last_trace()` to see the full context.
> rlang::last_trace()
<error/rlang_error>
Must group by variables found in `.data`.
* Column `mytempVar_` is not found.
Backtrace:
     x
  1. +-global::hrsThreshold(...)
  2. | \-`%>%`(...)
  3. +-dplyr::ungroup(.)
  4. +-dplyr::mutate(...)
  5. +-dplyr::mutate(...)
  6. +-dplyr::group_by(...)
  7. +-dplyr::mutate(., score = as_factor(score))
  8. +-dplyr::mutate(., percent = n_points/sum(n_points))
  9. +-dplyr::summarise(., n_points = n())
 10. +-dplyr::group_by(...)
 11. \-dplyr:::group_by.data.frame(...)
 12.   \-dplyr::group_by_prepare(.data, ..., .add = .add, caller_env = caller_env())

I've confirmed this is the issue that has been crashing my entire report application for days, in multiple functions. I only finally got it workoing by accidentally putting print statments in to debug the problem. Whereupon it mysteriously started to work. What on earth is going on?

Source File

library(tidyverse)
risk_assigned <- read.csv( text="hhi,device_id,rental_type,room,room_type,ts,temp,less_than_21,less_than_18,less_than_16,less_than_12
MyOrg,2C7970,Council Rental,Lounge,living,1598291349,18.3,1,NA,NA,NA
MyOrg,2C7970,Council Rental,Lounge,living,1598292249,18.3,1,NA,NA,NA
MyOrg,2C7970,Council Rental,Lounge,living,1598293149,18.3,1,NA,NA,NA
MyOrg,2C7970,Council Rental,Lounge,living,1598294049,18.3,1,NA,NA,NA
MyOrg,2C7970,Council Rental,Lounge,living,1598294949,18.2,1,NA,NA,NA
MyOrg,2C7970,Council Rental,Lounge,living,1598353449,18.5,1,NA,NA,NA
MyOrg,2C7970,Council Rental,Lounge,living,1598354354,18.5,1,NA,NA,NA
MyOrg,2C7970,Council Rental,Lounge,living,1598355254,18.5,1,NA,NA,NA
MyOrg,2C7970,Council Rental,Lounge,living,1598356154,18.5,1,NA,NA,NA
MyOrg,2C7970,Council Rental,Lounge,living,1598357054,18.5,1,NA,NA,NA
MyOrg,2C7970,Council Rental,Lounge,living,1598357954,18.5,1,NA,NA,NA
MyOrg,2C7971,Private Rental,Lounge,living,1598358854,18.5,1,NA,NA,NA
MyOrg,2C7971,Private Rental,Lounge,living,1598359754,18.6,1,NA,NA,NA
MyOrg,2C7971,Private Rental,Lounge,living,1598360654,18.5,1,NA,NA,NA
MyOrg,2C7971,Private Rental,Lounge,living,1598361554,18.5,1,NA,NA,NA
MyOrg,2C7971,Private Rental,Lounge,living,1598362454,18.4,1,NA,NA,NA
MyOrg,2C7971,Private Rental,Lounge,living,1598363354,18.4,1,NA,NA,NA
MyOrg,2C7971,Private Rental,Lounge,living,1598364254,18.4,1,NA,NA,NA
MyOrg,2C7971,Private Rental,Lounge,living,1598365154,18.4,1,NA,NA,NA
MyOrg,2C7971,Private Rental,Lounge,living,1598366054,18.3,1,NA,NA,NA
MyOrg,2C7971,Private Rental,Lounge,living,1598366954,18.2,1,NA,NA,NA
MyOrg,2C7971,Private Rental,Lounge,living,1598367850,18.2,1,NA,NA,NA
MyOrg,2C7971,Private Rental,Lounge,living,1598368750,18.1,1,NA,NA,NA
MyOrg,2C7971,Private Rental,Lounge,living,1598369650,18.1,1,NA,NA,NA
MyOrg,2C7971,Private Rental,Lounge,living,1598370550,18,1,NA,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598371450,17.9,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598372350,17.9,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598373250,17.8,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598374150,17.8,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598375050,17.8,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598375950,17.7,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598376850,17.6,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598377750,17.6,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598378650,17.6,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598379553,17.5,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598380453,17.5,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598381353,17.4,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598439851,17.7,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598440751,17.7,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598441651,17.6,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598442551,17.6,1,1,NA,NA
MyOrg,2C7972,Private Rental,Lounge,living,1598443451,17.5,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598444351,17.5,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598445251,17.4,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598446151,17.4,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598447051,17.3,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598447951,17.3,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598448851,17.1,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598449751,17.1,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598450651,17.1,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598451554,17.1,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598452454,17.1,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598453354,17.1,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598454254,17.1,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598455154,17.1,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598456054,17,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598456954,17,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598457854,16.9,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598458754,16.9,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598459654,16.9,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598460554,16.9,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598461454,16.8,1,1,NA,NA
MyOrg,2C7973,Council Rental,Lounge,living,1598462354,16.8,1,1,NA,NA
MyOrg,2C7974,Council Rental,Lounge,living,1598463254,16.7,1,1,NA,NA
MyOrg,2C7974,Council Rental,Lounge,living,1598464154,16.8,1,1,NA,NA
MyOrg,2C7974,Council Rental,Lounge,living,1598465051,16.7,1,1,NA,NA
MyOrg,2C7974,Council Rental,Lounge,living,1598465951,16.8,1,1,NA,NA
MyOrg,2C7974,Council Rental,Lounge,living,1598466851,16.7,1,1,NA,NA")

I could be completely off base but my guess is this is a case of lazy evaluation biting you. The print statement with force evaluation of something that might not have been evaluated otherwise. Once again - I didn't read the full question but this is my guess based on the details I skimmed. — Dason, Dec 31 '21 at 22:49
When you submit this it should go to the Rstudio forum. The code is pretty long and I’d suggest trying to narrow down the portion that is responsible and including a data object for it to chew on. — IRTFM, Jan 01 '22 at 05:37
When I search on mytempvar I get a deleted Q from you on SO. — IRTFM, Jan 01 '22 at 05:39
@IRTFM Yes -deleted question because it got closed by a couple of grumps. — monkey, Jan 01 '22 at 06:42
Do you realize that `mytempVar_` is not the same as `mytempVar`? — IRTFM, Jan 01 '22 at 07:11
@IRTFM Yes! Of course it's not the same. `mytempVar_ <- sym(mytempVar)` — monkey, Jan 02 '22 at 22:35
I think Dason's analysis is very likely correct. When you apply print to an object name it forces evaluation. So I suspect `mytempVar_` was only an unevaluated promise until you executed the print function which forced evaluation of the `group2` formal parameter which in turn forces evaluation of `mytempVar_`. There is a `force` function, although it often fails to achieve the results I expect of it. — IRTFM, Jan 03 '22 at 02:13
Perhaps this answers the question. https://stackoverflow.com/questions/51615206/evaluate-a-value-in-r-given-either-the-name-of-a-promise-or-the-formal-argument Cannot test. No MCVE as yet. Voting to close. — IRTFM, Jan 03 '22 at 02:29
@IRTFM Your above comment in support of Dason is actually helpful. But forgive me if I detect a strong negative undertone in your contribution. The minimal example is right there in front of you. By all means vote to close the question before we figure out what is causing the problem. But this forum is about learning and I believe there is a lesson here. If you disagree, I would urge you to move on to another question where you feel you can add value. — monkey, Jan 03 '22 at 09:20
I've edited it to make it a MCVE. I think you already have an explanation (infelicity in the tidy evaluation mechanism) but you just don't like it. I've also edited the tags to make an effort to attract someone to might understand the logical complexities induced by the byzantine NSE in the alternate un-"tidyverse". (I'd already voted to close it and after putting in the effort to make it reproducible it does appear to finally be such, so I retracted it.) — IRTFM, Jan 03 '22 at 15:42
@IRTFM. Thank you. I upvoted Dason's valuable comment, which I think answers the immediate question. I was just keen that this R trap be documented adequately to help future R new comers. — monkey, Jan 04 '22 at 20:49
It would have been helpful to reduce your post to a minimal example, removing as many lines of code as you can and as much data dependencies (function inputs) as possible. As it stands, your example requires too much analysis from contributors. — Lionel Henry, Jan 05 '22 at 09:42
@Dason's analysis was spot on. I've added an answer that provides a minimal example, an explanation, and two pointers to the tidy eval doc that explains this behaviour and why it can't be fixed. — Lionel Henry, Jan 05 '22 at 10:03

Lionel Henry · Accepted Answer · 2022-01-05T10:29:33.033

Why does this fail?

Here is a minimal example:

var <- rlang::sym("cyl")

fn <- function(var, force) {
  if (force) {
    force(var)
  }

  dplyr::group_by(mtcars, {{ var }})
}

The expected behaviour is:

fn(var, force = FALSE)
#> Error: Must group by variables found in `.data`.
#> * Column `var` is not found.

This is unexpected behaviour:

fn(var, force = TRUE)
#> # A tibble: 32 × 11
#> # Groups:   cyl [3]
#>     mpg   cyl  disp    hp  drat    wt  qsec    vs    am  gear  carb
#>   <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1  21       6   160   110  3.9   2.62  16.5     0     1     4     4
#> 2  21       6   160   110  3.9   2.88  17.0     0     1     4     4
#> 3  22.8     4   108    93  3.85  2.32  18.6     1     1     4     1
#> 4  21.4     6   258   110  3.08  3.22  19.4     1     0     3     1
#> # … with 28 more rows

There are two causes to this unexpected behaviour.

Using {{ out of context is equivalent to forcing an argument. So print({{ var }}) is equivalent to force(var). See https://rlang.r-lib.org/reference/topic-inject-out-of-context.html.
It is not possible to defuse a forced argument. And because the compiler unwraps constants from promises, we can't detect that case to throw an error instead. This is why forced arguments are injected rather than defused. See https://rlang.r-lib.org/reference/topic-embrace-non-args.html.

We can't fix either (1) or (2) because tidy evaluation is not implemented in the core language, it's implemented on top of it.

What should I do?

In your example, it seems that you would like to pass a column name to your function through an argument.

The first thing to note is that since that argument is passed to group_by() with {{, your function inherits all the behaviour of group_by(). It takes unquoted column names and supports all tidy eval features.

When you have a column name in a string, the canonical way to pass it is with the .data pronoun:

var <- "cyl"

fn <- function(var) {
  dplyr::group_by(mtcars, {{ var }})
}

fn(var)
#> Error: Must group by variables found in `.data`.
#> * Column `var` is not found.

fn(.data[[var]])
#> # A tibble: 32 × 11
#> # Groups:   cyl [3]
#>     mpg   cyl  disp    hp  drat    wt  qsec    vs    am  gear  carb
#>   <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1  21       6   160   110  3.9   2.62  16.5     0     1     4     4
#> 2  21       6   160   110  3.9   2.88  17.0     0     1     4     4
#> 3  22.8     4   108    93  3.85  2.32  18.6     1     1     4     1
#> 4  21.4     6   258   110  3.08  3.22  19.4     1     0     3     1
#> # … with 28 more rows

See https://rlang.r-lib.org/reference/dot-data.html.

Why does tidy evaluation throw an error unless an argument is printed or forced?

Code

Description of Issue

1 Answers1

Why does this fail?

What should I do?