Fast Transform and Compute Columns on a Data Frame
ftransform is a much faster version of transform and dplyr::mutate for data frames. It returns the data frame with new columns computed and/or existing columns modified or deleted. settransform does all of that by reference i.e. it modifies the data frame in the global environment. fcompute can be used to compute new columns from the columns in a data frame and returns only the computed columns.
# Modify and return data frame ftransform(.data, ...) ftransformv(.data, vars, FUN, ..., apply = TRUE) tfm(.data, ...) # Shortcut for ftransform tfmv(.data, vars, FUN, ..., apply = TRUE) # Modify data frame by reference settransform(.data, ...) settransformv(.data, vars, FUN, ..., apply = TRUE) settfm(.data, ...) # Shortcut for settransform settfmv(.data, vars, FUN, ..., apply = TRUE) # Replace/add modified columns in/to a data frame ftransform(.data) <- value tfm(.data) <- value # Shortcut for ftransform<- # Compute columns, returned as a new data frame fcompute(.data, ...)
.data |
a data frame or named list of columns. |
... |
further arguments of the form |
vars |
variables to be transformed by applying |
FUN |
a single function yielding a result of length |
apply |
logical. |
value |
a named list of replacements, it will be treated like an evaluated list of |
The ... arguments to ftransform are tagged
vector expressions, which are evaluated in the data frame
.data. The tags are matched against names(.data), and for
those that match, the values replace the corresponding variable in
.data, whereas the others are appended to .data. It is also possible to delete columns by assigning NULL to them, i.e. ftransform(data, colk = NULL) removes colk from the data. Note that names(.data) and the names of the ... arguments are checked for uniqueness beforehand, yielding an error if this is not the case.
Since collapse v1.3.0, is is also possible to pass a single named list to ..., i.e. ftransform(data, newdata). This list will be treated like a list of tagged vector expressions. Note the different behavior: ftransform(data, list(newcol = col1)) is the same as ftransform(data, newcol = col1), whereas ftransform(data, newcol = as.list(col1)) creates a list column. Something like ftransform(data, as.list(col1)) gives an error because the list is not named. See Examples.
The function ftransformv added in v1.3.2 provides a fast replacement for the functions dplyr::mutate_at and dplyr::mutate_if facilitating mutations of groups of columns (dplyr::mutate_all is already accounted for by dapply). See Examples.
The function settransform does all of that by reference, but uses base-R's copy-on modify semantics, which is equivalent to replacing the data with <- (thus it is still memory efficient but the data will have a different memory address afterwards).
The function fcompute works just like ftransform, but returns only the changed / computed columns without modifying or appending the data in .data.
The modified data frame .data, or, for fcompute, a new data frame with the columns computed on .data. All attributes of .data are preserved.
## ftransform modifies and returns a data.frame
head(ftransform(airquality, Ozone = -Ozone))
head(ftransform(airquality, new = -Ozone, Temp = (Temp-32)/1.8))
head(ftransform(airquality, new = -Ozone, new2 = 1, Temp = NULL)) # Deleting Temp
head(ftransform(airquality, Ozone = NULL, Temp = NULL)) # Deleting columns
# With collapse's grouped and weighted functions, complex operations are done on the fly
head(ftransform(airquality, # Grouped operations by month:
Ozone_Month_median = fmedian(Ozone, Month, TRA = "replace_fill"),
Ozone_Month_sd = fsd(Ozone, Month, TRA = "replace"),
Ozone_Month_centered = fwithin(Ozone, Month)))
# Grouping by month and above/below average temperature in each month
head(ftransform(airquality, Ozone_Month_high_median =
fmedian(Ozone, list(Month, Temp > fbetween(Temp, Month)), TRA = "replace_fill")))
## ftransformv can be used to modify multiple columns using a function
head(ftransformv(airquality, 1:3, log))
head(`[<-`(airquality, 1:3, value = lapply(airquality[1:3], log))) # Same thing in base R
head(ftransformv(airquality, 1:3, log, apply = FALSE))
head(`[<-`(airquality, 1:3, value = log(airquality[1:3]))) # Same thing in base R
# Using apply = FALSE yields meaningful performance gains with collapse functions
# This calls fwithin.default, and repeates the grouping by month 3 times:
head(ftransformv(airquality, 1:3, fwithin, Month))
# This calls fwithin.data.frame, and only groups one time -> 5x faster!
head(ftransformv(airquality, 1:3, fwithin, Month, apply = FALSE))
library(magrittr) # Pipe operators
# This also works for grouped and panel data frames (calling fwithin.grouped_df)
airquality %>% fgroup_by(Month) %>%
ftransformv(1:3, fwithin, apply = FALSE) %>% head
# But this gives the WRONG result (calling fwithin.default). Need option apply = FALSE!!
airquality %>% fgroup_by(Month) %>%
ftransformv(1:3, fwithin) %>% head
# For grouped modification of single columns in a grouped dataset, we can use GRP():
airquality %>% fgroup_by(Month) %>%
ftransform(W_Ozone = fwithin(Ozone, GRP(.)), # Grouped centering
sd_Ozone_m = fsd(Ozone, GRP(.), TRA = "replace"), # In-Month standard deviation
sd_Ozone = fsd(Ozone, TRA = "replace"), # Overall standard deviation
sd_Ozone2 = fsd(Ozone, TRA = "replace_fill"), # Same, overwriting NA's
sd_Ozone3 = fsd(Ozone)) %>% head # Same thing (calling rep())
rm(airquality)
## For more complex mutations we can use ftransform with compound pipes
airquality %>% fgroup_by(Month) %>%
ftransform(get_vars(., 1:3) %>% fwithin %>% flag(0:2)) %>% head
airquality %>% ftransform(STD(., cols = 1:3) %>% replace_NA(0)) %>% head
# The list argument feature also allows flexible operations creating multiple new columns
airquality %>% # The variance of Wind and Ozone, by month, weighted by temperature:
ftransform(fvar(list(Wind_var = Wind, Ozone_var = Ozone), Month, Temp, "replace")) %>% head
# Same as above using a grouped data frame (a bit more complex)
airquality %>% fgroup_by(Month) %>%
ftransform(fselect(., Wind, Ozone) %>% fvar(Temp, "replace") %>% add_stub("_var", FALSE)) %>%
fungroup %>% head
# This performs 2 different multi-column grouped operations (need c() to make it one list)
ftransform(airquality, c(fmedian(list(Wind_Day_median = Wind,
Ozone_Day_median = Ozone), Day, TRA = "replace"),
fsd(list(Wind_Month_sd = Wind,
Ozone_Month_sd = Ozone), Month, TRA = "replace"))) %>% head
## settransform(v) works like ftransform(v) but modifies a data frame in the global environment..
settransform(airquality, Ratio = Ozone / Temp, Ozone = NULL, Temp = NULL)
head(airquality)
rm(airquality)
# Grouped and weighted centering
settransformv(airquality, 1:3, fwithin, Month, Temp, apply = FALSE)
head(airquality)
rm(airquality)
# Suitably lagged first-differences
settransform(airquality, get_vars(airquality, 1:3) %>% fdiff %>% flag(0:2))
head(airquality)
rm(airquality)
# Same as above using magrittr::`%<>%`
airquality %<>% ftransform(get_vars(., 1:3) %>% fdiff %>% flag(0:2))
head(airquality)
rm(airquality)
# It is also possible to achieve the same thing via a replacement method (if needed)
ftransform(airquality) <- get_vars(airquality, 1:3) %>% fdiff %>% flag(0:2)
head(airquality)
rm(airquality)
## fcompute only returns the modified / computed columns, ...
head(fcompute(airquality, Ozone = -Ozone))
head(fcompute(airquality, new = -Ozone, Temp = (Temp-32)/1.8))
head(fcompute(airquality, new = -Ozone, new2 = 1))Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.