Title: | Autograph R for 'Tensorflow' |
---|---|
Description: | Translate R control flow expressions into 'Tensorflow' graphs. |
Authors: | Tomasz Kalinowski [aut, cre] |
Maintainer: | Tomasz Kalinowski <[email protected]> |
License: | GPL-3 |
Version: | 0.3.2.9000 |
Built: | 2024-11-06 02:55:33 UTC |
Source: | https://github.com/t-kalinowski/tfautograph |
TensorArray.write()
TensorArray.write()
## S3 replacement method for class 'tensorflow.python.ops.tensor_array_ops.TensorArray' ta[[i, ..., name = NULL]] <- value
## S3 replacement method for class 'tensorflow.python.ops.tensor_array_ops.TensorArray' ta[[i, ..., name = NULL]] <- value
ta |
a tensorflow |
i |
something castable to an int32 scalar tensor. 0-based. |
... |
Error if anything is passed to |
name |
A scalar string, name of the op |
value |
The value to write. |
## Not run: ta <- tf$TensorArray(tf$float32, size = 5L) for(i in 0:4) ta[[i]] <- i ta$stack() # You can use this to grow objects in graph mode accuracies_log <- tf$TensorArray(tf$float32, size = 0L, dynamic_size=TRUE) for(epoch in 0:4) accuracies_log[[epoch]] <- runif(1) acc <- accuracies_log$stack() acc ## End(Not run)
## Not run: ta <- tf$TensorArray(tf$float32, size = 5L) for(i in 0:4) ta[[i]] <- i ta$stack() # You can use this to grow objects in graph mode accuracies_log <- tf$TensorArray(tf$float32, size = 0L, dynamic_size=TRUE) for(epoch in 0:4) accuracies_log[[epoch]] <- runif(1) acc <- accuracies_log$stack() acc ## End(Not run)
tf.cond()
output structure when autographing if
This function can be used to specify the output structure from tf.cond()
when autographing an if
statement. In most use cases, use of this function
is purely optional. If not supplied, the if
output structure is
automatically built.
ag_if_vars( ..., modified = list(), return = FALSE, undefs = NULL, control_flow = 0 )
ag_if_vars( ..., modified = list(), return = FALSE, undefs = NULL, control_flow = 0 )
... |
Variables modified by the |
modified |
Variables names supplied as a character vector, or a list of
character vectors if specifying nested complex structures. This is an
escape hatch for the lazy evaluation semantics of |
return |
logical, whether to include the return value the evaluated R
expression in the |
undefs |
A bare character vector or a list of character vectors. Supplied names are exported as undefs in the parent frame. This is used to give a more informative error message when attempting to access a variable that can't be balanced between branches. |
control_flow |
An integer, the maximum number of control-flow statements
( |
If the output structure is not explicitly supplied via
ag_if_vars()
, then the output structure is automatically composed: The
true and false branches of the expression are traced into concrete
functions, then the output signature from the two branch functions are
balanced. Balancing is performed by either fetching a variable from an
outer scope or by reclassifying a symbol as an undef.
When dealing with complex composites (that is, nested structures where a
modified tensor is part of a named list or dictionary), care is taken to
prevent unnecessarily capturing other unmodified tensors in the structure.
This is done by pruning unmodified tensors from the returned output
structure, and then merging them back with the original object recursively.
One limitation of the implementation is that lists must either be fully
named with unique names, or not named at all, partially named lists or
duplicated names in a list throw an error. This is due to the conversion
that happens when going between python and R: named lists get converted to
python dictionaries, which require that all keys are unique. Additionally,
pruning of unmodified objects from an autographed if
is currently only
supported for named lists (python dictionaries). Unnamed lists or tuples
are passed as is (e.g, no pruning and merging done), which may lead to
unnecessarily bloat in the constructed graphs.
NULL
, invisibly
## Not run: # these examples only have an effect in graph mode # to enter graph mode easily we'll create a few helpers ag <- autograph # pass which symbols you expect to be modifed or created liks this: ag_if_vars(x) ag(if (y > 0) { x <- y * y } else { x <- y }) # if the return value from the if expression is important, pass `return = TRUE` ag_if_vars(return = TRUE) x <- ag(if(y > 0) y * y else y) # pass complex nested structures like this x <- list(a = 1, b = 2) ag_if_vars(x$a) ag(if(y > 0) { x$a <- y }) # undefs are for mark branch-local variables ag_if_vars(y, x$a, undef = "tmp_local_var") ag(if(y > 0) { y <- y * 100 tmp_local_var <- y + 1 x$a <- tmp_local_var }) # supplying `undef` is not necessary, it exists purely as a way to supply a # guardrail for defensive programming and/or to improve code readability ## modified vars can be supplied in `...` or as a named arg. ## these paires of ag_if_vars() calls are equivalent ag_if_vars(y, x$a) ag_if_vars(modified = list("y", c("x", "a"))) ag_if_vars(x, y, z) ag_if_vars(modified = c("x", "y", "z")) ## control flow # count number of odds between 0:10 ag({ x <- 10 count <- 0 while(x > 0) { ag_if_vars(control_flow = 1) if(x %% 2 == 0) next count <- count + 1 } }) ## End(Not run)
## Not run: # these examples only have an effect in graph mode # to enter graph mode easily we'll create a few helpers ag <- autograph # pass which symbols you expect to be modifed or created liks this: ag_if_vars(x) ag(if (y > 0) { x <- y * y } else { x <- y }) # if the return value from the if expression is important, pass `return = TRUE` ag_if_vars(return = TRUE) x <- ag(if(y > 0) y * y else y) # pass complex nested structures like this x <- list(a = 1, b = 2) ag_if_vars(x$a) ag(if(y > 0) { x$a <- y }) # undefs are for mark branch-local variables ag_if_vars(y, x$a, undef = "tmp_local_var") ag(if(y > 0) { y <- y * 100 tmp_local_var <- y + 1 x$a <- tmp_local_var }) # supplying `undef` is not necessary, it exists purely as a way to supply a # guardrail for defensive programming and/or to improve code readability ## modified vars can be supplied in `...` or as a named arg. ## these paires of ag_if_vars() calls are equivalent ag_if_vars(y, x$a) ag_if_vars(modified = list("y", c("x", "a"))) ag_if_vars(x, y, z) ag_if_vars(modified = c("x", "y", "z")) ## control flow # count number of odds between 0:10 ag({ x <- 10 count <- 0 while(x > 0) { ag_if_vars(control_flow = 1) if(x %% 2 == 0) next count <- count + 1 } }) ## End(Not run)
This can be used to manually specify which variables are to be included
explicitly as loop_vars
when autographing an expression into a
tf.while_loop()
call, or the loop_vars
equivalent when building a
dataset.reduce()
.
ag_loop_vars( ..., list = character(), include = character(), exclude = character(), undef = character() )
ag_loop_vars( ..., list = character(), include = character(), exclude = character(), undef = character() )
... |
Variables as bare symbol names |
list , include , exclude
|
optionally, the variable names as a character
vector (use this as an escape hatch from the |
undef |
character vector of symbols |
Use of this is usually not required as the loop variables are automatically
inferred. Inference is done by statically looking through the loop body and
finding the symbols that are the targets of the common assignment operators
from base R (<-
, ->
, =
), from package:zeallot (%<-%
and %->%
) and
package:magrittr (%<>%
).
In certain circumstances, this approach may capture variables that are
intended to be local variables only. In those circumstances it is also
possible to specify them preceded with a -
.
Note, the specified loop vars are expected to exist before the autographed expression, and a warning is issued otherwise (usually immediately preceding an error thrown when attempting to actually autograph the expression)
Only bare symbol names can be supplied as loop vars. In the future, support
may be expanded to allow for nested complex composites (e.g., specifying
variables that are nested within a more complex structure–passing
ag_loop_vars(foo$bar$baz)
is currently not supported.)
the specified hint invisibly.
The semantics of this function are inspired by base::rm()
## Not run: i <- tf$constant(0L) autograph({ ag_loop_vars(x, i) while(x > 0) { if(x %%2 == 0) i <- i + 1L x <- x - 1 } }) ## sometimes, a variable is infered to be a loop_var unnecessarily. For example x <- tf$constant(1:10) # imagine x is left over in the current scope from some previous calculations # It's value is not important, but it exists autograph({ for(i in tf$constant(1:6)) { x <- i * i tf$print(x) } }) # this will throw an error because `x` was infered to be a `loop_var`, # but it's shape witin the loop body is different from what it was before. # there are two solutions to prevent `x` from being captured as a loop_var ## 1) remove `x` from the current scope like so: rm(x) ## 2) provide a hint like so: ag_loop_vars(-x) ## if your variable names are being dynamically generated, there is an ## escape hatch for the lazy evaluation semantics of ... ag_loop_vars(exclude = "x") ## End(Not run)
## Not run: i <- tf$constant(0L) autograph({ ag_loop_vars(x, i) while(x > 0) { if(x %%2 == 0) i <- i + 1L x <- x - 1 } }) ## sometimes, a variable is infered to be a loop_var unnecessarily. For example x <- tf$constant(1:10) # imagine x is left over in the current scope from some previous calculations # It's value is not important, but it exists autograph({ for(i in tf$constant(1:6)) { x <- i * i tf$print(x) } }) # this will throw an error because `x` was infered to be a `loop_var`, # but it's shape witin the loop body is different from what it was before. # there are two solutions to prevent `x` from being captured as a loop_var ## 1) remove `x` from the current scope like so: rm(x) ## 2) provide a hint like so: ag_loop_vars(-x) ## if your variable names are being dynamically generated, there is an ## escape hatch for the lazy evaluation semantics of ... ag_loop_vars(exclude = "x") ## End(Not run)
This can be used before any autographed expression that results in the
creation of a tensor or op graph node. This can be used before for
(both with tensors and datasets), while
, and if
statements.
ag_name(x)
ag_name(x)
x |
A string |
x
, invisibly
## Not run: ## when you're in graph mode. (e.g, tf$executing_eagerly == FALSE) ag_name("main-training-loop") for(elem in dataset) ... ## End(Not run)
## Not run: ## when you're in graph mode. (e.g, tf$executing_eagerly == FALSE) ag_name("main-training-loop") for(elem in dataset) ... ## End(Not run)
tf.while_loop
optionsSee https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/while_loop for additional details.
ag_while_opts( ..., shape_invariants = NULL, parallel_iterations = 10L, back_prop = TRUE, swap_memory = FALSE, maximum_iterations = NULL )
ag_while_opts( ..., shape_invariants = NULL, parallel_iterations = 10L, back_prop = TRUE, swap_memory = FALSE, maximum_iterations = NULL )
... |
Ignored, used to ensure all arguments supplied are named. |
shape_invariants |
The shape invariants for the loop variables. |
parallel_iterations |
The number of iterations allowed to run in parallel. It must be a positive integer. |
back_prop |
Deprecated (optional). |
swap_memory |
Whether GPU-CPU memory swap is enabled for this loop. |
maximum_iterations |
Optional maximum number of iterations of the while
loop to run. If provided, the |
‘NULL“ invisibly, called for it’s side effect.
Use ag_name()
to supply name
and ag_loop_vars()
to supply
loop_vars
directly.
This is only applicable when autograph in graph mode, otherwise this has no effect.
## Not run: ## use tf_function() to enter graph mode: tf_function(autograph(function(n) { ag_name("silly-example") ag_while_opts(back_prop = FALSE) while(n > 0) n <- n - 1 })) ## End(Not run)
## Not run: ## use tf_function() to enter graph mode: tf_function(autograph(function(n) { ag_name("silly-example") ag_while_opts(back_prop = FALSE) while(n > 0) n <- n - 1 })) ## End(Not run)
Note, this documentation page is meant to serve as a technical reference, not
an introduction to autograph
. For the latter, please visit the
documentation website: (https://t-kalinowski.github.io/tfautograph/) or see
the package vignettes.
autograph(x)
autograph(x)
x |
a function supplied as a bare symbol, or an expression |
if x
is a function, then the the same function with a new parent
environment, package:tfautograph:ag_mask
, which is the autograph mask
that contains implementations of R control flow primitives that are capable
of handling tensorflow tensors. The parent of the
package:tfautograph:ag_mask
in turn is the original environment of x
.
if x
is an expression, then that expression is evaluated in a special
environment with the autograph mask ag_mask
active. If the result of that
expression included local assignment or modifications of variables, (for
example, via <-
), those modified variables are then exported into the
current frame. The return value of the expression is then returned.
A thin wrapper around tf$Assert()
that automatically constructs an
informative error message (passed on to data
argument), which includes the
expression passed to condition
, the values of the symbols found in the
expression, as well as the full R call stack at the time the tf$Assert()
node is created.
tf_assert( condition, ..., expr = substitute(condition), summarize = NULL, name = NULL )
tf_assert( condition, ..., expr = substitute(condition), summarize = NULL, name = NULL )
condition |
A boolean tensor |
... |
Additional elements passed on to |
expr |
A language object, provided in case |
summarize |
Print this many entries of each tensor. |
name |
A name for this operation (optional). |
## Not run: x <- tf$constant(-1) try(tf_assert(x > 0, "oopsies! x must be greater than 0")) ## End(Not run)
## Not run: x <- tf$constant(-1) try(tf_assert(x > 0, "oopsies! x must be greater than 0")) ## End(Not run)
This is a minimal wrapper around tf.case()
that allows you to supply the
pred_fn_pairs
using the ~
.
tf_case( ..., pred_fn_pairs = list(...), default = NULL, exclusive = FALSE, name = "case" )
tf_case( ..., pred_fn_pairs = list(...), default = NULL, exclusive = FALSE, name = "case" )
... , pred_fn_pairs
|
a list |
default |
a function, optionally specified with the |
exclusive |
bool, whether to evaluate all |
name |
a string, passed on to |
The result from tf$case()
## Not run: fizz_buzz_one <- function(x) { tf_case( x %% 15 == 0 ~ "FizzBuzz", x %% 5 == 0 ~ "Buzz", x %% 3 == 0 ~ "Fizz", default = ~ tf$as_string(x, precision = 0L) ) } fn <- tf_function(autograph(function(n) { for(e in tf$range(n)) tf$print(fizz_buzz_one(e)) })) x <- tf$constant(16) fn(x) ## End(Not run)
## Not run: fizz_buzz_one <- function(x) { tf_case( x %% 15 == 0 ~ "FizzBuzz", x %% 5 == 0 ~ "Buzz", x %% 3 == 0 ~ "Fizz", default = ~ tf$as_string(x, precision = 0L) ) } fn <- tf_function(autograph(function(n) { for(e in tf$range(n)) tf$print(fizz_buzz_one(e)) })) x <- tf$constant(16) fn(x) ## End(Not run)
This is a minimal wrapper around tf$cond()
that allows you to supply
true_fn
and false_fn
as lambda functions defined using the tilde ~
.
tf_cond(pred, true_fn, false_fn, name = NULL)
tf_cond(pred, true_fn, false_fn, name = NULL)
pred |
R logical or a tensor. |
true_fn , false_fn
|
a |
name |
a string, passed on to |
if cond is a tensor, then the result of tf.cond()
. Otherwise, if
pred
is an EagerTensor
or an R logical, then the result of either
true_fn()
or false_fn()
in Tensorflow version 1, the strict
keyword argument is supplied with
a value of TRUE
(different from the default)
## Not run: ## square if positive # using tf$cond directly: raw <- function(x) tf$cond(x > 0, function() x * x, function() x) # using tf_cond() wrapper tilde <- function(x) tf_cond(x > 0, ~ x * x, ~ x) ## End(Not run)
## Not run: ## square if positive # using tf$cond directly: raw <- function(x) tf$cond(x > 0, function() x * x, function() x) # using tf_cond() wrapper tilde <- function(x) tf_cond(x > 0, ~ x * x, ~ x) ## End(Not run)
tf.map_fn()
Thin wrapper around tf.map_fn()
with the following
differences:
accepts purrr
style ~
lambda syntax to define function fn
.
The order of elems
and fn
is switched to make it more pipe %>%
friendly and consistent with R mappers lapply()
and purrr::map()
.
tf_map( elems, fn, dtype = NULL, parallel_iterations = NULL, back_prop = TRUE, swap_memory = FALSE, infer_shape = TRUE, name = NULL )
tf_map( elems, fn, dtype = NULL, parallel_iterations = NULL, back_prop = TRUE, swap_memory = FALSE, infer_shape = TRUE, name = NULL )
elems |
A tensor or (possibly nested) sequence of tensors, each of which
will be unpacked along their first dimension. The nested sequence of the
resulting slices will be applied to |
fn |
An R function, specified using |
dtype |
(optional) The output type(s) of fn. If fn returns a structure of Tensors differing from the structure of elems, then dtype is not optional and must have the same structure as the output of fn. |
parallel_iterations |
(optional) The number of iterations allowed to run in parallel. When graph building, the default value is 10. While executing eagerly, the default value is set to 1. |
back_prop |
(optional) True enables support for back propagation. |
swap_memory |
(optional) True enables GPU-CPU memory swapping. |
infer_shape |
(optional) False disables tests for consistent output shapes. |
name |
(optional) Name prefix for the returned tensors. |
A tensor or (possibly nested) sequence of tensors. Each tensor packs the results of applying fn to tensors unpacked from elems along the first dimension, from first to last.
tf.switch_case
tf_switch( branch_index, ..., branch_fns = list(...), default = NULL, name = "switch_case" )
tf_switch( branch_index, ..., branch_fns = list(...), default = NULL, name = "switch_case" )
branch_index |
an integer tensor |
... , branch_fns
|
a list of function bodies specified with a |
default |
A function defined with a |
name |
a string, passed on to |
The result from tf.switch_case()
## Not run: tf_pow <- tf_function(function(x, pow) { tf_switch(pow, 0 ~ 1, 1 ~ x, 2 ~ x * x, 3 ~ x * x * x, default = ~ -1) }) # can optionally also omit the left hand side int, in which case the order of # the functions is used. tf_pow <- function(x, pow) { tf_switch(pow, ~ 1, ~ x, ~ x * x, ~ x * x * x, default = ~ -1) } # supply just some of the ints to override the default order tf_pow <- function(x, pow) { tf_switch(pow, 3 ~ x * x * x, 2 ~ x * x, ~ 1, ~ x, default = ~ -1) } # A slightly less contrived example: tf_norm <- tf_function(function(x, l) { tf_switch(l, 0 ~ tf$reduce_sum(tf$cast(x != 0, tf$float32)), # L0 norm 1 ~ tf$reduce_sum(tf$abs(x)), # L1 norm 2 ~ tf$sqrt(tf$reduce_sum(tf$square(x))), # L2 norm default = ~ tf$reduce_max(tf$abs(x))) # L-infinity norm }) ## End(Not run)
## Not run: tf_pow <- tf_function(function(x, pow) { tf_switch(pow, 0 ~ 1, 1 ~ x, 2 ~ x * x, 3 ~ x * x * x, default = ~ -1) }) # can optionally also omit the left hand side int, in which case the order of # the functions is used. tf_pow <- function(x, pow) { tf_switch(pow, ~ 1, ~ x, ~ x * x, ~ x * x * x, default = ~ -1) } # supply just some of the ints to override the default order tf_pow <- function(x, pow) { tf_switch(pow, 3 ~ x * x * x, 2 ~ x * x, ~ 1, ~ x, default = ~ -1) } # A slightly less contrived example: tf_norm <- tf_function(function(x, l) { tf_switch(l, 0 ~ tf$reduce_sum(tf$cast(x != 0, tf$float32)), # L0 norm 1 ~ tf$reduce_sum(tf$abs(x)), # L1 norm 2 ~ tf$sqrt(tf$reduce_sum(tf$square(x))), # L2 norm default = ~ tf$reduce_max(tf$abs(x))) # L-infinity norm }) ## End(Not run)
Visualizes the generated graph
view_function_graph( fn, args, ..., name = deparse(substitute(fn)), profiler = FALSE, concrete_fn = do.call(fn$get_concrete_fn, args), graph = concrete_fn$graph )
view_function_graph( fn, args, ..., name = deparse(substitute(fn)), profiler = FALSE, concrete_fn = do.call(fn$get_concrete_fn, args), graph = concrete_fn$graph )
fn |
TensorFlow function (returned from |
args |
arguments passed to |
... |
other arguments passed to |
name |
string, provided to tensorboard |
profiler |
logical, passed on to |
concrete_fn |
a |
graph |
a tensorflow graph (only used in graph mode, ignored with a warning if executing eagerly) |
## Not run: fn <- tf_function(function(x) autograph(if(x > 0) x * x else x)) view_function_graph(fn, list(tf$constant(5))) ## End(Not run)
## Not run: fn <- tf_function(function(x) autograph(if(x > 0) x * x else x)) view_function_graph(fn, list(tf$constant(5))) ## End(Not run)