Implicit Stochastic Gradient Descent Optimizer
This optimizer performs an implicit stochastic gradient descent
algorithm. It is mainly used within a bamlss call.
opt_isgd(x, y, family, weights = NULL, offset = NULL,
gammaFun = function(i) 1/(1 + i), shuffle = TRUE,
CFun = function(beta) diag(length(beta)),
start = NULL, i.state = 0)x |
For function |
y |
The model response, as returned from function
|
family |
A bamlss family object, see
|
weights |
Prior weights on the data, as returned from function |
offset |
Can be used to supply model offsets for use in fitting,
returned from function |
gammaFun |
Function specifying the step length. |
shuffle |
Should the data be shuffled? |
CFun |
Hessian approximating function. |
start |
Vector of starting values. |
i.state |
Added to |
tpf
For function opt_isgd() a list containing the following objects:
fitted.values |
A named list of the fitted values based on the last iteration of the modeled parameters of the selected distribution. |
parameters |
A matrix, each row corresponds to the parameter values of one iteration. |
sgd.summary |
The summary of the stochastic gradient descent algorithm which can be printed and plotted. |
CAUTION: Arguments weights and offset are not implemented yet!
Motivated by the lecture 'Regression modelling with large data sets' given by Ioannis Kosmidis in Innsbruck, January 2017.
Thorsten Simon
Toulis, P and Airoldi, EM (2015): Scalable estimation strategies based on stochastic approximations: Classical results and new insights. Statistics and Computing, 25, no. 4, 781–795. doi: 10.1007/s11222-015-9560-y
## Not run:
set.seed(111)
d <- GAMart(n = 10000)
f <- num ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b <- bamlss(f, data = d, optimizer = opt_isgd, sampler = FALSE)
plot(b, ask = F)
## loop over observations a 2nd time
b <- bamlss(f, data = d, optimizer = opt_isgd, sampler = FALSE, start = parameters(b),
i.state = b$model.stats$optimizer$sgd.summary$i.state)
plot(b, ask = F)
## try differeent gammaFuns, e.g.,
# gammaFun <- function(i) .3/sqrt((1+i)) + 0.001
## testing some families
f2 <- bin ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b2 <- bamlss(f2, data = d, optimizer = opt_isgd, sampler = FALSE, family = "binomial")
f3 <- cens ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b3 <- bamlss(f3, data = d, optimizer = opt_isgd, sampler = FALSE, family = "cnorm")
## End(Not run)Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.