CAST (version 0.9.0)

DItoErrormetric: Model the relationship between the DI and the prediction error

Description

Performance metrics are calculated for moving windows of DI values of cross-validated training data

Usage

DItoErrormetric(
  model,
  trainDI,
  multiCV = FALSE,
  length.out = 10,
  window.size = 5,
  calib = "scam",
  method = "L2",
  useWeight = TRUE,
  k = 6,
  m = 2
)

Value

A scam or linear model

Arguments

model

the model used to get the AOA

trainDI

the result of trainDI or aoa object aoa

multiCV

Logical. Re-run model fitting and validation with different CV strategies. See details.

length.out

Numeric. Only used if multiCV=TRUE. Number of cross-validation folds. See details.

window.size

Numeric. Size of the moving window. See rollapply.

calib

Character. Function to model the DI~performance relationship. Currently lm and scam are supported

method

Character. Method used for distance calculation. Currently euclidean distance (L2) and Mahalanobis distance (MD) are implemented but only L2 is tested. Note that MD takes considerably longer. See ?aoa for further explanation

useWeight

Logical. Only if a model is given. Weight variables according to importance in the model?

k

Numeric. See mgcv::s

m

Numeric. See mgcv::s

Author

Hanna Meyer, Marvin Ludwig

Details

If multiCV=TRUE the model is re-fitted and validated by length.out new cross-validations where the cross-validation folds are defined by clusters in the predictor space, ranging from three clusters to LOOCV. Hence, a large range of DI values is created during cross-validation. If the AOA threshold based on the calibration data from multiple CV is larger than the original AOA threshold (which is likely if extrapolation situations are created during CV), the AOA threshold changes accordingly. See Meyer and Pebesma (2021) for the full documentation of the methodology.

References

Meyer, H., Pebesma, E. (2021): Predicting into unknown space? Estimating the area of applicability of spatial prediction models. tools:::Rd_expr_doi("10.1111/2041-210X.13650")

See Also

aoa

Examples

if (FALSE) {

  library(CAST)
  library(sf)
  library(terra)
  library(caret)


  data(splotdata)
  splotdata <- st_drop_geometry(splotdata)
  predictors <- terra::rast(system.file("extdata","predictors_chile.tif", package="CAST"))

  model <- caret::train(splotdata[,6:16], splotdata$Species_richness, ntree = 10,
                        trControl = trainControl(method = "cv", savePredictions = TRUE))

  AOA <- aoa(predictors, model)

  errormodel <- DItoErrormetric(model, AOA)
  plot(errormodel)

  expected_error = terra::predict(AOA$DI, errormodel)
  plot(expected_error)


  # with multiCV = TRUE
  errormodel = DItoErrormetric(model, AOA, multiCV = TRUE, length.out = 3)
  plot(errormodel)

  expected_error = terra::predict(AOA$DI, errormodel)
  plot(expected_error)

  # mask AOA based on new threshold from multiCV
  mask_aoa = terra::mask(expected_error, AOA$DI > attr(errormodel, 'AOA_threshold'), maskvalues = 1)
  plot(mask_aoa)




}