h2o (version 3.40.0.4)

h2o.calculate_fairness_metrics: Calculate intersectional fairness metrics.

Description

Calculate intersectional fairness metrics.

Usage

h2o.calculate_fairness_metrics(
  model,
  frame,
  protected_columns,
  reference,
  favorable_class
)

Value

Dictionary of frames. One frame is the overview, other frames contain dependence of performance on threshold for each protected group.

Arguments

model

H2O Model

frame

Frame used to calculate the metrics.

protected_columns

List of categorical columns that contain sensitive information such as race, gender, age etc.

reference

List of values corresponding to a reference for each protected columns. If set to NULL, it will use the biggest group as the reference.

favorable_class

Positive/favorable outcome class of the response.

Examples

if (FALSE) {
library(h2o)
h2o.init()
data <- h2o.importFile(paste0("https://s3.amazonaws.com/h2o-public-test-data/smalldata/",
                              "admissibleml_test/taiwan_credit_card_uci.csv"))
x <- c('LIMIT_BAL', 'AGE', 'PAY_0', 'PAY_2', 'PAY_3', 'PAY_4', 'PAY_5', 'PAY_6', 'BILL_AMT1',
       'BILL_AMT2', 'BILL_AMT3', 'BILL_AMT4', 'BILL_AMT5', 'BILL_AMT6', 'PAY_AMT1', 'PAY_AMT2',
       'PAY_AMT3', 'PAY_AMT4', 'PAY_AMT5', 'PAY_AMT6')
y <- "default payment next month"
protected_columns <- c('SEX', 'EDUCATION')

for (col in c(y, protected_columns))
  data[[col]] <- as.factor(data[[col]])

splits <- h2o.splitFrame(data, 0.8)
train <- splits[[1]]
test <- splits[[2]]
reference <- c(SEX = "1", EDUCATION = "2")  # university educated man
favorable_class <- "0" # no default next month

gbm <- h2o.gbm(x, y, training_frame = train)

h2o.calculate_fairness_metrics(gbm, test, protected_columns = protected_columns,
                               reference = reference, favorable_class = favorable_class)
}