climb.tool.impl.data_suite.third_party.uq360.utils package

Subpackages

Submodules

climb.tool.impl.data_suite.third_party.uq360.utils.gbm_whitebox_features module

class climb.tool.impl.data_suite.third_party.uq360.utils.gbm_whitebox_features.GBM_WhiteboxFeatureExtractor(gbm)[source]

Bases: object

add_whitebox_feature(feature_name, value)[source]
compute_decision_distance(x_test, x_prod)[source]
compute_depth_sums(data)[source]
compute_gbm_internal_whitebox_features(x_test, y_test, x_prod)[source]
compute_leaf_frequencies(x_test, y_test, x_prod)[source]
compute_tree_depth_sums(estimator, data, max_depth)[source]
count_nodes(data, max_node_id, accuracy_map=None)[source]

climb.tool.impl.data_suite.third_party.uq360.utils.generate_1D_regression_data module

climb.tool.impl.data_suite.third_party.uq360.utils.generate_1D_regression_data.make_data_gap(seed, data_count=100)[source]
climb.tool.impl.data_suite.third_party.uq360.utils.generate_1D_regression_data.make_data_sine(seed, data_count=450)[source]

climb.tool.impl.data_suite.third_party.uq360.utils.hidden_markov_model module

class climb.tool.impl.data_suite.third_party.uq360.utils.hidden_markov_model.HMM(X, config_dict=None)[source]

Bases: object

A Hidden Markov Model with Gaussian observations with unknown means and known precisions.

clear_trellis()[source]
compute_hessian(params_one, weights_one)[source]
compute_jacobian(params_one, weights_one)[source]
fit(weights, init_params=None, num_random_restarts=1, verbose=False, maxiter=None)[source]
static get_indices_in_held_out_fold(T, pct_to_drop, contiguous=False)[source]
Parameters:

  • T – length of the sequence

  • pct_to_drop – % of T in the held out fold

  • contiguous – if True generate a block of indices to drop else generate indices by iid sampling

Returns:

o (the set of indices in the fold)

static get_prior_contrib(param_dict)[source]
initialize_params(seed=1234)[source]
static log_obs_lik(x, phi, Sigma)[source]
Parameters:

  • x – T*D*1

  • phi – 1*D*K

  • Sigma – D*D*K — precision matrices per state

Returns:

ll

loss_at_missing_timesteps(weights, params)[source]
Parameters:

  • weights – zeroed out weights indicate missing values

  • params – packed parameters

Returns:

static pack_params(params_dict)[source]
static synthetic_hmm_data(K, T, D, sigma0=None, seed=1234, varainces_of_mean=1.0, diagonal_upweight=False)[source]
Parameters:

  • K – Number of HMM states

  • T – length of the sequence

unpack_params(params)[source]
weighted_alpha_recursion(xseq, pi, phi, Sigma, A, wseq, store_belief=False)[source]

Computes the weighted marginal probability of the sequence xseq given parameters; weights wseq turn on or off the emissions p(x_t | z_t) (weighting scheme B) :param xseq: T * D :param pi: K * 1 :param phi: D * K :param wseq: T * 1 :param A: :return:

weighted_beta_recursion(xseq, pi, phi, Sigma, A, wseq, store_belief=False)[source]

Runs beta recursion; weights wseq turn on or off the emissions p(x_t | z_t) (weighting scheme B) :param xseq: T * D :param pi: K * 1 :param phi: D * K :param wseq: T * 1 :param A: :return:

weighted_loss(params, weights)[source]

For LOOCV / IF computation within a single sequence. Uses weighted alpha recursion :param params: :param weights: :return:

climb.tool.impl.data_suite.third_party.uq360.utils.latent_features module

class climb.tool.impl.data_suite.third_party.uq360.utils.latent_features.LatentFeatures(model: Callable, layer: Module | List[Module], post_processing_fn=None, out_device: str = 'cpu')[source]

Bases: object

extract(input)[source]

climb.tool.impl.data_suite.third_party.uq360.utils.logistic_regression module

class climb.tool.impl.data_suite.third_party.uq360.utils.logistic_regression.LogisticRegression[source]

Bases: object

compute_hessian(params_one, weights_one, inputs, targets)[source]
compute_jacobian(params_one, weights_one, inputs, targets)[source]
fit(weights, init_params, inputs, targets, verbose=True)[source]
get_test_acc(params, test_targets, test_inputs)[source]
loss(params, weights, inputs, targets)[source]
predict(X)[source]
set_parameters(params)[source]
static synthetic_lr_data(N=10000, D=10)[source]
climb.tool.impl.data_suite.third_party.uq360.utils.logistic_regression.get_num_train(inputs)
climb.tool.impl.data_suite.third_party.uq360.utils.logistic_regression.logistic_predictions(params, inputs)
climb.tool.impl.data_suite.third_party.uq360.utils.logistic_regression.sigmoid(x)

climb.tool.impl.data_suite.third_party.uq360.utils.misc module

class climb.tool.impl.data_suite.third_party.uq360.utils.misc.DummySklearnEstimator(num_classes, base_model_prediction_fn)[source]

Bases: ABC

fit()[source]
predict_proba(X)[source]
climb.tool.impl.data_suite.third_party.uq360.utils.misc.fitted_ucc_w_nullref(y_true, y_pred_mean, y_pred_lower, y_pred_upper)[source]

Instantiates an UCC object for the target predictor plus a ‘null’ (constant band) reference :param y_pred_lower: :param y_pred_mean: :param y_pred_upper: :param y_true: :return: ucc object fitted for two systems: target + null reference

climb.tool.impl.data_suite.third_party.uq360.utils.misc.form_D_for_auucc(yhat, zhatl, zhatu)[source]
climb.tool.impl.data_suite.third_party.uq360.utils.misc.generate_regression_data(seed, data_count=500)[source]

Generate data from a noisy sine wave. :param seed: random number seed :param data_count: number of data points. :return:

climb.tool.impl.data_suite.third_party.uq360.utils.misc.make_batches(n_data, batch_size)[source]
climb.tool.impl.data_suite.third_party.uq360.utils.misc.make_sklearn_compatible_scorer(task_type, metric, greater_is_better=True, **kwargs)[source]
Parameters:

  • task_type – (str) regression or classification.

  • metric – (str): choice of metric can be one of these - [aurrrc, ece, auroc, nll, brier, accuracy] for classification and [“rmse”, “nll”, “auucc_gain”, “picp”, “mpiw”, “r2”] for regression.

  • greater_is_better – is False the scores are negated before returning.

  • **kwargs – additional arguments specific to some metrics.

Returns:

sklearn compatible scorer function.

climb.tool.impl.data_suite.third_party.uq360.utils.optimizers module

climb.tool.impl.data_suite.third_party.uq360.utils.optimizers.adam(grad, x, callback=None, num_iters=100, step_size=0.001, b1=0.9, b2=0.999, eps=1e-08, polyak=False)[source]

Adapted from autograd.misc.optimizers

climb.tool.impl.data_suite.third_party.uq360.utils.significance_test module

class climb.tool.impl.data_suite.third_party.uq360.utils.significance_test.SignificanceTester(metric)[source]

Bases: object

Class for non-parametric significance testing. It has two main functions:

hypothesis_test: Assumes two vectors of paired observations (ie each point in measurement_1 is paired with the point with the same index in measurement_2, for example measurements of the length of the same set of objects using two different rulers). Performs a permutation test to determine if one set of measurements is statistically different (higher, lower, or either one, depending on the ‘tailed’ argument) from the other, and computes a p-value.

confidence_interval: Uses bootstrapping to compute a confidence interval (controlled by ‘alpha’ argument, default is 95% confidence interval) for a list of (1-dimensional) measurements.

confidence_interval(measurement, metric_payload={}, metric_kwargs={}, n_iter=10000, alpha=0.05, verbose=False)[source]
get_average(predictions)[source]
get_cost(predictions, deltas=[], ratio=0.5, exp=1)[source]
get_metric_function(metric)[source]
hypothesis_test(measurement_1, measurement_2, metric_payload={}, n_iter=1000, tailed='two', verbose=False)[source]
resample_compute(measurement, indices, metric_payload, metric_kwargs)[source]

climb.tool.impl.data_suite.third_party.uq360.utils.utils module

Module contents