climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks package

Subpackages

• climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.bnn_models package
  • Submodules
  • climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.bnn_models.bayesian_mlp module
    • BayesianClassificationNet
      • BayesianClassificationNet.likelihood()
      • BayesianClassificationNet.neg_elbo()
    • BayesianNN
      • BayesianNN.forward()
      • BayesianNN.get_weights()
      • BayesianNN.kl_divergence_w()
      • BayesianNN.prior_predictive_samples()
      • BayesianNN.set_weights()
    • BayesianRegressionNet
      • BayesianRegressionNet.get_noise_var()
      • BayesianRegressionNet.likelihood()
      • BayesianRegressionNet.mse()
      • BayesianRegressionNet.neg_elbo()
  • climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.bnn_models.horseshoe_mlp module
    • HshoeBNN
      • HshoeBNN.fixed_point_updates()
      • HshoeBNN.forward()
      • HshoeBNN.get_weights()
      • HshoeBNN.kl_divergence_w()
      • HshoeBNN.prior_predictive_samples()
      • HshoeBNN.set_weights()
    • HshoeClassificationNet
      • HshoeClassificationNet.likelihood()
      • HshoeClassificationNet.neg_elbo()
    • HshoeRegressionNet
      • HshoeRegressionNet.get_noise_var()
      • HshoeRegressionNet.likelihood()
      • HshoeRegressionNet.mse()
      • HshoeRegressionNet.neg_elbo()
  • Module contents

Submodules

climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.layer_utils module

Contains implementations of various utilities used by Horseshoe Bayesian layers

class climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.layer_utils.InvGammaHalfCauchyLayer(out_features, b)

Bases: Module

Uses the inverse Gamma parameterization of the half-Cauchy distribution. a ~ C^+(0, b) <==> a^2 ~ IGamma(0.5, 1/lambda), lambda ~ IGamma(0.5, 1/b^2), where lambda is an auxiliary latent variable. Uses a factorized variational approximation q(ln a^2)q(lambda) = N(mu, sigma^2) IGamma(ahat, bhat). This layer places a half Cauchy prior on the scales of each output node of the layer.

entropy()

Computes entropy of q(ln a^2) and q(lambda)

entropy_a2()

entropy_lambda()

expectation_wrt_prior()

Computes E[ln p(a^2 | lambda)] + E[ln p(lambda)]

fixed_point_updates()

initialize_from_prior()

Initializes variational parameters by sampling from the prior.

kl()

Computes KL(q(ln(a^2)q(lambda) || IG(a^2 | 0.5, 1/lambda) IG(lambda | 0.5, 1/b^2))

class climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.layer_utils.InvGammaLayer(a, b, out_features=1)

Bases: Module

Approximates the posterior of c^2 with prior IGamma(c^2 | a , b) using a log Normal approximation q(ln c^2) = N(mu, sigma^2)

entropy()

expectation_wrt_prior()

Computes E[ln p(c^2 | a, b)]

initialize_from_prior()

Initializes variational parameters by sampling from the prior.

kl()

Computes KL(q(ln(c^2) || IG(c^2 | a, b))

climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.layer_utils.diag_gaussian_entropy(log_std, D)

climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.layer_utils.inv_gamma_entropy(a, b)

climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.layer_utils.log_normal_entropy(log_std, mu, D)

climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.layers module

Contains implementations of various Bayesian layers

class climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.layers.BayesianLinearLayer(in_features, out_features, cuda=False, init_weight=None, init_bias=None, prior_stdv=None)

Bases: Module

Affine layer with N(0, v/H) or N(0, user specified v) priors on weights and fully factorized variational Gaussian approximation

forward(x, do_sample=True, scale_variances=False)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_parameters(init_weight, init_bias)

kl()

KL divergence (q(W) || p(W)) :return:

class climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.layers.HorseshoeLayer(in_features, out_features, cuda=False, scale=1.0)

Bases: BayesianLinearLayer

Uses non-centered parametrization. w_k = v*tau_k*beta_k where k indexes an output unit and w_k and beta_k are vectors of all weights incident into the unit

fixed_point_updates()

forward(x, do_sample=True, debug=False, eps_scale=None, eps_w=None)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

kl()

KL divergence (q(W) || p(W)) :return:

class climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.layers.NodeSpecificRegularizedHorseshoeLayer(in_features, out_features, cuda=False, scale=1.0, c_a=2.0, c_b=6.0)

Bases: RegularizedHorseshoeLayer

Uses the regularized Horseshoe distribution. The regularized Horseshoe soft thresholds the tails of the Horseshoe. For all weights w_k incident upon node k in the layer we have: w_k ~ N(0, (tau_k * v)^2 I) N(0, c_k^2 I), c_k^2 ~ InverseGamma(a, b). c_k^2 controls the scale of the thresholding. As c_k^2 -> infinity, the regularized Horseshoe -> Horseshoe Note that we now have a per-node c_k.

class climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.layers.RegularizedHorseshoeLayer(in_features, out_features, cuda=False, scale=1.0, c_a=2.0, c_b=6.0)

Bases: HorseshoeLayer

Uses the regularized Horseshoe distribution. The regularized Horseshoe soft thresholds the tails of the Horseshoe. For all weights w_k incident upon node k in the layer we have: w_k ~ N(0, (tau_k * v)^2 I) N(0, c^2 I), c^2 ~ InverseGamma(c_a, b). c^2 controls the scale of the thresholding. As c^2 -> infinity, the regularized Horseshoe -> Horseshoe.

forward(x, do_sample=True, **kwargs)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

kl()

KL divergence (q(W) || p(W)) :return:

climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.layers.reparam(mu, logvar, do_sample=True, mc_samples=1)

climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.misc module

climb.tool.impl.data_suite.third_party.uq360.models.bayesian_neural_networks.misc.compute_test_ll(y_test, y_pred_samples, std_y=1.0)

Computes test log likelihoods = (1 / Ntest) * sum_n p(y_n | x_n, D_train) :param y_test: True y :param y_pred_samples: y^s = f(x_test, w^s); w^s ~ q(w). S x Ntest, where S is the number of samples q(w) is either a trained variational posterior or an MCMC approximation to p(w | D_train) :param std_y: True std of y (assumed known)

Module contents