Skip to content

Estimators

The package implements a number of estimators with common API. For general usage instructions and examples, consult the general instructions. Although the API is organized alphabetically, the estimators have been grouped by types in the list of estimators together with relevant literature.

bmi.estimators.CCAMutualInformationEstimator

Bases: IMutualInformationPointEstimator

bmi.estimators.DonskerVaradhanEstimator

Bases: NeuralEstimatorBase

bmi.estimators.HistogramEstimator

Bases: IMutualInformationPointEstimator

__init__(n_bins_x=5, n_bins_y=None, standardize=True)

Parameters:

Name Type Description Default
n_bins_x int

number of bins per each X dimension

 5
n_bins_y Optional[int]

number of bins per each Y dimension. Leave to None to use n_bins_x

 None
standardize bool

whether to standardize the data set

 True

estimate(x, y)

MI estimate.

bmi.estimators.InfoNCEEstimator

Bases: NeuralEstimatorBase

bmi.estimators.KDEMutualInformationEstimator

Bases: IMutualInformationPointEstimator

The kernel density mutual information estimator based on

\(I(X; Y) = h(X) + h(Y) - h(X, Y)\),

where \(h(X)\) is the differential entropy \(h(X) = -\mathbb{E}[ \log p(X) ]\).

The logarithm of probability density function \(\log p(X)\) is estimated via a kernel density estimator (KDE) using SciKit-Learn.

Note

This estimator is very sensitive to the choice of the bandwidth and the kernel. We suggest to treat it with caution.

__init__(kernel_xy='tophat', kernel_x=None, kernel_y=None, bandwidth_xy='scott', bandwidth_x=None, bandwidth_y=None, standardize=True)

Parameters:

Name Type Description Default
kernel_xy _AllowedKernel

kernel to be used for joint distribution PDF \(p_{XY}\) estimation. See SciKit-Learn's KernelDensity object for more information.

 'tophat'
kernel_x Optional[_AllowedKernel]

kernel to be used for the :math:p_X estimation. If None (default), kernel_xy will be used.

 None
kernel_y Optional[_AllowedKernel]

similarly to kernel_x.

 None
bandwidth_xy _AllowedBandwith

kernel bandwidth to be used for joint distribution estimation.

 'scott'
bandwidth_x Optional[_AllowedBandwith]

kernel bandwidth to be used for \(p_X\) estimation. If set to None (default), then bandwidth_xy is used.

 None
bandwidth_y Optional[_AllowedBandwith]

similar to bandwidth_x

 None
standardize bool

whether to standardize the data points

 True

estimate_entropies(x, y)

Calculates differential entropies.

Note

Differential entropy is not invariant to standardization. In particular, if you want to estimate differential entropy of the original variables, you should use standardize=False.

bmi.estimators.KSGEnsembleFirstEstimator

Bases: IMutualInformationPointEstimator

An implementation of of the neighborhood-based KSG estimator.

We use the first approximation (i.e., equation (8) in the paper) and allow for using different neighborhood sizes. The final estimate is the average of the estimates using different neighborhood sizes.

__init__(neighborhoods=(5, 10), standardize=True, metric_x='euclidean', metric_y=None, n_jobs=1, chunk_size=10)

Parameters:

Name Type Description Default
neighborhoods Sequence[int]

sequence of positive integers, specifying the size of neighborhood for MI calculation

 (5, 10)
standardize bool

whether to standardize the data before MI calculation, by default true

 True
metric_x _AllowedContinuousMetric

metric on the X space

 'euclidean'
metric_y Optional[_AllowedContinuousMetric]

metric on the Y space. If None, metric_x will be used

 None
n_jobs int

number of jobs to be launched to compute distances. Use -1 to use all processors.

 1
chunk_size int

internal batch size, used to speed up the computations while fitting into the memory

 10
Note

If you use Chebyshev (\(\l_\infty\)) distance for both \(X\) and \(Y\) spaces, KSGChebyshevEstimator may be faster.

bmi.estimators.MINEEstimator

Bases: IMutualInformationPointEstimator

trained_critic: Optional[eqx.Module] property

Returns the critic function from the end of the training.

Note
  1. You need to train the model by estimating mutual information, otherwise None is returned.
  2. Note that the critic can have different meaning depending on the function used.

bmi.estimators.NWJEstimator

Bases: NeuralEstimatorBase