| class NestedSampler( object ) | [source] |
|---|
Nested Sampling is a novel technique to do Bayesian calculations.
Nested Sampling calculates the value of the evidence while it simultaneously produces samples from the posterior probability distribution of the parameters, given a Model and a dataset ( x-values, y-values and optionally weights ). The samples are collected in a SampleList.
NestedSampler is constructed according to the ideas of John Skilling and David MacKay ( ref TBD ).
Internally it contains an ensemble ( by default: 100 ) of trial samples, called walkers. Initially they are randomly distributed over the space available to the parameters. This randomness is distributed according to the prior distribution of the parameters. In most simple cases it will be uniform.
In an iterative process, the sample with the lowest likelihood is selected and replaced by a copy of one of the others. The parameters of the copy are randomly walked around under the condition that its likelihood stays larger than the selected lowest likelihood. A new independent trial sample is constructed. The original lowest sample is placed, appropriately weighted, into the SampleList for output.
This way the likelihood is climbed until the maximum is found. Along the way the integral of the likelihood function is calculated, which is equal to the evidence for the model, given the dataset.
There are different likelihood functions available: Gaussian (Normal), Laplace (Norm-1), Uniform (Norm-Inf), Poisson (for counting processes), Bernoulli (for categories), Cauchy (aka Lorentz) and Exponential (generalized Gaussian). A mixture of ErrorDistributions can also be defined. By default the Gaussian distribution is used.
Except for Poisson and Bernoulli , all others are ScaledErrorDistributions. The scale is a HyperParameter which can either be fixed, by setting the attribute errdis.scale or optimized, given the model parameters and the data. By default it is optimized for Gaussian and Laplace distributions. The prior for the noise scale is a JeffreysPrior.
The Exponential also has a power as HyperParameter which can be fixed or optimized. Its default is 2 (==Gaussian).
For the randomization of the parameters within the likelihood constraint so-called engines are written. By default only engines 1 and 2 is switched on.
-
GalileanEngine.
It walks all (super)parameters in a fixed direction for about 5 steps.
When a step ends outside the high likelihood region the direction is
mirrored on the lowLikelihood edge and continued. -
ChordEngine.
It draws a randomly oriented line through a point inside the region,
until it reaches outside the restricted likelihood region on both sides.
A random point is selected on the line until it is inside the likelihood region.
This process runs several times -
GibbsEngine.
It moves each of the parameters by a random step, one at a time.
It is a randomwalk. -
StepEngine.
It moves all parameters in a random direction. It is a randomwalk.
For dynamic models 2 extra engines are defined
-
BirthEngine.
It tries to increase the number of parameters.
It can only be switched on for Dynamic Models. -
DeathEngine.
It tries to decrease the number of parameters.
It can only be switched on for Dynamic Models.
For modifiable models 1 engine is defined.
- StructureEngine.
It alters the internal structure of the model.
It can only be switched on for Modifiable Models.
Attributes
- xdata : array_like
array of independent input values - model : Model
the model function to be fitted - ydata : array_like
array of dependent (to be fitted) data - weights : array_like (None)
weights pertaining to ydata - problem : Problem (ClassicProblem)
to be solved (container of model, xdata, ydata and weights) - distribution : ErrorDistribution
to calculate the loglikelihood - limits : None or [low,high]
limits to the hyperparameter(s) of the distribution - keep : None or dist of {int:float}
Indices of parameters that are kept fixed at the given value. - ensemble : int (100)
number of walkers - discard : int (1)
number of walkers to be replaced each generation - seed : int (80409)
seed of the Random number generator (rng) - rate : float (1.0)
speed of exploration - bestBoost : bool or Fitter (False) (deprecated)
IGNORED with Warning: It causes an erroneous evidence calculation - usePhantoms : bool (True)
Find starting point in the Phantoms (Walkers if False) - engines : list of Engine
Engine that move the walkers around within the given constraint: logL > lowLogL - maxsize : None or int
maximum size of the resulting sample list (None : no limit) - threads : bool ( False)
Use threads (only when discard > 1) - verbose : int
level of blabbering - repiter : int (100)
report every reiter iteration (when verbose=2) - minimumIterations : int (100)
minimum number of iterations (adapt when starting problems occur) - end : float (2.0)
stopping criterion - tolerance : float (-20)
stopping criterion: stop if log( dZ / Z ) < tolerance - rng : RandomState
random number generator - walkers : WalkerList
ensemble of walkers that explore the likelihood space - samples : SampleList
Samples resulting from the exploration - initialEngine : Engine
Engine that distributes the walkers over the available space - restart : StopStart (TBW)
write intermediate results to (optionally) start from.
Author Do Kester.
| NestedSampler( xdata=None, model=None, ydata=None, weights=None, accuracy=None, problem=None, distribution=None, limits=None, keep=None, ensemble=ENSEMBLE, discard=1, seed=80409, rate=RATE, bestBoost=False, usePhantoms=True, engines=None, maxsize=None, threads=False, verbose=1 ) | [source] |
|---|
Create a new class, providing inputs and model.
Either (model,xdata,ydata) needs to be provided or a completely filled problem.
Parameters
- xdata : array_like
array of independent input values - model : Model
the model function to be fitted
the model needs priors for the parameters and (maybe) limits - ydata : array_like
array of dependent (to be fitted) data - weights : array_like (None)
weights pertaining to ydata - accuracy : float or array_like
accuracy scale for the datapoints
all the same or one for each data point - problem : None or string or Problem
Defines the kind of problem to be solved.
None same as "classic"
"classic" ClassicProblem
"errors" ErrorsInXandYProblem
"multiple" MultipleOutputProblem
Problem Externally defined Problem.
When Problem has been provided, xdata, model, weights
and ydata are not used. - keep : None or dict of {int:float}
None of the model parameters are kept fixed.
Dictionary of indices (int) to be kept at a fixed value (float).
Hyperparameters follow model parameters.
The values will override those at initialization.
They are used in this instantiation, unless overwritten at the call to sample() - distribution : None or String or ErrorDistribution
Defines the ErrorDistribution to be used
When the hyperpar(s) are not to be kept fixed, they needPriorand maybe limits.
None same as "gauss"
"gauss" GaussErrorDistribution with (fixed) scale equal to 1.0
"laplace" LaplaceErrorDistribution with 1 hyperpar scale
"poisson" PoissonErrorDistribution no hyperpar
"cauchy" CauchyErrorDstribution with 1 hyperpar scale
"uniform" UniformErrorDistribution with 1 hyperpar scale
"bernoulli" BernoulliErrorDistribution no hyperpar
"exponential" ExponentialErrorDistribution with 2 hyperpar (scale, power)
ErrorDistribution Externally defined ErrorDistribution - limits : None or [low,high] or [[low],[high]]
None no limits implying fixed hyperparameters of the distribution
low low limit on hyperpars
high high limit on hyperpars
When limits are set the hyperpars are not fixed. - ensemble : int
number of walkers - discard : int
number of walkers to be replaced each generation - seed : int
seed of random number generator - rate : float
speed of exploration - bestBoost : bool or Fitter (False) (deprecated)
IGNORED with Warning: It causes an erroneous evidence calculation - usePhantoms : bool (True)
True Copy starting walkers from phantoms
False Copy starting walkers from walkers - engines : None or (list of) string or (list of) Engine
to randomly move the walkers around, within the likelihood bound.
None use a Problem defined selection of engines
"galilean" GalileanEngine move forward and mirror on edges
"chord" ChordEngine select random point on random line
"gibbs" GibbsEngine move one parameter at a time
"step" StepEngine move all parameters in arbitrary direction
For Dynamic models only
"birth" BirthEngine increase the parameter list of a walker by one
"death" DeathEngine decrease the parameter list of a walker by one
For Modifiable models only
"struct" StructureEngine change the (internal) structure.
Engine an externally defined (list of) Engine - maxsize : None or int
maximum size of the resulting sample list (None : no limit) - threads : bool (False)
Use Threads to distribute the diffusion of discarded samples over the available cores. - verbose : int (1)
0 silent
1 basic information
2 more about every 100th iteration
3 more about every iteration
>4 for debugging
| sample( keep=None, plot=False, **kwargs ) | [source] |
|---|
A additional result of this method is a SampleList which contains samples taken from the posterior distribution.
Parameters
- keep : None or dict of {int:float} (None)
Dictionary of indices (int) to be kept at a fixed value (float)
Hyperparameters follow model parameters
The values will override those at initialization.
They are only used in this call of fit. - plot : bool or str (False)
bool show a plot of the final results
"iter" show iterations
"all" show iterations and final result
"last" show final result
"test" plot iterations but dont show (for testing) - kwargs : dict
to be fed to the plot
| initSample( ensemble=None, keep=None ) | [source] |
|---|
Parameters
- ensemble : int or None
size of ensemble of walkers. None slects self.ensemble - keep : None or dict of {int:float}
Dictionary of indices (int) to be kept at a fixed value (float)
Hyperparameters follow model parameters
The values will override those at initialization.
They are only used in this call of fit.
| walkerLogL( w ) | [source] |
|---|
Parameters
- w : Walker
to get the logL from
| makeFitlist( keep=None ) | [source] |
|---|
Parameters
- keep : None or dict of {int : float}
dictionary of indices that need to be kept at the float value.
| initReport( keep=None ) | [source] |
|---|
Parameters
- keep : dict or None
dict of parameters to keep fixed.
| iterReport( kw, tail ) | [source] |
|---|
Parameters
- kw : int
walker index with low LogL - tail : int
number of params at the end to report (mostly hyperpars)
| printIterRep( kw, parfmt="%s", tail=0, max=None, indent=0, end="\n" ) | [source] |
|---|
Parameters
- kw : int
walker index to report - parfmt : format ("%s")
to print the parameters - tail : int
nuber of tail pars to print. (see Formatter) - max : int or None
maximum nr of pars to print. (see Formatter) - indent : int
number of initial spaces. (see Formatter)
| lastReport( kw, **kwargs ) | [source] |
|---|
Parameters
- kw : int
index of (last) walker. - kwargs : dict
for plotter
| setPlotters( plot ) | [source] |
|---|
| plot | doIterPlot | doLastPlot | comment |
|---|---|---|---|
| False | no plot | no plot | default; dont plot |
| True | no plot | plotSampleList | |
| "last" | no plot | plotSampleList | |
| "iter" | plotWalker | no plot | |
| "all" | plotWalker | plotSampleList | |
| "test" | plotWalker | plotSampleList | no show(), for tests |
The plot methods plotWalker and plotSampleList are part of the module Plotter.py
Parameters
- plot : str or bool (False)
as in table
| plotNot( kw, show=False ) | [source] |
|---|
| plotIter( kw, show=False ) | [source] |
|---|
| plotLast( kw, show=False, **kwargs ) | [source] |
|---|
| nextIteration( ) | [source] |
|---|
| optionalRestart( ) | [source] |
|---|
| optionalSave( ) | [source] |
|---|
| updateEvidence( worst ) | [source] |
|---|
The walkers need to be sorted to logL
Parameters
- worst : int
Number of walkers used in the update
| unitDomain( ) | [source] |
|---|
| copyWalker( worst ) | [source] |
|---|
Parameters
- worst : int
number of Walkers to copy
| copyWalkerFromPhantoms( worst ) | [source] |
|---|
Parameters
- worst : int
number of Walkers to copy
| updateWalkers( explorer, worst ) | [source] |
|---|
Parameters
- explorer : Explorer
Explorer object - worst : int
number of walkers to update
| setProblem( name, model=None, xdata=None, ydata=None, weights=None, accuracy=None ) | [source] |
|---|
If name is a Problem, then the keyword arguments (xdata,model,ydata,weights) are overwritten provided they are not None
Parameters
- name : string or Problem
name of problem
Problem Use this one - model : Model
the model to be solved - xdata : array_like or None
independent variable - ydata : array_like or None
dependent variable - weights : array_like or None
weights associated with ydata - accuracy : float or array_like or None
of the data
| setErrorDistribution( name=None, limits=None, scale=1.0, power=2.0 ) | [source] |
|---|
Parameters
- name : None or string or ErrorDistribution
None distribution is Problem dependent.
string name of distribution; one of
"gauss", "laplace", "poisson", "cauchy", "uniform",
"exponential", "gauss2d", "model", "bernoulli"
ErrorDistribution use this one - limits : None or [low,high] or [[low],[high]]
None no limits implying fixed hyperparameters (scale,power,etc)
low low limit on hyperpars (needs to be >0)
high high limit on hyperpars
when limits are set, the hyperpars are not fixed. - scale : float
fixed scale of distribution - power : float
fixed power of distribution
| setEngines( engines=None, enginedict=None ) | [source] |
|---|
Parameters
- engines : None or [Engine] or [string]
None engines is Problem dependent
[Engines] list of Engines to use
[string] list engine names - enginedict : dictionary of { str : Engine }
connecting names to the Engines
| setInitialEngine( ensemble, allpars, fitIndex, startdict=None ) | [source] |
|---|
Parameters
-
ensemble : int
length of the walkers list -
TBC : remove allpars and fitIndex
-
allpars : array_like
array of (hyper)parameters -
fitIndex : array_like
indices of allpars to be fitted -
startdict : dictionary of { str : Engine }
connecting a name to a StartEngine
| initWalkers( ensemble, allpars, fitIndex, startdict=None ) | [source] |
|---|
Parameters
- ensemble : int
length of the walkers list - allpars : array_like
array of (hyper)parameters - fitIndex : array_like
indices of allpars to be fitted - startdict : dictionary of { str : Engine }
connecting a name to a StartEngine
| report( ) | [source] |
|---|