| class NestedSampler( object ) | Source |
|---|
Nested Sampling is a novel technique to do Bayesian calculations.
Nested Sampling calculates the value of the evidence while it simultaneously produces samples from the posterior probability distribution of the parameters, given a Model and a dataset ( x-values, y-values and optionally weights ). The samples are collected in a SampleList.
NestedSampler is constructed according to the ideas of John Skilling and David MacKay ( ref TBD ).
Internally it contains an ensemble ( by default: 100 ) of trial samples, called walkers. Initially they are randomly distributed over the space available to the parameters. This randomness is distributed according to the prior distribution of the parameters. In most simple cases it will be uniform.
In an iterative process, the sample with the lowest likelihood is selected and replaced by a copy of one of the others. The parameters of the copy are randomly walked around under the condition that its likelihood stays larger than the selected lowest likelihood. A new independent trial sample is constructed. The original lowest sample is placed, appropriately weighted, into the SampleList for output.
This way the likelihood is climbed until the maximum is found. Along the way the integral of the likelihood function is calculated, which is equal to the evidence for the model, given the dataset.
There are different likelihood functions available: Gaussian (Normal), Laplace (Norm-1), Uniform (Norm-Inf), Poisson (for counting processes), Bernoulli (for categories), Cauchy (aka Lorentz) and Exponential (generalized Gaussian). A mixture of ErrorDistributions can also be defined. By default the Gaussian distribution is used.
Except for Poisson and Bernoulli , all others are ScaledErrorDistributions. The scale is a HyperParameter which can either be fixed, by setting the attribute errdis.scale or optimized, given the model parameters and the data. By default it is optimized for Gaussian and Laplace distributions. The prior for the noise scale is a JeffreysPrior.
The Exponential also has a power as HyperParameter which can be fixed or optimized. Its default is 2 (==Gaussian).
For the randomization of the parameters within the likelihood constraint so-called engines are written. By default only engines 1 and 2 is switched on.
-
GalileanEngine.
It walks all (super)parameters in a fixed direction for about 5 steps.
When a step ends outside the high likelihood region the direction is
mirrored on the lowLikelihood edge and continued. -
ChordEngine.
It draws a randomly oriented line through a point inside the region,
until it reaches outside the restricted likelihood region on both sides.
A random point is selected on the line until it is inside the likelihood region.
This process runs several times -
GibbsEngine.
It moves each of the parameters by a random step, one at a time.
It is a randomwalk. -
StepEngine.
It moves all parameters in a random direction. It is a randomwalk.
For dynamic models 2 extra engines are defined
-
BirthEngine.
It tries to increase the number of parameters.
It can only be switched on for Dynamic Models. -
DeathEngine.
It tries to decrease the number of parameters.
It can only be switched on for Dynamic Models.
For modifiable models 1 engine is defined.
- StructureEngine.
It alters the internal structure of the model.
It can only be switched on for Modifiable Models.
Attributes
- xdata : array_like
array of independent input values - model : Model
the model function to be fitted - ydata : array_like
array of dependent (to be fitted) data - weights : array_like (None)
weights pertaining to ydata - problem : Problem (ClassicProblem)
to be solved (container of model, xdata, ydata and weights) - distribution : ErrorDistribution
to calculate the loglikelihood - ensemble : int (100)
number of walkers - discard : int (1)
number of walkers to be replaced each generation - rng : RandomState
random number generator - seed : int (80409)
seed of rng - rate : float (1.0)
speed of exploration - maxsize : None or int
maximum size of the resulting sample list (None : no limit) - minimumIterations : int (100)
minimum number of iterations (adapt when starting problems occur) - end : float (2.0)
stopping criterion - tolerance : float (-12)
stopping criterion: stop if log( dZ / Z ) < tolerance - verbose : int
level of blabbering - walkers : WalkerList
ensemble of walkers that explore the likelihood space - samples : SampleList
Samples resulting from the exploration - engines : list of Engine
Engine that move the walkers around within the given constraint: logL > lowLogL - initialEngine : Engine
Engine that distributes the walkers over the available space - restart : StopStart (TBW)
write intermediate results to (optionally) start from.
Author Do Kester.
| NestedSampler( xdata=None, model=None, ydata=None, weights=None, accuracy=None, problem=None, distribution=None, limits=None, keep=None, ensemble=ENSEMBLE, discard=1, seed=80409, rate=RATE, bestBoost=False, engines=None, maxsize=None, threads=False, verbose=1 ) |
|---|
Create a new class, providing inputs and model.
Either (model,xdata,ydata) needs to be provided or a completely filled problem.
Parameters
- xdata : array_like
array of independent input values - model : Model
the model function to be fitted
the model needs priors for the parameters and (maybe) limits - ydata : array_like
array of dependent (to be fitted) data - weights : array_like (None)
weights pertaining to ydata - accuracy : float or array_like
accuracy scale for the datapoints
all the same or one for each data point - problem : None or string or Problem
Defines the kind of problem to be solved.
None same as "classic"
"classic" ClassicProblem
"errors" ErrorsInXandYProblem
"multiple" MultipleOutputProblem
Problem Externally defined Problem. When Problem has been provided,
xdata, model, weights and ydata are not used. - keep : None or dict of {int:float}
None of the model parameters are kept fixed.
Dictionary of indices (int) to be kept at a fixed value (float).
Hyperparameters follow model parameters.
The values will override those at initialization.
They are used in this instantiation, unless overwritten at the call to sample() - distribution : None or String or ErrorDistribution
Defines the ErrorDistribution to be used
When the hyperpar(s) are not to be kept fixed, they needPriorand maybe limits.
None same as "gauss"
"gauss" GaussErrorDistribution with (fixed) scale equal to 1.0
"laplace" LaplaceErrorDistribution with 1 hyperpar scale
"poisson" PoissonErrorDistribution no hyperpar
"cauchy" CauchyErrorDstribution with 1 hyperpar scale
"uniform" UniformErrorDistribution with 1 hyperpar scale
"bernoulli" BernoulliErrorDistribution no hyperpar
"exponential" ExponentialErrorDistribution with 2 hyperpar (scale, power)
ErrorDistribution Externally defined ErrorDistribution - limits : None or [low,high] or [[low],[high]]
None no limits implying fixed hyperparameters of the distribution
low low limit on hyperpars
high high limit on hyperpars
When limits are set the hyperpars are not fixed. - ensemble : int
number of walkers - discard : int
number of walkers to be replaced each generation - seed : int
seed of random number generator - rate : float
speed of exploration - bestBoost : bool or Fitter
False no updates of best logLikelihood
True boost the fit using LevenbergMarquardtFitter
fitter boost the fit using this fitter. - engines : None or (list of) string or (list of) Engine
to randomly move the walkers around, within the likelihood bound.
None use a Problem defined selection of engines
"galilean" GalileanEngine move forward and mirror on edges
"chord" ChordEngine select random point on random line
"gibbs" GibbsEngine move one parameter at a time
"step" StepEngine move all parameters in arbitrary direction
For Dynamic models only
"birth" BirthEngine increase the parameter list of a walker by one
"death" DeathEngine decrease the parameter list of a walker by one
For Modifiable models only
"struct" StructureEngine change the (internal) structure.
Engine an externally defined (list of) Engine - maxsize : None or int
maximum size of the resulting sample list (None : no limit) - threads : bool (False)
Use Threads to distribute the diffusion of discarded samples over the available cores. - verbose : int (1)
0 silent
1 basic information
2 more about every 100th iteration
3 more about every iteration
>4 for debugging
| sample( keep=None, plot=False ) |
|---|
A additional result of this method is a SampleList which contains samples taken from the posterior distribution.
Parameters
- keep : None or dict of {int:float}
Dictionary of indices (int) to be kept at a fixed value (float)
Hyperparameters follow model parameters
The values will override those at initialization.
They are only used in this call of fit. - plot : bool or str
bool show a plot of the final results
"iter" show iterations
"all" show iterations and final result
"last" show final result
"test" plot iterations but dont show (for testing)
| initSample( ensemble=None, keep=None ) |
|---|
| walkerLogL( w ) |
|---|
| makeFitlist( keep=None ) |
|---|
Parameters
- keep : None or dict of {int : float}
dictionary of indices that need to be kept at the float value.
| doIterPlot( plot ) |
|---|
int 0 no plot
1 plot
2 plot but dont show (for testing)
| doLastPlot( plot ) |
|---|
True when the last plot is requested (plot equals 'last' or 'all' or True) False otherwise
| initReport( keep=None ) |
|---|
| iterReport( kw, tail, plot=False ) |
|---|
| printIterRep( kw, parfmt="%s", tail=0, max=None, indent=0, end="\n" ) |
|---|
print( fmt( pl, max=None ) )
npars = self.walkers[kw].problem.npars
pmn, pmx = self.phantoms.getParamMinmax( self.lowLhood, npars )
print( fmt( pmn, max=None ) )
print( fmt( pmx, max=None ) )
print( fmt( np ), fmt( self.phantoms.length( npars ) ),
fmt( self.logdZ, format="%10.3g" ) )
for ky in self.phantoms.phantoms
print( fmt( ky ), fmt( len( self.phantoms.phantoms[ky] ) ) )
| lastReport( kw, plot=False ) |
|---|
| plotLast( ) |
|---|
| getMaxIter( ) |
|---|
| nextIteration( ) |
|---|
| optionalRestart( ) |
|---|
| optionalSave( ) |
|---|
| updateEvidence( worst ) |
|---|
The walkers need to be sorted to logL
Parameters
- worst : int
Number of walkers used in the update
| copyWalker( worst ) |
|---|
Parameters
- worst : int
number of Walkers to copy
| copyWalkerFromPhantoms( worst ) |
|---|
Parameters
- worst : int
number of Walkers to copy
| copyWalkerFromDynamicPhantoms( worst ) |
|---|
Parameters
- worst : int
number of Walkers to copy
| updateWalkers( explorer, worst ) |
|---|
Parameters
- explorer : Explorer
Explorer object - worst : int
number of walkers to update
| setProblem( name, model=None, xdata=None, ydata=None, weights=None, accuracy=None ) |
|---|
If name is a Problem, then the keyword arguments (xdata,model,ydata,weights) are overwritten provided they are not None
Parameters
- name : string or Problem
name of problem
Problem Use this one - model : Model
the model to be solved - xdata : array_like or None
independent variable - ydata : array_like or None
dependent variable - weights : array_like or None
weights associated with ydata - accuracy : float or array_like or None
of the data
| setErrorDistribution( name=None, limits=None, scale=1.0, power=2.0 ) |
|---|
Parameters
- name : None or string or ErrorDistribution
None distribution is Problem dependent.
string name of distribution; one of
"gauss", "laplace", "poisson", "cauchy", "uniform",
"exponential", "gauss2d", "model", "bernoulli"
ErrorDistribution use this one - limits : None or [low,high] or [[low],[high]]
None no limits implying fixed hyperparameters (scale,power,etc)
low low limit on hyperpars (needs to be >0)
high high limit on hyperpars
when limits are set, the hyperpars are not fixed. - scale : float
fixed scale of distribution - power : float
fixed power of distribution
| setEngines( engines=None, enginedict=None ) |
|---|
Parameters
- engines : None or [Engine] or [string]
None engines is Problem dependent
[Engines] list of Engines to use
[string] list engine names - enginedict : dictionary of { str : Engine }
connecting names to the Engines
| setInitialEngine( ensemble, allpars, fitIndex, startdict=None ) |
|---|
Parameters
-
ensemble : int
length of the walkers list -
TBC : remove allpars and fitIndex
-
allpars : array_like
array of (hyper)parameters -
fitIndex : array_like
indices of allpars to be fitted -
startdict : dictionary of { str : Engine }
connecting a name to a StartEngine
| initWalkers( ensemble, allpars, fitIndex, startdict=None ) |
|---|
Parameters
- ensemble : int
length of the walkers list - allpars : array_like
array of (hyper)parameters - fitIndex : array_like
indices of allpars to be fitted - startdict : dictionary of { str : Engine }
connecting a name to a StartEngine
| plotResult( walker, iter, plot=0 ) |
|---|
Parameters
- walker : Walker
the walker to plot - iter : int
iteration number - plot : int (one of (0,1,2)
0 immediate return, no action
1 plot
2 plot but dont show (for testing)
| report( ) |
|---|