BISP6 - Sixth Workshop on Bayesian Inference in Stochastic Processes
BISP6
Sixth Workshop on
BAYESIAN INFERENCE IN STOCHASTIC PROCESSES
Accademia Cusano, Bressanone/Brixen (BZ), Italy
June, 18-20, 2009
We examine the complexity of MCMC algorithms in high
dimensions. Earlier works in the literature have shown
that the step of Random-Walk Metropolis should be scaled
as 1/n (n being the dimensionality index); for the (Metropolised)
Langevin algorithm the scaling is 1/n^{1/3}. Both results
have been theoretically justified in the simplified scenario
of iid targets. We consider the so-called `hybrid Monte-Carlo'
MCMC algorithm used by physicists in molecular dynamics and other
applications.
Bridging the machinery employed above with tools from numerical analysis
we show that the MCMC trajectory of the hybrid algorithm
converges (when appropriately rescaled) to
a hypoelliptic SDE. Such a result provides a complete characterization
of the efficiency of the algorithm: we conclude that
the hybrid algorithm should be scaled as 1/n^{1/6}, with optimal
acceptance probability 0.743.
|
Paul G. Blackwell, Catlin E. Buck, L. A. Collins, R. Rothlisberger
and J. J. Wheatley
|
Environmental signals `trapped' in polar ice, such as atmospheric levels
of particular chemicals or isotopes, offer valuable information about past
climates. These signals can be measured from ice cores, but their
interpretation requires knowledge of the relationship between the depth within
the core and the actual age of the ice. One approach exploits the fact that
some signals have strong annual cycles, and so counting these cycles|`layer
counting'|can in principle give a precise date. In practice, layer counting
is currently done by eye, based on plotting the signal level against depth,
with no formal measure of uncertainty. In this paper, we describe a statistical approach which models the underlying signal as a sine wave of varying
level and amplitude. Measurements from the ice core are modelled as a sig nal
of this form plus errors coming from an autocorrelated continuous-time
(Ornstein-Uhlenbeck) process. The time-depth relationship is modelled as a
piecewise linear function. Bayesian analysis of this model, using reversible
jump Markov chain Monte Carlo, allows inference about the age-depth rela-
tionship, and specifically the number of years covered by a particular section
of ice core, including quantitative assessment of the uncertainty involved.
Kernel-based models offer a flexible and descriptive framework for
studying space-time processes. Nonstationary and anisotropic
covariance structures can be readily accommodated by allowing kernel
parameters to vary over space and time. In addition, dimension
reduction strategies make model fitting computationally feasible for
large datasets. We explore various properties of this class of
statistical models, as well as the implications of dimension reduction
strategies on these properties. In addition, we illustrate the use of
one version of these models in a study of carbonaceous aerosols over
mainland Southeast Asia. Finally, efficient MCMC strategies are
introduced to further facilitate model fitting and comparison.
|
Fernanda D'Ippoliti and Sara Pasquali
|
We present a Bayesian approach to parameter estimation in a stochastic
volatility model with contemporaneous and correlated jumps (SVCCJ) starting
from time series of financial data.
In the present work, we develop a Bayesian estimation methodology for the
SVCCJ model formulated in D'Ippoliti et al. (2008) that can be applied
to other stochastic volatility jump-diffusion processes. The asset pricing
model presented in D'Ippoliti et al. (2008) describes the evolution of the
underlying asset by a SDE with jumps and two diffusion terms; the first term
with constant volatility and the second one of Heston type. The dynamics
of the volatility follows a square-root process with jumps.
The choice of the value of parameters is a critical point in the specification
of a model: in financial setting, a good estimation of the parameters of the
model is an essential requirement for a good valuation of asset prices as well.
In financial market, prices are recorded at discrete times. It follows that the
problem of parameter estimation from time series can be brought back to
parametric inference for discretely observed diffusion processes with jumps.
While values of the underlying are available on the market, volatility values
are not given. Hence, we consider volatility values as missing data. Moreover,
if the number of observed underlying prices is small, it is necessary to
generate latent data between two consecutive observations of the underlying in
order to obtain reliable estimates for parameters. Volatility values and latent
observations for the underlying are considered as missing data and an MCMC
method is applied to simulate them.
The Bayesian method used to estimate parameters can be summarized as follows:
(i) draw a value for each parameter from its prior distribution, (ii) generate
missing data given the current value of the parameters, (iii) sample a value
for each parameter from its posterior distribution. One iteration of the Markov
chain is completed when missing data and parameters are updated. Applying
recursively the procedure, we obtain a sequence of values for each parameter
giving an approximation of its posterior distribution.
In recent years, the problem of parameter estimation in a Bayesian setting
for stochastic volatility models has been widely studied. Among others,
Eraker (2001) and Durham and Gallant (2002) consider both volatility and
underlying prices as latent data. To the best of our knowledge, the issue of
parameter estimation in stochastic volatility with jumps model has not been
addressed in this way. Asgharian and Bengtsson (2006), Eraker and al. (2003),
and Nakajima and Omori (in press) analyze the problem of parameter estimation
using a large number of observations and only volatility values are missing
data.
Starting from the MCMC algorithms proposed by Eraker et al. (2003) and Durham
and Gallant (2002) we extend the procedure to stochastic volatility models
with jumps. Moreover, some numerical results are provided to illustrate the
procedure.
REFERENCES
1.Asgharian H., and Bengtsson C., 2006, Jump Spillover in International
Equity Markets, Journal of Financial Econometrics, Vol. 4, No. 2, 167-203.
2.D'Ippoliti F., Moretto E., Pasquali S., and Trivellato S., 2008,
The effect of stochastic volatility and jumps on derivative pricing,
submitted.
3.Durham G. B., and Gallant A. R., 2002, Numerical Techniques for Maximum
Likelihood Estimation of Continuous-time Diffusion Processes, The Journal
of Business and Economic Statistics, Vol. 20, 297-316.
4.Eraker B., 2001, MCMC Analysis of Diffusion Models with Application to
Finance Journal of Business and Economic Statistics, Vol. 19-2 (April),
177-191.
5.Eraker B., Johannes M., and Polson N., 2003, The Impact of Jumps in
Volatilty and Returns, The Journal of Finance, Vol. LVIII, No. 3, June,
1269-1300.
6.Nakajima J., and Omori Y., 2007, Leverage, heavy-tails and correlated
jumps in a stochastic volatility models, in press.
|
Stefano Favaro, Alessandra Guglielmi and Stephen G. Walker
|
We define and investigate a new class of measure-valued Markov chains having
as unique invariant measure the law of a Dirichlet process. This class of
Markov chains includes as a particular case the well known Markov chain
introduced by Feigin and Tweedie (1989), which has been widely used in order
to characterize linear functionals of the Dirichlet process and to provide
approximation procedures for estimating the law of the mean of a Dirichlet
process.
|
Tom Fricker and Jeremy Oakley
|
An emulator is a statistical surrogate for an expensive computer model, used
to obtain a fast probabilistic prediction of the output. Emulators can be
constructed by considering the input-output relationship as an unknown
function and modelling the uncertainty using a Gaussian process prior on
the function.
If the computer model produces multiple outputs, the emulator must capture
two types of correlation: correlation over the input space, and correlation
between different outputs. We show that the usual mathematically-convenient
approach of treating the two types as separable can result in misspecified
input-space correlation functions for some or all of the outputs. We propose
an emulator with a nonseparable covariance, based on the linear model of
coregionalization (LMC) taken from the geostatistical literature. By allowing
different outputs to have different correlation functions, the LMC emulator can
provide better estimates of prediction uncertainty across the outputs. The
advantages of the LMC over a separable structure are demonstrated in the
emulation of a simple climate model.
|
Sylvia Frühwirth-Schnatter and Helga Wagner
|
State space models are widely used in time series analysis to deal with
processes which gradually change over time. Model specification, however, is
a difficult task as one has to decide first, which components to include
into the model, and second, whether these components are fixed or stochastic.
Using a Bayesian approach, one could determine the posterior probabilities of
each model separately, which requires estimation of the marginal likelihood for
each model by some numerical method. A modern approach to Bayesian model
selection is to apply some model space MCMC methods by sampling jointly model
indicators and parameters as is done, e.g., in the stochastic variable
selection approach for regression models.
In this talk we discuss model space MCMC for state space models. To this aim,
we rewrite the state space model in a non-centered version and extend the
stochastic variable selection approach to state space models. This allows to
choose appropriate components and to decide, if these components are
deterministic or stochastic. Details will be provided for time-varying
parameter models and unobserved component time series models. The method is
extended to non-Gaussian state space models for binary, multinomial or count
data, where we make use of auxiliary mixture sampling.
We devise a sequential Monte Carlo method, via particle
learning (PL), for on-line sampling from the posterior distribution of
a sequential process over trees. The mixing problems which typically
plague MCMC approaches to similar Bayesian CART models are
circumvented by an automatic mixing of global (re-sample) and local
(propagate) rules -- the cornerstones of PL. We consider regression
trees with both constant and linear mean functions at the leaves, and
thereby index a thrifty but powerful family of nonparametric
regression models. We further exploit the on-line nature of inference
with extensions for sequential design (a.k.a. active learning) under
tree models for both (improved) surface prediction and global
optimization applications. In both cases, we demonstrate how our
proposed algorithms provide better results compared to higher-powered
methods but use a fraction of the computational cost.
|
Peter Green and Fay Hosking
|
We propose a Bayesian modelling approach to the analysis of genome-wide
association studies based on single nucleotide polymorphism (SNP) data.
Our latent seed model combines various aspects of k-means clustering,
hidden Markov models (HMMs) and logistic regression into a fully
Bayesian model. It is fitted using Markov chain Monte Carlo methods,
with Metropolis-Hastings update steps. The approach is flexible, both in
allowing different types of genetic models, and because it can be easily
extended while remaining computationally feasible due to the use of fast
algorithms for HMMs. It allows for inference primarily on the location
of the causal locus and also on other parameters of interest. The latent
seed model is used here to analyse three data sets, using both synthetic
and real disease phenotypes with real SNP data, and shows promising
results. Our method is able to correctly identify the causal locus in
examples where single SNP analysis is both successful and unsuccessful
at identifying the causal SNP.
Financial data such as asset prices or stock indices have well-studied
features such as volatility clustering and heavy tails, which have been
traditionally modelled using stochastic volatility. However, it is
difficult to specify the dependence of the volatility. Initially,
stationary model were proposed. However, more recently there has been
interest in non-stationary models using long memory or Markov switching
processes. In this talk, we will use the framework of infinite
superpositions of shot-noise processes (also known as OU processes) to
give flexible specifications of the dependence structure and make
inference using nonparametric methods. This talk will cover modelling
and computational issues with applications to daily and high frequency data.
|
Timothy J. Heaton, Paul G. Blackwell and Caitlin E. Buck
|
For accurate radiocarbon dating, it is necessary to identify fluctuations in
the level of radioactive carbon 14C present in the atmosphere through time.
The processes underlying these variations are not understood and so a
data-based calibration curve is required.
In this talk we present a novel MCMC approach to the production of the
internationally agreed curve and the individual challenges involved. Our
methodology models the calibration data as noisy observations of a Wiener
process and updates sample paths through use of a Metropolis-within-Gibbs
algorithm.
Implementation of this algorithm is complicated by certain specific features
of the data used, namely that many data points:
We give an overview of these issues and discuss their implications for the
resulting sampler.
|
Amy H. Herring and David A. Savitz
|
In many applications, it is of interest to relate a longitudinal predictor
to a response variable. For example, one may obtain repeated ultrasound
measurements of fetal growth during pregnancy, and repeated growth
measurements in infancy, with interest focused on the relationship between
early growth and outcomes measured at birth and later in childhood development.
To characterize the growth trajectory, we propose a dynamic latent class
transition model, which generalizes infinite hidden Markov models to allow a
more flexible dependence structure. To build a flexible joint model for
prediction, we allow the response distribution to change nonparametrically
according to the time-varying state indicators, while favoring a sparse
structure. Efficient methods are developed for posterior computation relying
on slice sampling, and the methods are applied to data from an epidemiologic
study.
|
Chaitanya Joshi and Simon Wilson
|
Since, in real life most of the diffusion processes are observed only at
discrete time intervals not small enough, both Likelihood based and Bayesian
methods of inference become non-trivial. To overcome this problem Bayesian
inference is centred around introducing m latent data points between
every pair of observations. However it was shown that as mincreases,
one can make very precise inference about the diffusion coefficient of the
process via the quadratic variation. This dependence results in slow mixing
of the naive MCMC schemes and it worsens linearly as the amount of data
augmentation increases. Various different approaches have been proposed to
get around this problem. Some of them involve transforming the SDE, while
most others present innovative MCMC schemes.
We propose a new method to approximate Bayesian inference on the diffusion
process parameters. Our method is simple, computationally efficient, does
not involve any transformations, and is not based on the MCMC approach.
Principle features of this new method are Gaussian approximation proposed
by Durham and Gallant (2002) and a grid search to explore parameter space.
In this paper we first introduce our new method and then compare its
performance with recently proposed MCMC based schemes on several
diffusion processes.
|
Qiming Lv, Paul G. Backwell, Catlin Buck, M. Charles,
S. Colledge and G. Jones
|
Current archaeological wisdom views the spread of Neolithic agriculture
as a leap-frogging migration across a non-uniform landscape. To obtain holistic
insight into the rate and pattern of such spread, we model it as a
spatio-temporal process on an irregular network based on a Delaunay t
riangulation. The nodes in the network represent sites of statistical and
geographic importance, augmented by points chosen to ensure that the network
meets a minimum angle constraint. Edges indicate major corridors or geographical
boundaries, plus additional edges to complete the triangulation. Inference
about arrival times of the process combines the spatial structure of the
network with observed radiocarbon dates of cereal grains found at some sites
(using an existing Bayesian radiocarbon calibration framework). This novel
approach breaks down arrival times into travel times along edges, thus
explic itly disentangling the spatial dependence of the arrival time among
neighboring sites and allowing geographic information and environmental
conditions to enter as priors. Preliminary tests on European data show that
our model can reduce uncertainties on arrival times for nodes where data are
available and make predictions on the others. It therefore allows coherent
analysis of patterns and processes rather than isolated investigation of
individual sites.
|
Anandamayee Majumdar, Debashis Paul and Dianne Bautista
|
We propose a flexible class of nonstationary stochastic models for
multivariate spatial data. The method is based on a general version of
convolutions of spatially varying covariance kernels and produces
mathematically valid covariance structures. This method generalizes the
convolution approach suggested by Majumdar and Gelfand (2007) to extend
multivariate spatial covariance functions to the nonstationary case.
A Bayesian method for estimation of the parameters in the covariance
model based on a Gibbs sampler is proposed, and applied to simulated
data. Flexibility or robustness of the model is examined as well, using
simulations from several kinds of model-departures. Model comparison is
performed with the coregionalization model of Wackernagel (2003) which uses a
stationary bivariate model. Based on posterior prediction results, the
performance of our model is seen to be considerably better. Our model is then
applied to real data where the prediction coverage using this methods
results in significantly higher.
We will explore several possibilities of incorporating derivative
information in problems of calibration and validation of computer
models. A key ingredient in the analysis of computer
models is the specification of a statistical model relating the output
of the computer model and the reality it aims at reproducing, which
involves a discrepancy function. This idea was originally proposed by
Kennedy and O'Hagan (2001). We propose an alternative formulation of
this discrepancy function which involves the use of derivative
information of the computer model. This formulation is motivated by a
perturbation and linearization idea. Simulated examples are used to
assess the advantages and disadvantages of incorporating derivative
information in this class of problems.
In Lucantoni (1990) the term Batch Markovian Arrival process (BMAP) is
first used to describe the Versatile Markovian point process introduced
by Neuts (1979). In the BMAP arrivals are allowed to occur in batches
where different types of arrivals can have different batch size
distribution. The MAP is a special case of BMAP, where all batch sizes
are equal to one. The idea of a BMAP is to keep the tractability of the
Poisson arrival process but significantly generalizes it to allow the
inclusion of dependent interarrival times, non-exponential
interarrival-time distributions, and correlated batch sizes. This makes
the BMAP an effective and powerful Internet data traffic model: it is
able to capture dependence and correlation, one of the main features in
Internet-related data.
In this work we develop Bayesian inference for the 2-states MAP, where
times of arrivals depend on the current state of a hidden underlying
Markov Process. In practice, only the times when arrivals occur are
observed, and neither the values of the states at these time points or
the transient changes are available. We base our approach on a
Metropolis-Hastings scheme, where the likelihood function is derived
from what we define as the Effective Markovian Arrival process. We will
illustrate our methodology with simulated and a real teletraffic data
set. We will also delineate some ideas concerning the identifiability of
the MAP.
|
Gareth O. Roberts, Yvo Pokern, Omiros Papaspiliopoulos and Andrew Stuart
|
This presentation will describe recent progress on Bayesian non-parametric
analysis of diffusion drift functions given continuous data on a finite time
interval. It turns out that Gaussian processes can be used as conjugate priors,
and we describe methodology for characterising posterior mean and covariance
structure in terms of solutions to differential equations with coefficients
given as functions of the observed diffusion local time.
|
Abel Rodriguez and David B. Dunson
|
Constructing flexible models for stochastic processes is an important
challenge in a number of settings. We describe a novel class of Bayesian
nonparametric priors based on stick-breaking constructions where the
weights of the process are constructed as probit transformations of normal
random variables. We show that these priors are extremely flexible,
allowing us to generate a great variety of models while preserving
computational simplicity. Particular emphasis is placed on the
construction of rich temporal and spatial processes, which are applied to
two problems in finance and ecology.
|
Esther Salazar, Hedibert F. Lopes and Dani Gamerman
|
A new class of space-time models derived from standard dynamic factor models
is proposed. The temporal dependence is modeled by latent factors while the
spatial dependence is modeled by the factor loadings. Factor analytic
arguments are used to help identify temporal components that summarize most
of the spatial variation of a given region. The temporal evolution of the
factors is described in a number of forms to account for different aspects
of time variation such as trend and seasonality. The spatial dependence is
incorporated into the factor loadings by a combination of deterministic and
stochastic elements thus giving them more flexibility and generalizing
previous approaches. The new structure implies nonseparable space-time
variation to observables, despite its conditionally independent nature, while
reducing the overall dimensionality, and hence complexity, of the problem. The
number of factors is treated as another unknown parameter and fully Bayesian
inference is performed via a reversible jump Markov Chain Monte Carlo
algorithm. The new class of models is tested against one synthetic dataset
and applied to real data obtained from the Clean Air Status and Trends Network
(CASTNet). The factor model decompositionis shown to capture important
aspects of spatial and temporal behavior of the data.
|
Yuan Shen, Dan Cornford, Cedric Archambeau and Manfred Opper
|
There is increasing interest in Bayesian inference in diffusion processes, with
ongoing efforts to develop computationally efficient algorithms. We present
two extensions to a recently developed Variational Gaussian Process
Approximation (VGPA) method.
Aiming at high-dimensional systems, we adopt a model reduction approach to
VGPA. By adopting a basis function expansion, both the time-dependent control
parameters of the approximate process and its moment equations are projected
onto a lower-dimensional subspace. This allows us both to reduce the
computational complexity and to eliminate the time discretisation used in
the previous algorithm. We show an efficient smoother and compare this with
existing methods.
We go on to show how we combine our algorithms with Markov Chain Monte Carlo
methods. The optimised approximate process from VGPA is used to devise an
adaptive mixture of an independence and a random walk sampler, together with
a blocking strategy. We show this results in a sampling scheme with better
mixing than a state-of-art Hybrid Monte Carlo (HMC) path sampler.
Comparing with VGPA and HMC, the efficiency and accuracy of our approaches are
validated on the stochastic Lorenz system. We also report its application
to the 20-dimensional diffusion approximation of the turbulence like
Kumamoto-Shivashinsky system.
In this talk I will start with an overview presentation of particle physics, a
topic that has recently attracted the attention of many physicists, and which
has spawned several Noble Prizes. This will be followed by the introduction of
a mathematical problem that this topic brings forth. The problem is at the
heart of the very nature of what constitutes matter. Standard statistical
approaches to solving this problem have failed to produce satisfactory answers,
causing the physicists to call for a use of Bayesian methods. I have been able
to do this by first looking at the anatomy of the underlying equations which
characterize the Lattice QCD, and noticing a pattern. I then endow the
parameters of these equations with suitable proper elicited priors.
Statistically, the problem boils down to estimating an infinite number of
parameters using a finite number of equations. The role of MCMC has been
essential to obtain results that match those given by the theoretical
physicists. The computing work has been done by Josh Landon.
|
Mark F.J. Steel and Thais C.O. Fonseca
|
The aim of this work is to construct non-Gaussian and nonseparable
covariance functions for processes that vary continuously in space
and time. Stochastic modelling of phenomena over space and time
is important in many areas of application. Choosing an appropriate
model can be difficult as one must take care to use valid covariance
structures. We start from a general and flexible way of constructing
valid nonseparable covariance functions through mixing over separable
Gaussian covariance functions. We then generalize the resulting
models by allowing for individual outliers as well as regions with larger
variances. We induce this through scale mixing with separate positive-valued
processes. Smooth mixing processes are applied to the underlying
correlated Gaussian processes in space and in time, thus leading
to regions in space and time with increased spread. We also apply
a separate uncorrelated mixing process to the nugget effect to generate
individual outliers. We examine properties of these models and
consider posterior and predictive Bayesian inference. We implement
the latter through a Markov chain Monte Carlo sampler and apply
our modelling approach to simulated and real data (temperature data
from the Basque country and wind speed data from Brazil).
Kingman's coalescent process opens the door for estimation of population
genetics model parameters from molecular sequences. One paramount parameter
of interest is the effective population size. Temporal variation of this
quantity characterizes the demographic history of a population. Since
researchers are rarely able to choose a priori a deterministic model
describing effective population size dynamics for data at hand,
non-parametric curve fitting methods based on multiple change-point (MCP)
models have been developed. We propose an alternative to change-point
modeling that exploits Gaussian Markov random fields to achieve temporal
smoothing of the effective population size in a Bayesian framework. The
main advantage of our approach is that, in contrast to MCP models, the
explicit temporal smoothing does not require strong prior decisions. To
approximate the posterior distribution of the population dynamics, we use
efficient, fast mixing MCMC algorithms designed for highly structured
Gaussian models. In a simulation study, we demonstrate that the proposed
temporal smoothing method, named Bayesian skyride, successfully recovers
"true" population size trajectories in all simulation scenarios and
competes well with the MCP approaches without evoking strong prior
assumptions. We apply our Bayesian skyride method to two real data sets.
We analyze sequences of hepatitis C virus contemporaneously sampled in
Egypt, reproducing all key known aspects of the viral population dynamics.
Next, we estimate the demographic histories of human influenza A
hemagglutinin sequences, serially sampled throughout three flu
seasons. Finally, We extend models of temporal variation to include
spatial diffusion. Such spatial processes enable formal Bayesian hypothesis
testing about the relative contributions of geography and human transport
in infectious disease evolution. We demonstrate how to successfully embed
estimation of spatial diffusion in an already highly structured stochastic
model through fast computation using graphics processing units. These
stream processors yield over 100-fold run-time improvement for important
aspects of the spatial-temporal model.
Bayesian nonparametric methods have been previously employed in flexible
modeling for spatial event data that may be viewed as the realization of a
Poisson point process. When these spatial events occur over time, it is
common to incorporate temporal dependence by viewing the data as a single
realization of a point process in both space and time. However, in discrete
time settings -- when each time point corresponds to a set of spatially
distributed events -- it is more appropriate to model the data as arising
from multiple point processes that have an intensity function which is
changing in time. This is made possible by employing a novel version of the
Dependent Dirichlet process as a prior for time-indexed normalized process
intensities. Details of the development of the model and prior measure will
be provided, along with an example involving crime event data.
|
Elisa Varini and Renata Rotondi
|
We aim to define a model to study sequences of earthquake
occurrences in a region by including some features suggested by
geophysical and geological findings and by direct observation
of the seismic events.
Since the seismic rate is usually subject to clear temporal variations,
we assume that the physical system has different states in agreement
with these variations.
At least three states can be deduced from the literature corresponding
respectively to periods of low, medium and high seismic activity (e.g.
seismic quiescence, foreshocks and aftershocks).
We consider appropriate to propose a state-space model where the
observations come from a point process such that its intensity function
has a different expression for each state.
In a previous preliminary study we defined the state process to be a
homogeneous Markov process, that is the transition probabilities
from a state to another were constant in time.
In the present work, we consider the state process to be a
non-homogeneous process such that the transition probabilities
at time t depend on the observations up to t as well as the last
visited states.
We perform a Bayesian analysis of some real data sets by exploiting MCMC
and particle filtering methods.
|
Michail D. Vrettas, Dan Cornford, Yuan Shen and Manfred Opper
|
Stochastic differential equations (SDEs) have been used
extensively in modelling phenomena that exhibit intrinsic randomness or
unknown components. To tackle these problems a range of different methods
have been employed based on the Kalman filter and Monte Carlo approaches.
An alternative is based on a variational treatment of the inference
problem. Recently, a variational Bayesian algorithm has been proposed for
inference in diffusion processes that can be used for both state and
parameter estimation. In this paper we propose a new approach that is
based on a radial basis function (RBF) reparametrisation of this
variational approximation algorithm. We focus on estimating the (hyper-)
parameters in the drift and diffusion of the stochastic differential
equation considered. The new RBF approach is fast and shows great
robustness and stability. Here we validate our method on a
multidimensional (40 variable Lorentz) dynamical system and compare our
results with a state of the art ensemble Kalman filter and a recently
reported Markov Chain Monte Carlo based algorithm. We compare the
asymptotic behaviour of the algorithms as the observation density
increases, and discuss the reasons for the empirically observed
differences in performance of the methods.
Motivated by biological studies involving large data sets from flow cytometry
and fluorescent spectroscopy in immunology, we have developed a number of novel
Bayesian mixture models for density and intensity estimation, clustering
problems and spatio-temporal dynamic modelling. New models and computational
methods involve non-Gaussian component mixture modelling with new uses of old
nonparametric mixtures, and flexible mixtures for static and dynamic spatial
point processes. I will report on some of these developments and immunological
"big data" applications.