## Time Series, or Statistics for Stochastic Processes and Dynamical Systems

*08 Sep 2014 11:11*

Rates of convergence of estimators; analogs to VC-dimension results (see Meir's paper below). Large deviation techniques. Prediction schemes. Are there universal schemes which do not demand exponentially growing volumes of data?

If you have an ergodic process, then the sample-path mean for any nice statistic you care to measure will, almost surely, converge to the distributional mean. This is even true of trajectory probabilities (i.e., if you want to know the probability of a certain finite-length trajectory, simply count how often it happens). So "sit and count" is a reliable and consistent statistical procedure. If the process mixes sufficiently quickly, the rate of convergence might even be respectable. But this doesn't say anything about the efficiency of such procedures, which is surely a consideration. And what do you do for non-ergodic processes? (Take multiple runs and hope they're telling you about different ergodic components?) Non-stationary, even?

I need to learn more about frequency-domain approaches; despite being raised as a physicist, I find the time domain much more natural. After all, the frequency domain is effectively just one choice of a function basis, and there are infinitely many others, which might in some sense be more appropriate to the process at hand. But that's at least in part a rationalization against having to learn more math.

*LSE econometrics* and its "general-to-specific" modeling procedure
is very interesting, and I think possibly even related to stuff I've done, but
I need to understand it much better than I do.

(This notebook really needs subdivision.)

See also: Bootstrapping; Change-Point Problems; Classifiers and Clustering for Time Series; Control Theory; Dynamical Systems; Ergodic Theory; Filtering, State Estimation and Signal Processing; Grammatical Inference; Information Theory; Machine Learning, Statistical Inference and Induction; Markov Models and Hidden Markov Models; Neural Coding; Power Law Distributions, 1/f Noise and Long-Memory Processes; Recurrence Times of Stochastic Processes (also Hitting, Waiting, and First-Passage Times) Sequential Decisions Under Uncertainty; State-Space Reconstruction; Statistical Learning Theory with Dependent Data; Statistics; Stochastic Processes; Symbolic Dynamics; Universal Prediction Algorithms

- Recommended, big picture:
- M. S. Bartlett
- An Introduction to Stochastic Processes, with Special Reference to Methods and Applications [Mini-review]
- "Inference and Stochastic Processes", Journal
of the Royal Statistical Society A
**130**(1967): 457--478 [JSTOR] - "Chance or Chaos?", Journal of the Royal
Statistical Society A
**153**(1990): 321--347 [JSTOR]

- Ishwar V. Basawa and B. L. S. Prakasa Rao, Statistical Inference for Stochastic Processes [Assumes familiarity with normal theoretical statistics, i.e., you have to have already been taught to care about confidence intervals, hypothesis tests, estimation efficiency, etc. But good, given that background.]
- David R. Brillinger, "The 2005 Neyman Lecture: Dynamic Indeterminism in Science", Statistical Science
**23**(2008): 48--64, arxiv:0808.0620 [With discussions and response] - Jianqing Fan and Qiwei Yao, Nonlinear Time Series: Nonparametric and Parametric Methods [Review: Everyone Their Own Oracle]
- Peter Guttorp, Stochastic Modeling of Scientific Data [An introduction to statistical inference for many different kinds of dependent data, not just time series; can be used by scientists and statisticians.]
- Holger Kantz and Thomas Schreiber, Nonlinear Time Series Analysis [An excellent presentation of the nonlinear dynamical systems approach, which comes out of physics]
- Judy Klein, Statistical Visions in Time: A History of Time-Series Analysis, 1662--1938
- Robert Shumway and David Stoffer, Time Series Analysis and Its Applications: With R Applications [A standard applied statistics text, but better than many at creating pathways into theory, and realizing that ARIMA is not the alpha and omega of the subject!]
- Jorma Rissanen, Stochastic Complexity in Statistical
Inquiry
[Review: Review:
Less Is More, or,
*Ecce data!*] - David Ruelle, Chaotic Evolution and Strange Attractors: The Statistical Analysis of Deterministic Nonlinear Systems [From notes prepared by Stefano Isola]
- Norbert Wiener, Extrapolation, Interpolation and Smoothing of Stationary Time Series

- Recommended, closeups:
- Markus Abel, K. H. Andersen and Guglielmo Lacorata, "Hierarchical Markovian modeling of multi-time systems," nlin.CD/0201027
- Miika Ahdesmäki, Harri Lähdesmäki, Ron Pearson,
Heikki Huttunen, and Olli Yli-Harja, "Robust detection of periodic time series
measured from biological systems", BMC
Bioinformatics
**6**(2005): 117 [Open access, yay!] - Pierre Alquier and Olivier Wintenberger, "Model selection and
randomization for weakly dependent time series
forecasting", Bernoulli
**18**(2012): 883--913, arxiv:0902.2924 - Francesco Audrino and Peter Bühlmann, "Splines for Financial
Volatility", Journal of the Royal
Statistical Society B
**71**(2009): 655--670 - Jushan Bai, "Testing parametric conditional distributions of
dynamic
models", The
Review of Economics and Statistics
**85**(2003): 531--549 - Matthew J. Beal, Zoubin Ghahramani and Carl Edward Rasmussen, "The Infinite Hidden Markov Model", in NIPS 14 [Link]
- Patrick Billingsley, Statistical Inference for Markov Processes [Discrete-time and cadlag processes only]
- Denis Bosq, Nonparametric Statistics for Stochastic Processes
- Denis Bosq and Delphine Blanke, Inference and Prediction in Large Dimensions [Mini-review]
- David Brillinger
- "Remarks concerning graphical models for time series and
point processes," Revista de Econometria
**16**(1996): 1--23 - "Second-order moments and mutual information in the analysis of time series and point processes," Proceedings of the Conference Statistics 2001 Canada [online]
- "Does anyone know when the correlation coefficient is
useful?: A study of the times of extreme river flows,"
Technometrics
**43**(2001): 266--273 [JSTOR]

- "Remarks concerning graphical models for time series and
point processes," Revista de Econometria
- David R. Brillinger, Brent S. Stewart, Charles L. Littnan, "Three months journeying of a Hawaiian monk seal", pp. 246--264 of Deborah Nolan and Terry Speed (eds.), Probability and Statistics: Essays in Honor of David A. Freedman (2008), arxiv:0805.3019 [A pretty application]
- Prabir Burman, Edmond Chow and Deborah Nolan, "A Cross-Validatory
Method for Dependent Data", Biometrika
**81**(1994): 351--358 [JSTOR] - S. Caires and J. A. Ferreira, "On the Non-parametric Prediction of
Conditionally Stationary Sequences", Statistical Inference
for Stochastic Processes
**8**(2005): 151--184 - Luca Capriotti
- "A Closed-Form Approximation of Likelihood Functions for Discretely Sampled Diffusions: the Exponent Expansion", physics/0703180
- "The Exponent Expansion: An Effective Approximation of
Transition Probabilities of Diffusion Processes and Pricing Kernels of
Financial
Derivatives", physics/0602107
= International Journal of Theoretical and Applied
Finance
**9**(2006): 1179--1199

- Tianjiao Chu and Clark Glymour, "Search for Additive Nonlinear Time Series Causal Models", Journal of Machine Learning Research
**9**(2008): 967--991 - George Cybenko and Valentino Crespi, "Learning Hidden Markov Models Using
Nonnegative Matrix Factorization", IEEE
Transactions on Information Theory
**57**(2011): 3963--3970, arxiv:0809.4086 [Though it contains an error, at least in the preprint version, about the capacities of our CSSR algorithm --- we can get model structures right with much less data than they think, though we presented examples using more data than was strictly needed.] - Allan Dafoe, "First Do No Harm: The Risks of Modeling Temporal Dependence" [PDF preprint]
- R. Dahlhaus, "Fitting Time Series Models to Nonstationary
Processes",
Annals of Statistics
**25**(1997): 1--37 - Jérôme Dedecker, Paul Doukhan, Gabriel Lang, José Rafael León R., Sana Louhichi and Clémentine Prieur, Weak Dependence: With Examples and Applications [Mini-review]
- David Degras, "Nonparametric inference of a trend using functional data", arxiv:0812.2749
- Piet de Jong and Jeremy Penzer, "ARMA models in state space
form", Statistics
and Probability Letters
**70**(2004): 119--125 [preprint] - Piet De Jong, "A Cross-Validation Filter for Time Series
Models", Biometrika
**75**(1988): 594--600 [JSTOR] - Victor H. de la Pena, Rustam Ibragimov, and Shaturgun Sharakhmetov, "Characterizations of joint distributions, copulas, information, dependence and decoupling, with applications to time series", math.ST/0611166
- Franklin M. Fisher, "A Correspondence Principle for Simultaneous Equation Models", Econometrica
**38**(1970): 73--92 [When are simultaneous systems of equations legitimate limits of models of time-evolution?] - Emily B. Fox, Mike West, "Autoregressive Models for Variance Matrices: Stationary Inverse Wishart Processes", arxiv:1107.5239
- Andrew M. Fraser, Hidden Markov Models and Dynamical Systems [Review: Statistics of Moving Shadows]
- Neil Gershenfeld, B. Schoner and E. Metois, "Cluster-Weighted
Modelling for Time-Series Analysis," Nature
**397**(1999): 329--332 [Also described in Gershenfeld's incredible Nature of Mathematical Modeling] - Gershenfeld and Weigend (eds.), Time Series Prediction: Forecasting the Future and Understanding the Past
- Silvia Goncalves and Halbert White, "Maximum likelihood and the
bootstrap for nonlinear dynamic models",
Journal of
Econometrics
**119**(2004): 199--219 - Christian Gouriéroux and Alain Monfort, Simulation-Based Econometric Methods [Review: By Indirection Find Direction Out]
- Kevin D. Hoover and Stephen J. Perez, "Data-Mining Reconsidered:
Encompassing and the General-to-Specific Approach to Specification Search,"
Econometrics Journal
**2**(1999): 167--191 - Aapo Hyvärinen, Kun Zhang, Shohei Shimizu, Patrik O. Hoyer, "Estimation of a Structural Vector Autoregression Model Using Non-Gaussianity", Journal of Machine Learning Research
**11**(2010): 1709--1731 - Marc Joannides and Francois Le Gland, "Small Noise Asymptotics of
the Bayesian Estimator in Nonidentifiable Models", Statistical Inference
for Stochastic Processes
**5**(2002): 95--130 - M. L. Kleptsyna, A. Le Breton and M.-C. Roubaud, "Parameter
Estimation and Optimal Filtering for Fractional Type Stochastic
Systems", Statistical Inference for Stochastic Processes
**3**(2000): 173--182 - Rudolf Kulhavy, Recursive Nonlinear Estimation: A Geometric Approach [Includes, explicitly, estimation in time-series systems]
- Guglielmo Lacorata, Ruben A. Pasmanter and Angelo Vulpiani, "Markov-chain approach to a process with long-time memory," nlin.CD/0110010 [A special case of a more general result encompassed in my paper with Cris Moore]
- S. N. Lahiri, Resampling Methods for Dependent Data [Mini-review]
- R. Dean Malmgren, Jake M. Hofman, Luis A. N. Amaral, Duncan J. Watts, "Characterizing Individual Communication Patterns", arxiv:0905.0106
- Ron Meir, "Nonparametric Time Series Prediction Through Adaptive
Model Selection," Machine Learning
**39**(2000): 5--34 [PDF. Extending the "structural risk minimization" framework due to Vapnik to time series. Unfortunately Meir's approach demands knowledge of the mixing rate of the process, which we don't really know how to estimate, but this is a very encouraging first step.*Addendum*, 2013: we know how to estimate it now.] - Gusztáv Morvai, Sidney J. Yakowitz and Paul Algoet, "Weakly
Convergent Nonparametric Forecasting of Stationary Time Series," IEEE
Trans. Info. Theory
**43**(1997): 483--498 - Martin Nilsson, "Generalized Singular Spectrum Time Series Analysis," physics/0205094
- Andrey Novikov, "Optimal sequential multiple hypothesis tests",arxiv:0811.1297
- Maxim Raginsky, Roummel F. Marcia, Jorge Silva and Rebecca M. Willett, "Sequential Probability Assignment via Online Convex Programming Using Exponential Families" [ISIT 2009; PDF]
- Maxim Raginsky, Rebecca M. Willett, C. Horn, Jorge Silva and Roummel F. Marcia, "Sequential anomaly detection in the presence of noise and limited feedback", IEEE Transactions on Information Theory
**58**(2012): 5544--5562, arxiv:0911.2904 - James Ramsay, Giles Hooker, David Campbell and Jiguo Cao, "Parameter Estimation for Differential Equations: A Generalized Smoothing Approach", Journal of the Royal Statistical Society forthcoming (2007) [PDF preprint]
- Cavan Reilly and Angelique Zeringue, "Improved predictions of lynx trappings using a biological model", pp. 297--308 in Andrew Gelman and Xiao-Li Meng (eds.), Applied Bayesian Modeling and Causal Inference from Incomplete-Data Perspectives [PDF. "Improved" compared to using standard time-series models with no biological content.]
- P. A. Robinson, "Interpretation of scaling properties of
electroencephalographic fluctuations via spectral analysis and underlying
physiology," Physical
Review E
**67**(2003): 032902 [A polite but devastating demonstration that "detrended fluctuation analysis", per Gene Stanley and co., is an obfuscated way of looking at the power spectrum.] - George G. Roussas
- Contiguity of Probability Measures: Some Applications in Statistics [Asymptotic theory of approximation, estimation and testing, for discrete-time Markov processes on fairly general state-spaces. Mini-review]
- "Asymptotic distribution of the log-likelihood
function for stochastic processes," Zeitschrift für
Wahrscheinlickkeitstheorie und verwandte Gebiete
**47**(1979): 31--46 [Elegant solution of a basic problem for a pretty broad class of processes; extends work in his book.]

- Daniil Ryabko, "A criterion for hypothesis testing for stationary processes", arxiv:0905.4937
- Daniil Ryabko and Boris Ryabko, "Testing Statistical Hypotheses
About Ergodic
Processes", arxiv:0804.0510
[Appears to be the same as their "Nonparametric Statistical Inference for
Ergodic
Processes", IEEE
Transactions on Information Theory
**56**(2010): 1430--1435 - Nobusumi Sagara, "Nonparametric maximum-likelihood estimation of
probability measures: existence and consistency", Journal of
Statistical Planning and Inference
**133**(2005): 249--271 ["This paper formulates the nonparametric maximum-likelihood estimation of probability measures and generalizes the consistency result on the maximum-likelihood estimator (MLE). We drop the independent assumption on the underlying stochastic process and replace it with the assumption that the stochastic process is stationary and ergodic. The present proof employs Birkhoff's ergodic theorem and the martingale convergence theorem. The main result is applied to the parametric and nonparametric maximum-likelihood estimation of density functions."] - Christopher C. Strelioff and Alfred W. Hübler, "Medium-Term
Prediction of Chaos", Physical Review
Letters
**96**(2006): 044101 - Masanobu Taniguchi and Yoshihide Kakizawa, Asymptotic Theory of Statistical Inference for Time Series [Finally, a proper statistical treatment which doesn't confine itself to expletive-deleted ARMA processes. Neat information geometry too.]
- Halbert White, Estimation, Inference and Specification Analysis [Review]
- Andrew Gordon Wilson and Zoubin Ghahramani, "Copula Processes", arxiv:1006.1350 [Theoretically interesting, though on the real data example it does at most marginally better than the off-the-shelf GARCH model, at considerably higher computational cost]
- Simon N. Wood, "Statistical inference for noisy nonlinear ecological dynamic systems", Nature
**466**(2010): 1102--1104 - Wei Biao Wu
- "Nonlinear system theory: Another look at
dependence", Proceedings of the
National Academy of Sciences
**102**(2005): 14150--14154 [New measures of mixing, or decay of dependence, related to coupling arguments, and in principle easily checked from the model's specification.] - "Recursive estimation of time-average variance constants",
Annals of Applied Probability
**19**(2009): 1529--1552, arxiv:0908.4540 [This is an ingenious way of getting at the asymptotic variance of the mean, but it doesn't, from my reading, seem to have any*statistical*advantages over the bootstrap methods given by (e.g.) Lahiri (sec. 3.2.1). The computational advantage in*memory*would however be massive.]

- "Nonlinear system theory: Another look at
dependence", Proceedings of the
National Academy of Sciences

- Modesty forbids me to recommend:
- Daniel J. McDonald, CRS and Mark Schervish
- "Estimating $\beta$-mixing coefficients", AISTATS 2011, arxiv:1103.0941
- "Generalization error bounds for stationary autoregressive models", arxiv:1103.0942
- "Risk bounds for time series without strong mixing", arxiv:1106.0730
- "Estimating Beta-Mixing Coefficients via Histograms", arxiv:1109.5998 [The extended journal version of the AISTATS 2011 paper]
- "Time series forecasting: model evaluation and selection using nonparametric risk bounds", arxiv:1212.0463

- CRS, Causal Architecture, Complexity and Self-Organization in Time Series and Cellular Automata [Ph.D. thesis, UW-Madison, 2001]
- CRS, Abigail Z. Jacobs, Kristina Lisa Klinkner and Aaron Clauset, "Adapting to Non-stationarity with Growing Expert Ensembles", arxiv:1103.0949
- CRS and Kristina Lisa Klinkner, "Blind Construction of Optimal Nonlinear Predictors for Discrete Sequences", cs.LG/0406011 = pp. 504--511 of Uncertainty in Artificial Intelligence: Proceedings of the Twentieth Conference (UAI 2004)

- To read:
- Luis A. Aguirre, Ubiratan S. Freitas, Christophe Letellier and Jean
Maquet, "Structure-selection techniques applied to continuous-time nonlinear
models," Physica D
**158**(2001): 1--18 - Eduardo G. Altmann and Holger Kantz, "Recurrence time analysis, long-term correlations, and extreme events", physics/0503056
- Shun-ichi Amari, "Estimating Functions of Independent Component
Analysis for Temporally Correlated Signals," Neural Computation
**12**(2000): 2083--2107 - Oren Anava, Elad Hazan, Shie Mannor, Ohad Shamir, "Online Learning for Time Series Prediction", arxiv:1302.6927
- Heather M. Anderson, "Choosing Lag Lengths in Nonlinear Dynamic Models," Monash Econometric Working Paper [online]
- Claudia Angelini, Daniela Cavab, Gabriel Katul, and Brani
Vidakovic, "Resampling hierarchical processes in the wavelet domain: A case
study using atmospheric turbulence", Physica
D
**207**(2005): 24--40 - J. A. D. Aston, "Modeling macroeconomic time series via heavy tailed distributions", math.ST/0702844
- Francesco Audrino, Lorenzo Camponovo, "Oracle Properties and Finite Sample Inference of the Adaptive Lasso for Time Series Regression Models", arxiv:1312.1473
- Alexander Aue, Siegfried Hörmann, Lajos Horváth
and Matthew Reimherr, "Break detection in the covariance structure of multivariate time series models", Annals of Statistics
**37**(2009): 4046--4087 - Ishwar V. Basawa and D. J. Scott, Asymptotic Optimal Inference for Non-ergodic Models
- Sumanta Basu, George Michailidis, "Estimation in High-dimensional Vector Autoregressive Models", arxiv:1311.4175
- Nathaniel Beck and Jonathan N. Katz
- "What to Do (and Not to Do) with Time-Series Cross-Section Data", American Political Science Review
**89**(1995): 634--647 [JSTOR] - Commentary by the
authors, American Political Science
Review
**100**(2006): 676--677

- "What to Do (and Not to Do) with Time-Series Cross-Section Data", American Political Science Review
- Tadeusz Bednarski, "Fréchet differentiability in statistical inference for time series", Statistical Methods and Applications
**19**(2010): 517--528 - Jose Bento, Morteza Ibrahimi and Andrea Montanari
- "Learning Networks of Stochastic Differential Equations", NIPS 23 (2010), arxiv:1011.0415
- "Information Theoretic Limits on Learning Stochastic Differential Equations", arxiv:1103.1689
- Alain Berlinet and Gérar Biau, "Minimax Bounds in
Nonparametric Estimation of Multidimensional Deterministic Dynamical
Systems", Statistical Inference for Stochastic Processes
**4**(2001): 229--248 ["We consider the problem of estimating a multidimensional discrete deterministic dynamical system from the first n+1 observations. We exhibit the optimal rate function ... the near neighbor estimator achives this optimal rate.... optimal rate function is defined from multidimensonal spacings which are edge lengths of simplicies associated with a triangulation of the Voronoi cells built from the observations." Sounds very cool!] - Alain Berlinet and Christian Francq, "On the Identifiability
of minimal VARMA representations", Statistical Inference for Stochastic
Processes
**1**(1998): 1--15 - Patrice Bertail, Paul Doukhan and Philippe Soulier (eds.), Dependence in Probability and Statistics ["recent developments in the field of probability and statistics for dependent data... from Markov chain theory and weak dependence with an emphasis on some recent developments on dynamical systems, to strong dependence in times series and random fields. ... statistical estimation problems and specific applications"]
- D. Blanke, D. Bosq and D. Guegan, "Modelization and Nonparametric
Estimation for Dynamical Systems with Noise", Statistical Inference for
Stochastic Processes
**6**(2003): 267--290 - Adriaan Blommaert, Niel Hens, Philippe Beutels, "Data Mining for Longitudinal Data under Multicollinearity and Time Dependence using Penalized Generalized Estimating Equations", arxiv:1206.1425
- Luke Bornn, Marian Anghel, Ingo Steinwart, "Forecasting with Historical Data or Process Knowledge under Misspecification: A Comparison", arxiv:1205.3845
- David R. Brillinger, "The Nicholson blowfly experiments: some history and EDA", Journal of Time Series Analysis
**33**(2012): 718--723 - Noelle Bru, Laurence Despres and Christian Paroissin, "A comparison of statistical models for short categorical or ordinal time series with applications in ecology", math.ST/0702706
- Prabir Burman and Robert H. Shumway, "Estimation of trend in state-space models: Asymptotic mean square error and rate of convergence", arxiv:0911.3469 = Annals of Statistics
**37**(2009): 3715--3742 - Alexandre X. Carvalho and Martin A. Tanner, "Mixtures-of-Experts of
Autoregressive Time Series: Asymptotic Normality and Model
Specification", IEEE Transactions on Neural
Networks
**16**(2005): 39--56 - Carlos M. Carvalho, Michael S. Johannes, Hedibert F. Lopes, and Nicholas G. Polson, "Particle Learning and Smoothing", Statistical Science
**25**(2010): 88--106 - Kung-Sik Chan and Howell H. Tong, Chaos: A Statistical Perspective
- Ngai Hang Chan and Ching-Kang Ing, "Uniform moment bounds of Fisherâ€™s information with applications to time series", Annals of Statistics
**39**(2011): 1526--1550 - J.-R. Chazottes, P. Collet and B. Schmitt, "Statistical Consequences of Devroye Inequality for Processes. Applications to a Class of Non-Uniformly Hyperbolic Dynamical Systems", math.DS/0412167
- J.-R. Chazottes, E. Floriani and R. Lima, "Relative Entropy and
Identification of Gibbs Measures in Dynamical Systems," Journal of
Statistical Physics
**90**(1998): 697--725 - Zhuo Chen and Yuhong Yan, "Time Series Models for Forecasting:
Testing or Combining?", Studies in Nonlinear Dynamics and
Econometrics
**11:1**(2007): 3 - Christophe Chesneau, Jalal Fadili, Bertrand Maillot, "Adaptive estimation of an additive regression function from weakly dependent data", arxiv:1111.3994
- Zhiyi Chi, "Large deviations for template matching between point
processes", Annals of Applied
Probability
**15**(2005): 153--174 = math.PR/0503463 - P. Cizek, W. Hardle, V. Spokoiny, "Adaptive pointwise estimation in time-inhomogeneous conditional heteroscedasticity models", arxiv:0903.4620 [I'm more interested in the idea of adaptively estimating non-stationary time series here than the finance application...]
- Michael P. Clements and David F. Hendry (eds.), Companion to Economic Forecasting
- Todd P. Coleman and Sridevi S. Sarma, "A Computationally Efficient Method for Nonparametric Modeling of Neural Spiking Activity with Point Processes", Neural Computation
**22**(2010): 2002--2030 - P. Collet, S. Martinez and B. Schmitt, "Asymptotic distribution of
tests for expanding maps of the interval", Ergodic Theory and Dynamical
Systems
**24**(2004): 707--722 [Kolmogorov-Smironov-type results for the empirical distribution under the invariant measure of a dynamical system] - Daniel Commenges and Anne Gegout-Petit, "Likelihood inference for incompletely observed stochastic processes: ignorability conditions", math.ST/0507151 ["We define a general coarsening model for stochastic processes. We decribe incomplete data by means of sigma-fields and we give conditions of ignorability for likelihood inference."]
- Daan Crommelin, "Estimation of Space-Dependent Diffusions and Potential Landscapes from Non-equilibrium Data", Journal of Statistical Physics
**149**(2012): 220--233 - Colleen D. Cutler and Daniel T. Kaplan (eds.), Nonlinear Dynamics and Time Series: Building a Bridge between the Natural and Statistical Sciences
- Sophie Dabo-Niang, Ali Laksaci, "Conditional mode regression: Application to functional time series prediction", arxiv:0812.4882
- Sophie Dabo-Niang, Christian Francq and Jean-Michel Zakoïan,
"Combining Nonparametric and Optimal Linear Time Series Predictions",
Journal of the American Statistical
Association
**104**(2010): 1554--1565 - Serguei Dachian, Yury A. Kutoyants
- "Hypotheses Testing: Poisson Versus Self-exciting", arxiv:0903.4636
= Scandinavian Journal of Statistics
**33**(2006): 391 - "On the Goodness-of-Fit Tests for Some Continuous Time Processes", arxiv:0903.4642 ["We present a review of several results concerning the construction of the Cramer-von Mises and Kolmogorov-Smirnov type goodness-of-fit tests for continuous time processes. As the models we take a stochastic differential equation with small noise, ergodic diffusion process, Poisson process and self-exciting point processes"]

- "Hypotheses Testing: Poisson Versus Self-exciting", arxiv:0903.4636
= Scandinavian Journal of Statistics
- Rainer Dahlhaus and Wolfgang Polonik, "Empirical spectral processes for locally stationary time series", Bernoulli
**15**(2009): 1--39, arxiv:902.1448 - Arnak Dalalyan and Markus Reiss, "Asymptotic statistical equivalence for ergodic diffusions: the multidimensional case", math.ST/0505053
- Youri Davydov, "Remarks on Estimation Problem for Stationary
Processes in Continuous Time", Statistical Inference for Stochastic
Processes
**4**(2001): 1--15 - A. De Gregorio and S. M. Iacus, "Adaptive Lasso-type estimation for ergodic diffusion processes", arxiv:1002.1312
- D. Dehay and Yu. A. Kutoyants, "On confidence intervals for
distribution function and density of ergodic diffusion process", Journal of
Statistical Planning and Inference
**124**(2004): 63--73 - Miguel A. Delgado, Javier Hidalgo and Carlos Velasco, "Distribution
free goodness-of-fit tests for linear
processes", math.ST/0603043
= Annals of
Statistics
**33**(2005): 2568--2609 [i.e., goodness-of-fit for the autocorrelation function] - Holger Dette, Philip Preuss, and Mathias Vetter, "A Measure of Stationarity in Locally Stationary Processes With Applications to Testing", Journal of the American Statistical Association
**106**(2011): 1113--1124 - Vanessa Didelez, "Graphical models for marked point processes based on local independence", arxiv:0710.5874
- Thomas G. Dietterich, "Machine Learning for Sequential Data" [PDF. Thanks to Gustavo Lacerda for a pointer.]
- Dmitry Dolgopyat, Vadim Kaloshin, Leonid Koralov, "Sample path properties of the stochastic flows," math.PR/0111011
- Randal Douc, Eric Moulines and Tobias Ryden, "Asymptotic properties
of the maximum likelihood estimator in autoregressive models with Markov
regime", Annals
of Statistics
**32**(2004): 2254--2304 = math.ST/0503681 - Holger Drees, "Some aspects of extreme value theory under serial dependence", arxiv:0710.5879
- Pierre Duchesne, "On Testing for Serial Correlation with a
Wavelet-Based Spectral Density Estimator in Multivariate Time Series", Econometric
Theory
**22**(2006): 633--676 - K. Dzhaparidze, Parameter Estimation and Hypothesis Testing in Spectral Analysis of Stationary Time Series
- Michael Eichler
- "Fitting Graphical Interaction Models to Multivariate Time Series", UAI 2006, arxiv:1206.6839
- "Graphical modelling of multivariate time series", math.ST/0610654

- Enrique Figueroa-Lopez and Christian Houdre, "Nonparametric estimation for Levy processes with a view towards mathematical finance", math.ST/0412351
- D. Florens and H. Pham, "Large Deviations in Estimation of an
Ornstein-Uhlenbeck Model," Journal of Applied Probability
**36**(1999): 60--77 - Christian Francq and Jean-Michel Zakoian, "Bartlett's formula for
a general class of nonlinear processes", Journal of Time Series Analysis
**30**(2009): 449--465 - Marc K. Francke, Siem Jan Koopman, Aart F. De Vos, "Likelihood functions for state space models with diffuse initial conditions",
10.1111/j.1467-9892.2010.00673.xJournal of Time Series Analysis
**31**(2010): 407--414 - Jurgen Franke, Jens-Peter Kreiss and Enno Mammen,
"Bootstrap of Kernel Smoothing in Nonlinear Time Series",
Bernoulli
**8**(2002): 1--37 - T. D. Frank, "Delay Fokker-Planck equations, perturbation theory,
and data analysis for nonlinear stochastic systems with time delays",
Physical Review
E
**71**(2005): 031106 - Philip Hans Franses and Dick Van Dijk, Non-Linear Time Series Models in Empirical Finance
- Roland Fried and Vanessa Didelez, "Latent variable analysis and
partial correlation graphs for multivariate time series", Statistics and
Probability Letters
**73**(2005): 287--296 - Cheng-Der Fuh, "Efficient likelihood estimation in state space models", Annals of Statistics
**34**(2006): 2026--2068, arxiv:0611376 - Ben D. Fulcher, Max A. Little, Nick S. Jones, "Highly comparative time-series analysis: The empirical structure of time series and their methods",
Journal of the Royal Society
Interface
**10**(2013): 20130048, arxiv:1304.1209 - Irene Gannaz and Olivier Wintenberger, "Adaptative density estimation with dependent observations", math.ST/0510311
- Jiti Gao, Maxwell King, Zudi Lu and Dag Tjostheim, "Specification
testing in nonlinear and nonstationary time series autoregression",
Annals of Statistics
**37**(2009): 3893--3928, arxiv:0911.3736 - Basillis Gidas and Alejandro Murua, "Optimal transformations for
prediction in continuous-time stochastic processes: finite past and future", Probability Theory and
Related Fields
**131**(2005): 479--492 - German Gomez-Herrero, Wei Wu, Kalle Rutanen, Miguel C. Soriano, Gordon Pipa, Raul Vicente, "Assessing coupling dynamics from an ensemble of time series", arxiv:1008.0539
- Georg A. Gottwald and Ian Melbourne, "Testing for chaos in
deterministic systems with
noise", Physica
D
**212**(2005): 100--110 - Janez Gradisek, Silke Siegert, Rudolf Friedrich and Igor Grabec,
"Analysis of time series from stochastic processes," Physical Review
E
**62**(2000): 3146--3155 - Grassberger and Nadal (eds.), From Statistical Physics to Statistical Inference and Back
- Robert L. Grossman and Richard G. Larson, "State Space Realization Theorems for Data Mining", arxiv:0901.2735
- Diego Guarin, Alvaro Orozco, Edilson Delgado, "A new surrogate data method for nonstationary time series", arxiv:1008.1804
- David Gubbins, Time Series and Inverse Theory for Geophysicists
- Shota Gugushvili, Peter Spreij, "Parametric inference for stochastic differential equations: a smooth and match approach", arxiv:1111.1120
- Laszlo Gyorfi et al., Nonparametric Curve Estimation from Time Series
- Peter Hall, Soumendra Nath Lahiri and Jorg Polzehl,
"On Bandwidth Choice in Nonparametric Regression with Both Short- and
Long-Range Dependent Errors", Annals of Statistics
**23**(1995): 1921--1936 - Niels Richard Hansen, "Penalized maximum likelihood estimation for generalized linear point processes", arxiv:1003.0848
- Wolfgang Hardle, Helmut Lutkepohl, Rong Chen, "A Review of Nonparametric Time Series Analysis", International Statistical Review
**65**(1997): 49--72 [JSTOR] - Jeffrey D. Hart, "Automated Kernel Smoothing of Dependent
Data by using Time Series Cross-Validation", Journal of the
Royal Statistical Society B
**56**(1994): 529--542 [JSTOR] - David Harte, "PtProcess: An R Package for Modelling Marked Point
Processes Indexed by
Time", Journal of Statistical
Software
**35**(2010): 8 - Florian Hartig, Carsten F. Dormann, "Does "model-free" forecasting really outperform the "true" model? A reply to Perretti et al", arxiv:1305.3544
- Andrew Harvey et al (eds.), State Space and Unobserved Component Models: Theory and Applications
- Stefan Haufe, Guido Nolte, Klaus-Robert Mueller and Nicole Kraemer, "Sparse Causal Discovery in Multivariate Time Series", arxiv:0901.1234 [I am not altogether happy with defining "causes" as "has a non-zero coefficient in a vector autoregression"...]
- David Hendry, Econometrics: Alchemy or Science? [Review by Bruce Hansen]
- David F. Hendry and Bent Nielsen, [Econometric Modeling: A Likelihood Approach
- Nadine Hilgert, Vivien Rossi, Jean-Pierre Vila, Verene Wagner,
"Identification, Estimation, and Control of Uncertain Dynamic Systems: A
Nonparametric
Approach", Communications
in Statistics: Theory and Methods
**36**(2007): 2509--2525 - Junichi Hirukawa and Masanobu Taniguchi, "LAN theorem for
non-Gaussian locally stationary processes and its applications", Journal
of Statistical Planning and Inference
**136**(2006): 640--688 - Scott H. Holan, Robert Lund, and Ginger Davis, "The ARMA alphabet soup: A tour of ARMA model variants", Statistics Surveys
**4**(2010): 232--274 - Jinh Hu, Wen-wen Tung, Jianbo Gao and Yinhe Cao, "Reliability of
the 0-1 test for
chaos", Physical
Review E
**72**(2005): 056207 [On Gottwald and Melbourne] - Jianhua Z. Huang and Lijian Yang, "Identification
of Non-Linear Additive Autoregressive Models", Journal of
the Royal Statistical Society B
**66**(2004): 463--477 [JSTOR. Proves consistency under the assumption that the data-generating process is strictly stationary and strongly mixing.] - Stefano M. Iacus
- "Statistical analysis of stochastic resonance with ergodic diffusion noise," math.PR/0111153
- Simulation and Inference for Stochastic Differential Equations
- "On Lasso-type estimation for dynamical systems with small noise", arxiv:0912.5078

- Ching-Kang Ing, "Accumulated prediction errors, information criteria and optimal forecasting for autoregressive time series", Annals of
Statistics
**35**(2007): 1238--1277, arxiv:0708.2373 - Atsushi Inoue and Lutz Kilian, "In-sample or out-of-sample tests of predictability: which one should we use?", European Central Bank Working Paper [PDF]
- Akihiko Inoue and Yukio Kasahara, "Explicit representation of
finite predictor coefficients and its applications", math.ST/0405051 = Annals of
Statistics
**34**(2006): 973--993 - D. A. Ioannides and D. P. Papanastassiou, "Estimating the
distribution function of a stationary process involving measurement
errors", Statistical Inference for Stochastic
Processes
**4**(2001): 181--198 - E. L. Ionides, C. Breto and A. A. King, "Inference for nonlinear
dynamical
systems", Proceedings
of the National Academy of Sciences (USA)
**103**(2006): 18438--18443 - S. Ishii and M.-A. Sato, "Reconstruction of chaotic dynamics by
on-line EM algorithm," Neural Networks
**14**(2001): 1239--1256 - Christine Jacob, "Conditional least squares estimation in nonstationary nonlinear stochastic regression models", Annals of Statistics
**38**(2010): 566--597 - Òscar Jordà, "Simultaneous Confidence Regions for
Impulse Responses", The Review of Economics
and Statistics
**91**(2009): 629--647 - C. T. Jose, B. Ismail, S. Jayasekhar, "Trend, Growth Rate, and Change Point Analysis: A Data Driven Approach", Communications in Statistics: Simulation and Computation
**37**(2008): 498--506 - Joseph Tadjuidje Kamgaing, Hernando Ombao and Richard A. Davis,
"Autoregressive processes with data-driven regime switching",
Journal of Time Series Analysis
**30**(2009): 505--533 - Daniel M. Keenan, Xin Wang, Steven M. Pincus and Johannes D. Veldhuis, "Modelling the nonlinear time dynamics of multidimensional hormonal systems",
Journal of Time Series Analysis
**33**(2012): 779--796 - Matthew B. Kennel, "Testing time symmetry in time series
using data compression dictionaries", Physical Review E
**69**(2004): 056208 - Tae Yoon Kim and Sangyeol Lee, "Kernel density estimator for strong
mixing processes", Journal of
Statistical Planning and Inference
**133**(2005): 273--284 - Young Min Kim, Soumendra N. Lahiri and Daniel J. Nordman,
"A Progressive Block Empirical Likelihood Method for Time Series",
Journal of the American Statistical Association
**108**(2013): 1506--1516 - Jon Kleinberg, "Bursty and Hierarchical Structure in Streams" [PDF]
- D. Kleinhans, R. Friedrich, "Maximum Likelihood Estimation of Drift and Diffusion Functions", physics/0611102
- D. Kleinhans, R. Friedrich, A. Nawroth and J. Peinke, "An iterative
procedure for the estimation of drift and diffusion coefficients of Langevin
processes", Physics Letters
A
**346**(2005): 42--46 = physics/0502152 ["The analysis is based on an iterative procedure minimizing the Kullback-Leibler distance between measured and estimated two time joint probability distributions of the process."] - M. L. Kleptsyna and A. Le Breton, "Statistical Analysis of the
Fractional Ornstein-Uhlenbeck Type Process", Statistical Inference for
Stochastic Processes
**5**(2002): 229--248 - Jens-Peter Kreiss, Efstathios Paparoditis, Dimitris N. Politis, "On the range of validity of the autoregressive sieve bootstrap", Annals
of Statistics
**39**(2011): 2103--2130, arxiv:12016211 - Clemens Kreutz, Andreas Raue, Jens Timmer, "Likelihood based observability analysis and confidence intervals for predictions of dynamic models", arxiv:1107.0013
- D. Kugiumtzis, "Statically Transformed Autoregressive Process and Surrogate Data Test for Nonlinearity," nlin.CD/0110025
- Uwe Küchler and Michael Sørensen, Exponential Families of Stochastic Processes
- Hans R. Künsch, "State Space and Hidden Markov Models", pp. 109--173 in Ole E. Barndorff-Nielsen, David R. Cox and Claudia Klüppelberg (eds.), Complex Stochastic Systems
- Y. A. Kutoyants
- Statistical Inference for Ergodic Diffusion Processes
- "On the Goodness-of-Fit Testing for Ergodic Diffusion Processes", arxiv:0903.4550
- "Goodness-of-Fit Tests for Perturbed Dynamical Systems", arxiv:0903.4612
- "On Properties of Estimators in non Regular Situations for Poisson Processes", arxiv:0903.4613

- B. Lacaze, "Errorless uniform sampling of complex stationary
processes," Signal
Processing
**83**(2003): 913--917 - Guy Lebanon, Yang Zhao, and Yanjun Zhao, "Modeling temporal text streams using the local multinomial model", Electronic Journal of Statistics
**4**(2010): 566--584 - J. Lember and A. Koloydenko, "Adjusted Viterbi training", math.ST/0406237
- Daniel Lemire, "A Better Alternative to Piecewise Linear Time Series Segmentation", cs.DB/0605103
- Matthieu Lerasle, "Adaptive density estimation for stationary
processes", Mathematical Methods of Statistics
**18**(2009): 59--83, arxiv:0909.0999 - J. K. Lindsey, Statistical Analysis of Stochastic Processes in Time [old draft in Postscript; data and R code]
- Yu. N. Lin'kov, Asymptotic Statistical Methods for Stochastic Processes
- Weidong Liu and Wei Biao Wu, "Simultaneous nonparametric inference
of time
series", Annals
of Statistics
**38**(2010): 2388--2421 - E. Locherbach, "Likelihood Ratio Processes for Markovian Particle
Systems with Killing and Jumps", Statistical Inference for Stochastic
Processes
**5**(2002): 153--177 - Wei-Yin Loh and Wei Zheng, "Regression trees for longitudinal and multiresponse data", Annals of Applied Statistics
**7**(2013): 495--522 - Wei Lu, Namrata Vaswani, "The Wiener-Khinchin Theorem for Non-wide Sense stationary Random Processes" ["under certain assumptions, the power spectral density (PSD) of any random process is equal to the Fourier transform of the time-averaged autocorrelation function"]
- Zudi Lu, Dag Johan Steinskog, Dag Tjostheim and Qiwei Yao,
"Adaptively Varying-Coefficient Spatiotemporal Models", Journal of the Royal Statistical Society B
**71**(2009): 859--880 [PDF preprint] - Xiaodong Luo, Tomomichi Nakamura and Michael Small, "Surrogate data method applied to nonlinear time series", nlin.CD/0603004
- Xiaodong Luo, Jie Zhang, Junfeng Sun, Michael Small, Irene Moroz, "Asymptotically pivotal statistic for surrogate testing with extended hypothesis", nlin.CD/0701008
- Xiaodong Luo, Jie Zhang and Michael Small, "Exact nonparametric inference for detection of nonlinear determinism", nlin.CD/0507049 [More exactly, this is an exact test for linear stochasticity --- rejecting the null indicates either nonlinearity or determinism, or both.]
- Cesar Maldonado, "Fluctuation Bounds for Chaos Plus Noise in Dynamical Systems", Journal of Statistical Physics
**148**(2012): 548--564 - Enno Mammen and Swagata Nandi, "Change of the nature of a test when
surrogate data are applied", Physical Review
E
**70**(2004): 016121 - Heikki Mannila and Dmitry Rusakov, "Decomposition of Event Sequences into Independent Components" [short and long versions in PS]
- T. K. March, S. C. Chapman and R. O. Dendy, "Recurrence plot statistics and the effect of embedding", physics/0502042
- Inés P. Mariño, Joaquín Míguez, and Riccardo Meucci, "Monte Carlo method for adaptively estimating the unknown parameters and the dynamic state of chaotic systems", Physical Review E
**79**(2009): 056218 - Pierre-Francois Marteau, "Time Warp Edit Distances with Stiffness Adjustment for Time Series Matching", cs.IR/0703033
- Norbert Marwan and Jurgen Kurths, "Nonlinear analysis of bivariate data with cross recurrence plots," physics/0201061
- Norbert Marwan, M. Thiel, N. R. Nowaczyk, "Cross Recurrence Plot Based Synchronization of Time Series," physics/0201062
- Norbert Marwan, N. Wessel, U. Meyerfeldt, A. Schirdewan, J. Kurths, "Recurrence Plot Based Measures of Complexity and its Application to Heart Rate Variability Data," physics/0201064
- Ikuo Matsuba, Hiroshi Takahashi and shinya Wakasa, "Stochastically Equivalent Dynamical System Approach to Nonlinear Deterministic Prediction",
International
Journal of Bifurcation and Chaos
**16**(2006): 2721--2728 [I can't tell, from the abstract, if they're proposing to use stochastic systems to predict deterministic ones or vice versa; it'd be interesting either way!] - Muneya Matsui, "A characterization of ARMA and Fractional ARIMA models with infinitely divisible innovations", math.ST/0703731
- Brendan P. M. McCabe, Gael M. Martin, David Harris, "Efficient probabilistic forecasts for counts", Journal
of the Royal Statistical Society B
**73**(2011): 253--272 - Emma J. McCoy, Sofia C. Olhede, David A. Stephens, "Non-Regular Likelihood Inference for Seasonally Persistent Processes", arxiv:0709.0139
- Kevin McGoff, Sayan Mukherjee, Natesh S. Pillai, "Statistical inference for dynamical systems: a review", arxiv:1204.6265
- Patrick E. McSharry and Leonard A. Smith, "Consistent nonlinear
dynamics: identifying model inadequacy", nlin.CD/0401024 = Physica
D
**192**(2004): 1--22 - Jan Mielniczuk, Zhou Zhou and Wei Biao Wu, "On nonparametric prediction of linear processes", Journal of
Time Series Analysis
**30**(2009): 652--673 - Javier R. Movellan, Paul Mineiro, and R. J. Williams, "A Monte
Carlo EM Approach for Partially Observable Diffusion Processes: Theory and
Applications to Neural Networks," Neural Computation
**14**(20020: 1507--1544 - Eric Moulines, Pierre Priouret and Francois Roueff, "On recursive
estimation for time varying autoregressive
processes", math.ST/0603047
= Annals of
Statistics
**33**(2005): 2610--2654 - Jose M. F. Moura and Sanjoy K. Mitter, "Identification and Filtering: Optimal Recursive Maximum Likelihood Approach" [1986 technical report from MIT, found looking for something else, original URL now lost --- presumably long since published]
- Hans-Georg Muller and Ulrich Stadtmuller, "Generalized functional
linear models", math.ST/0505638 = Annals of
Statistics
**33**(2005): 774--805 ["We propose a generalized functional linear regression model for a regression situation where the response variable is a scalar and the predictor is a random function. A linear predictor is obtained by forming the scalar product of the predictor function with a smooth parameter function, and the expected value of the response is related to this linear predictor via a link function. If, in addition, a variance function is specified, this leads to a functional estimating equation which corresponds to maximizing a functional quasi-likelihood. This general approach includes the special cases of the functional linear model, as well as functional Poisson regression and functional binomial regression. The latter leads to procedures for classification and discrimination of stochastic processes and functional data. ... An essential step in our proposal is dimension reduction by approximating the predictor processes with a truncated Karhunen-Loeve expansion."] - Ursula U. Müller, Anton Schick and Wolfgang Wefelmeyer,
"Estimating the innovation distribution in nonparametric autoregression",
Probability Theory and Related Fields
**144**(2009): 53--77 ["We prove a Bahadur representation for a residual-based estimator of the innovation distribution function in a nonparametric autoregressive model. The residuals are based on a local linear smoother for the autoregression function."] - Tomomichi Nakamura, Yoshito Hirata, and Michael Small, "Testing for
correlation structures in short-term variabilities with long-term trends of
multivariate time
series", Physical
Review E
**74**(2006): 041114 - Tomomichi Nakamura, Xiaodong Luo, and Michael Small, "Testing for
nonlinearity in time series without the Fourier transform", Physical Review
E
**72**(2005): 055201 - Tomomichi Nakamura and Michael Small, "Small-shuffle surrogate
data: Testing for dynamics in fluctuating data with
trends", Physical
Review E
**72**(2005): 056216 - Yuval Nardi, Alessandro Rinaldo, "Autoregressive Process Modeling via the Lasso Procedure", arxiv:0805.1179
- Sahand Negahban, Pradeep Ravikumar, Martin J. Wainwright, Bin Yu, "A unified framework for high-dimensional analysis of $M$-estimators with decomposable regularizers", arxiv:1010.2731
- Ilia Negri, "Efficiency of a class of unbiased estimators for the invariant distribution function of a diffusion process", math.ST/0609590
- Ilia Negri and Yoichi Nishiyama, "Goodness of fit test for ergodic
diffusions by tick time sample scheme", Statistical Inference for stochastic Processes
**13**(2010): 81--95 - Yoichi Nishiyama, "Goodness-of-fit test for a nonlinear time
series", Journal of Time Series Analysis
**30**(2009): 674--681 - Jun Ohkubo, "Nonparametric model reconstruction for stochastic
differential equations from discretely observed time-series data",
Physical Review E
**84**(2011): 066702 - Jimmy Olsson, Olivier Cappe, Dandal Douc and Eric Moulines, "Sequential Monte Carlo smoothing with application to parameter estimation in non-linear state space models", math.ST/0609514
- Sorinel Adrian Oprisan, "An application of the least-squares method
to system parameters extraction from experimental data", Chaos
**12**(2002): 27--32 - Brahim Ouhbi and Nikolaos Limnios, "Nonparametric estimation for
semi-Markov processes based on its hazard rate functions", Statistical
Inference for Stochastic Processes
**2**(1999): 151--173 - P. Palaniyandi and M. Lakshmanan, "Estimation of System Parameters and Predicting the Flow Function from Time Series of Continuous Dynamical Systems", nlin.CD/0406027
- Milan Palus, "Coarse-grained
entropy rate for characterization of complex time series", Physica
D
**93**(1996): 64--77 [Thanks to Prof. Palus for a reprint] - Angeliki Papana and Dimitris Kugiumtzis, "Evaluation of Mutual Information Estimators for Time Series", arxiv:0904.4753
- Efstathios Paparoditis, "Validating Stationarity Assumptions in
Time Series Analysis by Rolling Local
Periodograms", Journal
of the American Statistical Association
**105**(2010): 839--851 - Emanuel Parzen, "Statistical inference on time series by RKHS methods" (1970)
- Charles T. Perretti, Stephan B. Munch, and George Sugihara,
"Model-free forecasting outperforms the correct mechanistic model for simulated
and experimental data", Proceedings of the National Academy of
Sciences
**110**(2013): 5253--5257 [See also response by Hartig and Dormann] - Raffaella Piccarreta, "Graphical and Smoothing Techniques for Sequence Analysis", Sociological Methods and Research
**41**(2012): 362--380 - Zacharias Psaradakis
- "A sieve bootstrap test for stationarity,"
Statistics and
Probability Letters
**62**(2003): 263--274 - "Blockwise bootstrap testing for stationarity",
Statistics and
Probability Letters
**76**(2006): 562--570

- "A sieve bootstrap test for stationarity,"
Statistics and
Probability Letters
- Zacharias Psaradakis, Martin Sola, Fabio Spagnolo and Nicola Spagnolo, "Selecting nonlinear time series models using information criteria",
Journal of
Time Series Analysis
**30**(2009): 369--394 - N. U. Prabhu and Ishawar V. Basawa (eds.), Statistical Inference in Stochastic Processes (1991)
- B. L. S. Prakasa Rao
- Semimartingales and Their Statistical Inference
- Statistical Inference for Diffusion-Type Processes

- E. Racca and A. Porporato, "Langevin equations from time series",
Physical Review
E
**71**(2005): 027101 - Ali Rahimi, Learning to Transform Time Series with a Few Examples [Ph.D. thesis, MIT dept. of electrical engineering and computer science, 2005. PDF]
- Ali Rahimi, Ben Recht and Trevor Darrell, "Learning to Transform Time Series with a Few Examples", tech report [PDF]
- M. B. Rajarshi, Statistical Inference for Discrete Time Stochastic Processes
- M. M. Rao, Stochastic Processes: Inference Theory
- Ramiro Rico-Martinez, K. Krischer, G. Flaetgen, J.S. Anderson and I. G. Kevrekidis, "Adaptive Detection of Instabilities: An Experimental Feasibility Study," nlin.CD/0202057
- Christoph Rieke, Ralph G. Andrzejak, Florian Mormann and Klaus
Lehnertz, "Improved statistical test for nonstationarity using recurrence time
statistics", Physical Review E
**69**(2004): 046111 [link] - Ricardo Ríos, Luis-Angel Rodríguez,
"Penalized estimate of the number of states in Gaussian linear AR with Markov regime", Electronic Journal of Statistics
**2**(2008): 1111--1128, arxiv:0807.2726 - John C. Robertson, Ellis W. Tallman and Charles H. Whiteman, "Forecasting using relative entropy," Federal Reserve Bank of Atlanta Working Paper 2002-20 [PDF]
- J. W. C. Robinson, J. Rung, A. R. Bulsara and M. E. Inchiosa,
"General measures for signal-noise separation in nonlinear dynamical
systems," Physical Review E
**63**2000: 011107 - Brois Ryabko, "Applications of Universal Source Coding to Statistical Analysis of Time Series", arxiv:0809.1226
- Boris Ryabko and Jaakko Astola
- "Universal Codes as a Basis for Time Series Testing", cs.IT/0602084
- "Universal Codes as a Basis for Nonparametric Testing of Serial Independence for Time Series", cs.IT/0506094

- Daniil Ryabko
- "Characterizing predictable classes of processes", arxiv:0905.4341
- "Discrimination between B-Processes is Impossible",
Journal of
Theoretical Probability
**23**(2010): 565--575 - "Clustering processes", arxiv:1005.0826

- Manuel S. Santos, "Consistency properties of a simulation-based estimator for dynamic processes", Annals of Applied Probability
**20**(2010): 196--213 - Suchi Saria, Daphne Koller, Anna Penn, "Discovering shared and individual latent structure in multiple time series", arxiv:1008.2028
- Joao R. Sato, Sergi Costafreda, Pedro A. Morettin, Michael John Brammer, "Measuring Time Series Predictability Using Support Vector Regression",
Communications in Statistics: Simulation
and Computation
**37**(2008): 1183--1197 - Nicola Scafeta, Patti Hamilton and Paolo Grigolini, "The Thermodynamics of Social Processes: The Teen Birth Phenomenon," cond-mat/0009020 [Not because I believe them about sociology, but because they claim to have new and powerful nonparametric methods for detecting and quantifying memory in time series]
- Thomas Schreiber and Andreas Schmitz, "Surrogate time series," chao-dyn/9909037
- Reiner Schulz and James A. Reggia, "Temporally Asymmetric Learning Supports Sequence Processing in Multi-Winner Self-Organizing Maps",
Neural
Computation
**16**(2004): 535--561 [the "model presented here raises the possibility that SOMs may ultimately prove useful as visualization tools for temporal sequences and as preprocessors for sequence pattern recognition systems."] - Xiaofeng Shao, "A self-normalized approach to confidence interval construction in time series", Journal of the
Royal Statistical Society B
**72**(2010): 343--366, arxiv:1005.2137 [Arxiv version includes an important correction to Assumption 2 and related theorems] - Xiaofeng Shao, Wei Biao Wu, "Asymptotic spectral theory for nonlinear time series", math.ST/0611029
- M. Siefert, J. Peinke and R. Friedrich, "On a quantitative method to analyse dynamical and measurement noise," physics/0108034
- Przemyslaw Sliwa and Wolfgang Schmid, "Monitoring the
cross-covariances of a multivariate time series", Metrika
**61**(2005): 89--115 - A. Sitz, U. Schwarz, J. Kurths, H. U. Voss, "Estimation of
parameters and unobserved components for nonlinear systems from noisy time
series," Physical Review E
**66**(2002): 016210 - Michael Small and Kevin Judd, "Detecting periodicity in experimental data using linear modeling techniques", physics/9810021
- Vadim N. Smelyanskiy and Dmitry G. Luchinsky, "Inference of stochastic nonlinear oscillators with applications to physiological problems", physics/0403121 [They present this as a Bayesian inference issue, but the core of their work appears, from skimming, to be an efficient method for computing the likelihood, so it'd apply equally well to maximum likelihood inference, for instance.]
- V. N. Smelyanskiy, D. A. Timucin, A. Bandrivskyy and D. G. Luchinsky, "Model reconstruction of nonlinear dynamical systems driven by noise", physics/0310062 [Same as earlier paper --- was this one submitted to PRL?]
- Dmitry A. Smirnov, Vladislav S. Vlaskin and Vladimir I.
Ponomarenko, "Estimation of parameters in one-dimensional maps from noisy
chaotic time series", Physics Letters
A
**336**(2005): 448--458 - Song Song and Peter J. Bickel, "Large Vector Auto Regressions", arxiv:1106.3915
- Eduardo D. Sontag, "For differential equations with r parameters, 2r+1 experiments are enough for identification," math.DS/0111135
- D. Sornette and V. F. Pisarenko, "Properties of a simple bilinear stochastic model: estimation and predictability", physics/0703217
- Ingo Steinwart, Marian Anghel, "Consistency of support vector machines for forecasting the evolution of an unknown ergodic dynamical system from observations with unknown noise", Annals of Statistics
**37**(2009): 841--875, arxiv:0707.0322 - Jean-Pierre Stockis, Jurgen Franke and Joseph Tadjuidje Kamgaing,
"On geometric ergodicity of CHARME
models", Journal
of Time Series Analysiscite>
**31**(2010): 141--152 - Tomoya Suzuki, Tohru Ikeguchi, and Masuo Suzuki, "Effects of data
windows on the methods of surrogate data", Physical Review
E
**71**(2005): 056708 - Alexander G. Tartakovsky, "Asymptotic Optimality of Certain
Multihypothesis Sequential Tests: Non-i.i.d. Case", Statistical Inference
for Stochastic Processes
**1**(1998): 265--295 - Marco Thiel, M. Carmen Romano and Jurgen Kurths, "How much
information is contained in a recurrence plot?", Physics Letters
A
**330**(2004): 343--349 - Bo Thiesson, David Maxwell Chickering, David Heckerman, Christopher Meek, "ARMA Time-Series Modeling with Graphical Models", UAI 2004, arxiv:1207.4162
- Madeleine B. Thompson, "A Comparison of Methods for Computing Autocorrelation Time", arxiv:1011.0175
- Tina Toni, David Welch, Natalja Strelkowa, Andreas Ipsen, Michael P.H. Stumpf, "Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems", arxiv:0901.1925
- Wilson Truccolo, John P. Donoghue, "Nonparametric Modeling of
Neural Point Processes via Stochastic Gradient Boosting Regression", Neural
Computation
**19**(2007): 672-705 - Masayuki Uchida and Nakahiro Yoshida, "Information Criteria in
Model Selection for Mixing Processes", Statistical Inference for
Stochastic Processes
**4**(2001): 73--98 ["The emphasis is put on the use of the asymptotic expansion of the distribution of an estimator based on the conditional Kullback-Leibler divergence for stochastic processes. Asymptotic properties of information criteria and their improvement are discusssed."] - Aad van der Vaart and Harry van Zanten, "Donsker theorems for
diffusions: Necessary and sufficient conditions", math.PR/0507412 = Annals of
Probability
**33**(2005): 1422--1451 - Harry van Zanten, "On Uniform Laws of Large Numbers for Ergodic
Diffusions and Consistency of Estimators", Statistical Inference for
Stochastic Processes
**6**(2003): 199--213 ["In contrast with uniform laws of large numbers for i.i.d. random variables, we do not need conditions on the 'size' of the class [of functions] in terms of bracketing or covering numbers. The result is a consequence of a number of asymptotic properties of diffusion local time that we derive."] - J. H. van Zanten, "On the Uniform Convergence of the Empirical
Density of an Ergodic Diffusion", Statistical Inference for
Stochastic Processes
**3**(2000): 251--262 - Paolo Vidoni, "A simple procedure for computing improved
prediction intervals for autoregressive models", Journal of Time Series Analysis
**30**(2009): 577--590 - Juan M. Vilar-Fernandez and Ricardo Cao, "Nonparametric Forecasting
in Time Series - A Comparative Study", Communications in
Statistics: Simulation and Computation
**36**(2007): 311--334 - R. Vilela Mendes, R. Lima and T. Araujo, "A Process-Reconstruction Analysis of Market Fluctuations," cond-mat/0102301 [I don't care about the market, but they claim to have a new method for identifying distributions over entire sequences]
- Divakar Viswanath, Xuan Liang, Kirill Serkh, "Metric Entropy and the Optimal Prediction of Chaotic Signals", arxiv:1102.3202
- Li Wang, Lijian Yang, "Spline-backfitted kernel smoothing of
nonlinear additive autoregression
model", Annals of
Statistics
**35**(2007): 2474--2503, arxiv:math/0612677 - Qiying Wang and Peter C. B. Phillips, "A specification test for nonlinear nonstationary models", Annals of Statistics
**40****(2012): 727--758** **Zijun Wang, "Finite Sample Performances of the Model Selection Approach in Nonparametric Model Specification for Time Series", Communications in Statistics: Theory and Methods****38**(2009): 2302--2330**Halbert White ,Asymptotic Theory for Econometricians****Wei Biao Wu**- and
Jan Mielniczuk, "Kernel Density Estimation for Linear Processes", Annals
of Statistics
**30**(2002): 1441--1459

- and
Jan Mielniczuk, "Kernel Density Estimation for Linear Processes", Annals
of Statistics
**Herwig Wendt, Patrice Abry and Stephane Jaffard, "Bootstrap for Empirical Multifractal Analysis", IEEE Signal Processing Magazine July 2007, pp. 38--48 [+ technical papers by these authors]****Han Xiao and Wei Biao Wu, "Covariance matrix estimation for stationary time series", Annals of Statistics****40**(2012): 466--493**Hongqi Xue, Hongyu Miao, and Hulin Wu, "Sieve estimation of constant and time-varying coefficients in nonlinear ordinary differential equation models by considering both numerical error and measurement error", Annals of Statistics****38**(2010): 2351--2387**A. Zeileis and G. Grothendieck, "zoo: S3 Infrastructure for Regular and Irregular Time Series", Journal of Statistical Software****14**(2005): 1--27 = math.ST/0505527 ["zoo is an R package providing an S3 class with methods for indexed totally ordered observations, such as discrete irregular time series. Its key design goals are independence of a particular index/time/date class and consistency with base R and the "ts" class for regular time series."]**Zhou Zhou, "Nonparametric inference of quantile curves for nonstationary time series", Annals of Statistics****38**(2010): 2187--2217**M. Zukovic, D. T. Hristopulos, "Spartan Random Processes in Time Series Modeling", 0709.3418**

- To write/finish:
- CRS, "Learning Rates and Recurrence Times" [a.k.a. "Wait and see"]
- CRS, "Algorithms for Inferring the Statistical Structure of Symbol Sequences: History and Review"

- Alain Berlinet and Gérar Biau, "Minimax Bounds in
Nonparametric Estimation of Multidimensional Deterministic Dynamical
Systems", Statistical Inference for Stochastic Processes