View source for Polynomial chaos

{{Short description|Method of representing a random variable}}
'''Polynomial chaos (PC)''', also called '''polynomial chaos expansion''' ('''PCE''') and '''Wiener chaos expansion''', is a method for [[Basis (linear algebra)|representing]] a [[random variable]] in terms of a [[Polynomial Function|polynomial function]] of other random variables. The polynomials are chosen to be [[Orthogonal polynomials|orthogonal]] with respect to the joint [[probability distribution]] of these random variables. Note that despite its name, PCE has no immediate connections to [[chaos theory]]. The word "chaos" here should be understood as "random".<ref>The use of the word "chaos" by [[Norbert Wiener]] in his 1938 publication precedes the use of "chaos" in the branch of mathematics called [[chaos theory]] by almost 40 years. [https://www.etymonline.com/word/chaos]</ref>
PCE was first introduced in 1938 by [[Norbert Wiener]] using [[Hermite polynomials]] to model [[stochastic]] processes with [[Gaussian]] [[random]] variables.<ref name=":1">{{Cite journal|last=Wiener|first=Norbert|date=1938|title=The Homogeneous Chaos|url=https://www.jstor.org/stable/2371268|journal=American Journal of Mathematics|volume=60|issue=4|pages=897–936|doi=10.2307/2371268|jstor=2371268}}</ref> It was introduced to the physics and engineering community by R. Ghanem and P. D. Spanos in 1991<ref name=":0">{{Citation|last1=Ghanem|first1=Roger G.|title=Stochastic Finite Element Method: Response Statistics|date=1991|url=http://dx.doi.org/10.1007/978-1-4612-3094-6_4|work=Stochastic Finite Elements: A Spectral Approach|pages=101–119|place=New York, NY|publisher=Springer New York|access-date=2021-09-29|last2=Spanos|first2=Pol D.|doi=10.1007/978-1-4612-3094-6_4|isbn=978-1-4612-7795-8}}</ref> and generalized to other orthogonal polynomial families by D. Xiu and [[George Karniadakis|G. E. Karniadakis]] in 2002.<ref>{{Cite journal|last1=Xiu|first1=Dongbin|last2=Karniadakis|first2=George Em|date=2002|title=The Wiener--Askey Polynomial Chaos for Stochastic Differential Equations|url=http://dx.doi.org/10.1137/s1064827501387826|journal=SIAM Journal on Scientific Computing|volume=24|issue=2|pages=619–644|doi=10.1137/s1064827501387826|bibcode=2002SJSC...24..619X |s2cid=10358251 |issn=1064-8275}}</ref> Mathematically rigorous proofs of existence and convergence of generalized PCE were given by O. G. Ernst and coworkers in 2011.<ref>{{Cite journal|last1=Ernst|first1=Oliver G.|last2=Mugler|first2=Antje|last3=Starkloff|first3=Hans-Jörg|last4=Ullmann|first4=Elisabeth|date=2011-10-12|title=On the convergence of generalized polynomial chaos expansions|journal=ESAIM: Mathematical Modelling and Numerical Analysis|volume=46|issue=2|pages=317–339|doi=10.1051/m2an/2011045|issn=0764-583X|doi-access=free}}</ref>
PCE has found widespread use in engineering and the applied sciences because it makes possible to deal with probabilistic uncertainty in the parameters of a system. In particular, PCE has been used as a [[surrogate model]] to facilitate [[Uncertainty quantification|uncertainty quantification analyses]].<ref>{{Cite journal |last1=Soize |first1=Christian |last2=Ghanem |first2=Roger |date=2004 |title=Physical Systems with Random Uncertainties: Chaos Representations with Arbitrary Probability Measure |url=http://dx.doi.org/10.1137/s1064827503424505 |journal=SIAM Journal on Scientific Computing |volume=26 |issue=2 |pages=395–410 |bibcode=2004SJSC...26..395S |doi=10.1137/s1064827503424505 |issn=1064-8275 |s2cid=39569403}}</ref><ref>O’Hagan, Anthony. "[https://web.archive.org/web/20230730211831/http://courses.washington.edu/me333afe/Polynomial-chaos_OHagan.pdf Polynomial chaos: A tutorial and critique from a statistician’s perspective]." ''SIAM/ASA J. Uncertainty Quantification'' 20 (2013): 1-20.</ref> PCE has also been widely used in [[Stochastic partial differential equation|stochastic]] [[Finite element method|finite element analysis]]<ref name=":0" /> and to determine the evolution of [[uncertainty]] in a [[dynamical system]] when there is probabilistic uncertainty in the system parameters.<ref>{{Cite journal |date=2013 |title=Wiener's Polynomial Chaos for the Analysis and Control of Nonlinear Dynamical Systems with Probabilistic Uncertainties [Historical Perspectives] |url=https://ieeexplore.ieee.org/document/6595094 |journal=IEEE Control Systems |volume=33 |issue=5 |pages=58–67 |doi=10.1109/MCS.2013.2270410 |s2cid=5610154 |issn=1066-033X}}</ref>
== Main principles ==
Polynomial chaos expansion (PCE) provides a way to represent a [[random variable]] <math>Y</math> with finite variance (i.e., <math>\operatorname{Var}(Y)<\infty</math>) as a function of an <math>M</math>-dimensional [[random vector]] <math>\mathbf{X}</math>, using a polynomial basis that is orthogonal with respect to the distribution of this random vector. The prototypical PCE can be written as:
: <math>Y = \sum_{i\in\mathbb{N}}c_{i}\Psi_{i}(\mathbf{X}).</math>
In this expression, <math>c_{i}</math> is a coefficient and <math>\Psi_{i}</math> denotes a polynomial basis function. Depending on the distribution of <math>\mathbf{X}</math>, different PCE types are distinguished.
=== Hermite polynomial chaos ===
The original PCE formulation used by [[Norbert Wiener]]<ref name=":1" /> was limited to the case where <math>\mathbf{X}</math> is a random vector with a Gaussian distribution. Considering only the one-dimensional case (i.e., <math>M=1</math> and <math>\mathbf{X}=X</math>), the polynomial basis function orthogonal w.r.t. the Gaussian distribution are the set of <math>i</math>-th degree [[Hermite polynomials]] <math>H_i</math>. The PCE of <math>Y</math> can then be written as:
: <math>Y = \sum_{i\in\mathbb{N}}c_{i}H_{i}(X)</math>.
=== Generalized polynomial chaos ===
Xiu (in his PhD under Karniadakis at Brown University) generalized the [[Cameron–Martin theorem|result of Cameron–Martin]] to various continuous and discrete distributions using [[orthogonal polynomials]] from the so-called [[Askey-scheme]] and demonstrated <math>L_2</math> convergence in the corresponding Hilbert functional space. This is popularly known as the generalized polynomial chaos (gPC) framework. The gPC framework has been applied to applications including stochastic [[fluid dynamics]], stochastic finite elements, solid [[mechanics]], nonlinear estimation, the evaluation of finite word-length effects in non-linear fixed-point digital systems and [[probabilistic]] robust control. It has been demonstrated that gPC based methods are computationally superior to [[Monte-Carlo method|Monte-Carlo]] based methods in a number of applications.<ref>{{cite journal |last1=Enstedt |first1=Mattias |last2=Wellander |first2=Niklas |title=Uncertainty Quantification of Radio Propagation Using Polynomial Chaos |journal=Progress in Electromagnetics Research M |date=2016 |volume=50 |pages=205–213 |doi=10.2528/PIERM16062101 |url=https://www.jpier.org/ac_api/download.php?id=16062101|doi-access=free }}</ref> However, the method has a notable limitation. For large numbers of random variables, polynomial chaos becomes very computationally expensive and Monte-Carlo methods are typically more feasible.<ref>{{cite book	| last1 = Dias | first1 = Fabio| author-link1 = Fabio Dias | last2 = Peters | first2 = Gareth W. | author-link2 = Gareth W. Peters | title = Option Pricing with Polynomial Chaos Expansion Stochastic Bridge Interpolators and Signed Path Dependence | year = 2020 | url = https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3588871	| page = 11}}</ref>
=== Arbitrary polynomial chaos ===
Recently chaos expansion received a generalization towards the arbitrary polynomial chaos expansion (aPC),<ref>{{cite journal | url=https://www.sciencedirect.com/science/article/pii/S0951832012000853 | doi=10.1016/j.ress.2012.05.002 | title=Data-driven uncertainty quantification using the arbitrary polynomial chaos expansion | year=2012 | last1=Oladyshkin | first1=S. | last2=Nowak | first2=W. | journal=Reliability Engineering & System Safety | volume=106 | pages=179–190 }}</ref> which is a so-called data-driven generalization of the PC. Like all polynomial chaos expansion techniques, aPC approximates the dependence of simulation model output on model parameters by expansion in an orthogonal polynomial basis. The aPC generalizes chaos expansion techniques towards arbitrary distributions with arbitrary probability measures, which can be either discrete, continuous, or discretized continuous and can be specified either analytically (as probability density/cumulative distribution functions), numerically as histogram or as raw data sets. The aPC at finite expansion order only demands the existence of a finite number of moments and does not require the complete knowledge or even existence of a probability density function. This avoids the necessity to assign parametric probability distributions that are not sufficiently supported by limited available data. Alternatively, it allows modellers to choose freely of technical constraints the shapes of their statistical assumptions. Investigations indicate that the aPC shows an exponential convergence rate and converges faster than classical polynomial chaos expansion techniques{{citation needed|date=March 2024}}. Yet these techniques are in progress but the impact of them on computational fluid dynamics (CFD) models is quite impressionable.
== Polynomial chaos and incomplete statistical information ==
In many practical situations, only incomplete and inaccurate statistical knowledge on uncertain input parameters are available. Fortunately, to construct a finite-order expansion, only some partial information on the probability measure is required that can be simply represented by a finite number of statistical moments. Any order of expansion is only justified if accompanied by reliable statistical information on input data. Thus, incomplete statistical information limits the utility of high-order polynomial chaos expansions.<ref>{{cite journal | url=https://www.sciencedirect.com/science/article/pii/S0951832016306263 | doi=10.1016/j.ress.2017.08.010 | title=Incomplete statistical information limits the utility of high-order polynomial chaos expansions | year=2018 | last1=Oladyshkin | first1=Sergey | last2=Nowak | first2=Wolfgang | journal=Reliability Engineering & System Safety | volume=169 | pages=137–148 }}</ref>
== Polynomial chaos and non-linear prediction ==
Polynomial chaos can be utilized in the prediction of non-linear [[Functional (mathematics)|functionals]] of [[Gaussian process|Gaussian]] [[stationary increments|stationary increment]] processes conditioned on their past realizations.<ref>{{Cite journal |doi=10.1080/01630563.2015.1065273 |title=Wiener Chaos Approach to Optimal Prediction |year=2015 |last1=Alpay |first1=Daniel |last2=Kipnis |first2=Alon |journal=Numerical Functional Analysis and Optimization |volume=36 |issue=10 |pages=1286–1306 |s2cid=54744829 |arxiv=1411.3032 }}</ref> Specifically, such prediction is obtained by deriving the chaos expansion of the functional with respect to a special [[wikt:basis|basis]] for the Gaussian [[Hilbert space]] generated by the process that with the property that each basis element is either measurable or independent with respect to the given samples. For example, this approach leads to an easy prediction formula for the [[Fractional Brownian motion]].
== Bayesian polynomial chaos ==
In a non-intrusive setting, the estimation of the expansion coefficients <math>c_{i}</math> for a given set of basis functions <math>\Psi_{i}</math> can be considered as a [[Bayesian regression]] problem by constructing a [[surrogate model]]. This approach has benefits in that analytical expressions for the data evidence (in the sense of [[Bayesian inference]]) as well as the uncertainty of the expansion coefficients are available.<ref name=":2">{{Cite journal|last1=Ranftl|first1=Sascha|last2=von der Linden|first2=Wolfgang|date=2021-11-13|title=Bayesian Surrogate Analysis and Uncertainty Propagation|journal=Physical Sciences Forum|volume=3|issue=1|pages=6|doi=10.3390/psf2021003006|issn=2673-9984|doi-access=free |arxiv=2101.04038}}</ref> The evidence then can be used as a measure for the selection of expansion terms and pruning of the series (see also [[Bayesian model comparison]]). The uncertainty of the expansion coefficients can be used to assess the quality and trustworthiness of the PCE, and furthermore the impact of this assessment on the actual quantity of interest <math>Y</math>.
Let <math>D= \{\mathbf{X}^{(j)}, Y^{(j)}\}</math> be a set of <math>j = 1,...,N_s</math> pairs of input-output data that is used to estimate the expansion coefficients <math>c_{i}</math>. Let <math>M</math> be the data matrix with elements <math>[M]_{ij} = \Psi_i(\mathbf{X}^{(j)})</math>, let <math>\vec Y = (Y^{(1)},..., Y^{(j)},...,Y^{(N_s)})^T</math> be the set of <math>N_s</math> output data written in vector form, and let be <math>\vec c = (c_1,...,c_i,...,c_{N_p})^T</math> the set of expansion coefficients in vector form.  Under the assumption that the uncertainty of the PCE is of [[Student's t-distribution|Gaussian type with unknown variance]] and a scale-invariant [[Prior probability|prior]], the [[expectation value]] <math>\langle \cdot \rangle</math> for the expansion coefficients is
<math>\langle \vec c \rangle = (M^T \;M)^{-1}\; M^T\; \vec Y</math>
With <math>H = (M^T M)^{-1}</math>, then the covariance of the coefficients is<ref name=":2">{{Cite journal|last1=Ranftl|first1=Sascha|last2=von der Linden|first2=Wolfgang|date=2021-11-13|title=Bayesian Surrogate Analysis and Uncertainty Propagation|journal=Physical Sciences Forum|volume=3|issue=1|pages=6|doi=10.3390/psf2021003006|issn=2673-9984|doi-access=free |arxiv=2101.04038}}</ref>
<math>\text{Cov}(c_m, c_n) = \frac{\chi_{\text{min}}^2}{N_s-N_p-2} H_{m,n}</math>
where <math>\chi_{\text{min}}^2= \vec Y^T( \mathrm{I}-M\; H^{-1}M^T) \;\vec Y</math> is the minimal misfit and <math>\mathrm{I}</math> is the identity matrix. The uncertainty of the estimate for the coefficient <math>n</math> is then given by <math>\text{Var}(c_m) = \text{Cov}(c_m, c_m)  </math>.Thus the uncertainty of the estimate for expansion coefficients can be obtained with simple vector-matrix multiplications. For a given input propability density function <math> p(\mathbf{X}) </math>, it was shown the second moment for the quantity of interest then simply is<ref name=":2">{{Cite journal|last1=Ranftl|first1=Sascha|last2=von der Linden|first2=Wolfgang|date=2021-11-13|title=Bayesian Surrogate Analysis and Uncertainty Propagation|journal=Physical Sciences Forum|volume=3|issue=1|pages=6|doi=10.3390/psf2021003006|issn=2673-9984|doi-access=free |arxiv=2101.04038}}</ref>
<math>\langle Y^2 \rangle = \underbrace{\sum_{m,m'} \int\Psi_m (\mathbf{X}) \Psi_{m'} (\mathbf{X} )
\langle c_m\rangle \langle c_{m'} \rangle  p(\mathbf{X})\;  dV_{\mathbf{X}} } _{=I_1}
+ \underbrace{ \sum_{m,m'} \int\Psi_m (\mathbf{X}) \Psi_{m'} (\mathbf{X} )\; \text{Cov}(c_m, c_{m'})\; p(\mathbf{X})\;  dV_{\mathbf{X}}} _{=I_2} </math>
This equation amounts the matrix-vector multiplications above plus the [[Marginalization (probability)|marginalization]] with respect to <math>\mathbf{X}</math>. The first term <math>I_1</math> determines the primary uncertainty of the quantity of interest <math>Y </math>, as obtained based on the PCE used as a surrogate. The second term <math>I_2</math> constitutes an additional [[Uncertainty quantification|inferential uncertainty]] (often of mixed aleatoric-epistemic type) in the quantity of interest <math>Y </math> that is due to a finite uncertainty of the PCE.<ref name=":2">{{Cite journal|last1=Ranftl|first1=Sascha|last2=von der Linden|first2=Wolfgang|date=2021-11-13|title=Bayesian Surrogate Analysis and Uncertainty Propagation|journal=Physical Sciences Forum|volume=3|issue=1|pages=6|doi=10.3390/psf2021003006|issn=2673-9984|doi-access=free |arxiv=2101.04038}}</ref>  If enough data is available, in terms of quality and quantity, it can be shown that <math>\text{Var}(c_m)  </math> becomes negligibly small and becomes small <ref name=":2">{{Cite journal|last1=Ranftl|first1=Sascha|last2=von der Linden|first2=Wolfgang|date=2021-11-13|title=Bayesian Surrogate Analysis and Uncertainty Propagation|journal=Physical Sciences Forum|volume=3|issue=1|pages=6|doi=10.3390/psf2021003006|issn=2673-9984|doi-access=free |arxiv=2101.04038}}</ref> This can be judged by simply building the ratios of the two terms, e.g. <math>\frac{I_1}{I_1+I_2}</math>.This ratio quantifies the amount of the PCE's own uncertainty in the total uncertainty and is in the interval <math>[0,1]</math>. E.g., if <math>\frac{I_1}{I_1+I_2} \approx 0.5</math>, then half of the uncertainty stems from the PCE itself, and actions to improve the PCE can be taken or gather more data. If<math>\frac{I_1}{I_1+I_2} \approx 1</math>, then the PCE's uncertainty is low and the PCE may be deemed trustworthy.
In a Bayesian surrogate model selection, the probability for a particular surrogate model, i.e. a particular set <math>S</math> of expansion coefficients <math>c_{i}</math> and basis functions <math>\Psi_{i}</math> , is given by the evidence of the data <math>Z_S</math>,
<math>Z_S = \Omega_{N_p} \mid H \mid^{-1/2} (\chi^2_{\text{min}})^{-\frac{N_s-N_p}{2}}
\frac{\Gamma\big(\frac{N_p}{2}\big) \Gamma \big( \frac{N_s-N_p}{2}\big)}{\Gamma \big(\frac{N_s}{2}\big)}</math>
where <math>\Gamma</math> is the [[Gamma function|Gamma-function]], <math>\mid H \mid</math> is the determinant of <math>H</math>, <math>N_s</math> is the number of data, and <math>\Omega_{N_p}</math> is the solid angle in <math>N_p</math> dimensions, where <math>N_p</math> is the number of terms in the PCE.
Analogous findings can be transferred to the computation of PCE-based [[Sensitivity analysis|sensitivity indices]]. Similar results can be obtained for [[Kriging]].<ref name=":2">{{Cite journal|last1=Ranftl|first1=Sascha|last2=von der Linden|first2=Wolfgang|date=2021-11-13|title=Bayesian Surrogate Analysis and Uncertainty Propagation|journal=Physical Sciences Forum|volume=3|issue=1|pages=6|doi=10.3390/psf2021003006|issn=2673-9984|doi-access=free |arxiv=2101.04038}}</ref>
==See also==
*[[Orthogonal polynomials]]
*[[Surrogate model]]
*[[Variance-based sensitivity analysis]]
*[[Karhunen–Loève theorem]]
*[[Hilbert space]]
*[[Proper orthogonal decomposition]]
*[[Bayesian regression]]
*[[Bayesian model comparison]]
== References ==
{{Reflist}}
{{DEFAULTSORT:Polynomial Chaos}}
[[Category:Stochastic processes]]