Wikipedia:Reference desk/Archives/Mathematics/2016 September 5

= September 5 =

piecewise polynomial least squares, the sequel
See Reference desk/Archives/Mathematics/2015 November 15. Thanks again to Meni for the answer there. Now: sometimes I want to put other constraints on the fit. (You may assume, for simplicity and also because it's what I'm most likely doing, that my function is piecewise quadratic, discontinuous only in the second derivative.) —Tamfang (talk) 08:30, 5 September 2016 (UTC)
 * How can I make it periodic, i.e. $$f(x_0) = f(x_m)$$ and $$f'(x_0) = f'(x_m)$$?
 * How can I fix $$f(x_0)$$ and $$f(x_m)$$ arbitrarily?
 * How can I specify that both end segments are linear? (It's easy to make the first segment linear, by not including $$(x-x_0)^2$$ as a basis function; or to do the same for the last, by turning things around.  Not obvious how I can so constrain both.)


 * An approach would be the following:


 * For a set of breakpoints $$x_i$$, and adjustable parameters $$a_{i,j}$$, let $$f(x) = \sum_i\sum_j f_{i,j}(x)$$, with $$f_{i,j}(x) = \begin{cases}

a_{i,j}\left(x-x_i\right)^j & x_i \le x < x_{i+1}, 0 \le i < m \\ a_{m,0} & x = x_m \\ 0 & \text{otherwise} \end{cases}$$


 * Continuity then requires: $$\forall i < m, \sum_j a_{i,j}\left(x_{i+1} - x_i\right)^j = a_{i+1,0}$$


 * First order differentiable requires: $$\forall i < m,\sum_jj\,a_{i,j}\left(x_{i+1} - x_i\right)^{j-1} = a_{i+1,1}$$


 * Periodic requires: $$\forall j, a_{0,j} = a_{m,j}$$


 * Linear ends requires: $$\forall j > 1, a_{0,j} = a_{m,j} = 0$$


 * Etc. Dragons flight (talk) 07:42, 6 September 2016 (UTC)


 * I hate to seem ungrateful but have you done anything here other than express my desiderata more formally? I already have a program that does a least squares fit with basis functions
 * $$f_{0,j}(x) = x^j, j < k $$
 * $$f_{i,k}(x) = \begin{cases} 0 & x < x_i \\

\left(x-x_i\right)^k & x_i \le x \end{cases}$$
 * (Continuity comes from having no $$f_{i,j}$$ for $$i>0,j<k$$. For this project, $$k=2$$.)  What I don't know is how to choose weights consistent with the least squares principle and the other constraints. —Tamfang (talk) 09:48, 7 September 2016 (UTC)
 * I assumed you knew how to do the fit since you already said you are doing the fit.
 * Given a target function $$g(x)$$, $$\chi^2 = \int_{x_0}^{x_m}\left(f(x)-g(x)\right)^2 dx = \int_{x_0}^{x_m}\left( \sum_i\sum_j f_{i,j}(x) -g(x)\right)^2 dx$$
 * (Here I assuming you want to measure the fit by moving uniformly in x, if instead you want to measure the fit by moving uniformly along the path length of g(x), then you would need to modify the definition of $$\chi^2$$ and adjust the following equations accordingly.)
 * For this project x is path length, so no worries! —Tamfang (talk) 10:22, 8 September 2016 (UTC)


 * This is minimized by solving the system of equation generated by:
 * $$0 = {\partial \over \partial a_{i,j}}\int_{x_0}^{x_m}\left( \sum_p\sum_q f_{p,q}(x) -g(x)\right)^2 dx = \int_{x_i}^{x_{i+1}}\left( \sum_q f_{i,q}(x) -g(x)\right) \left(x-x_i\right)^j dx$$
 * $$= \int_{x_i}^{x_{i+1}}\left( \sum_q a_{i,q}\left(x-x_i\right)^q - g(x)\right) \left(x-x_i\right)^j dx$$
 * $$\Rightarrow \sum_q a_{i,q} \int_{x_i}^{x_{i+1}} \left(x-x_i\right)^q \left(x-x_i\right)^j dx = \int_{x_i}^{x_{i+1}} g(x) \left(x-x_i\right)^j dx $$
 * For every choice of i and j you have one equation and one unknown, $$a_{i,j}$$, so it is simply a matter of solving the resulting matrix equation to find the unknown parameters.


 * The above describes the general version, with no constraints, not even including continuity. To add constraints, you take the constraint equations mentioned above and use them to replace unknowns in your matrix equation with the corresponding constraints expressed in other variables.  After replacement, you simply solve for the remaining unknowns to get your answer.  Dragons flight (talk) 11:34, 7 September 2016 (UTC)


 * Okay, I need to absorb this some. —Tamfang (talk) 18:13, 7 September 2016 (UTC)


 * I crossed out the linear constraint because, for the project I have in mind, I thought of how to do (better) without it. —Tamfang (talk) 18:13, 7 September 2016 (UTC)

For the periodic case, the basis functions can be 1 and (x_0-x_i)(x-x_i)(x_m-x) & x_i \le x \end{cases}$$ For most of my other constrained cases, I can use a three-part quadratic with double roots at $$x_i, x_{i+3}$$ and breaks at $$x_{i+1}, x_{i+2}$$. —Tamfang (talk) 04:14, 11 September 2016 (UTC)
 * $$f_i(x) = \begin{cases} (x_m-x_i)(x-x_0)(x_i-x) & x \le x_i \\

solutions
Eventually I did find a systematic way to cope with all sorts of boundary constraints. For the periodic case, start with piecewise basis functions for 0 ≤ p < n, where $$a_j$$ solves $$f_p^{(k)}(x_0) = f_p^{(k)}(x_m)$$ for 0 ≤ k < n, where n-1 is the required degree of continuity; for other kinds of end constraints, find any function that fits those constraints and subtract it from the "input" function. In any case, use as basis functions a family of (n+1)-piece solitons with $$g_j^{(k)}(x_j) = g_j^{(k)}(x_{j+n+1}) = 0, k x_j$$

Compactness Theorem and Model theory:
Can one prove, that every set - being a model of every finite sub-class of a given class of first-order axioms - is a model of that class?

Please notice that Compactness Theorem is not of help here. HOTmag (talk) 11:27, 5 September 2016 (UTC)
 * Modeling a collection of axioms is defined as modeling each individual axiom. So it's a triviality.--2406:E006:384B:1:2DF4:6B68:2A07:A694 (talk) 09:07, 6 September 2016 (UTC)