User:Rejones7/sandbox

the unconstrained equation $$\mathbf {X} \boldsymbol {\beta} = \mathbf {y}$$ must be fit as closely as possible (in the least squares sense) while ensuring that some other property of $$\boldsymbol {\beta}$$ is maintained

$$\mathbf {X} {\boldsymbol {\beta }}=\mathbf {y}$$

$$\mathbf {A} {\boldsymbol {x }}=\mathbf {y}$$

In constrained least squares one solves a linear least squares problem with one or more additional constraints on the solution. I.e., the unconstrained equation $$\mathbf {X} \boldsymbol {\beta} = \mathbf {y}$$ must be fit as closely as possible (in the least squares sense) while ensuring that some other property of $$\boldsymbol {\beta}$$ is maintained.

There are often special-purpose algorithms for solving such problems efficiently. Some examples of constraints are given below:


 * Equality constrained least squares: the elements of $$\boldsymbol {\beta}$$ must exactly satisfy $$\mathbf {L} \boldsymbol {\beta} = \mathbf {d}$$ (see Ordinary least squares).
 * Regularized least squares: the elements of $$\boldsymbol {\beta}$$ must satisfy $$\| \mathbf {L} \boldsymbol {\beta} - \mathbf {y} \| \le \alpha $$ (choosing $$\alpha$$ in proportion to the noise standard deviation of y prevents over-fitting).
 * Non-negative least squares (NNLS): The vector $$\boldsymbol {\beta}$$ must satisfy the vector inequality $$\boldsymbol {\beta} \geq \boldsymbol{0}$$ defined componentwise—that is, each component must be either positive or zero.
 * Box-constrained least squares: The vector $$\boldsymbol {\beta}$$ must satisfy the vector inequalities $$ \boldsymbol{b}_\ell \leq \boldsymbol{\beta} \leq \boldsymbol{b}_u$$, each of which is defined componentwise.
 * Integer-constrained least squares: all elements of $$\boldsymbol {\beta}$$ must be integers (instead of real numbers).
 * Phase-constrained least squares: all elements of $$\boldsymbol {\beta}$$ must be real numbers, or multiplied by the same complex number of unit modulus.