User:Shitikanth/sandbox/Multiplicative Weights Update Method

In computer science, multiplicative weights update method (MWU) is an algorithm design paradigm that is used for various optimization and learning problems. A multiplicative weights update algorithms works by maintaining a distribution over a set and using the multiplicative update rule to change these weights iteratively.

Multiplicative weights update method were proposed as an algorithm design paradigm by Arora, Hazan and Kale in 2008, who noticed that this idea was at the back of many algorithms in economics, machine learning and convex optimization.

The analysis of a multiplicative weights update algorithm usually involves quantifying the change in a potential function in each iteration.

Algorithm

 * 1) for i = 1 to n
 * $$ w_i (t) \leftarrow 1 $$
 * 1) $$ \Phi(1) \leftarrow n $$.
 * 2) for t = 1 to T
 * $$ p_i(t) = w_i(t)/\Phi(t) $$
 * Sample an i from the probability distribution $$p_i(t)$$ and follow expert i's advice.
 * Observe the loss obtained by the experts - $$m_i(t)$$
 * $$ w_i(t+1) = w_i(t).(1-\varepsilon)^{m_i(t)}$$
 * $$ \Phi(t+1) = \sum_i w_i(t+1)$$

Analysis
Theorem: For any $$ \varepsilon \le \frac 1 2 $$, the expected loss incurred by the above algorithm is bound as follows:

$$\sum_{t=1}^T \langle p(t), m(t) \rangle \le (1+\varepsilon) \sum_{t=1}^T m_i(t) + \frac{\ln n}{\varepsilon}$$

(The inequality holds for all i = 1 to n, and, in particular, for the expert that minimizes the losses).

Proof:

$$\Phi(t+1) = \sum_{i=1}^n w_i(t) ( 1 - \varepsilon)^ {m_i(t)} $$.

Since $$(1-\epsilon)^x \le (1-\epsilon x) $$, for $$ 0 \le x \le 1 $$,

$$\Phi(t+1) \le \sum_{i=1}^n w_i(t) ( 1 - \varepsilon {m_i(t)} ) = \Phi(t) - \varepsilon \langle w(t), m(t) \rangle = \Phi(t)(1 -\varepsilon \langle p(t), m(t) \rangle)$$.

Since $$ e^x \ge 1 + x$$ for all $$x$$,

$$\Phi(t+1) \le \sum_{i=1}^n w_i(t) ( 1 - \varepsilon {m_i(t)} ) = \Phi(t) - \varepsilon \langle w(t), m(t) \rangle = \Phi(t) (1-\varepsilon)^{ \langle p(t), m(t) \rangle}$$.

Therefore, by induction,

$$ \Phi(T+1) \ge n (1-\varepsilon)^{\sum_{i=1}^T \langle p(t), m(t) \rangle} $$

Furthermore, $$ \Phi(T+1) = \sum_{i=1}^n w_i(T+1) \ge w_i(T+1) = (1-\varepsilon)^{\sum_{t=1}^T m_i(t)} $$.

The theorem follows immediately from the above two inequalities.

Algorithm

 * 1) for i = 1 to n
 * $$ W(1) \leftarrow \mathbb I $$
 * 1) $$ \Phi(1) \leftarrow \text{Tr}(W(1)) = n $$.
 * 2) for t = 1 to T
 * $$ \rho(t) = W(t)/\Phi(t) $$
 * Follow expert's advice according to $$\rho(t) $$
 * Observe the loss matrix for day t - $$M(t)$$
 * $$ W(t+1) = \exp\Big(-\varepsilon \sum_{i=1}^t M(t) \Big)$$
 * $$ \Phi(t+1) = \text{Tr}(W(t+1))$$

Analysis
Theorem: For any $$ \varepsilon \le 1/2 $$ and for any $$ v \in \mathbb{C}^n $$, the expected loss incurred by the above algorithm is bound as follows:

$$(1-\varepsilon)\sum_{t=1}^T \langle \rho(t), M(t) \rangle \le \sum_{t=1}^T v^\ast M(t) v + \frac{\ln n}{\varepsilon}$$

Category:Algorithms