logsumexp

$\text{logsumexp}(x_1,x_2,\dots ,x_n)=\log \left({\sum }_{i=1}^n{e}^{x_i}\right)$

shifted with the logsumexp trick for stability

$\text{logsumexp}(x_1,x_2,\dots ,x_n)={\max }_i{x}_i+\log \left({\sum }_{i=1}^n{e}^{x_i-{\max }_j{x}_j}\right)$

proof:

$$\log \left(\sum _{i=1}^n{e}^{x_i}\right)$$ $$\log \left(\sum _{i=1}^n{e}^{x_i}\right)=\log \left(\sum _{i=1}^n{e}^{x_i-M}\cdot e^M\right)$$ $$=\log \left(e^M\sum _{i=1}^n{e}^{x_i-M}\right)$$ $$=\log (e^M)+\log \left(\sum _{i=1}^n{e}^{x_i-M}\right)$$ $$=M+\log \left(\sum _{i=1}^n{e}^{x_i-M}\right)$$ $$\log \left(\sum _{i=1}^n{e}^{x_i}\right)=\underset{i}{\max }{x}_i+\log \left(\sum _{i=1}^n{e}^{x_i-{\max }_j{x}_j}\right)$$