Paper笔记

Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting

Keyword: spatiotemporal sequence

The key equations of LSTM are:

where $\circ$ denotes the Hadamard product, also known as element-wise multiplication.
\[\begin{align*} & i_t = \sigma (W_{xi}x_t + W_{hi}h_{t-1} + W_{ci}c_{t-1} +b_i) \\ & \\ & f_t = \sigma (W_{xf}x_t + W_{hf}h_{t-1} + W_{cf}c_{t-1} +b_f) \\ & \\ & \bar{c_t} = tanh(W_{xc}x_t + W_{hc}h_{t-1} + b_c) \\ & \\ & c_t = f_t \circ c_{t-1} + i_t \circ \bar{c_t} \\ & \\ & o_t = \sigma ( W_{xo}x_t + W_{ho}h_{t-1} + W_{co}c_{t} +b_o ) \\ & \\ & h_t = o_t \circ tanh(c_t) \end{align*}\]
The key equations of ConvLSTM are:

where $\ast$ denotes the convolution operator and \circ denotes the Hadamard product, also known as element-wise multiplication.
\[\begin{align*} & i_t = \sigma (W_{xi} \ast X_t + W_{hi} \ast H_{t-1} + W_{ci} \circ C_{t-1} +b_i) \\ & \\ & f_t = \sigma (W_{xf} \ast X_t + W_{hf} \ast H_{t-1} + W_{cf} \circ C_{t-1} +b_f) \\ & \\ & \bar{C_t} = tanh(W_{xc} \ast X_t + W_{hc} \ast H_{t-1} + b_c) \\ & \\ & C_t = f_t \circ C_{t-1} + i_t \circ \bar{C_t} \\ & \\ & o_t = \sigma ( W_{xo} \ast X_t + W_{ho} \ast H_{t-1} + W_{co} \circ C_{t} +b_o ) \\ & \\ & H_t = o_t \circ tanh(C_t) \end{align*}\]

Keyword: LE-ConvMN

Retains the spatial and temporal information
Estimate the continuous movement of fingers with sEMG
More recent literature began to shift research focus from discrete movement classification to continuous movement regression
EMG-based continuous motion estimation approaches can be categorized as model-based and model-free approaches
This approach builds the connection between sEMG recordings and hand kinematics
LSTM has been proved more capable on the regression problem or continuous output fitting problem
The key equations of LSTM are: …
The key equations of ConvLSTM are: …
The structure of LE-ConvMN:

Chapter5(HARDWARE IMPLEMENTATION) is very important!!!

Keyword: ESE

memory footprint
- Memory footprint refers to the amount of main memory that a program uses or references while running.
- The word footprint generally refers to the extent of physical dimensions that an object occupies, giving a sense of its size.
- In computing, the memory footprint of a software application indicates its runtime memory requirements, while the program executes.
memory reference
- These instructions refer to memory address as an operand. The other operand is always accumulator.
- Specifies 12-bit address, 3-bit opcode (other than 111) and 1-bit addressing mode for direct and indirect addressing.
ESE optimizes LSTM computation across algorithm, software and hardware stack:
The speech recognition pipeline. LSTM takes more than 90% of the total execution time in the whole computation pipeline:
Load Balance-Aware Pruning
Weight and Activation Quantization: 神经网络模型量化方法简介
Encoding in CSC format and data align using zero-padding
CSC: compressed sparse column
SpMV: sparse matrix vector multiplication
El-emMul: element-wise multiplication
diagonal matrix: 对角矩阵
the overall ESE system
one channel with multiple PEs
the state flow of ESE
Memory management unit