@hackernews@die-partei.social

For those who want a quasi-preview::::

Contents
List of figures 5
Foreword 6
I Foundations 8
1 Machine Learning 9
1.1 Learning from data . . . . . . . 10
1.2 Basis function regression . . . . 12
1.3 Under and over-fitting . . . . . 13
1.4 Categories of models . . . . . . 15
2 Efficient computation 17
2.1 GPUs, TPUs, and batches . . . . 18
2.2 Tensors . . . . . . . . . . . . . . 20
3 Training 22
3.1 Losses . . . . . . . . . . . . . . 23
3.2 Autoregressive models . . . . . 26
3.3 Gradient descent . . . . . . . . 29
3.4 Backpropagation . . . . . . . . 33
3.5 Training protocols . . . . . . . 38
3.6 Training data . . . . . . . . . . 41
II Deep models 43
4 Model components 44
4.1 The notion of layer . . . . . . . 45
4.2 Linear layers . . . . . . . . . . . 47
4.3 Activation functions . . . . . . 56
4.4 Pooling . . . . . . . . . . . . . . 59
4.5 Dropout . . . . . . . . . . . . . 62
4.6 Normalizing layers . . . . . . . 64
4.7 Skip connections . . . . . . . . 68
4.8 Attention layers . . . . . . . . . 71
4.9 Token embedding . . . . . . . . 78
4.10 Positional encoding . . . . . . . 79
5 Architectures 81
5.1 Multi-Layer Perceptrons . . . . 82
5.2 Convolutional networks . . . . 84
5.3 Attention models . . . . . . . . 91
III Applications 98
6 Prediction 99
6.1 Image denoising . . . . . . . . . 100
6.2 Image classification . . . . . . . 102
6.3 Object detection . . . . . . . . . 103
6.4 Semantic segmentation . . . . . 108
6.5 Speech recognition . . . . . . . 111
6.6 Text-image representations . . . 113
7 Synthesis 116
7.1 Text generation . . . . . . . . . 117
7.2 Image generation . . . . . . . . 119
The missing bits 122
Afterword 126
Bibliography 127
Index 134