Follow
The original comparison of Adam vs AdamW is quite memorable.
On a related note, I absolutely recommend reading through reviews of that publication
https://openreview.net/forum?id=rk6qdGgCZ
