Okay, so it turns out there are two types of #NeoVim 
tutorials / documentations:
1. Just copy these lines to init.vim, run this command and you get what I have! (and if anything gets updated too much, or you want to add some additional stuff, well, you'll find an other tutorial for that that will not be compatible with this one)
2. These are the variables you can set and these Lua commands are available. (there is no default config, no clue what files to add...)
So I followed a guide that uses vim-plug to handle plug-ins, did some minimal config that's good enough for me, but one of the plug-ins was updated with breaking changes, and I wanted to add some debugging tools, and now I'm told to scrub all of my configs and just start from zero using "NeoVim Chad"...
So I guess in 2 years time when my needs change again, I'll just wipe the config again and start from zero with another manager...
@nicemicro one can choose the commit version of a plugin so that if a breaking change happens, your config still works
@barefootstache yeah I ended up going into the directory and checkout the last working version with git anyways.