@koreymoffett I've hosted my own email for years (and my first email was one hosted by one of my parents.) It's not as bad as people say although you do need to have certificates now (which means letsencrypt+certbot really) and dmarc (which means a decent domain registrar)
Also ISPs have blocked hosting email on residential connections for a very long time so you'll want a VPS.
I use OpenSMTPD (with a hackish custom patch to allow marked address so I can catch companies putting me on spam lists) but I've used postfix, and plan9's "qmail" (which is unrelated to the qmail package on most distros.) None of them are *too* hard. Also some domain registrars will host email on your domain for free or a very small fee.