Like seriously, otherwise you're opening a nation-state level attacker, or even just a manufacturer that *doesn't* happen to have your interests in mind, the possibility of backdooring like, everyone who's required to use these.
Hardware keys where the private key can't be pulled off them can be useful, *assuming* users can upload their own keys to them.