@mentallyalex Interesting idea... seems pretty plausible. Not sure ActivityPub would be the best protocol for it (it's geared more towards text/media posts; that's not to say that it couldn't be twisted into working for this purpose, though).
The biggest concern I'd have with that design would be the question of what happens when someone forks the code and introduces a server that doesn't play by the same rules as others? The interactive options in others' spaces would need to be necessarily limited to prevent abuse.