@worldsendless you might be able to use the profiler (M-x profiler-start, profiler-report) to identify the culprit. It's often an unexpectedly expensive hook. For example in my case ibuffer-auto-mode's post-command-hook was slowing things down so I ended up making a custom version.
