I see what Thing342 means now. The query I'm using takes 1.3s, compared with 0.16s for the next slowest I saw, with everything else < 0.1s.
Double that, for one active & one active+preview query, and ouch.
I'll leave this on
my own mirror of the site, maybe even make a commit to a separate branch, but not open a pull request just yet.
Eventually I may look into extra tables that can be created during the site update process to speed up the queries.