Turns out HTTP Host headers can have a trailing dot (totally valid too) and nginx will happily forward it to php-fpm making $_SERVER['HTTP_HOST'] have a trailing dot which then fails all WP multisite domain lookups...
First time seeing this on a fresh nginx + php 8.4 setup.