I was wrong. I'm running into an issue with my ssh config. I get some issue with ssh public key authentication:
When I try to connect, pubkey auth fails and I have to use a password (which I want to eventually disable).

On the client it fails in sign_and_send_pubkey stage saying agent refused operation.

On the server it fails because the host key type "they" sent are not supported.

The weird thing is: neither my client nor the server use the host key types listed in this log message - so I'm pretty much at the stage of trying to look into openssh's source code to figure out the exact conditions for this error and get pubkey ssh auth working.

See: babka