System Settings, Desktop & Dock, scroll down to “Click wallpaper to reveal desktop” and set it to “Only in Stage Manager”.

I’ve been using my M1 Air with a 55" 4k tv for the last 3 years and I have not experienced anything you’ve talked about here. Are you going through a native HDMI port or a dock? I use a dock and it’s been perfect, but I’ve heard there are a lot of bad docks that are extremely poor when using HDMI. I genuinely have zero complaints about the M1 Air.