I think it's something inherent to Mutter
The rounded edges you see in GNOME today are typical in GTK4+Libadwaita apps or GTK3+Libhandy apps, it's not something Mutter can do (yet, maybe in the future)
There are "artificial" ways to support rounded edges in all apps, at the cost of GPU consumption, which is Rounded Window Corners
macOS supports them because afaik only has 1 toolkit for applications, and on Windows 11 not all apps support them.
https://discourse.gnome.org/t/rounded-window-corner-support-for-all-non-gtk-apps/8756