Robots.txt is a nice convention but it's not law AFAIK. User agent strings are IMHO stupid - they're primarily about fingerprinting and tracking. Tailoring sites to device capabilities misses the point of having a layout engine in the browser and is overly relied upon.

I don't think most people want these 2 things to be legally mandated and binding.