Well, there never was any law that required robots.txt to be honored. Big players like Google do, but I am not aware of any consequences if they wouldnt (of course UNTIL it is then regulated).