I would have simply added a robots.txt to the site.