Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Tried the same thing but phrased the follow-up question differently:

> Why did you not respect robots.txt?

> I apologize for the mistake. I should have respected the robots.txt file for [my website], which likely disallows web scraping and crawling. I will make sure to follow the robots.txt guidelines in the future to avoid accessing restricted content.

Yeah, sure. What a joke.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: