Hacker News new | past | comments | ask | show | jobs | submit login Sourcehut will blacklist the Go module mirror ( sourcehut.org ) 216 points by Tomte 2 hours ago | hide | past | favorite | 102 comments ComputerGuru 1 hour ago | next [–] From the GitHub issue, by a Googler: > For boring technical reasons, it would be a fair bit of extra work for us to read robots.txt […] This is coming from one of the biggest, richest, most well-staffed companies on the planet. It’s too much work for them to read a robots.txt file like the rest of the world (and plenty of one-man teams) do before hammering a server with terabytes of requests. If this is too much for them then no wonder they won’t implement smarter logic like differential data downloads or traffic synchronization among peer nodes. reply psanford 50 minutes ago | parent | next [–] Why d...