<p dir="ltr">Hey Peter (B) and maintainers,</p>
<p dir="ltr">I spoke to Dago a few days ago, and we had a chat about a large portion of traffic from our main mirror being just the catalog files, that is, the files named 'catalog' that are downloaded and re-downloaded a countless number of times. The mirror can withstand it, but it's a constant stream of a few megabytes per second, day and night. Perhaps this can be helped by using the conditional GET with the possible HTTP 304 Not Modified response, or timestamping. wget has an option to timestamp files, and it can issue just a HEAD request to skip downloading the whole file. Here's some information I found:</p>
<p dir="ltr"><a href="http://www.gnu.org/software/wget/manual/wget.html#Time_002dStamping">http://www.gnu.org/software/wget/manual/wget.html#Time_002dStamping</a></p>
<p dir="ltr">Have we considered this in the past? I don't recall it. Maybe we should take a look, it could be simple to implement, and we would save some bandwidth on our main mirror and on other mirrors worldwide.</p>
<p dir="ltr">Maciej</p>