This is just a quick follow-up on my previous blog post about creating a dynamic robots.txt handler.Having a robots.txt tells the search engines what content to index, but it’s not a guarantee that a…