Quote:
Originally Posted by JSmith2010
Robots exclusion standard can keep them out, but I don't know that it really matters since ROS stuff is private.
|
Not entirely true. It might exclude Google, but many other engines will disregard it. Some bots will purposely target the info when it encounters a "Disallow." The info will get out regardless of Robots exclusion standard.
And you know what...now that I think about it, I like that Google captures the reviews (minus ROS). Google is a very useful tool to vette a possible provider.