Quote:
|
do I need to worry about them crawling my page or do I need to setup an htcacess file to keep them at bay?
|
To be safe, set them up in folders just for testing - then move the files to the main area. After all, there's various ways google can find a link to your site, eg your own carelessness. HTaccess is a bad idea because then if it gets crawled google has it pegged as blocked, and you have to wait for it to recrawl and who knows whether you'll get unlucky about the time delay. Leaving it as it is and taking a risk is also bad, simply because it is careless, unprofessional and shoddy.
This is a trivial question worth far less ascii than I have dedicated to it. I couldn't ever see myself ever having asked such trivial questions anywhere on my journey to development expertise... I think you need to try and rely more on your mind and more on your own drive and less on the (often incorrect) advice of those in an arena (social media / consumer generated media) with over 75% deceptive messages, written with the primary purpose of selling in mind.
|