Reply
Question About Robots and Adwords
Old 03-05-2008, 03:55 AM Question About Robots and Adwords
Junior Talker

Posts: 4
Name: Andew Sellers
I’ve been searching the forum and found any info about this question. From what can tell, Google penalizes organic rankings for duplicate content.
Our main site is being designed for SEO (we use index/follow tags). But we have several AdWords minisites that we use that have duplicate content of the main site (landing pages are modified a bit according to AdWords keywords). For these types of AdWords minsites, I’m wondering what would be the best index/follow tag to use? I don’t want the duplicate content on the minisites to compete/pentalize our main website, but I also want the AdWords spiders to make full use of them (to keep my AdWords Landing page quality score up, etc).
Is it suggested to use noindex/follow on these minisites so that the AdWords will index them, but the Search Engines won’t try to index them and see them as duplicate content? Or is it safe to use noindex/nofollow so that the engines will see that we are not trying to create duplicate sites, but will this tag still be safe to use with AdWords. Any suggestions would be greatly appreciated.

Last edited by chrishirst : 03-05-2008 at 04:20 AM. Reason: bad fonts - re-edited to make it readable
dsellers is offline
Reply With Quote
View Public Profile
 
Sponsored Links (We share ad revenue):
 
Old 03-05-2008, 04:24 AM Re: Question About Robots and Adwords
chrishirst's Avatar
Super Moderator

Posts: 10,765
Location: Blackpool. UK
Quote:
From what can tell, Google penalizes organic rankings for duplicate content.
Maybe your search skills need honing a little
http://www.webmaster-talk.com/the-go...e-content.html

Block your landing pages in robots.txt The adwords bot ignores robots.txt directives for landing pages that are specified in the ads.
__________________
Chris. ->> Links are advertising NOT optimising!! <<-
Indifference will be the downfall of mankind, but who cares?
Code Samples | People Counting System
chrishirst is online now
Reply With Quote
View Public Profile Visit chrishirst's homepage!
 
Old 03-05-2008, 04:36 AM Re: Question About Robots and Adwords
Junior Talker

Posts: 4
Name: Andew Sellers
Thanks for the info. Just to check, would this be the recommeneded meta tag to block robots on the adwords landing pages (I'm still new to this):

<meta name="robots" content="noindex,nofollow" />
dsellers is offline
Reply With Quote
View Public Profile
 
Old 03-08-2008, 04:15 PM Re: Question About Robots and Adwords
chrishirst's Avatar
Super Moderator

Posts: 10,765
Location: Blackpool. UK
Yes, but that may also stop the Adwords bot and lower the quality score

http://www.robotstxt.org/robotstxt.html

http://www.robotstxt.org/meta.html

https://adwords.google.com/support/b...y?answer=38197
__________________
Chris. ->> Links are advertising NOT optimising!! <<-
Indifference will be the downfall of mankind, but who cares?
Code Samples | People Counting System
chrishirst is online now
Reply With Quote
View Public Profile Visit chrishirst's homepage!
 
Old 03-08-2008, 04:21 PM Re: Question About Robots and Adwords
Junior Talker

Posts: 4
Name: Andew Sellers
I was just behind the times when it comes to robots.txt. I didn't know about it until today (I thought the meta tags controlled these things). Anyway, I found this help page:

How do I block or allow Googlebot?

Based on this, I think this what I should put in the robots.txt file of my ministes (and set the meta tags to follow/index):

User-agent: Adsbot-Google
Allow: /
User-agent: Googlebot
Disallow: /
User-agent: Googlebot-Mobile
Disallow: /
User-agent: Googlebot-Image
Disallow: /
User-agent: *
Disallow: /robots.txt


This will allow the google adwords bot in (so I won't be penalized for adwwords quality score), but it will keep other bots out so I won't be penalized for duplicate content in relation to SEO. I put Disallow: /robots.txt line so that the robots.txt won't get indexed and cached. If I'm not understanding any of this correctly, please let me know.
dsellers is offline
Reply With Quote
View Public Profile
 
Old 03-09-2008, 07:42 PM Re: Question About Robots and Adwords
chrishirst's Avatar
Super Moderator

Posts: 10,765
Location: Blackpool. UK
User-agent: *
Disallow: /robots.txt

remove that line as it effectively blocks bots from reading the robots.txt file once they have read it and cached it.
__________________
Chris. ->> Links are advertising NOT optimising!! <<-
Indifference will be the downfall of mankind, but who cares?
Code Samples | People Counting System
chrishirst is online now
Reply With Quote
View Public Profile Visit chrishirst's homepage!
 
Old 03-09-2008, 09:47 PM Re: Question About Robots and Adwords
Junior Talker

Posts: 4
Name: Andew Sellers
I'll definately look into that. I saw somewhere that that line prevents google from displaying the robots.txt file in organic searches, but come to think of it, when you do searches, you don't see a lot of files such as this
dsellers is offline
Reply With Quote
View Public Profile
 
Old 03-09-2008, 10:22 PM Re: Question About Robots and Adwords
ForrestCroce's Avatar
Half Man, Half Amazing

Latest Blog Post:
North Lake Union
Posts: 2,928
Name: Forrest Croce
Location: Seattle, WA
Google will never show a robots.txt file in their search results. The closes they'll ever come is to show pages that talk about the file when you search for it. It's specifically for 'bots, so that you can tell them how to crawl your site ... it's not a page.
ForrestCroce is offline
Reply With Quote
View Public Profile Visit ForrestCroce's homepage!
 
Sponsored Links (We share ad revenue):
 
Reply     « Reply to Question About Robots and Adwords
 

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off




   
RSS Feed  Feeds: RSS   JS   XML
RSS Feed  Feeds for this forum: RSS   JS   XML


Page generated in 0.14842 seconds with 15 queries