Hi,
I have a website (www.indioptions.com). Google has indexed some pages but not all. Approx 250 pages are indexed but Google has crawled over 2000 pages. I don't understand why Google crawls but does not index pages. Any idea?

Google Webmaster tools show crawl rate of ~1000 pages per day but only 3-5 get added on a day

Nitin

First, I would set up a redirect from your non www version to your www version.

Does Google webmaster tool show you any error in the html suggestions?

Do you mean redirecting http://indioptions.com to http://www.indioptions.com?

If yes, will surely set it up.

I see 1-2 entries in HTML suggestions. Not for every page that is not being indexed.

Anything else I should check.

Nitin

there are many other factors invoved with google indexed. you should check if there are some dup content in your websites.

Originally Posted by nitinsy

Hi,
I have a website (www.indioptions.com). Google has indexed some pages but not all. Approx 250 pages are indexed but Google has crawled over 2000 pages. I don't understand why Google crawls but does not index pages. Any idea?

Google Webmaster tools show crawl rate of ~1000 pages per day but only 3-5 get added on a day

Nitin

Have you already submitted sitemap on google webmaster tools?

I have submitted sitemap. URL count in sitemap is correct (1000+) but indexed URLs in sitemap is much less (< 50)

Nitin

Very interesting thread.
Regarding Google's Webmaster Tools is there a way to see which pages have been indexed ?
Eg my site map has 7 urls but only 6 have been indexed. Webmaster Tools does not report any errors. I would like to know which page has not been indexed !

in google you have only 387 rezult and you say the sitemap have 1000 link.
But in error wath thel to you?
i ask becouse i have this problem and the probem ar robot.txt ,dont wory you can submit 50000 link to google in sitemap

Don't worry after few days all your site pages are index by Google so no need to worry.

I think thats a normal activity of Google. Google crawl most of the pages and index only when it considers it important and helpful. So dont worry these figures changes often and as the site will grow old it will index all pages.

Regards

I have seen that happen before. Google seems to download pages much faster than it caches and indexes them. The four of five pages a day seems about the same as what happens to my blog. Eventually, Google usually indexes almost all of it, but it may take a while.

Trent Guidry

I think part of your problem is that the posts are scraped from forums, and the LARGE amount of tags on the posts details pages. Google looks at them as probably duplicate content ? Your current top post on the forums, when you view the details page has 6 lines, the post tags has around 100 lines ?

Duplicate content can be the major factor. If you want to get your duplicate contents index then get as much more links for that webpage

In principle if you have a site with lots of words and links, where the words lead to good relevant links on those words, helping users get what they want - then google, when it finds your site, will spider it intensively, exhaustively measure the relevance of your link and respond to the natural growth of human-made links to pages of yours. You will experience growth over time.

But if you make a page which just tries to mimic that, in the end you are against the grain, and what you build is built on sand.

Either way, you have to wait, and let Google digest data, follow up links, measure your credibility. If you are cheating them, no doubt it's an anxious wait! But if you build honestly, with effort not templates and mass databases, the wait is just a bit tiresome, but not angst ridden.

I remember how I found out about Google giving away fat traffic to responsible webmasters. I'd been building stuff I wanted to build and then out of the blue I found lots of my pages appearing in google and bringing in lots of happy users, happy customers.

I have strolled around your site - it fails on so many levels.

1. For humans it is a disgusting page - very annoying. Lots of "google-coloured" links (a bit of blue and green and black and white, your usual google colours) and various buttons for arsebook, myarse, twatter and so on. It is just a meaningless typical junk page, to the eye, with links which look too dull to click on.

2. For robots it is an even more disgusting page - a trail of external links, all of them dull and of a particularly annoying 'genre' - aka social media crap - tonnes of people's garbage, whether news sources or chat sources, listed in a long list to bore google's robots to death


Here's a little clue about what they actually like

1. Humans - love sites where when they go to your pages one thing they say is "wow! this is a page with information I can't find anywhere else"

2. Robots - love sites which give them a lot of well-structured link collections which have been categorized using a lot of effort and literacy, so that google's robots can enhance their primary advantage, lexical ability, just by "using" your pages - if you feed google's robot genuine intelligence, google feeds you valuable traffic which most of us can convert directly into cash one way or six others


There it is. That's the secret.

Secret 3 is - PATIENCE.

Be patient. Very patient.

Thanks everyone for the feedback. Will do some work on this over the next few weeks. Hopefully will churn out better organized and focussed information

Thanks again
Nitin

And update on the crawl:
Google is crawling average of 1100 pages per day
Indexed total of 1340 pages (site:www.indioptions.com)

Webmaster tools shows only 57 pages have been indexed from site maps. But I can see much more in actual search results. Don't know why the difference.

I agree so much with gunther. Take a bit of time to learn and redesign your site and just be patient. You'll get there, google is a mysterious beast.

it takes time to index pages , but google does it , Be patient..

I suggest submitting your sitemap to google and reindex or link build work site. Best is through forums and social bookmarking.

Yea if the problem is serious then only sitemaps and social bookmarking can help you. Connect to the best do follow sites with high PR you can also get some caching and PR juice.