|
specialist search engines are very lucrative, whether it's a local area or a category of site
i built an engine which during 2003 and 2004 sold £131,000 of goods for people whose pages were listed on my search engine...
the whole "there are only 4 search engines" thing is weak and i'd expect a more holistic approach from such hardened techies! still, more profit for me, the less people understand.
matey... who asked the question:
if you want a long list of search engines, take the ones you have found and look at the URLs, particularly at the end
eg if you wanted to make a list of 1000s of forums, you'd look on google for things like
showthread.php
and then add other terms to it, so you can get, for example, 500 forums about politics, and then 500 about cars, and then 500 about india, or whatever.
so to get a list of search engines, maybe there's a script, like search.cgi? (probably not, so find a real one, that's just an example i made up) and look for it. exhaustively.
you can easily build a list of 30,000+ search engines if you have the drive, the time and about a year! such a list would, moreover, be valuable for many many reasons, although perhaps as many as 5,000 would be gone inside a few years (although a further 15,000 would have sprouted in its place, for you to have found, ideally, at that stage, by an employee instead of your own painful labour)
there are easier ways to make money than all this hard work, but if you are interested in search engines, it's worth taking your interest as deep as it goes. the most profitable business in the world today after oil, guns etc, is a search engine (google)! and they're not very good at their job, in many ways - bill gates says so, and he's more objective than they are, about google that is.
use google to find your long list of search engines. google is currently the best source for finding search engines. don't worry if millions of generic spam sites end up in your list - you can throw away the crap later. the important thing is to make a full list of what's out there.
use perl, shell and/or similar stuff to help you manage your list, remove duplicates, etc
do NOT use scripts to search for data on google or any engine... that is bad behaviour
DO use scripts to go to individual websites, and crawl around the whole web, going from site to site, crawling millions of links yourself, on your home computer
if you want to pay me $1000 by paypal i'll do the whole thing for you in a month and anything i can't do in a month would be additional. but i don't recommend that. do the work yourself.
here's a tip - you are prob a windows user.
go and download a Perl environment for your windows. look it up on google. it's easy to find. i forget the name. you download that and you just double click, and you're ready to create a robot to crawl the web - in about 5 mins. if you download and install perl, for free, instantly, i can then give you a 4/5-part script to run, by just typing one command, and it will then start to crawl the entire web for you, gathering billions of sites if you leave it running for months, or run it every day, etc.
i'll charge you $5000 for use of those scripts. paypal. how about it? then you can find all the search engines you want. i don't use it for anything myself - like i said, there are other ways to get rich. but if this one's yours, feel free - 5k is a small investment if you are going to crawl billions of pages, eh?
|
but on the other hand I doubt the OP was looking for PPC search engines and the more specialised search tools given this bit of the post
