I need to run about 20,000 google searches on a single website....

[acemcmac]

this absolutley massive, static and totally custom coded website needs to be combed for orphan pages... I have a list of about 22,000 objects that haven't seen traffic in "a forever" from this site and the site's search engine is so slow... I'm doing the whole thing from google, but just typing in the blank on

<aTARGET="_blank"href=http://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=UTF-8&q=site:[target site]+[part number]</A>

is taking forever... anyone have any faster ideas? I've never been much the programmer, so if there's a VB solution, im totally in the dark...

edited to remove unnecessary info

 

[Eli]

Isn't doing it through google sorta, eh.. error prone?

 

[Cuda1447]

You quit just not do it...

 

[acemcmac]

im just trying to make sure the parts are listed... about 98% of them are... and google has this site totally crawled... I can deal with the missing ones easily, it's just a matter of finding them in the first place.

First I need to find which ones are and aren't listed... that's what im using google for.

Haven't quite figured out how im going to find out which ones are un-linked to... but then they shouldnt come up in the google search, automatically narrowing things down for me.

 

[acemcmac]

Originally posted by: Cuda1447
You quit just not do it...

huh? call me a grammar nazi, but i dont get what you're saying at all.

 

[MikePanic]

so you want to google bomb some stuff huh

 

[hevnsnt]

why not make a page with links to all of them, and just call that page from google and let the spider do all the work?

 

[KingNothing]

I'm confused. Are you building a webpage with links to all the google searches, or do you need to do something with the results of each search?

 

[Parrotheader]

You don't need Google for this (assuming looking for orphaned pages WITHIN the site is your objective.) Google rarely indexes all the pages on a site anyway; it might deep crawl SOME pages, but rarely indexes ALL pages; unless you're managing Amazon I doubt they're going to crawl 22,000 pages from your site. There's software out there specifically designed to check link integrity for big sites. We've used some in the past but I honestly can't remember what it was.

 

[SarcasticDwarf]

download Dreamweaver and use the built-in system?

 

[AgentEL]

MS Visio does the same thing