Hacking the 301 error

Post Reply
User avatar
captquirk
Site Admin
Posts: 297
Joined: Sun Apr 09, 2017 8:49 pm
Location: Arizona, USA
Contact:

Hacking the 301 error

Post by captquirk »

Many times, the 301 error which has been our nemesis, is "real" in that IS what is reported to Sphider. I can duplicate this by other methods. HOWEVER, in some cases there IS NOT relocation!

Wild guess... webmaster is sending a 301 to stop applications like Sphider.

I found a work around. First off, this work around basically IGNORES 301 errors, EVEN IF IT IS LEGITIMATE. So I definitely would not want to make this a part of a published version of Sphider.

Now to the "fix" --- more of a HACK, actually!

In Sphider 4.2.0 and 4.2.1, lines 334 and 335, or Sphiderlite 2.2.0 and 2.2.1, lines 311 and 312, find:
// $url_status['state'] = "ok";
// $url_status['content'] = "text";
Remove the "//" to uncomment the lines and save.

If there really is content on those (formerly) 301 pages, it will now index.

As I said, I wouldn't recommend this to be a formal part of Sphider, just a hacker's approach to index the un-indexable.

Remember to restore the "//" to comment those lines out again when you are finished! There IS such a thing as a genuine 301 error.
Post Reply