Sphider Help Forum

CPanel and mysqlnd ... This can get tricky. Read this Post - https://www.blog.worldspaceflight.com/2019/07/02/php-shared-hosting-and-mysqlnd/ Essentially, in CPanel you need to CHECK mysqlnd and UNCHECK mysqli! Now allow me to rant for a bit... Sphider requires BOTH mysqli AND mysqlnd. This is true ...

A 500 error is a hard to track server error. Since the installation went fine, and having gotten the login screen, install.php and auth.php ran perfectly. The crash occurred in admin.php. There is a script which can test your PHP installation. https://www.sphider.worldspaceflight.com/downloads/requi...

Many times, the 301 error which has been our nemesis, is "real" in that IS what is reported to Sphider. I can duplicate this by other methods. HOWEVER, in some cases there IS NOT relocation! Wild guess... webmaster is sending a 301 to stop applications like Sphider. I found a work around. ...

When an indexing run gets interrupted, Sphider has always had the ability to pick up where it left off. Re-indexing, however, is a different process. If a re-index gets interrupted, the only option has been to start over. Sphider 4.0.0, 4.0.1, and 4.0.2 (SphiderLite 2.0.0, 2.0.1, and 2.0.2) introduc...

Sorry for the long delay since the last post, but here is what I have found... Many times, the 301 error which has been our nemesis, is "real" in that IS what is reported to Sphider. I can duplicate this by other methods. HOWEVER, in some cases there IS NOT relocation! Wild guess... webmas...

Hopefully the sitemap.xml mod will help in indexing sites. If testing doesn't show any serious problems, the mod will be incorporated into the next release. What I HAVE noticed, however, then even when indexing with a sitemap, the very first (landing) page has to be able to be indexed, If that ends ...

In the case of https://yourstory.com/ , the Sphider nod found in the MODS section of this forum and titled "Index from sitemaps when sitemap is a list of sitemaps" will allow you to index using their sitemap. BUT be forewarned: the sitemap references 759 other sitemaps with a total of 61,3...

Sphider can index from sitemaps if they are simple sitemaps. If the initial sitemap is a list of links to additional sitemaps (popular with larger websites), it doesn't work. This mod shows promise to correct that. There may be situations which interfere in this working, but only testing will identi...

I have been looking into this. Basically, web pages are evolving in a manner which Sphider is incapable of understanding. I have a blog post addressing this and the ramifications.

https://www.blog.worldspaceflight.com/2 ... -obsolete/

The method of having a sitemap.xml consist of a list of additional sitemaps is perfectly valid. However, is is used primarily by very large sites in which a single sitemap would be HUGE! The maps are reduced to a manageable size, then referenced by a single master. I MAY look into a procedure to rea...