Search found 302 matches

by captquirk
Thu Aug 10, 2023 6:29 pm
Forum: Sphider Help
Topic: Sphider-5.0.0 - Indexing Keyword issues
Replies: 20
Views: 6072

Re: Sphider-5.0.0 - Indexing Keyword issues

It appears the FULL pages are not being indexed, only the top portion.
Now, I have to figure out WHY!!!
by captquirk
Thu Aug 10, 2023 4:24 pm
Forum: Sphider Help
Topic: Sphider-5.0.0 - Indexing Keyword issues
Replies: 20
Views: 6072

Re: Sphider-5.0.0 - Indexing Keyword issues

I am going to try a special scan from my end, focusing on a particular problematic page. If we can get those short words to index... fingers crossed! Your settings look good. Only suggestion, which has nothing to do with indexing, is to un-check "Display the RSS search form " in the "...
by captquirk
Thu Aug 10, 2023 4:20 pm
Forum: Sphider Help
Topic: Problem with indexing - Timeout occurred
Replies: 7
Views: 3631

Re: Problem with indexing - Timeout occurred

I have no experience with CloudFlare. However, I do know CloudFlare addresses web security, and one of those issues is DDoS. The timeout you are getting is a 524, which is specific to CloudFlare. Sooo, as someone who has no knowledge of the workings of CloudFlare, I am going to make a wild guess! Sp...
by captquirk
Wed Aug 09, 2023 11:56 pm
Forum: Sphider Help
Topic: Sphider-5.0.0 - Indexing Keyword issues
Replies: 20
Views: 6072

Re: Sphider-5.0.0 - Indexing Keyword issues

Post a snapshot of your settings. I'll see if anything jumps out at me.

Try spidering with a sitemap.xml, also. In a lab environment, creating a usable sitemap may be a a challenge, but give it a try. If you do that, it isw best done with an index and not a re-index.
by captquirk
Sat Jul 29, 2023 4:01 pm
Forum: Sphider Help
Topic: Sphider-5.0.0 - Indexing Keyword issues
Replies: 20
Views: 6072

Re: Sphider-5.0.0 - Indexing Keyword issues

A way of circumventing the issue with the index.html is to create a sitemap.xml, then do an index using that. Re-index using a sitemap does not work so well, but a clean index will. Re-index actually looks at existing pages for changes and adds new pages based upon an new references in those changed...
by captquirk
Fri Jul 28, 2023 4:19 pm
Forum: Sphider Help
Topic: Sphider-5.0.0 - Indexing Keyword issues
Replies: 20
Views: 6072

Re: Sphider-5.0.0 - Indexing Keyword issues

As things slowly calm down here, I can spend more time looking at this, although it seems you have made good progress. Contents of tables DO get indexed. Some words do not get indexed because they are in a file "include/common.txt". This probably is not the case for your acronyms, but ther...
by captquirk
Tue Jul 25, 2023 4:51 pm
Forum: Sphider Help
Topic: Sphider-5.0.0 - Indexing Keyword issues
Replies: 20
Views: 6072

Re: Sphider-5.0.0 - Indexing Keyword issues

Actually, the correct line 1131 is: ."link_log FROM ".$mysql_table_prefix."sites" Instead of removing that one "quote", you had to add a "quote" "dot" on the same line before "$mysql_table_prefix". You had the right idea, though! There were...
by captquirk
Sat Jul 22, 2023 4:21 pm
Forum: Announcements
Topic: Shphider 5.1.0 released
Replies: 0
Views: 4615

Shphider 5.1.0 released

Sphider 5.1.0 corrects a fatal error when using the indexAll() function. An incorrect number of arguments was being passed, due to some missing code. This has been corrected.

SphiderLite was unaffected.
by captquirk
Sat Jul 22, 2023 4:39 am
Forum: Sphider Help
Topic: Sphider-5.0.0 - Indexing Keyword issues
Replies: 20
Views: 6072

Re: Sphider-5.0.0 - Indexing Keyword issues

Okay. Definitely some code missing from the indexAll function. :oops: To correct, in spider.php line 1128, find: $stmt = $db->prepare( "SELECT url, spider_depth, required, disallowed, " ."usesitemap, ignore_robots, can_leave_domain, foreign_images FROM " .$mysql_table_prefix.&quo...
by captquirk
Sat Jul 22, 2023 3:37 am
Forum: Sphider Help
Topic: Sphider-5.0.0 - Indexing Keyword issues
Replies: 20
Views: 6072

Re: Sphider-5.0.0 - Indexing Keyword issues

HORROR!!!
Could I possibly have an error in the code?

That certainly does seem like a possibility, and your posting the error message will be of great aid finding it --- and correcting it.

I'll be back --- with solution in hand.