Search found 128 matches

by captquirk
Sat Apr 06, 2019 2:11 am
Forum: Sphider Help
Topic: Problem of HTML entities and quotes in search result
Replies: 4
Views: 729

Re: Problem of HTML entities and quotes in search result

Probably a filter somewhere. I'll take a look. -------------------------------------------- UPDATE 4/6: There seems to be a twofold issue. Using Sphider v2.3, none of the full text of pages from http://www.equiterre.fr is being stored. This is probably related to the truncation issue. Trying a diffe...
by captquirk
Sat Mar 16, 2019 6:00 pm
Forum: Sphider MODS
Topic: Cause spidering to pause after indexing a specified number of pages
Replies: 0
Views: 673

Cause spidering to pause after indexing a specified number of pages

If you wish to cause spidering to pause for a short time after indexing a certain number of pages, an easy edit can make this possible. Let's say that after indexing 30 pages, you want Sphider to take a five minute break, then resume for another 30 pages. Open spider.php. Find "function indexUrl". R...
by captquirk
Mon Mar 11, 2019 2:33 am
Forum: Sphider Help
Topic: "Show only the 2 most relevant links from each site" broken in 2.3.1
Replies: 0
Views: 554

"Show only the 2 most relevant links from each site" broken in 2.3.1

Another of the deprecated each() function replacements has come back to haunt Sphider 2.3.1. This has been corrected for the next release, but for those you can't wait, the fix is easy. Open the include/searchfuncs.php file and look for this code: $k = $domains_to_show[$domains[$key]][0]; $v = $doma...
by captquirk
Tue Feb 26, 2019 5:03 pm
Forum: Announcements
Topic: Sphider 2.3.0 released in legacy and PDO editions
Replies: 0
Views: 542

Sphider 2.3.0 released in legacy and PDO editions

Sphider 2.3.0 was released late 25 February 2019. Details can be seen in the change logs found at https://www.sphider.worldspaceflight.com.
by captquirk
Thu Jan 31, 2019 10:47 pm
Forum: Sphider Help
Topic: .com search at Sphider 1.42
Replies: 13
Views: 1722

Re: .com search at Sphider 1.42

Your code for limiting the number of links Sphider fetch from each domain works just fine! That's good. It has occurred to me that you may want to keep an eye on the 'temp' and 'pending' tables. Since indexing may stop before ALL the links are processed, these tables may contain "leftovers". "Clear...
by captquirk
Thu Jan 31, 2019 3:20 am
Forum: Sphider Help
Topic: .com search at Sphider 1.42
Replies: 13
Views: 1722

Re: .com search at Sphider 1.42

I mean to tell Sphider he has to stop when fetch 40 or 50 links from a domain.
In PDO 2.2.0, in spider.php, line 1009:

Code: Select all

while ($count < count($links)) {
You can try:

Code: Select all

while ($count < count($links) && $num < 50) {
I HAVE NOT TESTED THIS! NO GUARANTEE! It SEEMS like a good idea though. :lol:
by captquirk
Thu Jan 31, 2019 2:55 am
Forum: Sphider Help
Topic: .com search at Sphider 1.42
Replies: 13
Views: 1722

Re: .com search at Sphider 1.42

I mean to tell Sphider he has to stop when fetch 40 or 50 links from a domain. Hmmmm. Interesting. I'll have to dig into that one when I get some time. As far as eliminating advance options for an image search, that may be easier than I thought. In PDO 2.2.0, in search.php, find on line 331: global...
by captquirk
Wed Jan 30, 2019 10:41 pm
Forum: Sphider Help
Topic: .com search at Sphider 1.42
Replies: 13
Views: 1722

Re: .com search at Sphider 1.42

Is there a way to make the Images tab look like the Search tab? Without all the advanced choices? YES! In fact, that is one of the changes coming with Sphider 2.3. You will have the ability to mix and match with the search tabs, so an Image search can be a stand alone choice. [** I believe I misund...
by captquirk
Tue Jan 29, 2019 4:59 am
Forum: Sphider Help
Topic: .com search at Sphider 1.42
Replies: 13
Views: 1722

Re: .com search at Sphider 1.42

Here is the code translated from mysql to mysqli (like that found in 1.42). I have no way of testing this. My PHP version (7+) is such that 1.42 is too antiquated to run! But the code does look correct as far as the sql is concerned. The other code fragments do not need converting. It would just be ...
by captquirk
Tue Jan 29, 2019 2:50 am
Forum: Sphider Help
Topic: .com search at Sphider 1.42
Replies: 13
Views: 1722

Re: .com search at Sphider 1.42

Looking at the posted code, specifically the last section (everything before is preparation for making the last section work), I notice: // Search for URL's that were already indexed. What is actually being search for is URL's that have been indexed. That is already available. When you look on the S...