Search found 302 matches

by captquirk
Mon Jan 18, 2021 4:57 pm
Forum: Announcements
Topic: Sphider 4.0.0-MB and SphiderLite 2.0.0 released
Replies: 0
Views: 24644

Sphider 4.0.0-MB and SphiderLite 2.0.0 released

The latest releases feature a revised way of doing backups and restores, relying solely on MySQL backup and restore utilities. This increases speed and, more importantly, reliability of these processes. Also, a limited ability to recover from interrupted re-indexing has been implemented. More detail...
by captquirk
Fri Jan 01, 2021 12:07 am
Forum: Sphider Help
Topic: Missing lines in search ressults
Replies: 7
Views: 15358

Re: Missing lines in search ressults

Very happy that worked out for you. Upgrades are always encouraged as each new release is more liable to include minor improvements and bug fixes than new features. Upgrades are really quite easy. Download the newest version, extract it somewhere out of the way. Read the changelog. Note any files wh...
by captquirk
Thu Dec 31, 2020 9:36 am
Forum: Sphider Help
Topic: Missing lines in search ressults
Replies: 7
Views: 15358

Re: Missing lines in search ressults

Okay. It may be the the full text of the affected pages did not properly get put in the database. This is due to an issue discovered with determining page character sets. This is corrected in 3.6.0, but there is an easy hack for 3.5.2. In admin/spider.php, line 484, find: $fulltxt = removeEmoji($ful...
by captquirk
Thu Dec 31, 2020 3:37 am
Forum: Sphider Help
Topic: Missing lines in search ressults
Replies: 7
Views: 15358

Re: Missing lines in search ressults

BAscreen.png
BAscreen.png (95.04 KiB) Viewed 15354 times
settings.png
settings.png (116.44 KiB) Viewed 15354 times
This is what I get using the settings seen above. This is using SphiderLite 1.3.0.

If you use a different version and settings, let me know and I will check it.
by captquirk
Thu Dec 31, 2020 3:02 am
Forum: Sphider Help
Topic: Missing lines in search ressults
Replies: 7
Views: 15358

Re: Missing lines in search ressults

I will investigate and let you know what I find.
Which version are you running?
by captquirk
Wed Dec 23, 2020 9:55 pm
Forum: Sphider Help
Topic: All title tags are showing as Untitled document
Replies: 7
Views: 15126

Re: All title tags are showing as Untitled document

A definitive fix has been found! It seems that under certain circumstances, the regex on line 856 of spiderfuncs.php entered a runaway state. The way to fix this has FINALLY been found. spiderfuncs.php, line 856 should read: if (preg_match("@<title.*?>(.*?)<\/title.*?>@si", $file, $regs)) ...
by captquirk
Thu Dec 17, 2020 7:57 pm
Forum: Sphider Help
Topic: All title tags are showing as Untitled document
Replies: 7
Views: 15126

Re: All title tags are showing as Untitled document

SPECIFICALLY FOR kraisor: I found a way to capture the titles! The regex normally used gets mangled somewhere. It is a valid script, but at some point it stops working. I HAVE FOUND A REGEX WHICH DOES WORK! This is specifically for apnews.com and will not work for others. Edit admin/spiderfuncs.php,...
by captquirk
Thu Dec 17, 2020 4:30 pm
Forum: Sphider Help
Topic: All title tags are showing as Untitled document
Replies: 7
Views: 15126

Re: All title tags are showing as Untitled document

TO OTHER USERS WHO MAY EXPERIENCE THE SAME PROBLEM OF TITLES BEING BLANK: Sphider looks for the title tags, and the script (as of 3.5.2 and Lite 1.2.2) is looking for the form <title> or <title >. If you happen to have any attributes in the tag, the title won't be found. This is an easy fix: Edit s...
by captquirk
Thu Dec 17, 2020 1:25 am
Forum: Sphider Help
Topic: All title tags are showing as Untitled document
Replies: 7
Views: 15126

Re: All title tags are showing as Untitled document

Weird indeed! There is SOMETHING in the page that prevents Sphider from seeing the title tags. If I copy the source html into a local file and scan it with Sphider, the tags are not found. If I delete everything from the css down and save that fragment, title tags are found and read. I have been una...
by captquirk
Wed Dec 16, 2020 5:47 pm
Forum: Sphider Help
Topic: All title tags are showing as Untitled document
Replies: 7
Views: 15126

Re: All title tags are showing as Untitled document

Issue is confirmed! The 'title" column of the links table is not being populated for apnews.com. Now to figure out WHY! UPDATE: Sphider is unable to find the title tags (<title> and </title>). In the case of apnews.com, there are TWO causes for this. The first is a simple code fix, but the seco...