how can i tell if indexing is working
-
- Posts: 23
- Joined: Mon May 31, 2021 5:48 pm
how can i tell if indexing is working
OK REALLY dumb question I do I tell ind indexing is working??
Re: how can i tell if indexing is working
This is UNIX/Linux. No news is good news! LOL! (For Sphider, it seems to think ALL operating systems are Linux! )
Actually, depending on the browser being used, you should notice the page is active. For example, with Firefox you will notice a spinning activity symbol on the tab. Chromium (and I presume Chrome) have similar indicators.
Depending on the size of the site being indexed, indexing can take from less than a minute to several hours! As each page is indexed it should be indicated in the browser window. This is some caching involved with the browser, so there may appear to be nothing for a bit, then suddenly a burst of new lines followed by another apparent pause. There should be a message displayed when indexing is complete.
Another way would be to look in the admin/log directory. The most recent log file will continue to grow until indexing is complete.
In the event indexing dies, post the log file and we can diagnose from there,
EDIT: I might add, DO NOT DISTURB the browser tab or close the browser while doing an index run. That WILL kill the session.
Actually, depending on the browser being used, you should notice the page is active. For example, with Firefox you will notice a spinning activity symbol on the tab. Chromium (and I presume Chrome) have similar indicators.
Depending on the size of the site being indexed, indexing can take from less than a minute to several hours! As each page is indexed it should be indicated in the browser window. This is some caching involved with the browser, so there may appear to be nothing for a bit, then suddenly a burst of new lines followed by another apparent pause. There should be a message displayed when indexing is complete.
Another way would be to look in the admin/log directory. The most recent log file will continue to grow until indexing is complete.
In the event indexing dies, post the log file and we can diagnose from there,
EDIT: I might add, DO NOT DISTURB the browser tab or close the browser while doing an index run. That WILL kill the session.
-
- Posts: 23
- Joined: Mon May 31, 2021 5:48 pm
Re: how can i tell if indexing is working
Such a positive addttude.. My UNIX history has been that quiet is not always good.
so you recommences patience as this is what I have on my Chrome
As you can se your suggested added echo works fine.. I will remove it later
OK I will patiently wait.
so you recommences patience as this is what I have on my Chrome
And here we sit
Continuing suspended indexing.
[Back to admin]
Spidering http://kasdivi.com/
Disallowed files and directories in robots.txt:
http://kasdivi.com/cgi-bin/
http://kasdivi.com/tmp/
http://kasdivi.com/junk/
http://kasdivi.com/templates/
1. Retrieving: http://kasdivi.com/ at 16:47:07.
Size of page: 19.77kb. Starting indexing at 16:47:07.
As you can se your suggested added echo works fine.. I will remove it later
OK I will patiently wait.
Re: how can i tell if indexing is working
Doing a quick test of the site, indexing SHOULD proceed rather quickly.
If you are having problems with sphider exiting prematurely (before completion) and get a lot of "Continuing suspended indexing.", we need to dig deeper.
For starters, in admin/spider.php, lines 34 and 35 control error messages. Comment out 35, uncomment (enable) 34.
Any error messages will appear in the browser window but NOT in the log file. (Of course, they will also appear in the system's PHP error logging.)
Earlier versions of Sphider had trouble with pages presenting 403 and 404, but the latest version handles those pretty well. Timeouts are another issue. Let's hope you don't have those. They are more server related than PHP related can be a real pain in the behind!
If you are having problems with sphider exiting prematurely (before completion) and get a lot of "Continuing suspended indexing.", we need to dig deeper.
For starters, in admin/spider.php, lines 34 and 35 control error messages. Comment out 35, uncomment (enable) 34.
Any error messages will appear in the browser window but NOT in the log file. (Of course, they will also appear in the system's PHP error logging.)
Earlier versions of Sphider had trouble with pages presenting 403 and 404, but the latest version handles those pretty well. Timeouts are another issue. Let's hope you don't have those. They are more server related than PHP related can be a real pain in the behind!
Re: how can i tell if indexing is working
I performed a test index on the site; When I test, I tend to torture Sphider, so I am likely more aggressive than most users.
I allowed image indexing as well, which added considerable to the workload; The index completed after a marathon session of about 7.5 hours.
The site was larger than I had anticipated with:
2991 pages indexed
13639 images indexed (There was also a good number considered too small to index)
7556 keywords
A number of 404 (page not found) errors were also encountered, but this did not affect indexing.
I am sure NOT indexing images would speed up the process considerably! (Experience speaking here.)
Barring connection drops, all that should be required is patience.
I allowed image indexing as well, which added considerable to the workload; The index completed after a marathon session of about 7.5 hours.
The site was larger than I had anticipated with:
2991 pages indexed
13639 images indexed (There was also a good number considered too small to index)
7556 keywords
A number of 404 (page not found) errors were also encountered, but this did not affect indexing.
I am sure NOT indexing images would speed up the process considerably! (Experience speaking here.)
Barring connection drops, all that should be required is patience.
-
- Posts: 23
- Joined: Mon May 31, 2021 5:48 pm
Re: how can i tell if indexing is working
OK. ws able to correct this problem. Issue I was having was that my version of php (7,4.20). did not have iconv() so whole process was killed..
Now indexing semi happily. THANKS!! now I am having issue with images need to se what your documentation says on that
Great patience
Now indexing semi happily. THANKS!! now I am having issue with images need to se what your documentation says on that
Great patience
Last edited by captcurrent on Fri Jun 11, 2021 10:38 pm, edited 2 times in total.
Re: how can i tell if indexing is working
Good to hear you are indexing.
What is the image issue? Maybe I can help.
What is the image issue? Maybe I can help.
-
- Posts: 23
- Joined: Mon May 31, 2021 5:48 pm
Re: how can i tell if indexing is working
You have been a tremendous help already... My site is intended for images but to keep it responsive I can't use the size attributes the pages.
Imagick is therefore important. For php 7.4.20 thee appeal to be 4 different versions, or related ports/pkgs Not sure which one is correct. In working this I went from en error code to Sphider no longer seeing any install. I know you are Linuz so this might e a FreeBSD thingI
Thanks!!
Imagick is therefore important. For php 7.4.20 thee appeal to be 4 different versions, or related ports/pkgs Not sure which one is correct. In working this I went from en error code to Sphider no longer seeing any install. I know you are Linuz so this might e a FreeBSD thingI
Thanks!!
Last edited by captcurrent on Sat Jun 12, 2021 2:00 pm, edited 1 time in total.
Re: how can i tell if indexing is working
I know Imagick can be a pain. My version of Linux is Ubuntu, which is Debian based. Ubuntu has vast repositories which really simplify life.
Free BSD, on the other hand, is a real UNIX. My UNIX experience was on System V (AT&T) and that was several decades ago!
There was a time I installed Imagick on a Windows system... Now THAT was an adventure... of course doing anything on a Windows system can be an adventure.
One final thought... Images displayed in a search are dependent upon a tmp directory in Sphider root. It is different than admin/tmp. The web server has to be able to read/write to that directory, so ownership and permissions should be the same as the directories in admin.
Free BSD, on the other hand, is a real UNIX. My UNIX experience was on System V (AT&T) and that was several decades ago!
There was a time I installed Imagick on a Windows system... Now THAT was an adventure... of course doing anything on a Windows system can be an adventure.
One final thought... Images displayed in a search are dependent upon a tmp directory in Sphider root. It is different than admin/tmp. The web server has to be able to read/write to that directory, so ownership and permissions should be the same as the directories in admin.
-
- Posts: 23
- Joined: Mon May 31, 2021 5:48 pm
Re: how can i tell if indexing is working
i can no longer get sphider to recognize ANY of the 4 options.
Thanks for the tip on the tmp directory if I ever get that far
Di you know which version of Imagick sphider prefers??
Thanks for the tip on the tmp directory if I ever get that far
Di you know which version of Imagick sphider prefers??