"Stellent is not available" again

This forum is to discuss general things about X1.

Moderator: Mods

"Stellent is not available" again

Postby Kenward » Tue Oct 30, 2007 3:18 am

Now that the x1 6 forum is closed, I am raising here the resurrection of the "Stellent is not available" message in the Indexing Status.

I noticed it late last night, a few hours after installing the release version.

So, to rule out any carry over from the previous release, I cleared the index and had it restart.

It worked away over night. I now have 44,882 files with this status.

There was no sign of this behaviour in the previous release.

As it stands, X1 is now useless to me if it cannot index more than half of my PDF files.

Previous advice suggests that X1 is not likely to correct these errors without a complete reindex.

I seem to have permanently deleted the previous version. Is there any way that I can get another copy to see if it works better?
MK
X1 Search 8.6.1 - Build 6003fa (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4149
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Postby Kenward » Tue Oct 30, 2007 3:14 pm

I found the previous release. (Build 3563) But that is broken too.

22,018 PDF files successfully indexed.

52,238 "Stellent is not available"

Not what I would call encouraging.

This fault appeared late in the development of X1 version 6. A bit too late for comfort.
MK
X1 Search 8.6.1 - Build 6003fa (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4149
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Postby Kenward » Wed Oct 31, 2007 9:24 am

I may have found a possible explanation for this behaviour.

Yesterday, while X1 was in the middle of a yet another reindex brought about as a result of this issue, I was monitoring the indexing of PDF files. I did this with a set of custom searches for PDF files. These filtered the Indexing Status and created views for "Text extractor process" to find files where the "Text extractor process died".

I also have views for:

"=OK"

"-OK Skipped badfile="

"-OK"

"-OK Skipped protected"

and

"Stellent is not available"

It is this last one that is a sign that X1 has fallen over.

The other important one is "Text extractor process".

This finds just a handful of files. I took this to mean that I have a handful of PDF files (about half a dozen) that are OK but slightly damaged.

In the past I have just opened and resaved these files to create a new version that will index. This time, in the process of doing this the previously empty view "Stellent is not available" suddenly started to fill up at a rate of knots. It soon overtook the "=OK" search oin the number of bits.

At the suggestion of someone at X1, I sorted the general PDF view, with nothing in the "Indexing Status" slot, on "Date Last Indexed".

This showed that all was well until a file with the status "Text extractor process died" appeared in the list. All files before that were mostly "OK" while everything after was ""Stellent is not available".

My theory is that working on a "process died" file actually managed to kill the Text extractor completely, resulting in a different error.

If I am right, then the folks at X1 need to detect what is going on. If the extractor dies, and won't restart, X1 needs to strop indexing all together.

I am not going to test this theory by editing another "process died" file. I do, though, have some dodgy PDF files for further experimentation.

Does anyone else find PDF files of the "Text extractor process died" variety?
MK
X1 Search 8.6.1 - Build 6003fa (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4149
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Postby KVeil » Wed Oct 31, 2007 4:40 pm

(Apologies if this the wrong forum, but the RC II Build 3565 is locked!)

I hope that build 3565 fixes this and and also the "Application Error" crashes that I have been havng since Build 3560 ... :shock: (see http://forums.x1.com/viewtopic.php?p=10487#10487

Klaus
X1 V6.3575 | Win XP SP2 | Outlook 2003 | 2.0GHz | 1 GB RAM

i use skype - http://www.HL7.com.au/Skype.htm
KVeil
 
Posts: 19
Joined: Wed Jul 25, 2007 3:42 am

Postby Kenward » Thu Nov 01, 2007 3:03 am

Are you saying that you have seen the same issue?

Do you get a lot of files with ""Stellent is not available" as the indexing status?

So far, I seem to be the only victim.

If you do have this issue, can you post the details of your PC here? The X1 folks asked me for mine as a way of trying to pin down the issue.

I have not seen any "Application Error" crashes.
MK
X1 Search 8.6.1 - Build 6003fa (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4149
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Postby KVeil » Thu Nov 01, 2007 8:26 am

kenward,

No, I have not so far seen these error messages. My point was that since Build 3560 I have experienced (and documented here) a number of "Application Error" crashes.

I am now testing if Build 3565 is more stable...

Klaus
X1 V6.3575 | Win XP SP2 | Outlook 2003 | 2.0GHz | 1 GB RAM

i use skype - http://www.HL7.com.au/Skype.htm
KVeil
 
Posts: 19
Joined: Wed Jul 25, 2007 3:42 am

Postby Kenward » Sat Nov 03, 2007 10:04 am

Not the same issue at all then.

Perhaps you should wait and see if X1 6.0 solves your problems.

If not, you will probably get a more sensible response if you start your own new discussion with as much detail as you can must for the issue, and include as much information as possible about your system. This seems to help the X1 folks to pin down errors.
MK
X1 Search 8.6.1 - Build 6003fa (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4149
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Content of PDF files not indexed

Postby nobody » Thu Jan 10, 2008 12:56 am

I just downloaded and installed X1 ® Professional Client Version 6.0
(Build 3565) (Created: Thu Oct 25 10:46:55 PDT 2007) and PDF files are not having the content indexed.

Indexing Status - Stellent is not available.

There may be a work around by not using Acrobat until all indexing is completed, but that makes the program too unreliable to use on a regular basis.

Nice program. Too bad it doesn't work.
nobody
 
Posts: 2
Joined: Thu Jan 10, 2008 12:47 am

Postby Kenward » Thu Jan 10, 2008 3:31 am

This turned out to be an odd bug that the folks at X1 just could not replicate.

I spent quite some time trying to troubleshoot this issue. In the end, I think the solution turned out to be to create a view that showed me all PDF files that showed "Indexing Status - Stellent is not available" and to move them. X1 eventually managed to index them.

No an elegant solution, I know, but it worked. I no longer have any PDF files with this disease.
MK
X1 Search 8.6.1 - Build 6003fa (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4149
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Postby nobody » Thu Jan 10, 2008 9:10 am

Kenward wrote:
I think the solution turned out to be to create a view that showed me all PDF files that showed "Indexing Status - Stellent is not available" and to move them. X1 eventually managed to index them.


Kenward, Thanks for your suggestion.

The problem I'm experiencing seems to have been known for some time. Searching for "problem" files and then moving them is putting X1 in charge of my computer rather than having it work for me.

How can I trust searches by X1 if I may have files that might not have the content indexed? Searches would become 2-step. First search for files displaying the bad indexing status and then search again for what I want?

I have no commitment to X1. I'm looking at other search programs. The one that works is the one I'll use.
nobody
 
Posts: 2
Joined: Thu Jan 10, 2008 12:47 am

Postby Kenward » Thu Jan 10, 2008 2:42 pm

I understand your argument.

If this were a persistent problem, I might well join you. But ever since X1 completed its indexing, I have never seen this error again.

I agree completely that it is worrying when X1 misses files. I wonder, though, if the alternatives are also missing stuff but not reporting it back to us.

I mention this because I have a set of saved X1 searches that detect indexing errors. Some of these are because I have, for some reason, obtained broken PDF files. X1 can't index them. Then again, nothing can open them, not Adobe Reader or any of the alternatives.

I wonder if the alternatives to X1 can also provide this useful information. Maybe it just gags on the file. Passes over it, and leaves us all in the dark.

That would make an interesting extra bit in pgk's current analysis of X1 and Microsoft's alternative.
MK
X1 Search 8.6.1 - Build 6003fa (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4149
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Stellent is not available

Postby bruceDouglass » Wed Feb 20, 2008 6:08 pm

I am having the same problem - except that NONE of my files are being searched for content unless the search string is in the filename or path. If I look for the string "subsystem" for example, which appears in dozens of Word, Powerpoint, text, HTML, and PDF files, it only finds them if that term appears in the filename or in the path.

I am using version 5.2. I cannot upgrade to version 6 (I've tried a few of the more recent versions including the latest) because it won't run on my XP Home Media edition computer. It fails the first time I try to run it and never runs. It have to close it down with the task manager.

I added Indexing Status column to help debug the problem. In the file list (no search string), almost all have the status "Stellent is not available". What does this mean???

A few have the status - File Extension is not selected for content indexing. Folders have no indexing status. There are no other values for Indexing Status that I've seen.

A few months ago I changed out my XP Home computer for this new one and it worked fine before. I'm not sure exactly when X1 started to fail. I've tried completely uninstalling and removing the directories for X1 and reinstalling them - nothing works.

In the meantime I'm using Google Desktop but I really want to get X1 working again.

So, in summary:
1. What does "Stellent not available" mean anyway?
2. How do I fix this?
3. Did the attempt to upgrade cause the problem even though I uninstalled and did my best to remove residual files from the 6.x install?
4. Why can't I upgrade to a new version of X1?

Configuration: MS WIndows XP Pro
Version: 5.1.2600 Service Pack 2 Build 2600
Physical Memory: 2,048MB
Available virtual memory: 1.96GB
Available disk space: 700GB
bruceDouglass
 
Posts: 11
Joined: Wed Feb 20, 2008 8:13 am

Re: "Stellent is not available" again

Postby shapirojm » Mon Mar 07, 2011 9:22 am

Just so that there's a record in the forum that this problem hasn't gone away, I too discovered the "stellent is not available" error when the index failed to find Acrobat .pdf files that I know for certain, among others, had the information that I was searching for. These files were not damaged, and when opened, Acrobat's internal search function readily found the search terms. I am using Windows 7 - 64 Bit Professional and X1 Version 6.7.2, which is set to index my entire hard drive, with no exceptions. I tried deleting my old index and re-indexing the entire drive, but this didn't fix the problem. I also tried saving a duplicate of one "problem" file, which didn't solve the issue. Upon further investigation, I have found more than 4,000 .pdf, .doc and other standard documents on my drive that returned the "stellent not available" error in X1.

I have tried re-saving one of the offending files under a different name, but that had no impact. Will also try optimizing the file size in Acrobat to see if that helps. Have contacted X1 technical support, and they have indicated that they will look into the issue.
shapirojm
 
Posts: 25
Joined: Thu Mar 03, 2011 8:42 am

Re: "Stellent is not available" again

Postby barclaybrown » Sat Aug 20, 2011 12:33 pm

Problem is here too. This is the kind of problem that MANY people may have and not know it. The only way to find it is to notice that a search result that should be showing up is not showing up, and then know enough to add the indexing status column to your display to see how man items are affected. Last time I indexed, I had 284,007 items with "stellant is not available" out of a total of about 650,000 files. I deleted the indexes and at this moment 2,925 items have this status out of 89,214.

I see no replies from X1 on this. Did you get any reply from them?

Is there any way to force X1 to try to reindex an item?

Anyone found a better workaround, like indexing in safe mode or whatever?

Thanks!
barclaybrown
 
Posts: 6
Joined: Sat Aug 20, 2011 12:28 pm

Re: "Stellent is not available" again

Postby Kenward » Sun Aug 21, 2011 2:21 am

Which version of X1 do you have? My version – see signature – does not show anything with this error.

I do have 617 PDF files, out of 143,000 files, that do not show as "OK" in their indexing status. But these are for other reasons, including some that are password protected.

That error certainly used to be prevalent, but the last report of this was for one version before the most recent.
MK
X1 Search 8.6.1 - Build 6003fa (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4149
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Re: "Stellent is not available" again

Postby barclaybrown » Sun Aug 21, 2011 5:46 am

I have the very latest versions of everything -- X1, Win 7, very powerful computer, etc. Geek here.

OK, found a workaround, at least for me. Stop using the "everything" setting in the "What to scan for this folder" on any folder. I changed to only using the second option ("File names and sizes, plus content for specified extensions"). For the initial indexing, I left only the default filetypes. No "Stellant not available" status, at least so far. I'll be adding back file extensions that I find that it should be able to index (like "pwd" for the old pocket word from handheld computers of many years ago) and see how it does.

Actually I just noticed that I do have 17 pdfs that have a status of "text extractor process died". Is there any way to get X1 to re-try these files? They range in size from 1.8m to 25m. Would the X1 people like any of these to find out why they won't index? I also seem to have about 75 other files where the extractor died, mostly .exe and a few .htm files.

My guess at what's going on, from the past posts and my own experiences, is that when you use the "everything" option, X1 tries to parse the content of every file, and on some files the text extractor crashes. This somehow causes Stellant to not work, at least for a while and X1 isn't smart enough to go back and re-try to parse those failed files. Of course, better behavior would be for the extractor to not crash, but just to move on to the next file. My guess about why this happens is that some software uses a standard file extension (like say .doc, but probably not one as common as this) for a file OTHER than a normal .doc, and this is a case that X1 (or Stellant) isn't expecting, thus it crashes. Perhaps I have some files like this that most people don't have and so I triggered it. I also have has as many as 640,000+ files in my index, and some go back 14 years, so there's a wide selection and some might be unusual and from very old software.

X1 could work around Stellant here by being smart enough to re-try to parse failed files at least say 10 times, some on later runs of X1, like the next day, or at startup, etc. As it is, it seems once Stellant fails on a file, X1 never tries again.

Of course I don't know the internals, so these are guesses. But if you're getting the errors, try the workaround above.

I am also surprised that I'm just now finding out about all these files that X1 isn't indexing! I have been using X1 for over 5 years and the fact that X1 has failed on a number of files is WELL HIDDEN from the user! I had no idea there were thousands, now just dozens of files that are just not indexed. I had always wondered why some things that should have showed up, didn't. I would suggest that a little summary show up somewhere of the files that X1 has found, index, failed on, etc. so the user can know. In fact, it's STILL not easy to tell since I don't know all the Indexing Statuses, and which ones are failures vs. successes. Maybe there's another field that just shows whether it was indexed or not. But that's my point--you can't tell!

Barclay
barclaybrown
 
Posts: 6
Joined: Sat Aug 20, 2011 12:28 pm

Re: "Stellent is not available" again

Postby shapirojm » Sun Aug 21, 2011 10:44 am

I worked extensively with X1 technical support on this issue, and sent a couple sample files for review. I have to compliment Chris in tech support for spending a lot of time and effort on the issue.

The conclusion with respect to the sample files was that the .pdf's were not properly formed either due to processing issues with Acrobat or because a third-party .pdf program was used to create the file. The problem was missing "lookup" tables. Technical support referred me to the following article:

http://forums.adobe.com/thread/758316?tstart=0

from that post:

"As a PDF stores the character codes rather than the human-readable text, the fact you can see a letter "A" on the page doesn't mean Acrobat has any idea that it's an "A". The lookup tables make that connection, so if they're missing or corrupted there's no way to recreate the semantic connection unless you can re-fry the file with an original copy of the font."

The inability to index text shows up in X1, but the problem seems to be improperly built PDF files. I was able to confirm problems with the .pdf files that I had sent to X1 for review. I didn't check all of the problem files, of which there were many on my system...too many to check one by one.

Technical support indicated that they would continue looking to see if a work-around was possible. One other thing to bear in mind is that X1 doesn't index some secured .pdf files.

The only work-around that we discussed was printing the problem .pdf files to a post-script file and then printing them back to .pdf format (I didn't do this since I'm not familiar with how to print to a post-script file and didn't want to spend more time on this issue).

You can force a re-index by duplicating the file with another name, but I'm not sure if there's another way.
shapirojm
 
Posts: 25
Joined: Thu Mar 03, 2011 8:42 am

Re: "Stellent is not available" again

Postby Kenward » Sun Aug 21, 2011 11:18 am

shapirojm wrote:The only work-around that we discussed was printing the problem .pdf files to a post-script file and then printing them back to .pdf format (I didn't do this since I'm not familiar with how to print to a post-script file and didn't want to spend more time on this issue).

You can force a re-index by duplicating the file with another name, but I'm not sure if there's another way.

Nice research.

An alternative approach might be to put the offending files through an Acrobat batch that processes them and resaves. I expect Acrobat to create properly formed files.

I do this all the time with files that are way bigger than they need to be. So I put them through Acrobat's "optimise" routine. This can save as much as 90 per cent of the file's disk space.

I will try this with a few of my own dodgy files.

PS This seems to work. You just have to have Acrobat or something like it to process the files.
MK
X1 Search 8.6.1 - Build 6003fa (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4149
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Re: "Stellent is not available" again

Postby Kenward » Mon Aug 22, 2011 2:45 am

It might help to add other ways to "fix" PDF files that X1 sees as "broken", even though they work just fine in viewers.

As well as using Acrobat 7 to "shrink" files – I run this as a batch process – I have "mended" files using PDF Converter 7 to "Optimize" PDF files. This also removes them from X1's list of files that do not show as "OK" in "Indexing Status".

Anyone found other ways to make files, PDF or otherwise, behave properly when thrown at X1?
MK
X1 Search 8.6.1 - Build 6003fa (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4149
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Re: "Stellent is not available" again

Postby shapirojm » Thu Sep 01, 2011 10:54 am

Following up on the problem that I've had with the Stellent and .pdf files, I have a several files that I have tried to get indexed without success. In trying to resolve the problem, I have 1) Exported a file to an .xps image and OCR'd the image to Acrobat, and 2) Tried printing it as a postscript file and imported it back into Acrobat using a GhostScript program. I also tried two commercial Acrobat file repair utilities on the original and re-scanned versions of files, which indicated that they successfully "repaired" the files. Nevertheless, each version of these files that I've created ends up with the "text extractor process died" or "stellent not available" error messages. The various versions of the files range from 47 - 300MB, and other files that I have in this range index successfully.

Also, I'm wondering why X1 doesn't index .xps files? I tried printing sample .pdf files to .xps, but then found that X1 apparently doesn't index .xps format at all. I'm not that familiar with .xps, but text is searchable in .xps viewer, which means the format is more than just a "picture" image of a page that one would think X1 could handle.

All quite frustrating...
shapirojm
 
Posts: 25
Joined: Thu Mar 03, 2011 8:42 am

Re: "Stellent is not available" again

Postby Kenward » Fri Sep 02, 2011 2:01 am

shapirojm wrote:Also, I'm wondering why X1 doesn't index .xps files? I tried printing sample .pdf files to .xps, but then found that X1 apparently doesn't index .xps format at all. I'm not that familiar with .xps, but text is searchable in .xps viewer, which means the format is more than just a "picture" image of a page that one would think X1 could handle.

All quite frustrating...


It would be frustrating were it true.

Try this:

>>Tools
>>Options
>>Indexing
>>Files
>>Select Files to Scan
>>Specify File Types

In the pane at the bottom, add ".XPS"

Then see if X1 can find and display any files.

That works here.

And the good news is that unlike some file types X1 can actually preview xps files. Not as well as PDF files, and not always automatically, but it previews all the same.

It is odd that X1 has not included this in the standard set of file extensions. Maybe they can add it in a future version.

This one certainly needs some work. Even when it has indexed and found my xps file, the status line describes it as an "excluded extension". Maybe it needs time to catch up.

Thanks for making me go and look at that one. I did not realise that X1 had not indexed what looks like the only xps file on my PC.
MK
X1 Search 8.6.1 - Build 6003fa (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4149
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Re: "Stellent is not available" again

Postby shapirojm » Tue Sep 13, 2011 9:40 pm

I didn't recall that you could add a special suffix into the indexing list and had given up when I didn't see .xps in the checkbox options for file types to be scanned. Nevertheless, I'm unhappy to report that I got the "text extractor process died" error on the .xps version of the file that I'm continuing to troubleshoot (it's not the only problem file, just one that I've continued using in troubleshooting as my "case study"). Creating the .xps should have made an entirely new version of the file that I'm testing with, yet it still can't be indexed. I then went back from .xps and converted to .pdf...still failed. Tried postscript...failed. Tried going from postscript back to .pdf...failed. It makes no sense to me why any defect in the original file would not be purged by what I've done with these various attempts. If it weren't for the fact that I've got thousands of pages of these files, I'd print and scan them back in.
shapirojm
 
Posts: 25
Joined: Thu Mar 03, 2011 8:42 am

Re: "Stellent is not available" again

Postby Kenward » Wed Sep 14, 2011 1:42 am

It would help if there were something more useful in the way of a diagnostic message.

The bald "text extractor process died" isn't much to go on.

My thinking would lean toward an odd character set as the only common thing between those formats. But that seems most unlikely.
MK
X1 Search 8.6.1 - Build 6003fa (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4149
Joined: Tue Apr 20, 2004 2:35 am
Location: UK


Return to General Discussion

Who is online

Users browsing this forum: agrohimtme, Majestic-12 [Bot] and 42 guests

cron