Performance comments

X1 ® Professional Client Version 6.6

Moderator: Mods

Performance comments

Postby Vidyarthi » Wed Jul 01, 2009 1:03 pm

I installed 6.6 Build 3764 yesterday, and noticed some things that don't work as well as previously.

* Indexing Outlook 2003 is now slower than previously. My perception is that it takes as long to index a message as it does a PDF file, whereas messages were previously much faster.

[Added 7-9-2009: This aspect has improved markedly over time. It looks like the comment that the index needed time to be refreshed by the new software version was correct since this is back to normal now.]

* Right clicking to open an indexed file or to go to a folder didn't work at first but now it does. Response time is Ok, but I can't determine what changed from not working to working state.

[Added 7-9-2009: This aspect still seems slower than it used to be, but this is a subjective observation and not based on testing so it's accuracy is questionable.]

Possibly excluding x1 application and index files from on-access av scanning would probably help, but my software only excludes by individual file and not by folder.
Last edited by Vidyarthi on Thu Jul 09, 2009 6:16 am, edited 1 time in total.
Vidyarthi
 
Posts: 17
Joined: Thu Dec 15, 2005 8:03 am

Postby Kenward » Thu Jul 02, 2009 1:43 am

Which build did you have before?

I haven't seen any reports of AV slowing things down.

Did you get into a full reindex? That takes time, depending on how much email you have.

You might want to wait a bit before looking for causes. A big index can take a day or so.

Did you change the indexing settings for email? For example, do you "Keep copies of email ion index".
MK
X1 Search 8.5.2 - Build 6001si (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4068
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Re: Performance comments

Postby Vidyarthi » Fri Jul 03, 2009 11:29 am

Thanks for your comments.

I have just under 92,000 files included in the folders to be indexed excluding folders) and of those about 17,000 are unsupported file types. The rest are indexed up to 250MB in size. The x1 folder that holds the index files is 3.23GB.

The place that I notice the performance hit is when typing characters in a search box. For example when excluding folders from the count in the previous paragraph I started typing -fol and then the system froze for 10 seconds or so before I could type the rest.

There could be some memory paging out to the hard drive. My RAM is 2GB, but that is no different than before installing the newer version.

There is that pesky .NET family update that I haven't been able to install, so if X1 is based on .NET programming that might have something to do with it.

Scott
Vidyarthi
 
Posts: 17
Joined: Thu Dec 15, 2005 8:03 am

Postby Kenward » Fri Jul 03, 2009 2:29 pm

You are making seriously heavy demands on X1.

A lot of files, and a massive upper limit on their size. Do you really need to index files up to 250MB? My limit is 60MB. (Mostly PDF files, nearly 100,000 of them.) If I find anything bigger than that, X1 helps there, I try to slim it down.

Your index files are also pretty big. If you can bring yourself to do it, then maybe reindexing from scratch would help.

New versions of X1 do depend on .NET, but I would have thought that it would fail completely if it could not find what it needed on that front.

Unfortunately, I also find that X1 is slow to respond to typing in search terms. Always has been.
MK
X1 Search 8.5.2 - Build 6001si (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4068
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Postby tjh » Fri Jul 03, 2009 3:25 pm

I also find that closing X1 and using something like defraggler to defrag the X1 Databases helps performance somewhat. They get very heavily fragmented quickly.

You must be sure to close X1 fully before doing this though, otherwise you cause issues and X1 might reindex from scratch.
TiM
X1 Search v8.5.1 - 6001se (64 Bit)
tjh
X1 Super User
X1 Super User
 
Posts: 398
Joined: Sat Apr 12, 2008 4:12 am
Location: Napier, New Zealand

Postby Vidyarthi » Sun Jul 05, 2009 3:16 pm

I wasn't sure if my usage was typical or not.

In my work I aim towards using much less paper, so documents are printed to Acrobat first and then to paper if and when needed. I generally prefer PDF files over native format for whatever application creates them.

Virtually all scanned documents are OCR'd to make them searchable.

I regularly run SpeedDisk to minimize fragmentation. I also partition my harddrive to separate applications from data, and manage disk usage to maintain at least 25% free space. I'm up to a 320GB drive now, with 250GB dedicated to data.

Larger files make better use of disk space than a large number of small files. I've got just over 900 indexed files over 10MB, the largest just under 200MB.

I reformat and reinstall everything almost twice a year, and the drive I am currently using was set up at the beginning of June. Keeping all of the installation files and registration/activation information on a live drive helps immensely. Re-indexing is a pain, but the benefits of a periodic software refresh makes it very much worthwhile.

As far as choice of indexing software I can't say that I did an extensive search, but X1 was one of the first that I tried and it's been so great that I haven't felt the need to look for an alternative.

Unlike the filesync software that I use for backups. I had to switch from powersync to goodsync recently to better handle few-to-many relationships.

Scott
Vidyarthi
 
Posts: 17
Joined: Thu Dec 15, 2005 8:03 am

Postby Kenward » Mon Jul 06, 2009 1:42 am

Your work practice pretty well mirrors my own. But I use separate hard drives for programs and data. With a 1TB drive costing less than £60 now, around $100, this is the cheapest bit of the package.

I am still surprised by the size of your PDF files. My biggest is <100MB.

My PDFs come mostly from external sources, or from scanned newspaper cuttings.

I agree that this is a more efficient way of storing data than other formats. If you area a PDF fiend, then you really have to bite the bullet and get hold of a copy of Acrobat, despite the rip off prices that Adobe charges, especially for foreign buyers. Then you can get the software to reduce the size of your PDF files. I have seen truly sensational shrinkage of files in this way. You can knock as much as 90 per cent off the size of a file that was created with massive graphics.

Where you write "Larger files make better use of disk space than a large number of small files." you really need to factor in the file system you use. I don't think you will find much difference with an NTFS system. FAT16 was the real killer.

In any case, given the low cost of storage that I have already referred to, I am less interested in storage efficiency than in speed and such factors as the ease of working with pother software. X1 limits the size of indexed files for a reason. Big files slow it down. Likewise de-fragmenting is more efficient and quicker if it has small files to work on.

If anything, I would go for a lot of small files rather than fewer big ones.

I wonder if X1 has done any research on this.
MK
X1 Search 8.5.2 - Build 6001si (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4068
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Postby Vidyarthi » Wed Jul 08, 2009 5:46 am

One more thing, Kenward, in my office we have several copies (licensed of course!) of X1.

In general, I would like each one to use the same index settings for files and/or folders.

I've tried copying the [LocalFilesIndexingOptions] section from X1IndexService.ini, but the results haven't seemed to match my expectations.

Do you know of another technique, or am I just not recognizing success when I have it?

Thanks,

Scott
Vidyarthi
 
Posts: 17
Joined: Thu Dec 15, 2005 8:03 am

Postby Kenward » Thu Jul 09, 2009 1:28 am

I have never tried to transfer settings between PCs. I have X1 on desktop and laptop, and they are too different to warrant synchronised settings.

The only stuff that I transfer regularly is my set of "saved searches".

The idea of transferable, or group, settings seems like a good one.

The approach you have tried seems like the only option at the moment. Maybe you need to investigate other bits this and other .ini files.

Perhaps one of the X1 team can advise.
MK
X1 Search 8.5.2 - Build 6001si (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4068
Joined: Tue Apr 20, 2004 2:35 am
Location: UK


Return to X1 Professional Client 6.6

Who is online

Users browsing this forum: No registered users and 5 guests