Upper limit for v.8

Check in here for general discussion about Search Version 8

Moderator: Mods

Upper limit for v.8

Postby edsager » Sat Jul 20, 2013 12:16 am

Hello All,

I will be upgrading to v.8 from v.6.7 some time in the next 30-45 days or so and I wanted to be prepared for any upper limits to the index.

I tested X1 v.7 for awhile and found that it handled the large number of files it indexed better than 6.7, but still not very well. X1 would hang for anywhere from a few seconds to so long that I had to End Process on it to get to other work.

FYI: I have nearly 3 million files listed as having been indexed.

1. What is the upper limit of files which X1 can index?

2. What is the recommended upper limit?

Thanks,
edsager

Computer specs:
Windows 7 Pro 64-bit
16GB DDR3-1866
Total HDD memory: 2 TB
OS HDD: Enterprise Level WD 500GB 7200 rpm
edsager
 
Posts: 19
Joined: Wed Mar 26, 2008 1:19 pm

Re: Upper limit for v.8

Postby Coast » Sat Aug 17, 2013 2:25 pm

Well that's sort of exactly my question that I posted in a new topic. I hadn't seen yours. Sorry.

I have 4 GB of web files which generate 4,000,000 documents. I was up to 1,500,000 but had some problems. It may or may not have been the cause for X1 to pause for a long time. I decided to repair the problem and start over. I'm at 125,000 files now.
Coast
 
Posts: 5
Joined: Sat Aug 17, 2013 1:53 pm

Re: Upper limit for v.8

Postby Kenward » Sun Aug 18, 2013 8:25 am

I currently index 123GB GB of PDF files and web pages, including nearly 9GB of web pages, without any problems when it comes to performance.
MK
X1 Search 8.6.1 - Build 6003fa (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4149
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Re: Upper limit for v.8

Postby Coast » Sun Aug 18, 2013 8:27 am

Yea sorry, I meant 400 GB of files.

AMD 8 core 3.1 Ghz 16 GB RAM Wndows 7 Pro
Coast
 
Posts: 5
Joined: Sat Aug 17, 2013 1:53 pm

Re: Upper limit for v.8

Postby edsager » Sun Aug 18, 2013 11:31 pm

Thank you for your replies.

Nice to know others' experiences with large numbers of items indexed.

I would love to hear more from anyone.

Thanks,
edsager
edsager
 
Posts: 19
Joined: Wed Mar 26, 2008 1:19 pm

Re: Upper limit for v.8

Postby Kenward » Mon Aug 19, 2013 4:42 am

My experience is that while X1 can happily index a mountain of files, it is not as swift as it could be when you want to search through a large number of small files, web pages for example.

While it can quickly find things in a load of PDF files, searching in a much smaller (by size) set of many more (by number) web documents is slow and flaky.

I have a saved search of web pages, it is slow to load and is less stable than other document types.

And this is on a PC that many people would consider overpowered.
MK
X1 Search 8.6.1 - Build 6003fa (64-bit)
Windows 10 Pro 64-bit | Windows 10 Home 32-bit
No, I have nothing to do with X1, just a user since 2004.
Kenward
X1 Guru
X1 Guru
 
Posts: 4149
Joined: Tue Apr 20, 2004 2:35 am
Location: UK

Re: Upper limit for v.8

Postby renatosobral » Wed Sep 14, 2016 2:23 pm

Hi All,

I'm experiencing some problems while indexing 1,25TB of files, of many extensions.
Several errors are registred at X1ContentExtraction.log file and if I stop X1 Search, it doesn't save the last index. Last time, I lost 700.000 files!

Does anyone knows something about this behavior ?

Thanks

Renato
renatosobral
 
Posts: 1
Joined: Wed Sep 14, 2016 2:12 pm

Re: Upper limit for v.8

Postby poy » Fri Sep 16, 2016 6:10 am

Hi Renato,
your experience matches mine.
If you index Millions of items, X1 becomes pretty unpredictable.
Today I am at 5.3Mio items. Whenever I start my notebook, X1 reports to do an initial scan.
The info box reports "last indexed" from about a month ago.
Best,
()
- < ( Connection with X1 Service has been lost. A restart does not fix this Problem. Would you like to restart X1?
()

Intel® Core™ i7-3740QM Processor 3.70 GHz - 24GB RAM - 1.5TB SSD - Win7 Ultimate 64bit
poy
X1 Power User
X1 Power User
 
Posts: 62
Joined: Wed Jun 27, 2012 11:33 am


Return to General Discussion V8

Who is online

Users browsing this forum: agrohimdij, agrohimoqn, agrohimvay, jackrilt, Mihaelgur and 28 guests

cron