Stop reindexing archived PST files on every computer restart

[szchren]

I've used XI with Outlook for years and have 30+ archived PST files which do not change, but I do need them in the index to find information from time to time. Yet at each computer reboot all the archived PST files are re-indexed with the currently active OST or PST file and that takes 15 minutes or so. It would be so much better if only the active OST or PST file were re-indexed and added to the index of all those static, archived PST files unless an option demanded otherwise.

[Kenward]

How are these PST files connected to X1?

Like you, I have a raft of archived files and X1 does not seem to spend any time on them.

[szchren]

They were connected automatically (mostly) by XI when installed as Outlook profiles. So I have 40+ profiles each to an archived PST file. Is there a better way?

Thanks for responding.

[Kenward]

So I have 40+ profiles each to an archived PST file.

That's a lot of profiles. I'm surprised that Outlook does not complain.

You can add PST individually to X1 without attaching them to a profile.

Or you can create one profile – email accounts not needed – and attach to it as many PST files as you like. That is the tactic that I have used for years.

I don't know if this would solve the problem. It would certainly make life simpler.

[szchren]

So I have 40+ profiles each to an archived PST file.

That's a lot of profiles. I'm surprised that Outlook does not complain.

You can add PST individually to X1 without attaching them to a profile.
That's what X1 does automatically resulting in the 40+ profiles I have.

Or you can create one profile – email accounts not needed – and attach to it as many PST files as you like. That is the tactic that I have used for years.
Yes, I have looked into that; create a new outlook profile without an email account and then add each PST individually. But that's 40+ to add manually (I cannot see how to bulk add). So my question is: I could do that but will it index that much faster?

I don't know if this would solve the problem. It would certainly make life simpler.

[Kenward]

That's what X1 does automatically resulting in the 40+ profiles I have.

X1 does not add all PSTs automatically here. Perhaps because I have not told it to "Enable all PST files found..."

The X1 profile gets created when I manually add a PST file.

In X1 8.5 is is available in the Outlook Email but of the Configured Data Sources settings.

Yes, I have looked into that; create a new outlook profile without an email account and then add each PST individually. But that's 40+ to add manually (I cannot see how to bulk add). So my question is: I could do that but will it index that much faster?
[/color]

A good start might be to clean up your email system. 40 PSTs? I have about half a dozen.

[szchren]

That's what X1 does automatically resulting in the 40+ profiles I have.

X1 does not add all PSTs automatically here. Perhaps because I have not told it to "Enable all PST files found..."

The X1 profile gets created when I manually add a PST file.

In X1 8.5 is is available in the Outlook Email but of the Configured Data Sources settings.

Yes, I have looked into that; create a new outlook profile without an email account and then add each PST individually. But that's 40+ to add manually (I cannot see how to bulk add). So my question is: I could do that but will it index that much faster?
[/color]

A good start might be to clean up your email system. 40 PSTs? I have about half a dozen.

I have now tried it both ways, letting X1 create all 40 profiles, or creating one non-email profile in outlook and then adding each of the 40 PSTs one at a time, and then letting X1 find that one profile with all 40 PSTs in it. Make NO difference to the time to index on start-up and when re-indexing.

I'll try at some point to merge PSTs in larger files, but it begs the question: If a PST is archived and is only needed to when searching for past information (just to read not edit), why can't XI have a category for this so those files do not have indexed automatically each day unless manually requested. In other words, just index the current, dynamic files each day, and then add the archived files to the index. I'm sure such a task is technically feasible.

[Kenward]

Like I said, I just don't see this behaviour when X1 starts.

What indexing regime do you have for Outlook? Perhaps that is the difference.

I have enabled Real Time Indexing. Then it is set for "Every 30 minutes".

My files are on local disks, not a network.

Could something else be accessing those PST files and changing their "access" flags?

[szchren]

Like I said, I just don't see this behaviour when X1 starts.

What indexing regime do you have for Outlook? Perhaps that is the difference.
Not sure what you mean by regime, I set X1 to index every day at a specific time, which it does (in addition to whenever I restart Windows). My archived PST files are NOT loaded into Outlook, only the current ones.

I have enabled Real Time Indexing. Then it is set for "Every 30 minutes".
I have real time indexing set too, and it works as intended

My files are on local disks, not a network.
My PST files are split. Active PSTs (those that change/are added to all the time) are on an SST, drive C, local. I have created a virtual drive, G:, on that SST, so also local, to store the archived PSTs. I do this so that that are not included in the system backups I do on a daily and weekly basis (they never change).

Could something else be accessing those PST files and changing their "access" flags?

As best I know only XI accesses those archived PSTs on a daily basis. The "date modified" of those files tracks when I last accessed them in X1.

[Kenward]

Once again, you have an atypical arrangement of drives and data sources. I keep everything simple. So I have no experience of virtual drives and how X1 handles them.

You then say that you "set X1 to index every day at a specific time".

Do you know that X1 has ever completed a first index? What you see could be a symptom of that. If the index is not complete, then it will get back to work at startup.

[szchren]

Once again, you have an atypical arrangement of drives and data sources. I keep everything simple. So I have no experience of virtual drives and how X1 handles them.
Not really. My G: drive is just part of the SST for backup separation; it is sometimes a good idea to break up a hard drive or SST into partitions. I cannot see how it is important to X1 as long as disc access time are not elongated

You then say that you "set X1 to index every day at a specific time".
That is really no different that you setting it to every 30 min

Do you know that X1 has ever completed a first index? What you see could be a symptom of that. If the index is not complete, then it will get back to work at startup.

Yes, they are always completed. You can see % indexed as X1 is indexing, and I can verify that any of my thousands of emails are in the index by searching, and total count of items in the index that XI provides

Let me ask you some questions since you do no have this issue

[1] What is the total size in MB or GB of all your archive PSTs?

[2] Are those archived PSTs always loaded into Outlook, or not incorporated into Outlook but just added to XI via an Outlook profile?

[3] Do you do the set up manually or did X1 find the profiles and add then automatically?

[4] How long does the indexing take to complete when you run it every 30 min?

[Kenward]

What is the total size in MB or GB of all your archive PSTs?

something like 25GB.

X1 reports more than 1 million email items in Outlook > All Email.

Are those archived PSTs always loaded into Outlook, or not incorporated into Outlook but just added to XI via an Outlook profile?

The answer to the first bit is yes, always loaded into Outlook.

I don't understand the rest of the question. What does "incorporated into Outlook" mean?

Do you do the set up manually or did X1 find the profiles and add then automatically?

Again question not clear.

X1 finds all the profiles. I have also added a bunch of IMAP accounts manually.

How long does the indexing take to complete when you run it every 30 min?

0 seconds? I don't notice it. After all, I have I have RTI turned on.

[szchren]

What is the total size in MB or GB of all your archive PSTs?

something like 25GB.

X1 reports more than 1 million email items in Outlook > All Email.
I have about half that

Are those archived PSTs always loaded into Outlook, or not incorporated into Outlook but just added to XI via an Outlook profile?

The answer to the first bit is yes, always loaded into Outlook.
I don't understand the rest of the question. What does "incorporated into Outlook" mean?
The archive PSTs don't have to be loaded (i.e., "not incorporated") into Outlook for X1 to index them. I have now tried it both ways and there is no substantial difference in indexing time.

Do you do the set up manually or did X1 find the profiles and add then automatically?

Again question not clear.
X1 finds all the profiles. I have also added a bunch of IMAP accounts manually.
X1 will create profiles automatically or you can add the PST file manually. Makes no difference to indexing time in my test.

How long does the indexing take to complete when you run it every 30 min?

0 seconds? I don't notice it. After all, I have I have RTI turned on.

I have RTI turned on too but still it takes 10min+ to index my emails

So having explored all this with you, I really have no explanation for the differences. One last try: My archive PST files are structured with many folders and sub folders for different topics so I have hundreds of them and some emails have large attachments. Do your PSTs have a simpler structure with very few folders?

[Kenward]

One last try: My archive PST files are structured with many folders and sub folders for different topics so I have hundreds of them and some emails have large attachments. Do your PSTs have a simpler structure with very few folders?

Yes. I must clean it up some day. With X1 there, I don't really need a complicated filing system to find things.

By the way, my index, like X1, is on a m.2 SSD, which is way faster than a regular SSD. I also have a reasonably fast home-rolled 64-bit PC with 16GB of DDR4 memory. (The graphics are "on board" though which is bound to slow things down.)

My Outlook data is split between the SSD and an HDD.

For the PSTs, I do not Store Outlook Email body in the index.

[szchren]

One last try: My archive PST files are structured with many folders and sub folders for different topics so I have hundreds of them and some emails have large attachments. Do your PSTs have a simpler structure with very few folders?

Yes. I must clean it up some day. With X1 there, I don't really need a complicated filing system to find things.

By the way, my index, like X1, is on a m.2 SSD, which is way faster than a regular SSD. I also have a reasonably fast home-rolled 64-bit PC with 16GB of DDR4 memory. (The graphics are "on board" though which is bound to slow things down.)

My Outlook data is split between the SSD and an HDD.

For the PSTs, I do not Store Outlook Email body in the index.

Here is my set-up



So I don't think there is much difference in our set-ups so I have no idea how to shorten my index time. I do know that I schedule my index daily and you do it every 30 min. I've done all I know including deleting my index and building the index again--takes 2 hours or so. My CPU in a windows 10 home:

Out of ideas!

[cscott]

Hello all,

We have optimized Outlook incremental "quick" scans to avoid having to re-index folders that have not changed.

This should dramatically improve the speed of Outlook scans, and will certainly go a long way towards alleviating this issue.

This fix will be included in the 8.5.1 release, so stay tuned in the coming weeks!

[szchren]

Hello all,

We have optimized Outlook incremental "quick" scans to avoid having to re-index folders that have not changed.

This should dramatically improve the speed of Outlook scans, and will certainly go a long way towards alleviating this issue.

This fix will be included in the 8.5.1 beta, so stay tuned in the coming weeks!

@cscott
Thanks for stepping into the thread with that. I look forward to the update.

I have a licensed product but not a support subscription so can I still get the beta release? Currently have latest 8.5 6001qm (64-bit)

Also, while this update should greatly speed up indexing, it would still be nice to have an option to prevent indexing at startup and just rely on the scheduled updates we can set up.

Thank again

[dvlamis]

I found this post when looking at using X1 with my 20 PST files. I too am concerned about my backup program continuously backing up my (static) PST files. Is there a way in the new version to cause X1 to stop updating the time modified on the PST files? I'm OK with X1 NOT allowing me to change the PST files--they are supposed to be static!

I'm looking forward to faster indexing in 8.5.1, but I also want to solve my backup issue.

[szchren]

I found this post when looking at using X1 with my 20 PST files. I too am concerned about my backup program continuously backing up my (static) PST files. Is there a way in the new version to cause X1 to stop updating the time modified on the PST files? I'm OK with X1 NOT allowing me to change the PST files--they are supposed to be static!

I'm looking forward to faster indexing in 8.5.1, but I also want to solve my backup issue.

Best solution at the moment, and even if the dates are not changed when the new version is implemented, is to partition your hard drive to create a separate volume (say drive G:\) and then put all the archive PSTs into that volume. Then your backup of your main drive with never backup those files -- you can backup those of G:\ separately.

[dvlamis]

Thanks for the information. I do want to request that X1 consider an enhancement request to allow for "read-only" PSTs. I don't understand why indexing software wants to open a PST read/write. Unless I have it wrong, there is no reason to *write* to an archival PST file. Perhaps I just don't understand the product well enough. But in any case, I appreciate the suggestion and the clear answer.

[Kenward]

This fix will be included in the 8.5.1 release, so stay tuned in the coming weeks!

Any progress on that?

Weeks now run the risk of becoming months

[SuperFlyBoy]

I've used XI with Outlook for years and have 30+ archived PST files which do not change, but I do need them in the index to find information from time to time. Yet at each computer reboot all the archived PST files are re-indexed with the currently active OST or PST file and that takes 15 minutes or so. It would be so much better if only the active OST or PST file were re-indexed and added to the index of all those static, archived PST files unless an option demanded otherwise.

+1!

[Kenward]

+1!

-1 here.

Some sort of configuration issue?