Store.exe hangs up when performing an online backup with NTbackup.

starriol

Member
Jan 3, 2006
187
0
0
Good morning guys.

This is the second day that we encounter this beauty; the backup server performs an online backup of Exchange (remotely, they are 2 different PCs) and between 2 and 5 minutes later, Microsoft Exchange Information Store (store.exe) crashes.

The events went like this:

First day:


Event Type: Information
Event Source: ESE
Event Category: Logging/Recovery
Event ID: 220
Date: 14/01/2008
Time: 20:01:51
User: N/A
Computer: TS001
Description:
Information Store (4032) First Storage Group: Beginning the backup of the file D:\Exchsrvr\mdbdata\priv1.edb (size 31642 Mb).

For more information, click http://www.microsoft.com/contentredirect.asp.


************************
Event Type: Error
Event Source: Microsoft Exchange Server
Event Category: None
Event ID: 1000
Date: 14/01/2008
Time: 20:02:37
User: N/A
Computer: TS001
Description:
The description for Event ID ( 1000 ) in Source ( Microsoft Exchange Server ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: store.exe. 6.5.7650.28. 43dfe228. ese.dll. 6.5.7638.2. 43433bbb. 0. 00046f0a.
Data:
0000: 41 00 70 00 70 00 6c 00 A.p.p.l.
0008: 69 00 63 00 61 00 74 00 i.c.a.t.
0010: 69 00 6f 00 6e 00 20 00 i.o.n. .
0018: 46 00 61 00 69 00 6c 00 F.a.i.l.
0020: 75 00 72 00 65 00 20 00 u.r.e. .
0028: 20 00 73 00 74 00 6f 00 .s.t.o.
0030: 72 00 65 00 2e 00 65 00 r.e...e.
0038: 78 00 65 00 20 00 36 00 x.e. .6.
0040: 2e 00 35 00 2e 00 37 00 ..5...7.
0048: 36 00 35 00 30 00 2e 00 6.5.0...
0050: 32 00 38 00 20 00 34 00 2.8. .4.
0058: 33 00 64 00 66 00 65 00 3.d.f.e.
0060: 32 00 32 00 38 00 20 00 2.2.8. .
0068: 69 00 6e 00 20 00 65 00 i.n. .e.
0070: 73 00 65 00 2e 00 64 00 s.e...d.
0078: 6c 00 6c 00 20 00 36 00 l.l. .6.
0080: 2e 00 35 00 2e 00 37 00 ..5...7.
0088: 36 00 33 00 38 00 2e 00 6.3.8...
0090: 32 00 20 00 34 00 33 00 2. .4.3.
0098: 34 00 33 00 33 00 62 00 4.3.3.b.
00a0: 62 00 62 00 20 00 66 00 b.b. .f.
00a8: 44 00 65 00 62 00 75 00 D.e.b.u.
00b0: 67 00 20 00 30 00 20 00 g. .0. .
00b8: 61 00 74 00 20 00 6f 00 a.t. .o.
00c0: 66 00 66 00 73 00 65 00 f.f.s.e.
00c8: 74 00 20 00 30 00 30 00 t. .0.0.
00d0: 30 00 34 00 36 00 66 00 0.4.6.f.
00d8: 30 00 61 00 0d 00 0a 00 0.a.....
*******************************************
Event Type: Error
Event Source: Application Error
Event Category: (100)
Event ID: 1000
Date: 14/01/2008
Time: 20:13:36
User: N/A
Computer: TS001
Description:
Faulting application store.exe, version 6.5.7650.28, faulting module ese.dll, version 6.5.7638.2, fault address 0x00046f0a.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 41 70 70 6c 69 63 61 74 Applicat
0008: 69 6f 6e 20 46 61 69 6c ion Fail
0010: 75 72 65 20 20 73 74 6f ure sto
0018: 72 65 2e 65 78 65 20 36 re.exe 6
0020: 2e 35 2e 37 36 35 30 2e .5.7650.
0028: 32 38 20 69 6e 20 65 73 28 in es
0030: 65 2e 64 6c 6c 20 36 2e e.dll 6.
0038: 35 2e 37 36 33 38 2e 32 5.7638.2
0040: 20 61 74 20 6f 66 66 73 at offs
0048: 65 74 20 30 30 30 34 36 et 00046
0050: 66 30 61 f0a
*******************************************


Second day:

Event Type: Information
Event Source: ESE
Event Category: Logging/Recovery
Event ID: 220
Date: 15/01/2008
Time: 20:00:10
User: N/A
Computer: TS001
Description:
Information Store (3840) First Storage Group: Beginning the backup of the file D:\Exchsrvr\mdbdata\priv1.edb (size 31642 Mb).

For more information, click http://www.microsoft.com/contentredirect.asp.

***********************************************
Event Type: Error
Event Source: Microsoft Exchange Server
Event Category: None
Event ID: 1000
Date: 15/01/2008
Time: 20:01:43
User: N/A
Computer: TS001
Description:
The description for Event ID ( 1000 ) in Source ( Microsoft Exchange Server ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: store.exe. 6.5.7650.28. 43dfe228. ese.dll. 6.5.7638.2. 43433bbb. 0. 000047e0.
Data:
0000: 41 00 70 00 70 00 6c 00 A.p.p.l.
0008: 69 00 63 00 61 00 74 00 i.c.a.t.
0010: 69 00 6f 00 6e 00 20 00 i.o.n. .
0018: 46 00 61 00 69 00 6c 00 F.a.i.l.
0020: 75 00 72 00 65 00 20 00 u.r.e. .
0028: 20 00 73 00 74 00 6f 00 .s.t.o.
0030: 72 00 65 00 2e 00 65 00 r.e...e.
0038: 78 00 65 00 20 00 36 00 x.e. .6.
0040: 2e 00 35 00 2e 00 37 00 ..5...7.
0048: 36 00 35 00 30 00 2e 00 6.5.0...
0050: 32 00 38 00 20 00 34 00 2.8. .4.
0058: 33 00 64 00 66 00 65 00 3.d.f.e.
0060: 32 00 32 00 38 00 20 00 2.2.8. .
0068: 69 00 6e 00 20 00 65 00 i.n. .e.
0070: 73 00 65 00 2e 00 64 00 s.e...d.
0078: 6c 00 6c 00 20 00 36 00 l.l. .6.
0080: 2e 00 35 00 2e 00 37 00 ..5...7.
0088: 36 00 33 00 38 00 2e 00 6.3.8...
0090: 32 00 20 00 34 00 33 00 2. .4.3.
0098: 34 00 33 00 33 00 62 00 4.3.3.b.
00a0: 62 00 62 00 20 00 66 00 b.b. .f.
00a8: 44 00 65 00 62 00 75 00 D.e.b.u.
00b0: 67 00 20 00 30 00 20 00 g. .0. .
00b8: 61 00 74 00 20 00 6f 00 a.t. .o.
00c0: 66 00 66 00 73 00 65 00 f.f.s.e.
00c8: 74 00 20 00 30 00 30 00 t. .0.0.
00d0: 30 00 30 00 34 00 37 00 0.0.4.7.
00d8: 65 00 30 00 0d 00 0a 00 e.0.....

**********************************************
Event Type: Error
Event Source: Application Error
Event Category: (100)
Event ID: 1000
Date: 15/01/2008
Time: 20:04:55
User: N/A
Computer: TS001
Description:
Faulting application store.exe, version 6.5.7650.28, faulting module ese.dll, version 6.5.7638.2, fault address 0x000047e0.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 41 70 70 6c 69 63 61 74 Applicat
0008: 69 6f 6e 20 46 61 69 6c ion Fail
0010: 75 72 65 20 20 73 74 6f ure sto
0018: 72 65 2e 65 78 65 20 36 re.exe 6
0020: 2e 35 2e 37 36 35 30 2e .5.7650.
0028: 32 38 20 69 6e 20 65 73 28 in es
0030: 65 2e 64 6c 6c 20 36 2e e.dll 6.
0038: 35 2e 37 36 33 38 2e 32 5.7638.2
0040: 20 61 74 20 6f 66 66 73 at offs
0048: 65 74 20 30 30 30 30 34 et 00004
0050: 37 65 30 7e0
**************************************************

Didn't find much information on this, except doing an offline defrag of the database, using eseutil. This haven't been done for about 2 years, so it looks like a good idea, if only to recover wasted disk space.

Any other ideas?
 

starriol

Member
Jan 3, 2006
187
0
0
Found this KB article, http://support.microsoft.com/kb/944254

Looks right, I have SP2, run public folders, but I get only 2 of the 3 events listed there. Also, it sounds too much of a coincidence that this happens 2-5 minutes after ntbackup starts.

And I'm not fan of installing hot fixes on a production server like nothing.
 

starriol

Member
Jan 3, 2006
187
0
0
Bump, third day that does this and can't do online backups... this is starting to worry me.
Also, about the same time this problem started, we see with my coworkers some "ghost" mails in a shared mailbox. I see "2 new mails", for example. With outlook (2000 or 2007, we use both), you can't see the mails. But when we go to OWA, they are there.

Any clues?
 

spikespiegal

Golden Member
Oct 10, 2005
1,219
9
76
Question: have you actually done a test restore with NTBackup?

I've found the failure rate for the built in MS product to be absurdly high when it comes to restores.
 

RebateMonger

Elite Member
Dec 24, 2005
11,586
0
0
I had an SBS 2003 Server that started crashing every time an NTBackup file came from a remote Server 2003. It turned out to be a disk/storage issue that didn't show up with the local backups. Small files were fine, but large files (such an a backup file) would take it down quickly. Upgrading all the storage controller BIOSes and upgrading the Server to the very latest revisions fixed it.

I'm not saying that this is necessarily the cause of your problem. Just noting that a remote NTBackup can make storage system errors show up.
 

starriol

Member
Jan 3, 2006
187
0
0
^^^^ Rebate, I did a local backup yesterday and was OK (remote where crashing store.exe).

I'm going to be running ISINTEG and ESEUTIL this weekend, wish me good luck!