• We’re currently investigating an issue related to the forum theme and styling that is impacting page layout and visual formatting. The problem has been identified, and we are actively working on a resolution. There is no impact to user data or functionality, this is strictly a front-end display issue. We’ll post an update once the fix has been deployed. Thanks for your patience while we get this sorted.

Any exchange Guru's in the house

Doboji

Diamond Member
Please....

Exchange server is down.... System Attendant can't be stopped... Information Store can't be started....

This happened after a DC went down.... anybody got any fantastic ideas?

-Max

*UPDATE*

The issue apparantly was 3 things....

The first problem was the whole ad was just completely losing it's mind and all the error messages I was getting were inconsistent intermitten and utterly unhelpful. I followed Freshprince's advice... shut down everything and brought it back up DCs first followed by Exchange... that calmed things down, and I finally could get some consistent usefull error messages.

The primary cause was the Manage Auditing and Security log right (SeSecurity Privilege) was not granted for the Exchange Enterprise Servers domain local group on the 2 remaining DCs, apparantly it was only on the DC that went down. I have no idea why it didnt replicate properly... but it appears to be working now.

I was able to add the SeSecurity privilege by running domainprep, and forcing replication across the DCs.

The secondary problem was the Information store AND Transaction logs were screwy... and I needed to run isinteg on the Information store... however, in order to do that I needed to mount and dismount the information store, which I couldn't do because the transaction logs were screwed. So I backed up, and then deleted the transaction logs. This allowed me to mount and dismount the info store. Then I ran isinteg, got the info store consistent, mounted the store, and voila... everything came up. As far as I can tell I didnt lose a single email

Got home around 5:30... crashed hard at 6... was back here at work at 10.... man what a night.

-Max
 
If information store is stopped, then that means the databases were dismounted properly. So isn't it safe to restart the server?
 
Originally posted by: TommyVercetti
If information store is stopped, then that means the databases were dismounted properly. So isn't it safe to restart the server?

Did that... it comes back up wit the SA still trying to start unsuccesfully....
Whole system seems hung up on the SA.
-Max
 
More info:

The DC that went down was a 2003 DC with a GC on it. All other DCs in the environment are 2000. The Exchange 2003 is running on a 2000 box.

Standby for event log errors....

-Max
 
I thought according to Microsoft themselves Exchange 2003 could only run on Server 2003? Anyway... have you tried any batch files to stop and start the SA? I got 2 of them you can try.. hold on...
 
Application Log Messages

MSExchangeDSAccess

Process INETINFO.EXE (PID = 1064) Topology Discovery failed, error 0x80040a02

MSExchangeMU

Metabase update failed to read the config namespace properly from the domain controller Error Code 80040a01

-Max
 
More app log messages:

MSExchangeSA

One of the system attendant's task is blocked Function:CMADframe: MonitorDS: Thread

MSExchangeSA

One of the system attendant's task is blocked Function:CMADframe: MonitorDS: Work
 
If you have a clean backup within the last 12-24 hours... I would try rebooting the exchange server... the services have already stopped and the information store has been brought down less then gracefully... We do daily backup of the IS at aroufn 12-1pm so at most you lose a days worth of mail. Assuming you have done some more research I would say try a reboot. Things can't get much worse and the DC is up currently so you never know...
 
Originally posted by: JRock
If you have a clean backup within the last 12-24 hours... I would try rebooting the exchange server... the services have already stopped and the information store has been brought down less then gracefully... We do daily backup of the IS at aroufn 12-1pm so at most you lose a days worth of mail. Assuming you have done some more research I would say try a reboot. Things can't get much worse and the DC is up currently so you never know...

done the reboot.... no dice....
 
It sounds more like a topology problem, is the Global Catalog server still down? (and please tell me you have more than one Global Catalog server) Does the GC that went down contain any of the FSMO roles (specifically infrastructure or schema master)?
 
this is what microsoft said to do:

1. Shut down all DSAccess processes. On a functional Exchange 2000 or Exchange 2003 server, the following processes typically have DSAccess loaded:

? Mad.exe (MSExchangeSA)
? Emsmta.exe (MSExchangeMTA)
? Store.exe (MSExchangeIS)
? Winmgmt.exe (WinMgmt)
? Inetinfo.exe (Several Services within)
? Exmgmt.exe (MSExchangeMGMT)

If you cannot restart the server, you can still stop all services that use DSAccess by running the following commands:

? net stop msexchangesa /y
? net stop iisadmin /y
? net stop winmgmt /y

To confirm that all processes using DSAccess have stopped, run the following command:
tlist -m dsaccess.dll
When you see the following output, you have successfully shut down all processes using DSAccess:
No tasks found using DSACCESS.DLL
2. Turn on tracing.For additional information about how to turn on tracing, click the article number below to view the article in the Microsoft Knowledge Base:
238614 (http://support.microsoft.com/kb/238614/EN-US/) XCON: How to Set Up Regtrace for Exchange 2000
3. Start Exchange System Attendant (to start Exchange System Attendant from the command line, type net start msexchangesa).
4. Wait for the failing DSAccess topology events to go by in Event Viewer, stop the RegTrace procedure, and then contact Microsoft Product Support Services (PSS) to interpret the output. For more information about how to contact PSS, visit the Microsoft Product Services Web site: http://support.microsoft.com/directory/question.asp?sd=gn&fr=0

 
Originally posted by: lobadobadingdong
It sounds more like a topology problem, is the Global Catalog server still down? (and please tell me you have more than one Global Catalog server) Does the GC that went down contain any of the FSMO roles (specifically infrastructure or schema master)?

FSMO roles on the dead server were Schema Infrastructure and RID.... all of which were siezed elsewhere

 
Originally posted by: lobadobadingdong
What ended up as your resolution? (in case I ever have the same problem, we'll be moving from exchange 5.5 to 2003 at the end of the year)

The issue apparantly was 3 things....

The first problem was the whole ad was just completely losing it's mind and all the error messages I was getting were inconsistent intermitten and utterly unhelpful. I followed Freshprince's advice... shut down everything and brought it back up DCs first followed by Exchange... that calmed things down, and I finally could get some consistent usefull error messages.

The primary cause was the Manage Auditing and Security log right (SeSecurity Privilege) was not granted for the Exchange Enterprise Servers domain local group on the 2 remaining DCs, apparantly it was only on the DC that went down. I have no idea why it didnt replicate properly... but it appears to be working now.

I was able to add the SeSecurity privilege by running domainprep, and forcing replication across the DCs.

The secondary problem was the Information store AND Transaction logs were screwy... and I needed to run isinteg on the Information store... however, in order to do that I needed to mount and dismount the information store, which I couldn't do because the transaction logs were screwed. So I backed up, and then deleted the transaction logs. This allowed me to mount and dismount the info store. Then I ran isinteg, got the info store consistent, mounted the store, and voila... everything came up. As far as I can tell I didnt lose a single email🙂

Got home around 5:30... crashed hard at 6... was back here at work at 10.... man what a night.

-Max
 
Originally posted by: Doboji
Originally posted by: lobadobadingdong
What ended up as your resolution? (in case I ever have the same problem, we'll be moving from exchange 5.5 to 2003 at the end of the year)

The issue apparantly was 3 things....

The first problem was the whole ad was just completely losing it's mind and all the error messages I was getting were inconsistent intermitten and utterly unhelpful. I followed Freshprince's advice... shut down everything and brought it back up DCs first followed by Exchange... that calmed things down, and I finally could get some consistent usefull error messages.

The primary cause was the Manage Auditing and Security log right (SeSecurity Privilege) was not granted for the Exchange Enterprise Servers domain local group on the 2 remaining DCs, apparantly it was only on the DC that went down. I have no idea why it didnt replicate properly... but it appears to be working now.

I was able to add the SeSecurity privilege by running domainprep, and forcing replication across the DCs.

The secondary problem was the Information store AND Transaction logs were screwy... and I needed to run isinteg on the Information store... however, in order to do that I needed to mount and dismount the information store, which I couldn't do because the transaction logs were screwed. So I backed up, and then deleted the transaction logs. This allowed me to mount and dismount the info store. Then I ran isinteg, got the info store consistent, mounted the store, and voila... everything came up. As far as I can tell I didnt lose a single email🙂

Got home around 5:30... crashed hard at 6... was back here at work at 10.... man what a night.

-Max

that's IT for ya....

great job!

another advice to all IT folks....

DON"T INSTALL server 2003 SP1!!!!!

if you manage multiple sites, and you link-up with VPN, you know that SP1 blows chunks. There are ways around it, but it's best not to install it. we made that mistake and could not recover...had to reinstall the server :|

 
Back
Top