Quantcast
Channel: THWACK: Message List
Viewing all articles
Browse latest Browse all 20019

Re: MSMQ crashing NPM

$
0
0

The post is a little old...but not marked "answered" so .... maybe here is another way this can occur, or at least some more info on what to look out for.

 

I ran two upgrades on NPM 11.0.1 to upgrade IPAM from 4.1 to 4.2, then another to upgrade to 4.3.  Everything looked "ok"...except it wasn't.

 

I saw some application event log entries (warning severity) on the Orion app server that indicated MSMQ was full.  All Orion services were running, etc. and everything was "green"  but it turns out no alerts were being generated (or new data collection, gaps appeared in all the CPU graphs, etc.).  Basically the system had stopped functioning and there were 0 indicators of this that were "obvious" until you let it sit for awhile and noticed no alerts or gaps in the data graphs.

 

I ran a little "alert test" using the advanced alert manager, and it worked. It was the "test alert button".  Yeah, I know I should not have trusted it as it's not a "real" test.  I guess it really only tests SMTP functionality.

 

I looked at the MSMQ queue folder (listed above) and there were several thousand files. I restarted the MSMQ service and the files cleared...

 

I thought about it and after some thwack reading I decided to reboot both the Orion database and app servers (I have two servers to support Orion.  App Is Win  2008, DB is 2012 R2) as I did not do that after the IPAM upgrade.  I shut down Orion (and everything associated with it from the Orion services manager) and the MSMQ service, then rebooted the database server. Once it came up I rebooted the Orion server.  After that things appeared to be working again.

 

Lesson learned: The status of MSMQ is very important so monitor it any way you can.  And maybe: always reboot after an upgrade of any component?


(edit: A typo.  had SNMP. changed to SMTP)


Viewing all articles
Browse latest Browse all 20019

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>