It's all about the answers!

Ask a question

Why does WebSphere Application Server hang on stopping CCM application?


Mike Chang (112) | asked Mar 18 '20, 4:13 a.m.

 When I stop the CCM application on WAS console, WAS hangs and the CCM application is not stopped.

I can see errors in WAS log repeatedly. Anyone has the same experience?

[2020/3/18   15:55:27:950 CST] 000000fa SystemErr     R   java.lang.NullPointerException
[2020/3/18   15:55:27:950 CST] 000000fa SystemErr     R    at org.eclipse.equinox.weaving.hooks.WeavingLoaderDelegateHook.postFindResource(WeavingLoaderDelegateHook.java:131)
[2020/3/18   15:55:27:950 CST] 000000fa SystemErr     R    at org.eclipse.osgi.internal.loader.BundleLoader.searchHooks(BundleLoader.java:454)
[2020/3/18   15:55:27:950 CST] 000000fa SystemErr     R    at org.eclipse.osgi.internal.loader.BundleLoader.findResource(BundleLoader.java:592)
[2020/3/18   15:55:27:950 CST] 000000fa SystemErr     R    at org.eclipse.osgi.internal.loader.ModuleClassLoader.getResource(ModuleClassLoader.java:201)
[2020/3/18   15:55:27:950 CST] 000000fa SystemErr     R    at org.eclipse.osgi.internal.framework.ContextFinder.getResource(ContextFinder.java:144)
[2020/3/18   15:55:27:950 CST] 000000fa SystemErr     R    at com.ibm.db2.jcc.am.vd.run(vd.java:49)
[2020/3/18   15:55:27:950 CST] 000000fa SystemErr     R    at java.security.AccessController.doPrivileged(AccessController.java:678)
[2020/3/18   15:55:27:950 CST] 000000fa SystemErr     R    at com.ibm.db2.jcc.am.GlobalProperties.a(GlobalProperties.java:146)
[2020/3/18   15:55:27:950 CST] 000000fa SystemErr     R    at com.ibm.db2.jcc.am.GlobalProperties.d(GlobalProperties.java:100)
[2020/3/18   15:55:27:950 CST] 000000fa SystemErr     R    at com.ibm.db2.jcc.am.ar.run(ar.java:124)
[2020/3/18   15:55:27:950 CST] 000000fa SystemErr     R    at java.util.TimerThread.mainLoop(Timer.java:566)
[2020/3/18   15:55:27:950 CST] 000000fa SystemErr     R    at java.util.TimerThread.run(Timer.java:516)
[2020/3/18   15:55:44:583 CST] 00000121 SystemErr     R   java.lang.NullPointerException
[2020/3/18   15:55:44:583 CST] 00000121 SystemErr     R    at org.eclipse.equinox.weaving.hooks.WeavingLoaderDelegateHook.postFindResource(WeavingLoaderDelegateHook.java:131)
[2020/3/18   15:55:44:583 CST] 00000121 SystemErr     R    at org.eclipse.osgi.internal.loader.BundleLoader.searchHooks(BundleLoader.java:454)
[2020/3/18   15:55:44:583 CST] 00000121 SystemErr     R    at org.eclipse.osgi.internal.loader.BundleLoader.findResource(BundleLoader.java:592)
[2020/3/18   15:55:44:583 CST] 00000121 SystemErr     R    at org.eclipse.osgi.internal.loader.ModuleClassLoader.getResource(ModuleClassLoader.java:201)
[2020/3/18   15:55:44:583 CST] 00000121 SystemErr     R    at org.eclipse.osgi.internal.framework.ContextFinder.getResource(ContextFinder.java:144)
[2020/3/18   15:55:44:583 CST] 00000121 SystemErr     R    at com.ibm.db2.jcc.am.vd.run(vd.java:49)
[2020/3/18   15:55:44:583 CST] 00000121 SystemErr     R    at java.security.AccessController.doPrivileged(AccessController.java:678)
[2020/3/18   15:55:44:583 CST] 00000121 SystemErr     R    at com.ibm.db2.jcc.am.GlobalProperties.a(GlobalProperties.java:146)
[2020/3/18   15:55:44:583 CST] 00000121 SystemErr     R    at com.ibm.db2.jcc.am.GlobalProperties.d(GlobalProperties.java:100)
[2020/3/18   15:55:44:583 CST] 00000121 SystemErr     R    at com.ibm.db2.jcc.am.ar.run(ar.java:124)
[2020/3/18   15:55:44:583 CST] 00000121 SystemErr     R    at java.util.TimerThread.mainLoop(Timer.java:566)
[2020/3/18   15:55:44:583 CST] 00000121 SystemErr     R    at java.util.TimerThread.run(Timer.java:516)
[2020/3/18   15:56:27:951 CST] 000000fa SystemErr     R   java.lang.NullPointerException
[2020/3/18   15:56:27:951 CST] 000000fa SystemErr     R    at org.eclipse.equinox.weaving.hooks.WeavingLoaderDelegateHook.postFindResource(WeavingLoaderDelegateHook.java:131)
[2020/3/18   15:56:27:951 CST] 000000fa SystemErr     R    at org.eclipse.osgi.internal.loader.BundleLoader.searchHooks(BundleLoader.java:454)
[2020/3/18   15:56:27:951 CST] 000000fa SystemErr     R    at org.eclipse.osgi.internal.loader.BundleLoader.findResource(BundleLoader.java:592)
[2020/3/18   15:56:27:951 CST] 000000fa SystemErr     R    at org.eclipse.osgi.internal.loader.ModuleClassLoader.getResource(ModuleClassLoader.java:201)
[2020/3/18   15:56:27:951 CST] 000000fa SystemErr     R    at org.eclipse.osgi.internal.framework.ContextFinder.getResource(ContextFinder.java:144)
[2020/3/18   15:56:27:951 CST] 000000fa SystemErr     R    at com.ibm.db2.jcc.am.vd.run(vd.java:49)
[2020/3/18   15:56:27:951 CST] 000000fa SystemErr     R    at java.security.AccessController.doPrivileged(AccessController.java:678)
[2020/3/18   15:56:27:951 CST] 000000fa SystemErr     R    at com.ibm.db2.jcc.am.GlobalProperties.a(GlobalProperties.java:146)
[2020/3/18   15:56:27:951 CST] 000000fa SystemErr     R    at com.ibm.db2.jcc.am.GlobalProperties.d(GlobalProperties.java:100)
[2020/3/18   15:56:27:951 CST] 000000fa SystemErr     R    at com.ibm.db2.jcc.am.ar.run(ar.java:124)
[2020/3/18   15:56:27:951 CST] 000000fa SystemErr     R    at java.util.TimerThread.mainLoop(Timer.java:566)
[2020/3/18   15:56:27:951 CST] 000000fa SystemErr     R    at java.util.TimerThread.run(Timer.java:516)


RTC version is 6.0.6.1
WAS 8.5.5.17
Database is Db2 11.1


Comments
Ian Wark commented Mar 20 '20, 3:20 a.m. | edited Mar 20 '20, 3:21 a.m.
To narrow down possible causes...

Are you able to stop the server by some other means?

Are you able to stop other ELM applications successfully?

What happens if you stop RTC from command line by going into the profile itself? Is it successful?

What OS are you using? If Windows, do you have a Windows service to start and stop the server? Does that successfully stop the server?

When did this start happening? Does that provide any clues to the cause?

One answer



permanent link
Mike Chang (112) | answered Mar 20 '20, 4:35 a.m.

 Hi Ian,


Thanks for your response. About your questions. I detailed the answers as below:

1. Are you able to stop the server by some other means?
Ans: No. When the CCM application hang, I can't stop the WAS service either. I have to kill the process from the task manager.

2. Are you able to stop other ELM applications successfully?
Ans: I can stop RS、DCC、JTS applications in the integrated console of WAS.

3. What happens if you stop RTC from command line by going into the profile itself? Is it successful?
Ans: No. The server is unable to stop due to the RTC application is hung during the stop process.


4. What OS are you using? If Windows, do you have a Windows service to start and stop the server? Does that successfully stop the server?
Ans: The OS is Windows. And WAS is installed as a service.



Comments
Ian Wark commented Mar 22 '20, 10:30 p.m.
That information is quite helpful. I think it shows that the hang is happening specifically with the CCM application.

My first suggestion would be to clear the WebSphere caches. Particular the temp, wstemp, logs and class caches. But you need to stop the server and take care before doing so. I found these tech notes.

https://www.ibm.com/support/pages/things-know-deleting-temporary-cache-and-log-files-websphere-application-server

If that doesn't work, I think you should obtain 5 javacores on shutdown on the CCM application server once every 30 seconds. Obtain the ISADC during that period to cross-reference too. If you can clear logs while the server is stopped that would help. Raise a case to customer support, (if you can) to get that analyzed.


Here is a document about obtaining javacores for WAS base on Windows:
https://www.ibm.com/support/pages/node/331089

ISADC:
https://www.ibm.com/support/knowledgecenter/SSYMRC_6.0.6.1/com.ibm.team.concert.doc/topics/t_using_the_isal.html


Ian Wark commented Mar 23 '20, 4:02 a.m.

Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.