Memory leak with RTC run as WAS application on Linux
We have our RTC 2.0.0.2 iFix2 run as WebSphere Enterprise application with WAS 7.0.0.7 on 64 bit Red Hat Enterprise Linux Server release 5.3 and have been seen bad memory leak which forced us to keep rebooting our server every week due to OOM (out of memeory) even though we have 12GB physical RAM and 2GB swap allocated.
After working with RTC and WAS support for more than two months and we finally start to put the leak under control with the following WAS confoguration:
1. "Disabling AIO (Asynchronous Input/Output) in WebSphere Application Server"
http://www.ibm.com/support/docview.wss?uid=swg21366862
2. In the WAS Admin Console:
Servers -> Application Servers -> serverName -> Web Container
Settings -> Web Container -> Custom Properties:
Press New:
Add the following pair:
Name: com.ibm.ws.webcontainer.channelwritetype
Value: sync
Press OK, and then save the configuration.
After restarted WAS with the above configuration and let the server run for almost 4 days, we observe that the java process's VIRT from top has been stablized at 4GB for 3 days, looks like a very promising solution.
We are worndering if there are other users on the same boat with us and want to share our case with you and see if you have other ways to solve this issue.
After working with RTC and WAS support for more than two months and we finally start to put the leak under control with the following WAS confoguration:
1. "Disabling AIO (Asynchronous Input/Output) in WebSphere Application Server"
http://www.ibm.com/support/docview.wss?uid=swg21366862
2. In the WAS Admin Console:
Servers -> Application Servers -> serverName -> Web Container
Settings -> Web Container -> Custom Properties:
Press New:
Add the following pair:
Name: com.ibm.ws.webcontainer.channelwritetype
Value: sync
Press OK, and then save the configuration.
After restarted WAS with the above configuration and let the server run for almost 4 days, we observe that the java process's VIRT from top has been stablized at 4GB for 3 days, looks like a very promising solution.
We are worndering if there are other users on the same boat with us and want to share our case with you and see if you have other ways to solve this issue.
5 answers
We are now certain that the above fix has stopped the memory leak completely after leaving our server up and running for more than three weeks and the VIRT value from top data is just moving around 4200MB.
We are wondering if someone from Jazz team should add this to the RTC's documentation for users to run jazz as WebSphere Application Server on Linux.
We are wondering if someone from Jazz team should add this to the RTC's documentation for users to run jazz as WebSphere Application Server on Linux.
Hello,
a couple of thoughts here.
After working with RTC and WAS support for more than two months...
--> Do you have the PMR number?
After your investigation, would you say the mem leak was in RTC?
Did you take / analyze (JVM) dumps?
Did you open a jazz.net work item?
Please contact me directly via email - eric_jodet@fr.ibm.com
Thanks.
Eric.
Le 6/8/2010 4:08 PM, ghu a crit :
a couple of thoughts here.
After working with RTC and WAS support for more than two months...
--> Do you have the PMR number?
After your investigation, would you say the mem leak was in RTC?
Did you take / analyze (JVM) dumps?
Did you open a jazz.net work item?
Please contact me directly via email - eric_jodet@fr.ibm.com
Thanks.
Eric.
Le 6/8/2010 4:08 PM, ghu a crit :
We have our RTC 2.0.0.2 iFix2 run as WebSphere Enterprise application
with WAS 7.0.0.7 on 64 bit Red Hat Enterprise Linux Server release
5.3 and have been seen bad memory leak which forced us to keep
rebooting our server every week due to OOM (out of memeory) even
though we have 12GB physical RAM and 2GB swap allocated.
After working with RTC and WAS support for more than two months and we
finally start to put the leak under control with the following WAS
confoguration:
1. "Disabling AIO (Asynchronous Input/Output) in WebSphere
Application Server"
http://www.ibm.com/support/docview.wss?uid=swg21366862
2. In the WAS Admin
Console:
Servers -> Application Servers -> serverName -> Web
Container
Settings -> Web Container -> Custom
Properties:
Press
New:
Add the following
pair:
Name:
com.ibm.ws.webcontainer.channelwritetype
Value:
sync
Press OK, and then save the configuration.
After restarted WAS with the above configuration and let the server
run for almost 4 days, we observe that the java process's VIRT from
top has been stablized at 4GB for 3 days, looks like a very promising
solution.
We are worndering if there are other users on the same boat with us
and want to share our case with you and see if you have other ways to
solve this issue.
Hello,
a couple of thoughts here.
After working with RTC and WAS support for more than two months...
--> Do you have the PMR number?
After your investigation, would you say the mem leak was in RTC?
Did you take / analyze (JVM) dumps?
Did you open a jazz.net work item?
Please contact me directly via email - eric_jodet@fr.ibm.com
Thanks.
Eric.
The PMR 43289,550,000 was raised against WebSpher as RTC support confirmed it is not an RTC issue.
We generated both java and system cor dumps as being requested by the WAS support team.
We haven't raise a jazz.net work item as it is an issue in WAS configuration after all.
What we want is that for future RTC users who want to run RTC as applicaion server in WAS, this kind of WAS configuration sould be mentioned in RTC documentation to avoid what we got to happen again.
Following this thought, I think I'll raise a work item for it.
Yes, please do log a work item against any bug or misconfiguration of
one of the sub-systems used by RTC (e.g. WebSphere, DB2, Oracle, etc.)
that results in a problem for RTC, so that the RTC doc team can warn
about it in the RTC documentation ... if it happened to you, it's very
likely to happen to other folks.
Thanks,
Geoff
On 7/9/2010 10:37 AM, ghu wrote:
one of the sub-systems used by RTC (e.g. WebSphere, DB2, Oracle, etc.)
that results in a problem for RTC, so that the RTC doc team can warn
about it in the RTC documentation ... if it happened to you, it's very
likely to happen to other folks.
Thanks,
Geoff
On 7/9/2010 10:37 AM, ghu wrote:
ejodetwrote:
Hello,
a couple of thoughts here.
After working with RTC and WAS support for more than two months...
--> Do you have the PMR number?
After your investigation, would you say the mem leak was in RTC?
Did you take / analyze (JVM) dumps?
Did you open a jazz.net work item?
Please contact me directly via email - eric_jodet@fr.ibm.com
Thanks.
Eric.
The PMR 43289,550,000 was raised against WebSpher as RTC support
confirmed it is not an RTC issue.
We generated both java and system cor dumps as being requested by the
WAS support team.
We haven't raise a jazz.net work item as it is an issue in WAS
configuration after all.
What we want is that for future RTC users who want to run RTC as
applicaion server in WAS, this kind of WAS configuration sould be
mentioned in RTC documentation to avoid what we got to happen again.
Following this thought, I think I'll raise a work item for it.