Is there any fix for this : CRJAZ1165I Marshalling exception processing request for service
We are noticing that our RTC server is going down very often. The reason was, the DB user is getting locked, due to multiple number of unsuccessful logins. What I presume is, until this exception throws all the RTC users are trying to revoke the session, which eventually gets failed after this exception is thrown. This is what I understood, please correct me if I am wrong here.
I searched for this, I could not find any fix for this. Please let me know how to fix this issue. Thanks! CRJAZ1165I Marshalling exception processing request for service "com.ibm.team.repository.common.service.IQueryService". CRJAZ1170I The request was made by user "xxxx@yy.zz.com" from "xxx.xxx.xxx.xxx".CRJAZ1166I The stack trace hash is 3601A6B65EE2D4314B71A07F3F1E7F90B91A0D97. com.ibm.team.repository.common.internal.marshal.MarshallingException: Async operation timed out at com.ibm.team.repository.common.internal.marshal.impl.EObjectMarshaller.demarshalInputStreamToObject(EObjectMarshaller.java:579) at com.ibm.team.repository.common.internal.marshal.impl.EObjectMarshaller.demarshalInputStreamToObject(EObjectMarshaller.java:599) at com.ibm.team.repository.common.internal.marshal.impl.WebServicesMarshaller.demarshalInputStreamToServiceRequest(WebServicesMarshaller.java:139) at com.ibm.team.repository.servlet.AbstractTeamServerServlet.doPost(AbstractTeamServerServlet.java:690) at javax.servlet.http.HttpServlet.service(HttpServlet.java:738) at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleRequest2(AbstractTeamServerServlet.java:2358) at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleRequest(AbstractTeamServerServlet.java:2155) at com.ibm.team.repository.servlet.AbstractTeamServerServlet.access$0(AbstractTeamServerServlet.java:2140) at com.ibm.team.repository.servlet.AbstractTeamServerServlet$1.service(AbstractTeamServerServlet.java:224) at com.ibm.team.repository.internal.service.auth.impl.JAuthHandler$1.run(JAuthHandler.java:109) at com.ibm.team.repository.servlet.AbstractTeamServerServlet.service(AbstractTeamServerServlet.java:1813) at javax.servlet.http.HttpServlet.service(HttpServlet.java:831) at org.eclipse.equinox.http.registry.internal.ServletManager$ServletWrapper.service(ServletManager.java:180) at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61) at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:126) at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:76) at javax.servlet.http.HttpServlet.service(HttpServlet.java:831) at org.eclipse.equinox.servletbridge.BridgeServlet.service(BridgeServlet.java:120) at com.ibm.team.repository.server.servletbridge.JazzServlet.service(JazzServlet.java:68) at javax.servlet.http.HttpServlet.service(HttpServlet.java:831) at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165) at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217) at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161) at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138) at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204) at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775) at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905) at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1646) Caused by: java.net.SocketTimeoutException: Async operation timed out at com.ibm.ws.tcp.channel.impl.AioTCPReadRequestContextImpl.processSyncReadRequest(AioTCPReadRequestContextImpl.java:189) at com.ibm.ws.tcp.channel.impl.TCPReadRequestContextImpl.read(TCPReadRequestContextImpl.java:111) at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext.read(SSLReadServiceContext.java:265) at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.fillABuffer(HttpServiceContextImpl.java:4171) at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.readSingleBlock(HttpServiceContextImpl.java:3403) at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.readBodyBuffer(HttpServiceContextImpl.java:3509) at com.ibm.ws.http.channel.inbound.impl.HttpInboundServiceContextImpl.getRequestBodyBuffer(HttpInboundServiceContextImpl.java:1782) at com.ibm.ws.webcontainer.channel.WCCByteBufferInputStream.bufferIsGood(WCCByteBufferInputStream.java:373) at com.ibm.ws.webcontainer.channel.WCCByteBufferInputStream.read(WCCByteBufferInputStream.java:266) at com.ibm.ws.webcontainer.srt.http.HttpInputStream.read(HttpInputStream.java:325) at java.io.FilterInputStream.read(FilterInputStream.java:127) at com.ibm.team.repository.servlet.internal.CountingInputStream.read(CountingInputStream.java:80) at com.ibm.team.repository.common.internal.marshal.impl.EObjectMarshaller.demarshalInputStreamToObject(EObjectMarshaller.java:567) ... 48 more Caused by: com.ibm.io.async.AsyncTimeoutException(Async operation timed out, [Timeout, rc=0]) at com.ibm.io.async.AbstractAsyncFuture.waitForCompletion(AbstractAsyncFuture.java:359) at com.ibm.io.async.AsyncFuture.getByteCount(AsyncFuture.java:218) at com.ibm.ws.tcp.channel.impl.AioSocketIOChannel.readAIOSync(AioSocketIOChannel.java:215) at com.ibm.ws.tcp.channel.impl.AioTCPReadRequestContextImpl.processSyncReadRequest(AioTCPReadRequestContextImpl.java:182) |
One answer
|
Hi Donald, Yes the DB user gets locked, whenever RTC server goes down. As and when RTC goes down, we first see whether the DB user is locked or not, and yes it gets locked, then we unlock that user, from where RTC comes up. When we checked the log, it mentioned that teamserver.properties is having issues, but we didnt see any issue or change there.
And also xxxx@yy.zz.com is the RTC admin user which is configured in LDAP. It is just an admin user which is used for rtc build related purposes. |
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.
Comments
The user "xxxx@yy.zz.com" looks more like an LDAP user, or a JTS/RTC user, not a "DB" user. Are you really sure the issue is caused by the DB user being locked? The DB user is specified in the teamserver.properties file. Unless the DB user is also used somewhere else (other than the RTC server), with a wrong password, I can't imagine how it can get locked during the operation.
Hi Donald, Yes the DB user gets locked, whenever RTC server goes down. As and when RTC goes down, we first see whether the DB user is locked or not, and yes it gets locked, then we unlock that user, from where RTC comes up. When we checked the log, it mentioned that teamserver.properties is having issues, but we didnt see any issue or change there.
And also xxxx@yy.zz.com is the RTC admin user which is configured in LDAP. It is just an admin user which is used for rtc build related purposes.
"It mentioned that teamserver.properties is having issues" - this is where you should go a bit deeper. If the RTC server fails to read the teamserver.properties file, it may get a wrong/corrupted DB user password. If it does work like that, the DB user will for sure get locked when the RTC server repeatedly tries to connect with the wrong/corrupted password.