Jazz Forum Welcome to the Jazz Community Forum Connect and collaborate with IBM Engineering experts and users

JTS causes CPU jump (not bad ... yet)

I have a JTS running under WAS 7.0.0.19 and this morning I noticed some mild performance slowdown and found that the WAS is clipping along at 25% cpu all by itself. Looking at the Active Services page I see 3 of these

com.ibm.team.repository.service.internal.auth.proxy.AuthProxy.findAppCreds

running for 4hr now sometimes showing this stack:


org.eclipse.emf.ecore.util.EcoreUtil$UUID.generate(EcoreUtil.java:3634)
org.eclipse.emf.ecore.util.EcoreUtil.generateUUID(EcoreUtil.java:3616)
com.ibm.team.repository.common.UUID.generate(UUID.java:51)
com.ibm.team.repository.service.internal.rdb.TransactionalCache$Invalidator.(TransactionalCache.java:1735)
com.ibm.team.repository.service.internal.rdb.TransactionalCache$CacheEntry.(TransactionalCache.java:1328)
com.ibm.team.repository.service.internal.rdb.TransactionalCache$DistributedTransactionalBackingMap.mergeEntries(TransactionalCache.java:2515)
com.ibm.team.repository.service.internal.rdb.TransactionalCache$DistributedTransactionalBackingMap.get(TransactionalCache.java:2569)


I tried to find the so-called Smokin' Gun (props to Robert Cray) in the jts logs but don't see a lot. I do see this often:



2012-04-11 09:27:59,204 [ WebContainer : 25] ERROR ibm.team.dashboard.internal.web.IFrameProxyServlet - POST https://jts02
:9444/jts/_iframeproxy/https%253A%252F%252Frtp-rtc10.tivlab.raleigh.ibm.com%253A9443%252Fjazz/service/com.ibm.te
am.workitem.common.internal.rest.IQueryRestService/getResultSet
HEADER: Host = jts02:9444
HEADER: User-Agent = Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.28) Gecko/20120306 AskTbFTB/3.14.1.20007 (CK-IBM) Firefox
/3.6.28 ( .NET CLR 3.5.30729; .NET4.0E)
HEADER: Accept = text/json
HEADER: Accept-Language = en-us,en;q=0.5
HEADER: Accept-Encoding = gzip,deflate
HEADER: Accept-Charset = ISO-8859-1,utf-8;q=0.7,*;q=0.7
HEADER: Keep-Alive = 115
HEADER: Connection = keep-alive
HEADER: X-com-ibm-team-configuration-versions = LATEST
HEADER: X-Jazz-CSRF-Prevent = 0000TM3TFlcd6syis3BZebh7B_E:-1
HEADER: Content-Type = application/x-www-form-urlencoded; charset=utf-8
HEADER: X-Requested-With = XMLHttpRequest
HEADER: Referer = https://jts02:9444/jts/_iframeproxy/https%253A%252F%252Frtp-rtc10.tivlab.raleigh.ibm.com%253A9
443%252Fjazz/_ajax-modules/com.ibm.team.dashboard.web.iframe/com.ibm.team.workitem.viewlets.web.ui.internal.WorkItemsViewlet?_proxyURL=
%2Fjts%2F_iframeproxy%2Fhttps%25253A%25252F%25252Frtp-rtc10.tivlab.raleigh.ibm.com%25253A9443%25252Fjazz
HEADER: Content-Length = 453
HEADER: Cookie = net-jazz-ajax-cookie-rememberUserId=; JazzFormAuth=Form; UnicaNIODID=lsTrro7DH4o-Wo3f6lv; JSESSIONID=0000TM3TFlcd6syis
3BZebh7B_E:-1; LtpaToken2=/NRU+y+bCAWbaoZoDn1+1mxXJymqiohJU4tLy+nkybiKhgSIppRuNmpGQU0K+FOSfC2OpRRkBiUXLVZPD/mNZ5aCEG/R6X9kc4U6U4W+d5jLI
Fx6bu6a7J2DWU+AVuKUMfHNudLIhuYeeVdRCRKJ7XmGZWx+cj0XB/gsOsh6KQeVtyjDW2JialJtToNK8RP8ILfSPBT06SiKhRS/egKuB06L6ro+6XmBKy9t0zNCVyza5y7Bt5FG
zY6TtCw6eXIGhp8vAOCFRl4Lqrctu2eDhkTSTx3pmg7CG7CTeDgUH+wiTI9lNiMCevrQhMfqO54AYZNzwQs1ISQHHdd1C0Uq/Mhh5iR2+7FpbJNCkFzVpGYt1XKpEMhYKxZV8Z4
h4rJn6IuYAZzxGFNQhTnC1gKy9t14pEoS5Y3OgqR13PBBYBHDnIJuRqITZuDGNnS9pfn817biEgTv4MA4nYrlY3XexKn1K9HtCnpTXemgAKJXrZ8IjknlQSwQWAtw7k+451oMuJ
27AnrpZUao93uWkaHcYQye+AkcCpek05VlGaMR2OcEsbC1I50Bg9kgYa29mKPyMNrUr5TFwjpV+czbc6EwryITLABJ9ta3K8c5z4f0CuZ+8RHDX5Ytp0nL3FOJxzB1TEhGt1qiM
6A5VAUQd6KYYXMtpVuZch3VY47Xb4g4FalcLVHQFuB6teKAjgcs/cS9yaqF2e1Uxz+fl24wLZgjHw==; WASReqURL=https://:9444/jts/authenticated/identity?red
irectPath=%252Fjts%252Fwhoami%253FitemId%253D_4EDxUF1DEeG3x-nGmNdbWw%2526projectAreaItemId%253Dtrue
HEADER: Pragma = no-cache
HEADER: Cache-Control = no-cache
Parameter: columnIdentifiers = [workItemType, summary, creator, owner, creationDate, duration, category, target, projectArea, internalT
ags, internalState]
Parameter: filterAttribute = []
Parameter: maxResults = [15]
Parameter: skipOAuth = [true]
Parameter: absoluteURIs = [true]
Parameter: filterValue = []
Parameter: projectAreaItemId = [true]
Parameter: itemId = [_GF8wUM5NEeCP58g2HCYqew]
Parameter: startIndex = [0]

java.lang.StringIndexOutOfBoundsException: String index out of range: 13
at java.lang.String.substring(String.java:1072)
at org.eclipse.equinox.http.servlet.internal.HttpServletRequestAdaptor.getPathInfo(HttpServletRequestAdaptor.java:60)
at javax.servlet.http.HttpServletRequestWrapper.getPathInfo(HttpServletRequestWrapper.java:146)
at org.eclipse.equinox.servletbridge.BridgeServlet.service(BridgeServlet.java:89)


I know I can probably rectify the CPU load by bouncing the JTS, but it has a number of dependent CCM.

Any suggestions ?

0 votes



One answer

Permanent link
I have a JTS running under WAS 7.0.0.19 and this morning I noticed some mild performance slowdown and found that the WAS is clipping along at 25% cpu all by itself. Looking at the Active Services page I see 3 of these

com.ibm.team.repository.service.internal.auth.proxy.AuthProxy.findAppCreds

running for 4hr now sometimes showing this stack:


org.eclipse.emf.ecore.util.EcoreUtil$UUID.generate(EcoreUtil.java:3634)
org.eclipse.emf.ecore.util.EcoreUtil.generateUUID(EcoreUtil.java:3616)
com.ibm.team.repository.common.UUID.generate(UUID.java:51)
com.ibm.team.repository.service.internal.rdb.TransactionalCache$Invalidator.(TransactionalCache.java:1735)
com.ibm.team.repository.service.internal.rdb.TransactionalCache$CacheEntry.(TransactionalCache.java:1328)
com.ibm.team.repository.service.internal.rdb.TransactionalCache$DistributedTransactionalBackingMap.mergeEntries(TransactionalCache.java:2515)
com.ibm.team.repository.service.internal.rdb.TransactionalCache$DistributedTransactionalBackingMap.get(TransactionalCache.java:2569)


I tried to find the so-called Smokin' Gun (props to Robert Cray) in the jts logs but don't see a lot. I do see this often:



2012-04-11 09:27:59,204 [ WebContainer : 25] ERROR ibm.team.dashboard.internal.web.IFrameProxyServlet - POST https://jts02
:9444/jts/_iframeproxy/https%253A%252F%252Frtp-rtc10.tivlab.raleigh.ibm.com%253A9443%252Fjazz/service/com.ibm.te
am.workitem.common.internal.rest.IQueryRestService/getResultSet
HEADER: Host = jts02:9444
HEADER: User-Agent = Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.28) Gecko/20120306 AskTbFTB/3.14.1.20007 (*-IBM) Firefox
/3.6.28 ( .NET CLR 3.5.30729; .NET4.0E)
HEADER: Accept = text/json
HEADER: Accept-Language = en-us,en;q=0.5
HEADER: Accept-Encoding = gzip,deflate
HEADER: Accept-Charset = ISO-8859-1,utf-8;q=0.7,*;q=0.7
HEADER: Keep-Alive = 115
HEADER: Connection = keep-alive
HEADER: X-com-ibm-team-configuration-versions = LATEST
HEADER: X-Jazz-CSRF-Prevent = 0000TM3TFlcd6syis3BZebh7B_E:-1
HEADER: Content-Type = application/x-www-form-urlencoded; charset=utf-8
HEADER: X-Requested-With = XMLHttpRequest
HEADER: Referer = https://jts02:9444/jts/_iframeproxy/https%253A%252F%252Frtp-rtc10.tivlab.raleigh.ibm.com%253A9
443%252Fjazz/_ajax-modules/com.ibm.team.dashboard.web.iframe/com.ibm.team.workitem.viewlets.web.ui.internal.WorkItemsViewlet?_proxyURL=
%2Fjts%2F_iframeproxy%2Fhttps%25253A%25252F%25252Frtp-rtc10.tivlab.raleigh.ibm.com%25253A9443%25252Fjazz
HEADER: Content-Length = 453
HEADER: Cookie = net-jazz-ajax-cookie-rememberUserId=; JazzFormAuth=Form; UnicaNIODID=lsTrro7DH4o-Wo3f6lv; JSESSIONID=0000TM3TFlcd6syis
3BZebh7B_E:-1; LtpaToken2=/NRU+y+bCAWbaoZoDn1+1mxXJymqiohJU4tLy+nkybiKhgSIppRuNmpGQU0K+FOSfC2OpRRkBiUXLVZPD/mNZ5aCEG/R6X9kc4U6U4W+d5jLI
Fx6bu6a7J2DWU+AVuKUMfHNudLIhuYeeVdRCRKJ7XmGZWx+cj0XB/gsOsh6KQeVtyjDW2JialJtToNK8RP8ILfSPBT06SiKhRS/egKuB06L6ro+6XmBKy9t0zNCVyza5y7Bt5FG
zY6TtCw6eXIGhp8vAOCFRl4Lqrctu2eDhkTSTx3pmg7CG7CTeDgUH+wiTI9lNiMCevrQhMfqO54AYZNzwQs1ISQHHdd1C0Uq/Mhh5iR2+7FpbJNCkFzVpGYt1XKpEMhYKxZV8Z4
h4rJn6IuYAZzxGFNQhTnC1gKy9t14pEoS5Y3OgqR13PBBYBHDnIJuRqITZuDGNnS9pfn817biEgTv4MA4nYrlY3XexKn1K9HtCnpTXemgAKJXrZ8IjknlQSwQWAtw7k+451oMuJ
27AnrpZUao93uWkaHcYQye+AkcCpek05VlGaMR2OcEsbC1I50Bg9kgYa29mKPyMNrUr5TFwjpV+czbc6EwryITLABJ9ta3K8c5z4f0CuZ+8RHDX5Ytp0nL3FOJxzB1TEhGt1qiM
6A5VAUQd6KYYXMtpVuZch3VY47Xb4g4FalcLVHQFuB6teKAjgcs/cS9yaqF2e1Uxz+fl24wLZgjHw==; WASReqURL=https://:9444/jts/authenticated/identity?red
irectPath=%252Fjts%252Fwhoami%253FitemId%253D_4EDxUF1DEeG3x-nGmNdbWw%2526projectAreaItemId%253Dtrue
HEADER: Pragma = no-cache
HEADER: Cache-Control = no-cache
Parameter: columnIdentifiers = [workItemType, summary, creator, owner, creationDate, duration, category, target, projectArea, internalT
ags, internalState]
Parameter: filterAttribute = []
Parameter: maxResults = [15]
Parameter: skipOAuth = [true]
Parameter: absoluteURIs = [true]
Parameter: filterValue = []
Parameter: projectAreaItemId = [true]
Parameter: itemId = [_GF8wUM5NEeCP58g2HCYqew]
Parameter: startIndex = [0]

java.lang.StringIndexOutOfBoundsException: String index out of range: 13
at java.lang.String.substring(String.java:1072)
at org.eclipse.equinox.http.servlet.internal.HttpServletRequestAdaptor.getPathInfo(HttpServletRequestAdaptor.java:60)
at javax.servlet.http.HttpServletRequestWrapper.getPathInfo(HttpServletRequestWrapper.java:146)
at org.eclipse.equinox.servletbridge.BridgeServlet.service(BridgeServlet.java:89)


I know I can probably rectify the CPU load by bouncing the JTS, but it has a number of dependent CCM.

Any suggestions ?


I restarted JTS which cleared the issue.

0 votes

Your answer

Register or log in to post your answer.

Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.

Search context
Follow this question

By Email: 

Once you sign in you will be able to subscribe for any updates here.

By RSS:

Answers
Answers and Comments
Question details

Question asked: Apr 11 '12, 9:40 a.m.

Question was seen: 4,260 times

Last updated: Apr 11 '12, 9:40 a.m.

Confirmation Cancel Confirm