jfs-rdfindex growing to big
Hello,
in our current rtc 3.0 installation the conf/ccm/indicies/jfs-rdfindex directory is growing really fast. Our ccm database (db2) has < 4gb size, but last week the jfs-rdfindex killed our server with a size of 30gb. shutting down the server, starting a repotools-ccm.sh -reindex gets the index size down to 240mb. But since then the index has grown again to 20gb and I am getting a little nervous. Environment: RTC 3.0 on tomcat with db2 Suse 64bit linux JRE 1.6.0 IBM J9 2.4 Linux amd64-64 (upgrade from 1.5 to 1.6 as I read about a problem with big index files on linux 64bit machines) There is also a context aware search engine running on the main source parts, which current index size is 1.4 gb. I haven't found any sizing informations for the indicies in the 3.0 infocenter. So I have no information how I should built up my server. I am aware of http://www-01.ibm.com/support/docview.wss?rs=3488&uid=swg21430319, but this should not apply as we use 3.0. So I would like some information 1. what these indicies are good for 2. how I can stop this one to grow beyond 6x the size of the database size it relies on 3. where can I get sizing information to scale my server Thanks, Stefan |
2 answers
Ralph Schoon (63.1k●3●36●46)
| answered Mar 30 '11, 1:03 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
Hi,
I would suggest to create a PMR or a work item. If it is a production system definitely a PMR and mention it is a production system. You can re-index using the repotools in your case the repotools for jfs. Not sure what the cause could be. There are several indexes used to quickly search for content within RTC, e.g. work item fulltext, and other information. You could also have a look at the logs if there is unusual activity. Thanks, Ralph Hello, Comments I'm seeing a slow but steady growth of jfs-rdfindex directory on one of the dozens of RTC applications we host. This one is hitting 20G and others are much less ( the next biggest is under 3G. One thing different about this repository: It's got Z/OS "heritage" in it. I haven't tried the -reindex, but would Z/OS source queries potentially inflate the size as I'm seeing ?
|
|
Hello,
Here is a update: the ccm/jfs-rdfindex has grown to 40gb. With the "Active Services" page under http://server/ccm/admin I can see a lot of JenaQueryService Threads, which are just waiting. Is this normal or could it be related to my problem? Following an examle Stacktrace of a JenaQueryService.service thread and a list of the active services... sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:184) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:822) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:952) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1272) java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:698) com.ibm.team.jfs.rdf.internal.jenatdb.BaseJenaProvider.enterLock(BaseJenaProvider.java:281) com.ibm.team.jfs.rdf.internal.jenatdb.BaseJenaProvider.query(BaseJenaProvider.java:701) com.ibm.team.jfs.rdf.internal.jenatdb.JenaQueryService.performQuery(JenaQueryService.java:483) com.ibm.team.jfs.rdf.internal.jenatdb.JenaQueryService.perform_GET(JenaQueryService.java:219) com.ibm.team.repository.service.TeamRawService.service(TeamRawService.java:82) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) java.lang.reflect.Method.invoke(Method.java:611) org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370) org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356) org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56) $Proxy284.service(Unknown Source) com.ibm.team.repository.servlet.AbstractTeamServerServlet.doRestService(AbstractTeamServerServlet.java:813) com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleRequest2(AbstractTeamServerServlet.java:1829) com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleRequest(AbstractTeamServerServlet.java:1688) com.ibm.team.repository.servlet.AbstractTeamServerServlet.access$0(AbstractTeamServerServlet.java:1673) com.ibm.team.repository.servlet.AbstractTeamServerServlet$1.service(AbstractTeamServerServlet.java:195) com.ibm.team.repository.service.internal.oauth.OAuthServiceProvider.handleProxiedRequest(OAuthServiceProvider.java:421) com.ibm.team.repository.service.internal.oauth.OAuthServiceProvider.handleOAuthRequest(OAuthServiceProvider.java:210) sun.reflect.GeneratedMethodAccessor509.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) java.lang.reflect.Method.invoke(Method.java:611) org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370) org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356) org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56) $Proxy237.handleOAuthRequest(Unknown Source) com.ibm.team.repository.service.internal.oauth.OAuthHandler.handledRequest(OAuthHandler.java:79) sun.reflect.GeneratedMethodAccessor207.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) java.lang.reflect.Method.invoke(Method.java:611) org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370) org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356) org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56) $Proxy437.handledRequest(Unknown Source) com.ibm.team.repository.servlet.AbstractTeamServerServlet.service(AbstractTeamServerServlet.java:1545) javax.servlet.http.HttpServlet.service(HttpServlet.java:729) org.eclipse.equinox.http.registry.internal.ServletManager$ServletWrapper.service(ServletManager.java:180) org.eclipse.equinox.http.servlet.internal.ServletRegistration.handleRequest(ServletRegistration.java:90) org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:111) org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:75) javax.servlet.http.HttpServlet.service(HttpServlet.java:729) org.eclipse.equinox.servletbridge.BridgeServlet.service(BridgeServlet.java:120) com.ibm.team.repository.server.servletbridge.JazzServlet.service(JazzServlet.java:62) javax.servlet.http.HttpServlet.service(HttpServlet.java:729) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:563) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) org.apache.catalina.authenticator.SingleSignOn.invoke(SingleSignOn.java:393) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174) org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200) org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291) org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:775) org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:704) org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:897) org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689) java.lang.Thread.run(Thread.java:736) Active Services com.ibm.team.jfs.rdf.internal.jenatdb.JenaQueryService.service by ADMIN, 2 days, 01:37:16:607 running time com.ibm.team.jfs.rdf.internal.jenatdb.JenaQueryService.service by ADMIN, 1 days, 01:20:42:028 running time com.ibm.team.jfs.rdf.internal.jenatdb.JenaQueryService.service by ADMIN, 1 days, 01:25:42:130 running time com.ibm.team.jfs.rdf.internal.jenatdb.JenaQueryService.service by ADMIN, 2 days, 01:52:17:470 running time com.ibm.teamz.supa.server.internal.v1.SUPAService.call by buildusr, 00:00:31:004 running time com.ibm.team.repository.service.internal.ServerStatusRestService.getActiveServiceInfo by sho, 00:00:00:000 running time com.ibm.team.jfs.rdf.internal.jenatdb.JenaQueryService.service by ADMIN, 01:10:41:701 running time com.ibm.team.jfs.rdf.internal.jenatdb.JenaQueryService.service by ADMIN, 1 days, 01:27:16:279 running time com.ibm.team.jfs.rdf.internal.jenatdb.JenaQueryService.service by ADMIN, 1 days, 01:32:16:426 running time com.ibm.team.jfs.rdf.internal.jenatdb.JenaQueryService.service by ADMIN, 01:05:41:583 running time com.ibm.team.jfs.rdf.internal.jenatdb.JenaQueryService.service by ADMIN, 2 days, 01:47:17:338 running time com.ibm.team.repository.service.internal.ServerQueryService.queryItems by null, 00:00:00:000 running time com.ibm.team.jfs.rdf.internal.jenatdb.JenaQueryService.service by ADMIN, 01:15:41:845 running time com.ibm.team.jfs.rdf.internal.jenatdb.JenaRdfService.delete by ADMIN, 3 days, 00:52:04:210 running time com.ibm.team.jfs.rdf.internal.jenatdb.JenaQueryService.service by ADMIN, 2 days, 01:42:16:732 running time com.ibm.team.jfs.rdf.internal.jenatdb.JenaQueryService.service by ADMIN, 01:00:40:905 running time |
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.