r13 - 2015-07-06 - 13:09:31 - GeraldMitchellYou are here: TWiki >  Deployment Web > DeploymentMonitoring > CLMServerMonitoring > CLMServerMonitoringPlugin

new.png CLM Server Monitoring plug-in

constantchange.png Authors: MichaelTabb JorgeAlbertoDiaz BorisKuschel GeraldMitchell
Build basis: Collaborative Lifecycle Management 4.0.6 and higher

Wip CLM Server 6.0 updates pending

Overview

The goal of CLM Server Monitoring is to be able to identify performance or hot spot areas in a running CLM server to aid customers, support, and developers alike. The server side of the CLM Server Monitoring is bundled and shipped with the Jazz Foundation server architecture and included with the CLM Applications.

This section will provide overall design and guidance of the CLM Server Monitoring plug-in.

The CLM Server Monitoring plug-in is the main CLM specific solution component, responsible for gathering the CLM application performance data and metrics. Note that, despite the name, you do not need to install additional components in the CLM application.

The core "plug-in" component resides in the CLM application JVM process space as part of the standard installation for CLM 4.0.6 and later, and is disabled by default.

The plugin uses byte-code weaving to measure and inspect CLM applications in the JVM, and collect system and JVM information.

Data is collected and provided to the CSM Agent or other monitoring solution via JMX.

You can also access the CLM Server Monitoring Plugins through JMX, using the CLM Server Monitoring Agent provided in CLM 5.0.2 and later, tools such as JConsole, or monitoring solutions like IBM Tivoli Monitoring.

CSM Plugin technology

The monitoring feature of CLM, the CLM Server Monitoring plug-in, runs in the same JVM as the CLM application it monitors, exposing the information gathered and alerts generated via Java Management Extensions (JMX).

The CLM Server Monitoring component of the Jazz Foundation Server is an Aspect Oriented Programming (AOP) based piece of software that enables instrumentation of existing foundation server code.

CLM Server Monitoring uses a combination of AspectJ and Java MBeans.

AspectJ - AspectJ is used to provide pointcuts and advice to insert itself into the call stack of a normal program flow. The various pointcuts and advice are primarily used to insert itself before and after measurable aspects of server behavior. For example we may want to measure how long a SQL call takes. To accomplish that, an advice would be placed before and after that sql call. As the before advice fires, the clock starts, when the after advice fires, the clock is stopped and the metric is rolled up into a statistical collector to be used for later computation.

Java MBean - Many of the monitors and various other components of the CLM Server Monitoring are exposed as Java MBeans. Enables a multitude of clients to harvest the wealth of information or to remotely invoke an operation.

Utilizing the CSM Plugin

The plugin must be currently be enabled at the the start of the application server, and requires some configuration of the application server before the server monitoring will be activated.

For more information on the process for setup and configuration of the application server to enable CLM Server Monitoring Plugin, see official CLM Server Monitoring 6.0 documentation.

Connecting to the CLM Server Monitoring Plugin running in the Application server MBean Provider through an appropriate monitoring view facility, such as the CLM Server Monitoring agent or JConsole, is documented official CLM Server Monitoring 6.0 documentation and won't be covered in this section. See CLM monitoring through ITM for instructions on how to monitor CLM Server Monitoring through IBM Tivoli Monitoring.

When you have a successful connection to the MBeanProvider on the Application server, you will see the available MBean domains and from there be able to interact with the monitors available for that domain.

    • See Enabling CLM monitoring for instructions on how to enable CLM Server Monitoring through JConsole. This uses the monitoring configuration through the Jonsole visual interface to the JMX visible Mbeans, and the configuration can be seen below for reference in the monitor configuration section.

CSM Object Names

Object naming for MBeans in JMX consists of the domain and the key value pairing for the type and name. The monitors have types and specific named monitors exist as names under that type. Each application space is allocated a domain based on a schema.

For CLM Server Monitoring starting in 6.0, the domain schema is team.server.monitoring.

The application identifier is the application name assigned URI space then an underscore followed by the port at which the application is available.

Currently, only those applications created on the Jazz Foundation base are monitor-enabled through CLM Server Monitoring. Currently the listed Jazz-based applications for CLM Server Monitoring 6.0 are:

  • JTS (Jazz Team Server)
  • CCM (Change and Configuration Management - Rational Team Concert)
  • RM (Requirements Management - Rational DOORS Next Generation)
  • QM (Quality Management - Rational Quality Manager)
  • GC (Global Configuration)
  • DCC (Data Collection)
  • RELM (Rational Engineering Lifecycle Manager)

An example would be team.server.monitoring.jts_9443 or team.server.monitoring.ccm_9445.

jconsole_csm_domain.png

Note: Prior to version 6.0.0 of CLM Server Monitoring Plugin, the schema was "team.server." with the applications shown as the application url and @ then port, such as: team.server./jazz@9443. This was changed in CLM 6.0.0 in order to separate from other JMX available CLM Server data and use more conventional characters not affected by encoding concerns.

Monitors_Jconsole.png

All of the monitors for an application will be collated under the domain for the application. This allows for multiple instances of the application in the same JVM to be distinguished. for instance, there can be a CCM1 and a CCM2 within the same JVM, team.server.monitoring.ccm1_9443 and team.server.monitoring.ccm2_9443 or distinguish between virtual hosting of 2 instances of the same application in the same JVM by port when both team.server.monitoring.ccm_9444 and team.server.monitoring.ccm1_9445 are needed.

CSM Monitor Types

A monitor meters and tracks activity inside its given domain.

There are a few interesting areas that have been identified which could useful performance metrics in a running server.

All monitors are registered as managed beans (MBeans) in the Java bean server. This allows for remote management of a Java interface. There are 2 elements which make up a monitor and its MBean:

  • Configuration - Provides the monitors with options which helps it meter its domain, and some light problem detection abilities
  • Counters - A roll-up of the activity that has been metered inside its domain. This may differ from monitor to monitor depending on what is being watched. At a minimum a monitor should be able to provide things like max,min, and avg functionality.

NOTE Roll-up is only considered prototype level capability, not supported in production environments, due to the resulting performance characteristics. Each monitor has the ability to breakout statistics into separate rollups per unique instance. For example in a request monitor, a user may want to see statistics for the /scr and /rootservices URLs separately. This is possible by enabling the RollupEnabled property on the monitors.

To view the monitor data or to change the monitor attributes, you can use a tool call JConsole. JConsole is a standalone GUI that comes with a JDK. The JMX server must be configured to allow remote connections. See the How-tos section below.

Once the JMX server is running, JConsole can be used to connect and introspect the available beans. This is what the monitoring beans look like through JConsole:

jconsole_domains_csm6_0_0.png

NOTE: MQTT and Scenario may be listed as monitors in the JMX, but are not supported as monitors directly in current releases of CLM Server Monitoring.

The data is presented in a tree structure. The nodes can be expanded and traversed to find the attributes and their values as defined for each of the following monitors:

Each monitor Has some basic information that it can convey. The Object Name is provided by a type of monitor, and any specific monitor instance that might exist beyond the general monitor.

  • Object Name: _type=[required]__[optional _,name = x.y.z__]
  • Management interface: com.ibm.team.server.monitoring.management.*
  • Attributes:
    Name Type Description
    Count Long A counter for the total number of instances of activity monitored by the particular monitor
    Duration Long A counter for the total time in milliseconds spent handling all instances of the type of monitor
    Enabled Boolean Whether or not the monitor should be enabled. This allows this domain to be actively tracked
    Name String The name of the monitor
    RollupEnabled Boolean Whether or not to allow roll-ups to be calculated on a per unique event basis _Roll-up is not supported
    Threshold Long If a an individual activity of the type of monitor exceeds this value in milliseconds, an issue is raised
    TimeSinceLastReset Long A counter for the amount of time that has passed since the counters for this monitor have been reset
  • Operations:
    Name Parameters Description
    dismissAllProblems   Dismisses all problems that are of the monitor type
    reset   Resets all possible counters in the attribute table

AsynchTask monitor

This monitor watches known Asynchronous tasks on the server
  • Object Name: type=AsynchTask
  • Management interface: com.ibm.team.server.monitoring.management.request.RequestManagedMonitorMBean
  • Attributes:
    Name Type Description
    Count Long The total number of tasks
    Duration Long The total time spent handling all tasks
    Enabled Boolean Whether or not the monitor should be enabled. This allows this domain to be actively tracked
    MaximumResponseTime Long The longest time it took the server to handle a task
    MinimumResponseTime Long The shortest time it tool the server to handle a task
    AverageResponseTime Long The average time it took the server to handle a task
    Name String The name of the monitor
    RollupEnabled Boolean Whether or not to allow rollups to be calculated on a per unique task
    Threshold Long If a taskduration exceeds this value, an issue is raised
    TimeSinceLastReset Long The amount of time that has passed since the counters for the monitor have been reset

  • Operations:
    Name Parameters Description
    dismissAllProblems   Dismisses all problems that are of a asynchronous task nature
    reset   Resets all possible counters in the above mentioned attribute table

More information on using the AsynchTask Monitoring can be found at Monitoring asynchronous tasks

Cache monitor

Monitors all activity for the transactional cache subsystem that is included in the 0.6 SDK.

  • Object Name: type=Cache
  • Management interface: com.ibm.team.server.monitoring.management.cache.CacheManagedMonitorMBean
  • Attributes:
    Name Type Description
    AutoUpdated Long The number of times the cache has been autoupdated
    CacheAdded Long The number of times an object has been added to cache
    CacheHit Long The number of times the cache has had a successful hit
    CacheInvalidated Long The number of times the cache has invalidated an entry
    CacheMethodStats Tabular Statistics for the various methods used on the cache API
    CacheMiss Long The number of times the cache had a miss
    Count Long The total number of service calls
    Duration Long The total time spent handling all service calls
    Enabled Boolean Whether or not the monitor should be enabled. This allows this domain to be actively tracked
    MaximumResponseTime Long The longest time it took the server to handle a service call
    MinimumResponseTime Long The shortest time it tool the server to handle a service call
    AverageResponseTime Long The average time it took the server to handle a service call
    Name String The name of the monitor
    RollupEnabled Boolean Whether or not to allow rollups to be calculated on a per unique service call
    Threshold Long If a service call duration exceeds this value, an issue is raised
    TimeSinceLastReset Long The amount of time that has passed since the counters for this monitor have been reset

  • Operations:
    Name Parameters Description
    dismissAllProblems   Dismisses all problems that are of a service nature
    reset   Resets all possible counters in the above mentioned attribute table

More information on using the Cache Monitoring can be found at Monitoring caches

DB Statement monitor

Monitors all SQL statement activity with the database (DB).

  • Object Name: type=DB Statement
  • Management interface: com.ibm.team.server.monitoring.management.ResourceMonitorMBean
  • Attributes:
    Name Type Description
    Count Long The total number of SQL statement executions
    Duration Long The total time spent handling all SQL statement executions
    Enabled Boolean Whether or not the monitor should be enabled. This allows this domain to be actively tracked
    MaximumResponseTime Long The longest time it took the server to handle a SQL statement execution
    MinimumResponseTime Long The shortest time it tool the server to handle a SQL statement execution
    AverageResponseTime Long The average time it took the server to handle a SQL statement execution
    Name String The name of the monitor
    RollupEnabled Boolean Whether or not to allow roll-ups to be calculated on a per unique SQL query statement
    Threshold Long If a SQL query execution duration exceeds this value, an issue is raised
    TimeSinceLastReset Long The amount of time that has passed since the counters for this monitor have been reset

  • Operations:
    Name Parameters Description
    dismissAllProblems   Dismisses all problems that are of a SQL query statement nature
    reset   Resets all possible counters in the above mentioned attribute table

More information on using the DB Statement Monitor can be found at Monitoring database statements

HTTP Client Request monitor

Monitors calls made from an Apache HTTP client. Usually these calls are for server-to-server communication. Note that this is monitoring outgoing requests from the server.

  • Object Name: type=HTTP Client Request
  • Management interface: com.ibm.team.server.monitoring.management.ResourceMonitorMBean
  • Attributes:
    Name Type Description
    Count Long The total number of HTTP client requests
    Duration Long The total time spent handling all HTTP client requests
    Enabled Boolean Whether or not the monitor should be enabled. This allows this domain to be actively tracked
    MaximumResponseTime Long The longest time it took the server to handle an HTTP client request
    MinimumResponseTime Long The shortest time it tool the server to handle an HTTP client request
    AverageResponseTime Long The average time it took the server to handle an HTTP client request
    Name String The name of the monitor
    RollupEnabled Boolean Whether or not to allow rollups to be calculated on a per unique HTTP client request URL
    Threshold Long If an HTTP client request duration exceeds this value, an issue is raised
    TimeSinceLastReset Long The amount of time that has passed since the counters for this monitor have been reset

  • Operations:
    Name Parameters Description
    dismissAllProblems   Dismisses all problems that are of an HTTP client request nature
    reset   Resets all possible counters in the above mentioned attribute table

More information on using the HTTP Client Request Monitor can be found at Monitoring internal HTTP client requests

Pool monitor

This monitor watches known pools used by the server.
  • Object Name: type=Pool
  • Management interface: com.ibm.team.server.monitoring.management.request.RequestManagedMonitorMBean
  • Attributes:
    Name Type Description
    Count Long The total number of pool requests
    Duration Long The total time spent handling all pool requests
    Enabled Boolean Whether or not the monitor should be enabled. This allows this domain to be actively tracked
    MaximumResponseTime Long The longest time it took the server to handle a pool request
    MinimumResponseTime Long The shortest time it tool the server to handle a pool request
    AverageResponseTime Long The average time it took the server to handle a pool request
    Name String The name of the monitor
    RollupEnabled Boolean Whether or not to allow rollups to be calculated on a per unique request
    Threshold Long If a pool request duration exceeds this value, an issue is raised
    TimeSinceLastReset Long The amount of time that has passed since the counters for the pool monitor have been reset

  • Operations:
    Name Parameters Description
    dismissAllProblems   Dismisses all problems that are of a pool nature
    reset   Resets all possible counters in the above mentioned attribute table
More information on using the Pool Monitoring can be found at Monitoring pools

Request monitor

This monitor tracks incoming HTTP requests to the server.

  • Object Name: type=Request
  • Management interface: com.ibm.team.server.monitoring.management.request.RequestManagedMonitorMBean
  • Attributes:
    Name Type Description
    MinimumRequestSize Long The minimum size of the request payload. This only applies to requests that allow it such as put or post. This is computed using all historical requests while the server is up
    MaximumRequestSize Long The maximum size of the request payload. This only applies to requests that allow it such as put or post. This is computed using all historical requests while the server is up
    AverageRequestSize Long The average request size. This is computed using the total number of requests with a payload
    Count Long The total number of requests
    Duration Long The total time spent handling all requests
    Enabled Boolean Whether or not the monitor should be enabled. This allows this domain to be actively tracked
    MaximumResponseTime Long The longest time it took the server to handle a request
    MinimumResponseTime Long The shortest time it tool the server to handle a request
    AverageResponseTime Long The average time it took the server to handle a request
    Name String The name of the monitor
    RollupEnabled Boolean Whether or not to allow rollups to be calculated on a per unique request
    Threshold Long If a requests duration exceeds this value, an issue is raised
    TimeSinceLastReset Long The amount of time that has passed since the counters for this monitor have been reset

  • Operations:
    Name Parameters Description
    dismissAllProblems   Dismisses all problems that are of a request nature
    reset   Resets all possible counters in the above mentioned attribute table

More information on using the Request Monitoring can be found at Monitoring requests

SPARQL monitor

Monitors SPARQL query execution.

  • Object Name: type=SPARQL
  • Management interface: com.ibm.team.server.monitoring.management.ResourceMonitorMBean
  • Attributes:
    Name Type Description
    Count Long The total number of SPARQL query executions
    Duration Long The total time spent handling all SPARQL query executions
    Enabled Boolean Whenther or not the monitor should be enabled. This alllows this domain to be actively tracked
    MaximumResponseTime Long The longest time it took the server to handle a SPARQL query execution
    MinimumResponseTime Long The shortest time it tool the server to handle a SPARQL query execution
    AverageResponseTime Long The average time it took the server to handle a SPARQL query execution
    Name String The name of the monitor
    RollupEnabled Boolean Whether or not to allow rollups to be calculated on a per unique SPARQL query
    Threshold Long If a SPARQL query execution duration exceeds this value, an issue is raised
    TimeSinceLastReset Long The amount of time that has passed since the counters for this monitor have been reset

  • Operations:
    Name Parameters Description
    dismissAllProblems   Dismisses all problems that are of a SPARQL query nature
    reset   Resets all possible counters in the above mentioned attribute table

More information on using the SPARQL Monitor can be found at Monitoring SPARQL queries

Service monitor

Monitors calls made to 0.6 SDK-style services. These are services that have been declared within the SDK framework.

  • Object Name: type=Service
  • Management interface: com.ibm.team.server.monitoring.management.ResourceMonitorMBean
  • Attributes:
    Name Type Default Description notes
    Count Long 0 The total number of service calls
    Duration Long 0 The total time spent handling all service calls
    Enabled Boolean 0 Whether or not the monitor should be enabled. This allows this domain to be actively tracked
    MaximumResponseTime Long   The longest time it took the server to handle a service call
    MinimumResponseTime Long   The shortest time it tool the server to handle a service call
    AverageResponseTime Long   The average time it took the server to handle a service call
    Name String   The name of the monitor
    RollupEnabled Boolean false Whether or not to allow rollups to be calculated on a per unique service call Not supported in production
    Threshold Long   If a service call duration exceeds this value, an issue is raised
    TimeSinceLastReset Long 0 The amount of time that has passed since the counters for this monitor have been reset

  • Operations:
    Name Parameters Description
    dismissAllProblems   Dismisses all problems that are of a service nature
    reset   Resets all possible counters in the above mentioned attribute table

More information on using the Services Monitoring can be found at Monitoring services

MQTT monitor

This is not supported for use in the current CLM Server Monitoring releases.

Scenario monitor

This is not supported for use in the current CLM Server Monitoring releases.

Other MBeans for CLM Server Monitoring

Application

Application specific information
  • Object Name: type=Application
  • Management interface: com.ibm.team.server.monitoring.SimpleManagedMonitor
  • Attributes:
    Name Type Description
    Links TabularDataSupport Internal use only, not a CLM Server Monitoring supported interface
Application Configuration MBean
Application configuration specific information
  • Object Name: type=Application,name=Configuration
  • Management interface: com.ibm.team.server.resources.impl.ApplicationResource
  • Attributes:
    Name Type Description
    StartTime Date Time since the Application was started on the application server
    AllApplicationProperties TabularDataSupport Internal use only, not a CLM Server Monitoring supported interface

Monitoring Monitor

This Monitor has CLM Server monitoring specific information
  • Object Name: __type=Monitoring
  • Management interface: com.ibm.team.server.monitoring.jmx.SmarterServerMonitor
  • Attributes:
    Name Type Description
    ServerVersion String The version of the CLM Server Monitoring plug-in installed on the server.
  • Operations:
    Name Parameters Description
    dismissAllProblems none Dismisses all problems in the JVM
    dismissAllTrackedSessions none Dismisses all Tracked Sessions in the JVM
Monitoring Configuration MBean
This MBean contains configuration information for the CLM Server Monitoring feature.
  • Object Name: type=Monitoring,name=Configuration
  • Management interface: com.ibm.team.server.monitoring.jmx.beans.SmarterServerConfigurationMBean
  • Attributes:
    Name Type Description
    AllMonitorsEnabled Boolean Acts as a one stop monitor enablement. Making this true turns on all monitors, regardless if the individual monitor is disabled
    CacheDiskStoreDirectory String A directory to store cached monitoring artifacts
    CacheMaxBytesLocalDisk String The maximum amount of disk space allocated for the cache
    CacheMaxEntriesLocalHeap Long The maximum amount of objects to keep in the Java heap
    CachingEnabled Boolean Whether or not to enable caching (should be deprecated. Is never used)
    DiagnosticsEnabled Boolean Exposes server side diagnostics that come with a Jazz server base
    MaxProblemsToKeep Long The total number of problems to keep in memory before they get scrubbed
    ProblemCacheMaxBytesLocalDisk Long The max number of bytes to allocate on disk for the problem cache
    ProblemCacheMaxEntriesLocalHeap Long Max number of problems to keep on JVM heap
    ProblemScrubberInterval Long The interval to run the problem scrubber to remove excessive problems that go over the MaxProblemsToKeep threshold
    ProblemScrubberStrategy String Can either be lifo or fifo. This determines which problems to get rid of first when scrubbing
    ActiveResourceProblemCheckinterval Long interval between checks for active resource problems
Monitoring Rules Issue MBean
  • Object Name: type=Monitoring,name=Rules, rulesType=Issue
  • Management interface: com.ibm.team.server.monitoring.resources.expressions.ProblemRulesAdminBean
  • Operations:
    Name Parameters Description
    installRuleSet String ruleSetXML, boolean persist installs the given XML rule set (String) in to the monitoring and if to persist the rules (boolean)
    listRules none returns the current rule set in XML as a string

CLM Server Monitoring rules

Rules are inputted differently depending on version. 4.0.6 behaves differently than 5.0 and later.

CLM Server Monitoring rules 5.0+

The issue rules are defined in an xml document and sent to the server to install into the rules engine. This provides the engine a way to evaluate resources as they are executing. A rule is comprised of the target resource information (type and name) followed by 1..n number of action elements. The action elements contain the logic which tests or evaluates the resource.

Rule Schema

See CLM Server Monitoring Rules page for the example rules and schema.

Adding/Removing Rule

See CLM Server Monitoring Rules page for how to manipulate the rules in the Plugin

CLM Server Monitoring data dictionary

The CLM Server Monitoring plug-in exposes performance monitoring information and management operations via its JMX interface. A data dictionary describes the JMX interface in detail and how the different generated JMX objects relate to each other and to the resources monitoring domains. The CLM Server Monitoring data dictionary wiki page contains such detailed information.

Related topics: None

External links:

  • None

Additional contributors: None

Edit | Attach | Printable | Raw View | Backlinks: Web, All Webs | History: r13 < r12 < r11 < r10 < r9 | More topic actions
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Contributions are governed by our Terms of Use. Please read the following disclaimer.
Ideas, requests, problems regarding the Deployment wiki? Create a new task in the RTC Deployment wiki project