March 22, 2018
IBM Rational Build Forge
Automate and accelerate build and release processes
Rational Build Forge 7.1.3
This page contains the following sections:
- New features
- New articles on jazz.net
- Agent changes
- Job runs
- Websphere Application Server Configuration
- Rational Automation Framework for Websphere
- Incubators: Evaluate and influence future functionality
- Operating system support. Support for the following operating systems has been added in this release:
- AIX 6.1 on POWER 7: the console and the agent are now supported on this platform.
- AIX 7.1 on POWER: the console and the agent are now supported on this platform.
- z/Linux: support for console has been added for Standard and Enterprise editions.
- FireFox 4 and higher: this browser version is now supported for use as a web client.
- Database support. Support for the following databases has been added in this release:
- Microsoft SQLServer 2008 SP2
- Microsoft SQLServer 2008 R2
- New Feature: User Types. Users may now be classified by type. The types are as follows:
- Normal: Normal users can access the console and perform any action for which they have permissions.
- Read-only: Read-only users can modify their personal settings but cannot modify the system. They cannot create or edit objects like environments, projects, and servers. They cannot modify system settings. They may belong to access groups that otherwise grant write permissions, but those permissions are not granted to them.
- API: API users may use an API client to access the console. They cannot log on to the console through the web client. Multiple clients may now share the same session automatically and consume only one license.
For more information, see the Information Center at Administering > Administration > Users.
- New Feature: Account Lockout. Users may be subject to a maximum number of unsuccessful logins. If they reach the threshold, you can configure the system to deny login attempts for a specified amount of time or deny all login attempts until a root administrator grants them access again. Account lockout is activated and controlled by new system configuration settings.
For more information, see the Information Center at Administering > Administration > Users > Account Lockout.
- New Feature: Automated Agent Update and Deployment: With this new utility, you can install or upgrade Rational Build Forge agents on multiple computers at the same time.
For information on system requirements and usage information, see the Information Center at Installing > Installing agents > Agent update and deployment utility.
- New Feature: Audit logging: With this new feature, you can:
- set audit policy levels through the Audit Policy system configuration setting
- view audit logs using the bfauditlog command-line utility
For more information, see the Information Center at Administering > Administration > Audit logging
- New feature: bfschema options (includes practice run): The bfschema command has two new options:
- Prints every SQL query that is being run.
- Only prints SQL queries--does not run them.
Use this "practice run" option to print, but not run, the SQL queries that bfschema would have run during operations such as schema creation, update, data population, and so on.
- New Feature: Dashboard. Rational Build Forge now provides a dashboard to give you a high-level overview of various data in your environment. Access the dashboard through the Dashboard tab. It is the default view when you log in to the system for the first time.
The dashboard includes data about these items:
- Job Success - The number of jobs that passed or failed over the last four weeks. You can focus on a single project.
- Schedules - The scheduled jobs.
- Server States - The number of available and unavailable servers, by collector.
- Job Queue - The number of active and waiting jobs.
- Purge Queue - The number of active and waiting purges.
- License Seats - The number of licenses in use.
For more information, see the Information Center at Administering > Dashboard tab.
The following figure shows the dashboard.
- Integration Documentation. The Integrating section of the documentation has been reorganized for better usability. In addition, the following areas have been expanded and clarified:
- Rational Build Forge and Rational ClearCase - setup requirements and features of integration using an adaptor
- Rational Build Forge and Rational ClearQuest - setup requirements and features of integration using an adaptor; features in Rational Build Forge that support record updates in Rational ClearQuest
- Rational Build Forge and Rational Team Concert - clarification of support for Rational Team Concert version 2.x and version 3.x. The documentation includes links to detailed setup instructions on the jazz.net wiki.
The article "Tutorial: Getting started with Rational Build Forge 7.1.3" has been posted. It contains six tutorials on basic usage of Rational Build Forge.
- The agent SHELL will now get set to the user's logon shell, even when #! has been used to change the shell for the step command's script.
Agent connection retries
The engine now has a more robust way to check for agent availability during a job run. It should help prevent jobs from hanging. It employs two internal calls in a cycle to retry a connection to the agent.
- Ready() tests to see if the connection to the agent is active.
- Reset() attempts to re-establish connection to the agent. It attempts to connect 10 times at 2-second intervals.
Each cycle (Ready + Reset) counts as one retry. The step_max_retries limits the number of retries. The default is 10 retries. In testing it takes approximately 10 minutes for a step to fail when an agent becomes permanently unavailable.
Note that the agent connection retry is not related to step timeout limits.
System messages during job runsThe number of messages produced by a job run has been reduced. You may want to adjust one or more AutoClean system configuration settings.
- JobStarted - (bfproject.pl) "Job [BUILD_52] for Project [asdf] started"
- JobFinished - (bfproject.pl) "Job [BUILD_52] for Project [asdf] finished"
Old messages (removed):
- StepStartPart - (Step.pm) "Job [%1$s] Step [%2$s] starting command part %3$d of %4$d."
- StepStopPart - (Step.pm) "Job [%1$s] Step [%2$s] completed command part %3$d of %4$d [%5$s]."
- SLConnectionWaiting - (buildforge.pl) "Attempting to establish BuildForge Services Connection..."
- SLConnectionEstablished - (ServicesHandler.pm, Step.pm) "Established connection to Build Forge Services."
- SLEventQueued - (SysEvent.pm) "Services queued event '%s' [%s]."
- LastRunSet - (Project.pm) "Job [%1$s] Last run set from build [%2$s] at [%3$s] [%4$d] [%5$s]."
- BuildStepUsingSelector - (Step.pm) "Job [%s] step [%s] using selector [%s]."
- StepCreateProcess - (Process.pm) "Job [%s] Step %s Created process [%s] with argument [%s]."
- CreateProcess - (Process.pm) "Created process [%s] with argument [%s]."
- BuildStarted - (Build.pm) "Build [%s] started."
- DaemonStarted - (bfproject.pl) "Process [%d] started."
- StartedProcess - (buildforge.pl) "Started process [%1$d] for id [%2$s] [%3$s]."
- JobCompleted - (buildforge.pl) "Job [%s] [%d] completed."
- BuildJobCreated - system message when queueing a new build programmatically (during chaining)
The Job Audit Log BOM section has been removed from the build logs. It contained messages that have now been removed from the system.
Agent communicationsThe agent is now contacted only when there is a need to communicate. The following are conditions where there is a need to communicate:
- The step has obtained a job slot and is now ready to run the step.
- The step is in between step parts and is resetting the connection to prepare for the next step part.
- The engine needs to send data to the agent. This is typically part of running a step command.
- The engine needs to receive data from the agent.
The changes reduce overall network traffic between the engine and the agent. Job execution behavior is not affected. >
Step logsThe following changes were made to the way step logs are produced and the way users can view step logs.
- Manifest entries are sorted. Wherever manifest entries are produced, the block of entries is sorted alphabetically. Example:
1 6/1/11 8:26 AM STEP Step using selector 'any'. 2 6/1/11 8:27 AM MANIFEST BF_AGENT_VERSION=126.96.36.199-0-0006 3 6/1/11 8:27 AM MANIFEST BF_JOBS=1 4 6/1/11 8:27 AM MANIFEST BF_LAST_REFRESH=1306934716 5 6/1/11 8:27 AM MANIFEST BF_LAST_UPDATE=1306934716 6 6/1/11 8:27 AM MANIFEST BF_LOADRATIO=0.0666666666666667 7 6/1/11 8:27 AM MANIFEST BF_NAME=linux1
- Change in step log display.
When viewing step logs, you can select Auto-Paginate or Display all to specify how many lines to display. The setting is a toggle.
When you select Auto-Paginate, 100 lines are displayed by default. You can specify a larger number. That number is used for any re-display of a paginated log. It is also used when displaying a new step log.
Database connection retriesBoth the engine and the services layer can tolerate momentary lapses in the connection to the database. As always, jobs that are running fail if the database connection is lost.
.retry works with exit codesThe .retry command now works on both an exit code failure as well as a log filter failure. This will affect warning and fail filters only as well as a RESULT of non-zero.
The filter types that are now considered for .retry are:
- Set Fail
- Set Fail\ Halt
Error conditionsThe _ERROR_THRESHOLD variable affects how filters behave. The _EXITCODE_MAP variable affects how exit codes are interpreted in order to make a job pass or fail.
- _ERROR_THRESHOLD number
- This variable is meant to count number errors in the same command and then act upon the result. Exceeding this threshold for a filter a cancels the current step. The job result is set to FAILED. This threshold takes precedence over any .retry attempts that are available.
- This variable maps nonzero exit codes to a successful exit code. It contains a list of space-separated entries in the form RESULT(N), where N is the code to be remapped to 0. You may also use commas, semicolons, or colons as separators. When a mapped exit code is returned, no .retry is executed. The only time .retry executes is if there is a failure or a warning set by filters. Remapping a nonzero exit code to 0 means there is no failure and as a result .retry is not needed or attempted. It is at your discretion what error codes can be meaningfully remapped to zero.
Addition of supported email address formatsThe following new formats are supported in email From and To fields.
First Last "You can use the new formats anywhere emails are used in templates. You can use multiple formats in the System Alert Email and System Alert Source system settings.
" First Last <"firstname.lastname@example.org"> First Last First Last email@example.com First Last "firstname.lastname@example.org" First Last@host.com
Engine writes to db.logThe engine now logs errors to the db.log file on both Windows and UNIX platforms, regardless of whether they are running as a service or in the foreground.
When you run the engine as a Windows service, any variable you want written to the log also needs to be set as a SYSTEM variable. You must restart Windows in order for the new system variables to take effect.
If you run the engine in the foreground, the variables are both displayed and written to the db.log, dbmain.log, db_pid.log depending on the debug options used.
Note: There are still some variables that are not written to the db.log. These variables are ones that are used by both the engine and the Perl API client, including the folllowing variables:
BFDEBUG_EX_DETAIL BFDEBUG_LOCALE BFDEBUG_SECURITY BFDEBUG_SVC_AUDIT BFDEBUG_SVC_PROTO
Levels added to BFDEBUG_SECURITY for SSL debuggingBFDEBUG_SECURITY now has debugging levels for diagnosing SSL connections.
- 1 = entry/exit trace (actually enabled for any non-zero value)
- 2 = base-64 encode/decode
- 4 = AES CBC encryption
- 8 = AES CBC decryption
The BuildForge::Utilities::BfCrypt now calls BfCrypt::set_debug(DEBUG) to pass on the value from BFDEBUG_SECURITY, so setting BFDEBUG_SECURITY=15 enables all of the debug trace.
New environment variables for simplified debuggingNew environment variables have been added to group sets of individual environment variables. Setting them also sets all of the environment variables in the group.
BF_ENGINE_DEBUG_FIREHOSE BF_ENGINE_DEBUG_FIREHOSE_SQL BF_ENGINE_DEBUG_FIREHOSE_API BF_ENGINE_DEBUG_FIREHOSE_EXTRA BF_ENGINE_FILES
To make them active, set them to 1 (or any non-null value). You do not need to set more than one. Use with caution: each causes more data to be added to the log. The individual variables set by each grouping variable are listed below.
This variable sets the following individual debug variables in the engine:
BF_ACTIVITY_DEBUG BF_DEBUG_TABLELOCK BF_ADAPTOR_DEBUG BF_ENGINE_DEBUG BF_AGENT_DEBUG BF_ENV_DEBUG BF_AGENT_DEBUGEXPAND BF_LIC_DEBUG BF_AGENT_DEBUGREAD BF_MAIL_DEBUG BF_AGENT_DEBUGSEND BF_PROC_DEBUG BF_AUTH_DEBUG BF_RESERVE_DEBUG BF_CONNECT_DEBUG BF_STEPHANDLER_DEBUG BF_DEBUG_INLINE BF_STREAM_DEBUG BF_DEBUG_QUEUE BF_TAG_DEBUG BF_DEBUG_STATE BF_USER_DEBUG
- This variable sets the following individual debug variables in the engine:
BFDEBUG_SQL_DO BFDEBUG_SQL_EXEC BFDEBUG_SQL_PREPARE
- This variable sets the following individual debug variables in the engine:
BFDEBUG_EX_DETAIL BFDEBUG_SECURITY BFDEBUG_SVC_AUDIT BFDEBUG_SVC_PROTO
This variable sets the following individual debug variables in the engine:
BFCQDEBUG BFDEBUG_LOCALE BF_GUID_DEBUG BF_STORE_DEBUG
This variable causes two files to be created.
- bfmain.log: all debug output is written in this file. This is similar to the old behavior of db.log with the exception that now the debug messages are also placed here.
- bf_pid.log: a file of this form is created for each engine PID. That means that each file contains information for one job. It is meant as an improvement on using the tee command.
Pausing and resuming stepsYou can now pause and resume running steps through the APIs. See the JavaDocs and PerlDocs provided.
Perl API - new aliasesThese aliases are available for using SSL and password encryption with the command line Perl API client. Use them in a shell environment.
- BFCLIENT_CONF - use anywhere you would use com.buildforge.client.config
- BFPWCRYPT_CONF - use anwhere you would use com.buildforge.password.encryption.file
Do the following when setting up Websphere Application Server to be the application server that Build Forge uses:
- In Manage Modules, select A Services Layer Login Servlet
- Find Locate Class Loader Order in the list box.
- Change the value to Classes loaded with local class loader first That causes the parent class loader to be loaded last.
See also the Websphere Information Center: "For example, under default class loader policies, a Web module has its own Web module (WAR) class loader to load its artifacts, which are typically in the WEB-INF/classes and WEB-INF/lib directories. An application module class loader is the immediate parent of this WAR class loader. To ensure that the Web module class loader searches these paths for a particular class or resource first, before delegating the load operation to the application module class loader, set the Class loader mode of the Web module to Parent last."
This release does not include IBM Rational Automation Framework for WebSphere.
If you are upgrading to Rational Build Forge version 7.1.3, any data that was created for Rational Automation Framework for Websphere is zipped and stored in
/Manager/rafw. The data was created only if you installed and used Rational Automation Framework for Websphere in your previous version of Rational Build Forge.
Incubators provide trial versions of technology that might be incorporated in a future release of Rational Build Forge. Download and install the incubator software to better understand and plan for potential upcoming enhancements. You can also make suggestions and help guide how the functionality is ultimately implemented.
The current incubator provides reports so that you can analyze the configurations, builds, and build logs in a Rational Build Forge environment.
To download the incubator, visit Rational Build Forge 7.1.2 Downloads. The 7.1.2 incubator works with Rational Build Forge 7.1.3.