Hi all,

First of all I'd like to thank the Jazz support team for helping us
resolve that issue: it was really appreciated!

Here's the story:
Earlier today our developers, were not able anymore to deliver nor
accept changes in RTC.

The panic started and spreaded fast!

We found the following message in the Jazz Web Application logs:

00000023 SystemOut O 14:09:35,173
ERROR sqlExceptionLogger
- Virtual storage or database resource is not
available.
SQL: Insert Into REPOSITORY.ITEM_STATES (ITEM_UUID, KEY_UUID,
MODIFIED, ITEM_TYPE_DBID, VAL_ENCODING, ITEM_VALUE) VALUES (?, ?, ?,
?, ?, ?)
SQL Exception #1
SQL Message: DB2 SQL error: SQLCODE: -289, SQLSTATE:
57011, SQLERRMC: CONTENTTS
SQL State: 57011
Error Code: -289

com.ibm.db2.jcc.c.SqlException: DB2 SQL error: SQLCODE:
-289, SQLSTATE: 57011, SQLERRMC: CONTENTTS
at com.ibm.db2.jcc.c.kh.b(kh.java:1367)
at com.ibm.db2.jcc.c.kh.c(kh.java:1354)
at com.ibm.db2.jcc.b.db.k(db.java:352)
at com.ibm.db2.jcc.b.db.a(db.java:60)
at com.ibm.db2.jcc.b.t.a(t.java:52)
at com.ibm.db2.jcc.b.tb.b(tb.java:202)
at com.ibm.db2.jcc.c.lh.X(lh.java:1842)
at com.ibm.db2.jcc.c.lh.d(lh.java:2411)
at com.ibm.db2.jcc.c.lh.T(lh.java:465)
at com.ibm.db2.jcc.c.lh.executeUpdate(lh.java:448)

We couldn't get a hold for a DB2 DBA so we thought that we were
stocked at that point.

We were able to get a hold for some help at the Jazz support team and
here's what we found out (as non DBAs)

the SQL ERROR Code is actually refering to problems with the DB's
Table Space.

We followed these steps:

First of all, we stopped the Jazz instance in WebSphere.

We listed the content of our teamserver.properties file and we found
out the following information: the Content Tablespace and the size
limit for it

In teamserver.properties we set the following properties:

# DB2 requires a path where the content tablespace should be created,
examples for Win and Linux follow
com.ibm.team.repository.db.db2.content.tablespace.location=content_tablespace
# Configure the size of the tablespace to be created, in 4K pages.
# This is a hard limit and will need to be raised for a production
database.
# Your DB2 DBA will need to monitor the utilization of the CONTENTS
tablespace and grow it as needed.
# Defaulting to 250,000 pages, or 1GB.
com.ibm.team.repository.db.db2.content.tablespace.numpages=500000

We then opened a session as the DB Instance owner:
db2inst1

and we typed the following commands:
We've connected to the JAZZ DB

db2 connect to jazzvoda

We've looked for details on the CONTENTTS Tablespace:
db2 list tablespaces show detail

..


Tablespace ID = 3
Name = CONTENTTS
Type = Database managed space
Contents = All permanent data. Large
table space.
State = 0x0000
Detailed explanation:
Normal
Total pages =
500000
Useable pages = 499952
Used pages =
499943
Free pages =
17
High water mark (pages) = 522208
Page size (bytes) = 4096
Extent size (pages) = 32
Prefetch size (pages) = 32
Number of containers = 1

..

At that point we realized that no more space was available in the
allocated table space where the Jazz's artifacts are stored (the
Content TS)

We decided to increase it by the third of its size after making sure
that we had enough disk space in that environment (for the mount
point where the Jazz DB was defined).

db2 "ALTER TABLESPACE CONTENTTS RESIZE (FILE
'content_tablespace' 750000)"

This operation took a few minutes to complete.

db2 list tablespaces show detail

..

Tablespace ID = 3
Name = CONTENTTS
Type = Database managed space
Contents = All permanent data. Large
table space.
State = 0x0000
Detailed explanation:
Normal
Total pages = 750000
Useable pages = 749952
Used pages = 522208
Free pages =
227744
High water mark (pages) = 522208
Page size (bytes) = 4096
Extent size (pages) = 32
Prefetch size (pages) = 32
Number of containers = 1

..

We increased the tablespace size and we now have more space in that
TS.

We altered the teamserver.properties to increase the property

n teamserver.properties we set the following properties:

#
# Configure the size of the tablespace to be created, in 4K pages.
# This is a hard limit and will need to be raised for a production
database.
# Your DB2 DBA will need to monitor the utilization of the CONTENTS
tablespace and grow it as needed.
# Defaulting to 250,000 pages, or 1GB.
com.ibm.team.repository.db.db2.content.tablespace.numpages=750000

We restarted the Jazz Instance and reconnected the RTC clients.
We were able to Accept/Deliver our pending changes.

I hope that this scenario will be documented in the Trouble shooting
guide: it will help non DB2 DBA people to get back on track really
fast!

I salute you all, I thank you again Jazz project members!

I don't know when your DB2 tablespace was initially created but, if I