Jazz Forum Welcome to the Jazz Community Forum Connect and collaborate with IBM Engineering experts and users

Where does RTC SCM store large files, inside database?

Hi,

Our clients would submit large files through RTC. We use RTC SCM for version control. The database is SQL Server. Does RTC SCM store the files directly in database or on file systems?

That is, suppose we have a stream on the Team Server. When the off-shore clients deliver the files to the stream, are those files stored inside database, for example as BLOB? Or the pointers are stored in the database, but the actual files are somewhere on the hard drive?

Another situation is, suppose we assign a work item to a client, they put large files as attachment of the work item and resolve it. When we navigate to the Links tab of the work item page at server side, we see the attachment but we don't save it, at this moment where are the attachment stored?

Sorry for the long question. We need to consider this to determine the hardware environement. Any suggestion is appreciated. Thank you very much.

Lin

0 votes



10 answers

Permanent link
Raistlin wrote:
Hi,

Our clients would submit large files through RTC. We use RTC SCM for
version control. The database is SQL Server. Does RTC SCM store the
files directly in database or on file systems?

That is, suppose we have a stream on the Team Server. When the
off-shore clients deliver the files to the stream, are those files
stored inside database, for example as BLOB? Or the pointers are
stored in the database, but the actual files are somewhere on the
hard drive?

Another situation is, suppose we assign a work item to a client, they
put large files as attachment of the work item and resolve it. When
we navigate to the Links tab of the work item page at server side, we
see the attachment but we don't save it, at this moment where are the
attachment stored?

Sorry for the long question. We need to consider this to determine the
hardware environement. Any suggestion is appreciated. Thank you very
much.

Lin


Jazz Source Control stores the data as BLOBs in the database, and will
be kept in the database so long as there are file versions that
reference them.

Likewise, the attachment content is also stored in the database.

Thanks
JohnC
SCM Server

2 votes

Comments

Hi,

Thank you very much for the reply.

Have a good day.
Lin

Raistlin wrote:
Hi,

Our clients would submit large files through RTC. We use RTC SCM for
version control. The database is SQL Server. Does RTC SCM store the
files directly in database or on file systems?

That is, suppose we have a stream on the Team Server. When the
off-shore clients deliver the files to the stream, are those files
stored inside database, for example as BLOB? Or the pointers are
stored in the database, but the actual files are somewhere on the
hard drive?

Another situation is, suppose we assign a work item to a client, they
put large files as attachment of the work item and resolve it. When
we navigate to the Links tab of the work item page at server side, we
see the attachment but we don't save it, at this moment where are the
attachment stored?

Sorry for the long question. We need to consider this to determine the
hardware environement. Any suggestion is appreciated. Thank you very
much.

Lin


Jazz Source Control stores the data as BLOBs in the database, and will
be kept in the database so long as there are file versions that
reference them.

Likewise, the attachment content is also stored in the database.

Thanks
JohnC
SCM Server


Permanent link
Let's say I do the following:

1. Create new work item.
2. Go to Links tab, add attachment.
Either:
3. Abandon work item, or
3. Attempt to save work item, save fails because some required property not set, then abandon work item.

Is the attachment still saved in the database? If so, is there any way to delete such "orphaned" attachments?

Thanks, Gary

0 votes


Permanent link
Not yet. We don't scrub attachments removed from work items as they are still accessible in the history.

Cheers,
Jean-Michel

0 votes


Permanent link
Thanks.

But what about the case I describe, where the work item itself is never actually "real?" It never is assigned an ID, for example, as it is never actually successfully saved.

Gary

0 votes


Permanent link
They get cleaned-up automatically - we may actually upload the attachement then save is atomic and if it fails the content/attachment is not claimed and will be cleaned-up.
Cheers,
Jean-Michel

0 votes


Permanent link
Hi,

I am using a Jazz server with a Derby database. I have accidentally uploaded large logs along with some build results. I have either manually removed the logs or deleted the build results but did not notice any decrease in the DB size.
I would say that build results do not have history, so I would expect the logs be cleared from the database. Is this correct?

Thank you,
Csaba

0 votes


Permanent link

Jazz Source Control stores the data as BLOBs in the database, and will
be kept in the database so long as there are file versions that
reference them.

Likewise, the attachment content is also stored in the database.

Thanks
JohnC
SCM Server


Hi JohnC,

I would like to know more about how Jazz Source Control stores its data...
Is there a wiki page or articles somewhere that describes the SCM in more details?

Thanks
- Hafiz

0 votes


Permanent link
Csaba, sorry for the delay. When you delete a build result, or download or log contribution, the contributions and their content blobs do not get deleted right away, but rather get marked for pruning. A separate pruning task runs every so often to actually delete the items and content. By default this runs every 15 minutes.

If you don't see any size decrease in the derby footprint after 15 minutes or so, it sounds like there's a problem with either Jazz or Derby, and it should be reported as a work item.

0 votes


Permanent link
Hi,

I have a question from a prospect:
Is there any way to completely cleanup large files from RTC SCM?
Otherwise the database is just endlessly growing, true?

Thanks,
Eitan

0 votes


Permanent link
That is correct, files under SCM are stored in the DB, and are kept around forever unless explicitly deleted. Deletion is supported as of 4.0. See plan item 169816: Support (destructive) deleting file contents from Source Control and the corresponding video.

Some DB admin actions may be needed to get the DB to actually reclaim the space freed up by deleting the content blobs.

0 votes

Your answer

Register or log in to post your answer.

Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.

Search context
Follow this question

By Email: 

Once you sign in you will be able to subscribe for any updates here.

By RSS:

Answers
Answers and Comments
Question details

Question asked: Jan 14 '10, 9:28 a.m.

Question was seen: 13,273 times

Last updated: Sep 18 '13, 7:13 a.m.

Confirmation Cancel Confirm