It's all about the answers!

Ask a question

Where does RTC SCM store large files, inside database?


Lin Ye (30632611) | asked Jan 14 '10, 9:28 a.m.
Hi,

Our clients would submit large files through RTC. We use RTC SCM for version control. The database is SQL Server. Does RTC SCM store the files directly in database or on file systems?

That is, suppose we have a stream on the Team Server. When the off-shore clients deliver the files to the stream, are those files stored inside database, for example as BLOB? Or the pointers are stored in the database, but the actual files are somewhere on the hard drive?

Another situation is, suppose we assign a work item to a client, they put large files as attachment of the work item and resolve it. When we navigate to the Links tab of the work item page at server side, we see the attachment but we don't save it, at this moment where are the attachment stored?

Sorry for the long question. We need to consider this to determine the hardware environement. Any suggestion is appreciated. Thank you very much.

Lin

10 answers



permanent link
John Camelon (1.7k14) | answered Jan 14 '10, 11:23 a.m.
JAZZ DEVELOPER
Raistlin wrote:
Hi,

Our clients would submit large files through RTC. We use RTC SCM for
version control. The database is SQL Server. Does RTC SCM store the
files directly in database or on file systems?

That is, suppose we have a stream on the Team Server. When the
off-shore clients deliver the files to the stream, are those files
stored inside database, for example as BLOB? Or the pointers are
stored in the database, but the actual files are somewhere on the
hard drive?

Another situation is, suppose we assign a work item to a client, they
put large files as attachment of the work item and resolve it. When
we navigate to the Links tab of the work item page at server side, we
see the attachment but we don't save it, at this moment where are the
attachment stored?

Sorry for the long question. We need to consider this to determine the
hardware environement. Any suggestion is appreciated. Thank you very
much.

Lin


Jazz Source Control stores the data as BLOBs in the database, and will
be kept in the database so long as there are file versions that
reference them.

Likewise, the attachment content is also stored in the database.

Thanks
JohnC
SCM Server

Comments
Lin Ye commented Jan 14 '10, 11:37 a.m. | edited Sep 18 '13, 7:13 a.m.

Hi,

Thank you very much for the reply.

Have a good day.
Lin

Raistlin wrote:
Hi,

Our clients would submit large files through RTC. We use RTC SCM for
version control. The database is SQL Server. Does RTC SCM store the
files directly in database or on file systems?

That is, suppose we have a stream on the Team Server. When the
off-shore clients deliver the files to the stream, are those files
stored inside database, for example as BLOB? Or the pointers are
stored in the database, but the actual files are somewhere on the
hard drive?

Another situation is, suppose we assign a work item to a client, they
put large files as attachment of the work item and resolve it. When
we navigate to the Links tab of the work item page at server side, we
see the attachment but we don't save it, at this moment where are the
attachment stored?

Sorry for the long question. We need to consider this to determine the
hardware environement. Any suggestion is appreciated. Thank you very
much.

Lin


Jazz Source Control stores the data as BLOBs in the database, and will
be kept in the database so long as there are file versions that
reference them.

Likewise, the attachment content is also stored in the database.

Thanks
JohnC
SCM Server


permanent link
Gary Mullen-Schultz (28725536) | answered Apr 28 '10, 9:54 p.m.
Let's say I do the following:

1. Create new work item.
2. Go to Links tab, add attachment.
Either:
3. Abandon work item, or
3. Attempt to save work item, save fails because some required property not set, then abandon work item.

Is the attachment still saved in the database? If so, is there any way to delete such "orphaned" attachments?

Thanks, Gary

permanent link
Jean-Michel Lemieux (2.5k11) | answered Apr 28 '10, 9:56 p.m.
JAZZ DEVELOPER
Not yet. We don't scrub attachments removed from work items as they are still accessible in the history.

Cheers,
Jean-Michel

permanent link
Gary Mullen-Schultz (28725536) | answered Apr 28 '10, 10:00 p.m.
Thanks.

But what about the case I describe, where the work item itself is never actually "real?" It never is assigned an ID, for example, as it is never actually successfully saved.

Gary

permanent link
Jean-Michel Lemieux (2.5k11) | answered Apr 29 '10, 8:42 a.m.
JAZZ DEVELOPER
They get cleaned-up automatically - we may actually upload the attachement then save is atomic and if it fails the content/attachment is not claimed and will be cleaned-up.
Cheers,
Jean-Michel

permanent link
Csaba Koncz (611) | answered Jun 01 '10, 4:04 a.m.
Hi,

I am using a Jazz server with a Derby database. I have accidentally uploaded large logs along with some build results. I have either manually removed the logs or deleted the build results but did not notice any decrease in the DB size.
I would say that build results do not have history, so I would expect the logs be cleared from the database. Is this correct?

Thank you,
Csaba

permanent link
Ahmad Hafiz Hafiz (161) | answered Jun 28 '10, 9:37 p.m.

Jazz Source Control stores the data as BLOBs in the database, and will
be kept in the database so long as there are file versions that
reference them.

Likewise, the attachment content is also stored in the database.

Thanks
JohnC
SCM Server


Hi JohnC,

I would like to know more about how Jazz Source Control stores its data...
Is there a wiki page or articles somewhere that describes the SCM in more details?

Thanks
- Hafiz

permanent link
Nick Edgar (6.5k711) | answered Jul 09 '10, 2:27 p.m.
JAZZ DEVELOPER
Csaba, sorry for the delay. When you delete a build result, or download or log contribution, the contributions and their content blobs do not get deleted right away, but rather get marked for pruning. A separate pruning task runs every so often to actually delete the items and content. By default this runs every 15 minutes.

If you don't see any size decrease in the derby footprint after 15 minutes or so, it sounds like there's a problem with either Jazz or Derby, and it should be reported as a work item.

permanent link
Eitan Shomrai (9154) | answered Apr 28 '12, 12:52 p.m.
Hi,

I have a question from a prospect:
Is there any way to completely cleanup large files from RTC SCM?
Otherwise the database is just endlessly growing, true?

Thanks,
Eitan

permanent link
Nick Edgar (6.5k711) | answered Apr 28 '12, 2:38 p.m.
JAZZ DEVELOPER
That is correct, files under SCM are stored in the DB, and are kept around forever unless explicitly deleted. Deletion is supported as of 4.0. See plan item 169816: Support (destructive) deleting file contents from Source Control and the corresponding video.

Some DB admin actions may be needed to get the DB to actually reclaim the space freed up by deleting the content blobs.

Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.