It's all about the answers!

Ask a question
QuestionsTagsUsersBadges
Follow this question
Question details

×42,788
×10,828
×1,198
×21

question asked: Jun 03 '15, 4:08 a.m.
question was seen: 4,103 times
last updated: Jun 03 '15, 7:57 a.m.

Related questions

FAQ - How this Forum works

Questions about storing binaries in SCM and as attachments in RTC

0
Karthik Krishnan (882●5●118●163) | asked Jun 03 '15, 4:08 a.m.
We use RTC 5.0.2 with Oracle as the back end DB

We want to do continuous build and store the compiled binaries / libraries  as part of streams

Are there any disadvantages of storing compiled binaries are part of streams ?

Oracle sure can hold bigger data size but I would like to know if there are any disadvantages of storing them in streams.

Some time back I read that it is not recommend  to store large file sizes as attachments. Both attachments and scm are part of DB so I am curious to know about this

Thank you!



Accepted answer

2
permanent link
Ralph Schoon (63.1k●3●36●46) | answered Jun 03 '15, 4:37 a.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
You should be very, very careful what you store in an SCM system.
Binaries and object files and the like can be stored in Streams. For example the JKE Banking example stores the ECJ compiler in one component. Some customers also want to store other development tools and libraries there.

This is acceptable if it changes infrequently.

My 2 cent on your case: If you store each build with all binaries in RTC, you will only make the hardware supplier of your hard drives happy. You will fill your database with a lot of data that should not be in there and you will overload your system and make it slow.

I would suggest other solutions solutions like an network storage to store the binaries for each build.
See: Build Artifacts Publishing Using HTTP Servers in Rational Team Concert
and
Automated Build Output Management Using the Plain Java Client Libraries

for some ideas.
Karthik Krishnan selected this answer as the correct answer
Comments
1
Ralph Schoon commented Jun 03 '15, 4:39 a.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

As per the description above, I would not store the binaries in the build results either, but rather refer to them as demonstrated.
Attachments on work items should be used for documentation relevant to the work item, screenshots and the like. You don't want to attach a full release code for a build there either.

Karthik Krishnan commented Jun 03 '15, 5:23 a.m.

Thanks Ralph.

The reason I brought in "Attachments" is just to understand the difference between data types stored in DB for Work items and SCM.

Of course we don't intend to store them as attachments.

Also we use File system for storing build results due to the size of the build output

One main reason for storing the libraries in RTC SCM is to share them instead of source code and make developers build always. If they have prebuilt libraries then local builds are faster

1
Ralph Schoon commented Jun 03 '15, 5:49 a.m. | edited Jun 03 '15, 5:50 a.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

I understand the reasoning an I wish there was a better answer I could give. I would not try to put them into SCM. I would look into Binary Repository Managers such as Artifactory and Nexus to support the build process.

I have no experience myself how to integrate them into the build process, but these names have come up in discussions.

1
Ralph Schoon commented Jun 03 '15, 5:53 a.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

Note, external libraries and the like, where you don't have the source code, would likely be put into SCM, but they change very infrequently. I just don't believe that RTC SCM would handle the load of binaries checked in each time. For source, it can find the delta and store that compressed. There is no way to do that for binaries.

Another problem is, where to store them. You don't want them in the development stream, because you would not even realize a file is not build in certain error scenarios, because the old binary is still there.

Karthik Krishnan commented Jun 03 '15, 6:24 a.m.

Thank you Ralph.

To understand better why is it not recommended to store the binaries in RTC SCM? they do behave like every other file types right ?

1
Ralph Schoon commented Jun 03 '15, 7:39 a.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

Binaries can not be merged and behave different in storage. See https://jazz.net/forum/questions/73442/rtc-compression-for-binary-files-vs-clearcase

1
sam detweiler commented Jun 03 '15, 7:54 a.m.

scm is a lousy place for binaries as noted.
You should use Nexus, or Artifactory.. these both have facilities to get specific versions or 'latest'.

If you use maven or gradle as the build mechansim, then they both leverage these library repositories.

for non-maven builds you can also use ant plugins or curl/wget scripts to push/pull artifacts.

both repos are also supported by the leading deployment tools Jenkins, Urbancode, Ansible, DeployXL and can trigger deployment orchestration, all SCM independent.

also, binaries really mess up the file loading to users/build machines, as they are significantly bigger than source files and have a tendency to cause buffering problems in the web app server running RTC. this is one of the reasons we implemented the squid caching proxy servers to prevent repeated large file loads from impacting RTC performance.

2
sam detweiler commented Jun 03 '15, 7:57 a.m.


Nexus also has te ability to create 'Staging' repositories, so you can build continuously, into either snapshot or lowest staging tier, and promote without recompile as the code moves thru the test cycle.
Maven Release forces a recompile to move from snapshot to release, which is always fraught with danger, IMHO

showing 5 of 8 show 3 more comments

Your answer

Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.