It's all about the answers!

Ask a question

[closed] Migrate Jira Comments/Attachments to RTC


VK L (8177127159) | asked Nov 25 '14, 11:39 a.m.
closed Dec 30 '14, 4:57 a.m.
 Hi All,
CSV Exported from Jira - does not contain comments field. So, we need to use an alternate approach to get it.
Our Jira version is 3.13 - I checked on using Bugzilla as an intermediate , but looks tedious with 1 bug-file per jira-item.
Also, xml exported from Jira has comments in it. I am looking to convert this xml to a csv format with accurate data.

Please advise on the easiest approach to automate this.

Thanks.

The question has been closed for the following reason: "The question is answered, right answer was accepted" by valli Dec 30 '14, 4:57 a.m.

Accepted answer


permanent link
sam detweiler (12.4k6173200) | answered Nov 25 '14, 12:51 p.m.
the bugzilla import isn't that bad.. the jira export makes them, and then the importer reads them..
same approach as with csv.
VK L selected this answer as the correct answer

Comments
VK L commented Nov 25 '14, 10:42 p.m.

 Hi Sam,

If I export Jira data in XML format, its a single XML file.There looks to be an additional conversion required for bugzilla-xml formation. And finally, the bugzilla XML to be used for RTC import.
Please advise on the Jira to bugzilla part.
Thanks.

5 other answers



permanent link
VK L (8177127159) | answered Nov 26 '14, 10:12 a.m.
  Hi Sam,
If I export Jira data in XML format, its a single XML file.There looks to be an additional conversion required for bugzilla-xml formation. And finally, the bugzilla XML to be used for RTC import.
Please advise on the Jira to bugzilla part.
Thanks.

Comments
sam detweiler commented Nov 26 '14, 12:23 p.m. | edited Nov 26 '14, 12:23 p.m.

VK L commented Nov 26 '14, 1:09 p.m.

 Thanks Sam. I saw this ruby script - just that I have to configure bugzilla,ruby just for this migration. 

I will give it a try. 
Thanks.


sam detweiler commented Nov 26 '14, 1:20 p.m. | edited Nov 26 '14, 1:21 p.m.

you do not need bugzilla, just ruby..

you are putting the data in the same format that bugzilla export would do
bugzilla format is 1 file per issue, like excel is one line per issue

I'm pretty sure you can do this with shell scripting too


VK L commented Nov 26 '14, 1:30 p.m.

 Currently, we exported CSV from Jira - to import normal attributes. 

Exported XML from Jira - for comments/attachments and imported into excel.
I hope the retrieved comments can be imported via CSV along with other attributes
For attachments, plain-java code to upload att. to work items.
If the ruby solution can do all of these in 1 shot, I would like to give it a try and see. Our Jira tickets have multiple comments/attachments , version is 3.13, and I haven't done attribute comparison between XML and CSV exports


sam detweiler commented Nov 26 '14, 2:46 p.m.

CSV import of comments is really a pain.. they wrap across line ends, and will completely mess up the csv if you attempt to edit it. the xml format was much better


permanent link
VK L (8177127159) | answered Dec 01 '14, 12:04 p.m.
edited Dec 01 '14, 12:04 p.m.
 Hi Sam,
I was trying to use curl to download each jira-issue into an xml using below curl command:

xml = %x[curl -u username:password http://jira.wiley.com:9090/sr/jira.issueviews:searchrequest-xml/BUG-#{i}/BUG-#{i}.xml]

I took this URL from Jira->Find issues by creating a search filter for all the required items (Around 89). Not sure how this can be accessed using BUG-#i, as i is a sequence from 1 to 1000 in below script [*from the ruby solution-link]:

# Extract JIRA issues. (Figure out how many issues to include.)
(1..1000).each do |i|
  # Use curl with credentials to extract each JIRA issue's XML.
  xml = %x[curl -u username:password http://jira.wiley.com:9090/sr/jira.issueviews:searchrequest-xml/BUG-#{i}/BUG-#{i}.xml]
  # Save the XML to a file.
  f=File.new("BUG-#{i}.xml","w")
  f.write(xml)
  f.close
end 

Please advise on how this Jira individual issue-link is formed


permanent link
VK L (8177127159) | answered Dec 01 '14, 12:41 p.m.
 URL that i get on xml view of the filtered Jira issues is:
http://jira.wiley.com:9090/sr/jira.issueviews:searchrequest-xml/11234/SearchRequest-11234.xml?tempMax=1000

Please advise on how to refer the individual items in this view into an xml

Comments
sam detweiler commented Dec 01 '14, 5:26 p.m.

the sample ruby script gets ALL tickets from 1 to 1000.. skipping none.

if you have a short non-consecutive list you would have to build some other loop mechanism..

in bash shell script you can use

for x in 1 4 8 99 105
do
   curl command using $x as part of filename
done

the list of ticket numbers could be built as a string (somehow) and substituted in the for loop.

sorry I can't help more


permanent link
VK L (8177127159) | answered Dec 02 '14, 2:17 p.m.
 Thanks Sam. It worked with issues.XML URL using key-ids.
While downloading attachments, have you tried .jpg,.xls and so on? Only text files looks to work for me. Other files are not in a correct format type.
Please advise.
Thanks.

Comments
sam detweiler commented Dec 02 '14, 2:47 p.m. | edited Dec 02 '14, 2:48 p.m.

see here http://peggylin.com/2013/11/18/exporting-ticket-comments-attachments-from-jira/

you need a different URL for each attachment in CURL.. and may have to set the datatype (http accept header)

 $attachments = $item->attachments->attachment;
    if ($attachments) {
      foreach($attachments as $attachment) {
        // echo 'attachment: ' . print_r($attachment) .  "\n";
        $attachment_id = $attachment['id'];
        $attachment_name = $attachment['name'];
        $url = "https://myjirasite.com/jira/secure/attachment/" . $attachment_id . "/" . urlencode($attachment_name) ;



importing attachments into RTC is MUCH more difficult..  we ended up writing our own utilities.


VK L commented Dec 02 '14, 3:01 p.m.

 Thanks Sam. The ruby code to get each of the attachments is working - just that the downloaded file content for excel and images - either doesnt open or format is incorrect. Maybe, I have to use different options with curl. %x to take contents and write in a file looks to cause data issue


sam detweiler commented Dec 02 '14, 3:17 p.m.

I think you need to use an accept-header to set the datatype to binary (not text, not json...)  you will probably have to set this differently for each attachmrnt request.  (jpg, vs pdf, vs excel, vs text,) ...  from the name of the attachment in the xml extract


permanent link
VK L (8177127159) | answered Dec 03 '14, 12:51 p.m.
Hi Sam,
Used open-uri in binary mode and all file-types look to work. 
I used bugzilla importer with limited fields mainly comments & attachments. Attachments works, but comments has html,ascii encoding issues and doesnt completely reflect the actual comment-text in JIRA.
For comments, did you perform html,ascii encoding and validations in the ruby script? Please advise on how best it can be rendered in RTC.
If comments contains tag-based text like <title>testing<testing>, RTC is not able to import it. The comment is rendered empty. 

Thanks.

Comments
sam detweiler commented Dec 03 '14, 1:22 p.m.

the RTC comment text field does not support all the same function that the Jira comment does.. so you will have to make some adjustments.

we wrote a java utility to do all the comments and attachments.


VK L commented Dec 05 '14, 10:17 a.m.

 Thanks Sam. If possible, Please share the code-snippet for "Comments extraction from XML" and the "HTML Encoding/decoding & other processing-part [to convert comment into RTC-Supported format". 

*Bugzilla importer fails to update certain comments which are not suitable for RTC-Format.

Thanks.


VK L commented Dec 05 '14, 10:17 a.m.

Thanks Sam. If possible, Please share the code-snippet for "Comments extraction from XML" and the "HTML Encoding/decoding & other processing-part [to convert comment into RTC-Supported format". 

*Bugzilla importer fails to update certain comments which are not suitable for RTC-Format.

Thanks.


sam detweiler commented Dec 05 '14, 10:38 a.m. | edited Dec 05 '14, 11:37 a.m.

I am unable to share that code, sorry

this litle shell script command used as part of a command pipeline can remove all the html tags from the content

sed 's/<[^>]*>//g'

that doesn't make the message any better..(loose formatting, etc.. ) but RTC comments aren't HTML format