It's all about the answers!

Ask a question

How to fetch compilation errors and warnings when using ant+msbuild in RTC


Pascal van Kempen (23126) | asked Mar 15 '13, 5:48 a.m.
Hi

When building a visual studio solution file using the "Jazz Build for Microsoft Visual Studio Solution", using MSBuild , all compilation errors and warnings are nicely displayed in the compilation tab of the build result.

Now I'm trying to achieve the same result using an ANT file for building (as I want to combine the build with several customized pre- and post steps).
Building goes fine, but I can't find out how to process the buildlog so it is displayed the same way (showing the compilation errors and warnings)

I tried several things using antlib:org.apache.ant.dotnet and dn:msbuild, but also with <exec executable="msbuild.exe" ....  and so on.

So, basically 3 questions
  1. is it possible anyway to have the msbuild log processed when using ant so the compile results are shown in the compile tab of the build result?
  2. if so, does anyone has an example of the ant build statement using msbuild ?
  3. is it also possible to have the compile errors and warnings shown when using devenv instead of msbuild?


(running Nunit tests and publishing those results goes fine by the way, so it's only this part)


Thanks for your support.


Comments
Scott Cowan commented Mar 18 '13, 9:45 a.m.
JAZZ DEVELOPER

Hi Pascal,

Have you seen the Jazz build Ant task reference?  Maybe there's something in there that can help.

Scott


Pascal van Kempen commented Mar 18 '13, 10:41 a.m. | edited Mar 18 '13, 1:15 p.m.

Hi Scott

From the reference you mention I found already out that jdtCompileLogPublisher is the one that publishes the log that is shown in the compilation tab of the eclipse client.

But I'm looking more into which logger or whatsoever is needed (and how to use it) to create the right content for that logfile, so that the warnings and errors are shown.

I see my component now in the tab, and although there are several warnings, the counters are still on "0".

Regards, Pascal


Scott Cowan commented Mar 18 '13, 11:20 a.m.
JAZZ DEVELOPER

Hi Pascal,

I'm thinking we'd need an msbuildLogPublisher that can properly parse an msbuild.log file.  I see we have an mstestLogPublisher.  I'm thinking this is currently unsupported.  I'll ask around.

Scott

Accepted answer


permanent link
Nick Edgar (6.5k711) | answered Mar 27 '13, 9:45 a.m.
JAZZ DEVELOPER
edited Mar 27 '13, 9:54 a.m.
 Hi Pascal,

My apologies for the delay in responding, I somehow lost track of your message and dropped the ball.
I've sent an example output file from ecj by email.  It has warnings, not errors, but it should be pretty obvious from this what an error would look like.

Our parser looks for /compiler/sources/source elements and extracts info from them and any child problems/problem elements and their message and source_context elements.
You can ignore the command_line and options and class_paths elements under /compiler.  For classfile elements under source, we do parse it but don't show this info in the UI, so you can skip it.

(I wanted to paste an example here, but the forum software is mangling it, removing key XML elements, probably thinking it's a script injection or something.  The example's in the email I sent.)

For source file / path, we extract the last segment of the source/@path attribute (ignoring the rest) and pull the package from the source/@package attribute.  This is fairly Java-centric, but I'm sure you can adapt as appropriate.
For problem elements, but message and source_context children are expected.  It will fail if message is missing, but just output an error and continue if source_context is missing.  The arguments element is ignored.

To publish with jdtCompileLogPublisher task, just point it at the XML file or directory containing them (it looks for .xml files).  See the Help for more info on the attributes/args it takes, or look at the examples included in the build toolkit (buildtoolkit/examples/compile-and-test)

Hope this helps.  Good luck working on the XSLT mapping, and yes, it would likely be helpful to others if you were to post your results back to the forum.

Regards,
Nick

Pascal van Kempen selected this answer as the correct answer

3 other answers



permanent link
Pascal van Kempen (23126) | answered Apr 03 '13, 7:43 a.m.
edited Apr 03 '13, 7:54 a.m.
Hi all

Many many thanks to Nick, as he pointed me into the right direction with the example he provided.
So, what is the scenario:

in Ant:
- I use MSBuild to build my solution files (and underlying visual studio project files)
- I create an XML build log using a MSBuild extension pack (found at http://msbuildextensionpack.codeplex.com/; see also http://www.msbuildextensionpack.com/help/4.0.5.0/html/242ab4fd-c2e2-f6aa-325b-7588725aed24.htm)
- translate the xml logging (using ant xslt statement)
- publish the translated xml via the jdtCompileLogPublisher

The translation sheet I used (probably not perfect XLST, but at least it works for the tests I performed):

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" indent="yes" encoding="UTF-8" version="1.0"/>

<xsl:strip-space elements="*"/>
 
<xsl:template match="/build">   
    <xsl:text>&#10;</xsl:text>
    <compiler>
        <sources>
                <xsl:apply-templates select="//project" />
        </sources>
    </compiler>
</xsl:template>

<xsl:template match="project">
    <xsl:if test="substring(@file, string-length(@file) - 8) != '.metaproj' and substring(@file, string-length(@file) - 3) != '.sln'">
   <source package="{@file}" path="{@file}">
       <problems>
      <xsl:apply-templates/>
     </problems>
   </source>
  </xsl:if>
</xsl:template>

<!-- skip all the message lines in the original output -->
<xsl:template match="message">
    <xsl:apply-templates/>
</xsl:template>

<!-- handle warnings -->
<xsl:template match="warning">
    <problem id="{@code}" line="{@line}" severity="WARNING">
        <xsl:variable name="TheMessageContent">
            <xsl:value-of disable-output-escaping="yes" select="."/>
        </xsl:variable>
        <message value="reports warning {@code}: {$TheMessageContent}"/>   
        <source_context value="{@file}"/>
        <xsl:apply-templates/>
    </problem>
</xsl:template>

<!-- handle errors -->
<xsl:template match="error">       
    <problem id="{@code}" line="{@line}" severity="ERROR">
        <xsl:variable name="TheMessageContent">
            <xsl:value-of disable-output-escaping="yes" select="."/>
        </xsl:variable>
        <message value="reports error {@code}: {$TheMessageContent}"/>   
        <source_context value="{@file}"/>
        <xsl:apply-templates/>
    </problem>               
 </xsl:template>

</xsl:stylesheet>           

(hmm, looks the forum editor screws up the layout a bit :( )
Hopefully others can use this too.
For me, it shows now clearly the amount of error / warnings per visual studio project file, and the number of compiler warnings can be shown using the appropriate widget.

Once again, thanks  2 all for your support and pointing me in the right direction.

Regards, Pascal


Comments
Nick Edgar commented Apr 03 '13, 9:15 a.m.
JAZZ DEVELOPER

It's good to hear that you were able to spike this through and get it going! 


Nick Edgar commented May 03 '13, 5:52 p.m.
JAZZ DEVELOPER

permanent link
Nick Edgar (6.5k711) | answered Mar 18 '13, 1:26 p.m.
JAZZ DEVELOPER
edited Mar 18 '13, 1:28 p.m.
Scott is correct: we don't currently have an msbuildLogPublisher Ant task.  

For an MS Build build definition, the JBE participant (extension) that runs msbuild passes it a custom logger DLL that outputs a file which we then parse for the compilation results, so any Ant task we provided would need to do similarly and be responsible for running msbuild, not just publishing its regular output.

If you need to move to the more general Ant build definition for more complex sequencing, one option would be to invoke JBE as a one-off invocation to run just the msbuild phase of the build.
This would be similar to how the JazzJBE adapter for Build Forge builds invokes JBE to run the SCM phase of the build.  It does the following.
<execute>$Build_Engine_Path/jbe -userId $Build_User -pass $Build_Password -repository $Repository_Address -buildResultUUID $buildResultUuid -engineUUID $engineUUID -participants com.ibm.team.build.jazzscm -noComplete -verbose</execute>
<resultsblock>
<match pattern="RC\=[0]">
<setenv name="Changes" value="TRUE" type="temp append\n" />
</match>
<match pattern="RC\=[1]">
<setenv name="Failure" value="TRUE" type="temp append\n" />
</match>
<match pattern="RC\=[2]">
<setenv name="Delete" value="TRUE" type="temp append\n" />
</match>
</resultsblock>

For the msbuild participant, you'd pass com.ibm.team.build.msbuild instead of com.ibm.team.build.jazzscm.
Although it uses the engine UUID in the above snippet, you can still identify the engine by id as you do normally: -engine <build engine id>.  The key difference from regular polling mode JBE is to pass: -noComplete which tells JBE it shouldn't complete the build when done, and -participants <participant ids> which tells JBE which participants to run (if unspecified it will run all participants appropriate for the build).



Comments
Nick Edgar commented Mar 18 '13, 1:30 p.m.
JAZZ DEVELOPER

The latter <resultsblock> part tells Build Forge to pattern match against the output of JBE.  RC=0 indicates the build ran, successfully.  RC=1 indicates it failed.  RC=2 indicates the build was scheduled but there were no new changes to build.


Unfortunately, JBE doesn't currently set the actual exit code (there's a work item for this), so this may be hard for you to deal with from Ant.


Nick Edgar commented Mar 18 '13, 1:37 p.m.
JAZZ DEVELOPER

 I've filed enhancement request Provide an msbuildLogPublisher Ant task (256426)


Nick Edgar commented Mar 18 '13, 1:38 p.m.
JAZZ DEVELOPER

 >  RC=2 indicates the build was scheduled but there were no new changes to build.

That's specific to the SCM participant, so wouldn't happen in the msbuild case.


Pascal van Kempen commented Mar 18 '13, 1:47 p.m. | edited Mar 18 '13, 2:57 p.m.

Hi all

first of all: thanks for your help so far !

From this post: https://jazz.net/forum/questions/105512/build-setup-for-net-projects
I was also looking at the JbeMsBuildLogger.dll as log filter. I also tried the extension pack of MSBuild. The latter generates great XML, but RTC does not fetch the right info from it.

But to be honest I'm a bit surprised that there is no parser for the msbuild log, in that sense that I get the expected output when invoking msbuild directly from the buildengine (using buildengine type "Jazz Build for Microsoft Visual Studio Solution"). This raised the expectation to me that there was a parser.

I'll dig into Nicks answer tomorrow when I'm back at work.






In that case.



Nick Edgar commented Mar 18 '13, 3:00 p.m.
JAZZ DEVELOPER

Another option would be to use JbeMsBuildLogger.dll or the extension pack to output the compilation results as XML, then map that XML format to JDT's ECJ (Eclipse Compiler for Java) XML format e.g. using XSLT, then use the jdtCompileLogPublisher Ant task.



Pascal van Kempen commented Mar 18 '13, 5:43 p.m.

Hi Nick

This sounds like the easiest way forward. The XML of the extension pack seems to be quite complete, so I have a good feeling about it.
Do you perhaps have an ant example how to implement this and/or a xlst file that can be used for conversion ?

Thanks a lot for the support!


Nick Edgar commented Mar 19 '13, 9:42 a.m.
JAZZ DEVELOPER

Sorry, I don't have any good examples of XSLT for this.  This thread may help for pointers, but it's for transforming NUnit XML to JUnit XML from before the time we had nunitLogPublisher:


If it would help, I can send an example XML file for ECJ results.  Just fire a note to me at nick_edgar@ca.ibm.com.


Pascal van Kempen commented Mar 27 '13, 3:52 a.m. | edited Mar 27 '13, 9:41 a.m.

Hi Nick

I dropped you a mail last week. Did you already find an example, as I've been digging in the translator already?
I managed to translate the sheet into a different format, and all I need now is to know to which format/style to translate it.

Thanks and regards

Pascal

showing 5 of 8 show 3 more comments

permanent link
Rachel Dichner (27259) | answered May 05 '13, 2:16 p.m.
 Hi Nick
In all answers I see that we need to use MSBuild in order to get the results as XML.
I don't use MSBuild, but if I will have the XML format I can plant the results(errors, warning and success) in that XML and then I can map that XML with the XSLT as described earlier, then use the jdtCompileLogPublisher.
Can someone post the XML format.
Thanks

Your answer


Register or to post your answer.