It's all about the answers!

Ask a question

Collecting Data from Jazz for ETL Purposes


Boris Kuschel (331113) | asked Feb 02 '07, 9:18 a.m.
JAZZ DEVELOPER
Hi,

I am a member of the Polaris project and we are interested in Jazz
collecting metrics related to quality and schedule. The intent is to
collect Jazz project information and roll it up into a warehouse in
Polaris for portfolio management purposes. More information can be found
on our community source web site:
https://cs.opensource.ibm.com/projects/polaris/

What is the best recommended approach for collecting data out of Jazz?
From what i understand, direct database access is not supported so we
would need to go through some sort of Java API. This is definately an
option but i was wondering if there were some other more direct means for
collecting for ETL purposes into a warehouse.

Boris

11 answers



permanent link
Rafik Jaouani (5.0k16) | answered Feb 02 '07, 5:57 p.m.
JAZZ DEVELOPER
Boris,

Jazz has a reporting component that creates a data warehouse to collect
metrics similar to what you need.
We use the java APIs to collect the data. We also register asynch tasks on
the server to keep the data up to date.

Rafik

<kuschel@ca.ibm.com> wrote in message
news:epvv8v$u3d$1@localhost.localdomain...
Hi,

I am a member of the Polaris project and we are interested in Jazz
collecting metrics related to quality and schedule. The intent is to
collect Jazz project information and roll it up into a warehouse in
Polaris for portfolio management purposes. More information can be found
on our community source web site:
https://cs.opensource.ibm.com/projects/polaris/

What is the best recommended approach for collecting data out of Jazz?
From what i understand, direct database access is not supported so we
would need to go through some sort of Java API. This is definately an
option but i was wondering if there were some other more direct means for
collecting for ETL purposes into a warehouse.

Boris

permanent link
Boris Kuschel (331113) | answered Feb 02 '07, 6:19 p.m.
JAZZ DEVELOPER
Rafik,

That is good to hear because it will be easier for us to provide drill
through to data hosted by the Jazz warehouse.

Is this warehousing component part of the standard build that i can
download from jazz.net? Also, where can i find documentation or could
you point me in the right direction (Java API Javadoc, jar file,
Warehouse DDL) to be able to do some poking around myself?

Thanks,

Boris

Rafik Jaouani wrote:
Boris,

Jazz has a reporting component that creates a data warehouse to collect
metrics similar to what you need.
We use the java APIs to collect the data. We also register asynch tasks on
the server to keep the data up to date.

Rafik

kuschel@ca.ibm.com> wrote in message
news:epvv8v$u3d$1@localhost.localdomain...
Hi,

I am a member of the Polaris project and we are interested in Jazz
collecting metrics related to quality and schedule. The intent is to
collect Jazz project information and roll it up into a warehouse in
Polaris for portfolio management purposes. More information can be found
on our community source web site:
https://cs.opensource.ibm.com/projects/polaris/

What is the best recommended approach for collecting data out of Jazz?
From what i understand, direct database access is not supported so we
would need to go through some sort of Java API. This is definately an
option but i was wondering if there were some other more direct means for
collecting for ETL purposes into a warehouse.

Boris


permanent link
Boris Kuschel (331113) | answered Feb 02 '07, 6:22 p.m.
JAZZ DEVELOPER
Rafik,

That is good to hear because it will be easier for us to provide drill
through to data hosted by the Jazz warehouse.

Is this warehousing component part of the standard build that i can
download from jazz.net? Also, where can i find documentation or could
you point me in the right direction (Java API Javadoc, jar file,
Warehouse DDL) to be able to do some poking around myself?

Thanks,

Boris

Rafik Jaouani wrote:
Boris,

Jazz has a reporting component that creates a data warehouse to collect
metrics similar to what you need.
We use the java APIs to collect the data. We also register asynch tasks on
the server to keep the data up to date.

Rafik

kuschel@ca.ibm.com> wrote in message
news:epvv8v$u3d$1@localhost.localdomain...
Hi,

I am a member of the Polaris project and we are interested in Jazz
collecting metrics related to quality and schedule. The intent is to
collect Jazz project information and roll it up into a warehouse in
Polaris for portfolio management purposes. More information can be found
on our community source web site:
https://cs.opensource.ibm.com/projects/polaris/

What is the best recommended approach for collecting data out of Jazz?
From what i understand, direct database access is not supported so we
would need to go through some sort of Java API. This is definately an
option but i was wondering if there were some other more direct means for
collecting for ETL purposes into a warehouse.

Boris


permanent link
Rafik Jaouani (5.0k16) | answered Feb 02 '07, 10:15 p.m.
JAZZ DEVELOPER
Yes, this should be part of the jazz downloads.

Check the com.ibm.team.datawarehouse projects in server jar, these expose
extension points for registring snapshot providers who are responsile in a
declarative for defining the schema of the data warehouse and how it should
be kept up to date.

The com.ibm.team.reports projects expose the jazz data warehouse to Birt and
the ui.

Rafik Jaouani

permanent link
James Moody (3.3k24) | answered Feb 08 '07, 3:19 p.m.
JAZZ DEVELOPER
Rafik Jaouani wrote:
Yes, this should be part of the jazz downloads.

Check the com.ibm.team.datawarehouse projects in server jar, these expose
extension points for registring snapshot providers who are responsile in a
declarative for defining the schema of the data warehouse and how it should
be kept up to date.

The com.ibm.team.reports projects expose the jazz data warehouse to Birt and
the ui.

Rafik Jaouani



Just as a followup to Rafik's comments...

We will shortly be starting a couple of articles. The first will be on
how to write your own BIRT report that takes advantage of the data in
the data warehouse. The second will be on how to extend the data
warehouse to add whatever kind of data you like, and schedule snapshot
jobs to update that data periodically. Stay tuned for more information
about this.

james

permanent link
Thanh Nguyen (76) | answered Jul 25 '07, 8:17 p.m.
We have to collect data from Jazz for analysis as well. So far we've
been writing Eclipse plug-ins that uses different queries in the API to
get the data and serialize them.

I didn't know the datawarehouse is providing these queries. I will look
into it.

Here is a question: If I want create a snapshot all workitem in a
project area, can I extend the datawarehouse component to do that
without explicitly writing queries to the repository?

Thank you,

Thanh



James Moody wrote:
Rafik Jaouani wrote:
Yes, this should be part of the jazz downloads.

Check the com.ibm.team.datawarehouse projects in server jar, these
expose extension points for registring snapshot providers who are
responsile in a declarative for defining the schema of the data
warehouse and how it should be kept up to date.

The com.ibm.team.reports projects expose the jazz data warehouse to
Birt and the ui.

Rafik Jaouani


Just as a followup to Rafik's comments...

We will shortly be starting a couple of articles. The first will be on
how to write your own BIRT report that takes advantage of the data in
the data warehouse. The second will be on how to extend the data
warehouse to add whatever kind of data you like, and schedule snapshot
jobs to update that data periodically. Stay tuned for more information
about this.

james

permanent link
James Moody (3.3k24) | answered Jul 26 '07, 9:57 a.m.
JAZZ DEVELOPER
Hi Thanh,

The data warehouse is extensible, yes. We have recently written a draft
article in the wiki that describes how to extend the data warehouse to
add your own fact and dimension tables, and populate it with data.

See here for details:

https://jazz.net/wiki/bin/view/Main/DataWarehouseExtending

There is other reporting information available on the main Reports page
in the wiki:

https://jazz.net/wiki/bin/view/Main/ReportsMain

If you have any questions just post them here, or if you have
enhancement requests or bugs to report, feel free to open work items for us.

james

Thanh Nguyen wrote:
We have to collect data from Jazz for analysis as well. So far we've
been writing Eclipse plug-ins that uses different queries in the API to
get the data and serialize them.

I didn't know the datawarehouse is providing these queries. I will look
into it.

Here is a question: If I want create a snapshot all workitem in a
project area, can I extend the datawarehouse component to do that
without explicitly writing queries to the repository?

Thank you,

Thanh



James Moody wrote:
Rafik Jaouani wrote:
Yes, this should be part of the jazz downloads.

Check the com.ibm.team.datawarehouse projects in server jar, these
expose extension points for registring snapshot providers who are
responsile in a declarative for defining the schema of the data
warehouse and how it should be kept up to date.

The com.ibm.team.reports projects expose the jazz data warehouse to
Birt and the ui.

Rafik Jaouani


Just as a followup to Rafik's comments...

We will shortly be starting a couple of articles. The first will be on
how to write your own BIRT report that takes advantage of the data in
the data warehouse. The second will be on how to extend the data
warehouse to add whatever kind of data you like, and schedule snapshot
jobs to update that data periodically. Stay tuned for more information
about this.

james

permanent link
Thanh Nguyen (76) | answered Jul 27 '07, 2:29 p.m.
Hi James,

Thanks for the lead. I was trying to follow the tutorial on extending
the data warehouse package. It seem to be a very good way to extract
data for research. I am wondering if there is anyway to get the reports
via the client API instead of going through the web interface?

Thank you,

Thanh

James Moody wrote:
Hi Thanh,

The data warehouse is extensible, yes. We have recently written a draft
article in the wiki that describes how to extend the data warehouse to
add your own fact and dimension tables, and populate it with data.

See here for details:

https://jazz.net/wiki/bin/view/Main/DataWarehouseExtending

There is other reporting information available on the main Reports page
in the wiki:

https://jazz.net/wiki/bin/view/Main/ReportsMain

If you have any questions just post them here, or if you have
enhancement requests or bugs to report, feel free to open work items for
us.

james

Thanh Nguyen wrote:
We have to collect data from Jazz for analysis as well. So far we've
been writing Eclipse plug-ins that uses different queries in the API
to get the data and serialize them.

I didn't know the datawarehouse is providing these queries. I will
look into it.

Here is a question: If I want create a snapshot all workitem in a
project area, can I extend the datawarehouse component to do that
without explicitly writing queries to the repository?

Thank you,

Thanh



James Moody wrote:
Rafik Jaouani wrote:
Yes, this should be part of the jazz downloads.

Check the com.ibm.team.datawarehouse projects in server jar, these
expose extension points for registring snapshot providers who are
responsile in a declarative for defining the schema of the data
warehouse and how it should be kept up to date.

The com.ibm.team.reports projects expose the jazz data warehouse to
Birt and the ui.

Rafik Jaouani


Just as a followup to Rafik's comments...

We will shortly be starting a couple of articles. The first will be
on how to write your own BIRT report that takes advantage of the data
in the data warehouse. The second will be on how to extend the data
warehouse to add whatever kind of data you like, and schedule
snapshot jobs to update that data periodically. Stay tuned for more
information about this.

james

permanent link
James Moody (3.3k24) | answered Jul 28 '07, 9:36 a.m.
JAZZ DEVELOPER
Check out the API in IReportRestService. This is the API that is called
from the web UI (as well as from the rich client). getRenderedReport, if
supplied all of the parameter values, will return you a hunk of HTML
that is the rendered report (keep in mind though that the HTML
references images which are on the server). Other API on the REST
service can retrieve the valid values for parameters, etc.

We're currently working on more API that will let you download a
complete report, either in HTML (with PNGs), PDF or perhaps other
formats that BIRT supports. Stay tuned.

james

Thanh Nguyen wrote:
Hi James,

Thanks for the lead. I was trying to follow the tutorial on extending
the data warehouse package. It seem to be a very good way to extract
data for research. I am wondering if there is anyway to get the reports
via the client API instead of going through the web interface?

Thank you,

Thanh

James Moody wrote:
Hi Thanh,

The data warehouse is extensible, yes. We have recently written a
draft article in the wiki that describes how to extend the data
warehouse to add your own fact and dimension tables, and populate it
with data.

See here for details:

https://jazz.net/wiki/bin/view/Main/DataWarehouseExtending

There is other reporting information available on the main Reports
page in the wiki:

https://jazz.net/wiki/bin/view/Main/ReportsMain

If you have any questions just post them here, or if you have
enhancement requests or bugs to report, feel free to open work items
for us.

james

Thanh Nguyen wrote:
We have to collect data from Jazz for analysis as well. So far we've
been writing Eclipse plug-ins that uses different queries in the API
to get the data and serialize them.

I didn't know the datawarehouse is providing these queries. I will
look into it.

Here is a question: If I want create a snapshot all workitem in a
project area, can I extend the datawarehouse component to do that
without explicitly writing queries to the repository?

Thank you,

Thanh



James Moody wrote:
Rafik Jaouani wrote:
Yes, this should be part of the jazz downloads.

Check the com.ibm.team.datawarehouse projects in server jar, these
expose extension points for registring snapshot providers who are
responsile in a declarative for defining the schema of the data
warehouse and how it should be kept up to date.

The com.ibm.team.reports projects expose the jazz data warehouse to
Birt and the ui.

Rafik Jaouani


Just as a followup to Rafik's comments...

We will shortly be starting a couple of articles. The first will be
on how to write your own BIRT report that takes advantage of the
data in the data warehouse. The second will be on how to extend the
data warehouse to add whatever kind of data you like, and schedule
snapshot jobs to update that data periodically. Stay tuned for more
information about this.

james

permanent link
Thanh Nguyen (76) | answered Aug 01 '07, 2:01 p.m.
Hi James,

Is it possible to deploy reports using the Eclipse client alone (with or
without admin right to the repository)?

Thank you,

Thanh

James Moody wrote:
Check out the API in IReportRestService. This is the API that is called
from the web UI (as well as from the rich client). getRenderedReport, if
supplied all of the parameter values, will return you a hunk of HTML
that is the rendered report (keep in mind though that the HTML
references images which are on the server). Other API on the REST
service can retrieve the valid values for parameters, etc.

We're currently working on more API that will let you download a
complete report, either in HTML (with PNGs), PDF or perhaps other
formats that BIRT supports. Stay tuned.

james

Thanh Nguyen wrote:
Hi James,

Thanks for the lead. I was trying to follow the tutorial on extending
the data warehouse package. It seem to be a very good way to extract
data for research. I am wondering if there is anyway to get the
reports via the client API instead of going through the web interface?

Thank you,

Thanh

James Moody wrote:
Hi Thanh,

The data warehouse is extensible, yes. We have recently written a
draft article in the wiki that describes how to extend the data
warehouse to add your own fact and dimension tables, and populate it
with data.

See here for details:

https://jazz.net/wiki/bin/view/Main/DataWarehouseExtending

There is other reporting information available on the main Reports
page in the wiki:

https://jazz.net/wiki/bin/view/Main/ReportsMain

If you have any questions just post them here, or if you have
enhancement requests or bugs to report, feel free to open work items
for us.

james

Thanh Nguyen wrote:
We have to collect data from Jazz for analysis as well. So far we've
been writing Eclipse plug-ins that uses different queries in the API
to get the data and serialize them.

I didn't know the datawarehouse is providing these queries. I will
look into it.

Here is a question: If I want create a snapshot all workitem in a
project area, can I extend the datawarehouse component to do that
without explicitly writing queries to the repository?

Thank you,

Thanh



James Moody wrote:
Rafik Jaouani wrote:
Yes, this should be part of the jazz downloads.

Check the com.ibm.team.datawarehouse projects in server jar, these
expose extension points for registring snapshot providers who are
responsile in a declarative for defining the schema of the data
warehouse and how it should be kept up to date.

The com.ibm.team.reports projects expose the jazz data warehouse
to Birt and the ui.

Rafik Jaouani


Just as a followup to Rafik's comments...

We will shortly be starting a couple of articles. The first will be
on how to write your own BIRT report that takes advantage of the
data in the data warehouse. The second will be on how to extend the
data warehouse to add whatever kind of data you like, and schedule
snapshot jobs to update that data periodically. Stay tuned for more
information about this.

james

Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.