Collecting Data from Jazz for ETL Purposes

Hi,

I am a member of the Polaris project and we are interested in Jazz
collecting metrics related to quality and schedule. The intent is to
collect Jazz project information and roll it up into a warehouse in
Polaris for portfolio management purposes. More information can be found
on our community source web site:
https://cs.opensource.ibm.com/projects/polaris/

What is the best recommended approach for collecting data out of Jazz?
From what i understand, direct database access is not supported so we
would need to go through some sort of Java API. This is definately an
option but i was wondering if there were some other more direct means for
collecting for ETL purposes into a warehouse.

Boris

Boris,

Jazz has a reporting component that creates a data warehouse to collect
metrics similar to what you need.
We use the java APIs to collect the data. We also register asynch tasks on
the server to keep the data up to date.

Rafik

kuschel@ca.ibm.com> wrote in message
news:epvv8v$u3d$1@localhost.localdomain...
Hi,

I am a member of the Polaris project and we are interested in Jazz
collecting metrics related to quality and schedule. The intent is to
collect Jazz project information and roll it up into a warehouse in
Polaris for portfolio management purposes. More information can be found
on our community source web site:
https://cs.opensource.ibm.com/projects/polaris/

What is the best recommended approach for collecting data out of Jazz?
From what i understand, direct database access is not supported so we
would need to go through some sort of Java API. This is definately an
option but i was wondering if there were some other more direct means for
collecting for ETL purposes into a warehouse.

Boris

Boris,

Jazz has a reporting component that creates a data warehouse to collect
metrics similar to what you need.
We use the java APIs to collect the data. We also register asynch tasks on
the server to keep the data up to date.

Rafik

kuschel@ca.ibm.com> wrote in message
news:epvv8v$u3d$1@localhost.localdomain...
Hi,

I am a member of the Polaris project and we are interested in Jazz
collecting metrics related to quality and schedule. The intent is to
collect Jazz project information and roll it up into a warehouse in
Polaris for portfolio management purposes. More information can be found
on our community source web site:
https://cs.opensource.ibm.com/projects/polaris/

What is the best recommended approach for collecting data out of Jazz?
From what i understand, direct database access is not supported so we
would need to go through some sort of Java API. This is definately an
option but i was wondering if there were some other more direct means for
collecting for ETL purposes into a warehouse.

Boris

Yes, this should be part of the jazz downloads.

Check the com.ibm.team.datawarehouse projects in server jar, these expose
extension points for registring snapshot providers who are responsile in a
declarative for defining the schema of the data warehouse and how it should
be kept up to date.

The com.ibm.team.reports projects expose the jazz data warehouse to Birt and
the ui.

Rafik Jaouani

Rafik Jaouani wrote:
Yes, this should be part of the jazz downloads.

Check the com.ibm.team.datawarehouse projects in server jar, these
expose extension points for registring snapshot providers who are
responsile in a declarative for defining the schema of the data
warehouse and how it should be kept up to date.

The com.ibm.team.reports projects expose the jazz data warehouse to
Birt and the ui.

Rafik Jaouani


Just as a followup to Rafik's comments...

We will shortly be starting a couple of articles. The first will be on
how to write your own BIRT report that takes advantage of the data in
the data warehouse. The second will be on how to extend the data
warehouse to add whatever kind of data you like, and schedule snapshot
jobs to update that data periodically. Stay tuned for more information
about this.

james

We have to collect data from Jazz for analysis as well. So far we've
been writing Eclipse plug-ins that uses different queries in the API to
get the data and serialize them.

I didn't know the datawarehouse is providing these queries. I will look
into it.

Here is a question: If I want create a snapshot all workitem in a
project area, can I extend the datawarehouse component to do that
without explicitly writing queries to the repository?

Thank you,

Thanh



James Moody wrote:
Rafik Jaouani wrote:
Yes, this should be part of the jazz downloads.

Check the com.ibm.team.datawarehouse projects in server jar, these
expose extension points for registring snapshot providers who are
responsile in a declarative for defining the schema of the data
warehouse and how it should be kept up to date.

The com.ibm.team.reports projects expose the jazz data warehouse to
Birt and the ui.

Rafik Jaouani


Just as a followup to Rafik's comments...

We will shortly be starting a couple of articles. The first will be on
how to write your own BIRT report that takes advantage of the data in
the data warehouse. The second will be on how to extend the data
warehouse to add whatever kind of data you like, and schedule snapshot
jobs to update that data periodically. Stay tuned for more information
about this.

james

Hi Thanh,

The data warehouse is extensible, yes. We have recently written a draft
article in the wiki that describes how to extend the data warehouse to
add your own fact and dimension tables, and populate it with data.

See here for details:

https://jazz.net/wiki/bin/view/Main/DataWarehouseExtending

There is other reporting information available on the main Reports page
in the wiki:

https://jazz.net/wiki/bin/view/Main/ReportsMain

If you have any questions just post them here, or if you have
enhancement requests or bugs to report, feel free to open work items for
us.

james

Thanh Nguyen wrote:
We have to collect data from Jazz for analysis as well. So far we've
been writing Eclipse plug-ins that uses different queries in the API
to get the data and serialize them.

I didn't know the datawarehouse is providing these queries. I will
look into it.

Here is a question: If I want create a snapshot all workitem in a
project area, can I extend the datawarehouse component to do that
without explicitly writing queries to the repository?

Thank you,

Thanh



James Moody wrote:
Rafik Jaouani wrote:
Yes, this should be part of the jazz downloads.

Check the com.ibm.team.datawarehouse projects in server jar, these
expose extension points for registring snapshot providers who are
responsile in a declarative for defining the schema of the data
warehouse and how it should be kept up to date.

The com.ibm.team.reports projects expose the jazz data warehouse to
Birt and the ui.

Rafik Jaouani


Just as a followup to Rafik's comments...

We will shortly be starting a couple of articles. The first will be
on how to write your own BIRT report that takes advantage of the data
in the data warehouse. The second will be on how to extend the data
warehouse to add whatever kind of data you like, and schedule
snapshot jobs to update that data periodically. Stay tuned for more
information about this.

james

Hi James,

Thanks for the lead. I was trying to follow the tutorial on extending
the data warehouse package. It seem to be a very good way to extract
data for research. I am wondering if there is anyway to get the reports
via the client API instead of going through the web interface?

Thank you,

Thanh

James Moody wrote:
Hi Thanh,

The data warehouse is extensible, yes. We have recently written a
draft article in the wiki that describes how to extend the data
warehouse to add your own fact and dimension tables, and populate it
with data.

See here for details:

https://jazz.net/wiki/bin/view/Main/DataWarehouseExtending

There is other reporting information available on the main Reports
page in the wiki:

https://jazz.net/wiki/bin/view/Main/ReportsMain

If you have any questions just post them here, or if you have
enhancement requests or bugs to report, feel free to open work items
for us.

james

Thanh Nguyen wrote:
We have to collect data from Jazz for analysis as well. So far we've
been writing Eclipse plug-ins that uses different queries in the API
to get the data and serialize them.

I didn't know the datawarehouse is providing these queries. I will
look into it.

Here is a question: If I want create a snapshot all workitem in a
project area, can I extend the datawarehouse component to do that
without explicitly writing queries to the repository?

Thank you,

Thanh



James Moody wrote:
Rafik Jaouani wrote:
Yes, this should be part of the jazz downloads.

Check the com.ibm.team.datawarehouse projects in server jar, these
expose extension points for registring snapshot providers who are
responsile in a declarative for defining the schema of the data
warehouse and how it should be kept up to date.

The com.ibm.team.reports projects expose the jazz data warehouse to
Birt and the ui.

Rafik Jaouani


Just as a followup to Rafik's comments...

We will shortly be starting a couple of articles. The first will be
on how to write your own BIRT report that takes advantage of the
data in the data warehouse. The second will be on how to extend the
data warehouse to add whatever kind of data you like, and schedule
snapshot jobs to update that data periodically. Stay tuned for more
information about this.

james

Check out the API in IReportRestService. This is the API that is called
from the web UI (as well as from the rich client). getRenderedReport, if
supplied all of the parameter values, will return you a hunk of HTML
that is the rendered report (keep in mind though that the HTML
references images which are on the server). Other API on the REST
service can retrieve the valid values for parameters, etc.

We're currently working on more API that will let you download a
complete report, either in HTML (with PNGs), PDF or perhaps other
formats that BIRT supports. Stay tuned.

james

Thanh Nguyen wrote:
Hi James,

Thanks for the lead. I was trying to follow the tutorial on extending
the data warehouse package. It seem to be a very good way to extract
data for research. I am wondering if there is anyway to get the
reports via the client API instead of going through the web interface?

Thank you,

Thanh

James Moody wrote:
Hi Thanh,

The data warehouse is extensible, yes. We have recently written a
draft article in the wiki that describes how to extend the data
warehouse to add your own fact and dimension tables, and populate it
with data.

See here for details:

https://jazz.net/wiki/bin/view/Main/DataWarehouseExtending

There is other reporting information available on the main Reports
page in the wiki:

https://jazz.net/wiki/bin/view/Main/ReportsMain

If you have any questions just post them here, or if you have
enhancement requests or bugs to report, feel free to open work items
for us.

james

Thanh Nguyen wrote:
We have to collect data from Jazz for analysis as well. So far we've
been writing Eclipse plug-ins that uses different queries in the API
to get the data and serialize them.

I didn't know the datawarehouse is providing these queries. I will
look into it.

Here is a question: If I want create a snapshot all workitem in a
project area, can I extend the datawarehouse component to do that
without explicitly writing queries to the repository?

Thank you,

Thanh



James Moody wrote:
Rafik Jaouani wrote:
Yes, this should be part of the jazz downloads.

Check the com.ibm.team.datawarehouse projects in server jar, these
expose extension points for registring snapshot providers who are
responsile in a declarative for defining the schema of the data
warehouse and how it should be kept up to date.

The com.ibm.team.reports projects expose the jazz data warehouse
to Birt and the ui.

Rafik Jaouani


Just as a followup to Rafik's comments...

We will shortly be starting a couple of articles. The first will be
on how to write your own BIRT report that takes advantage of the
data in the data warehouse. The second will be on how to extend the
data warehouse to add whatever kind of data you like, and schedule
snapshot jobs to update that data periodically. Stay tuned for more
information about this.

james