OSLC and Reportable Rest API behaviour with DOORS NG
I am working on some internal company tooling that can automatically post test artefacts to ETM and create links with DOORS NG requirement objects, so I am once again looking into Rest and OSLC APIs, and I've found some strange behaviour that I'm struggling to understand and get the results I want - hopefully someone out there can help me with some tips.
For example, I'm trying to automatically locate a DOORS module by its name (I know I can find the ID via the web browser, but I want users to be able to just put their project / module names into their pipeline, not some magic numbers), and I'm struggling.
Here's what I've tried:
* Using OSLC Service Discovery, I've found the project, component, and folder tree, and can locate the folder containing my module. This has an oslc:serviceProvider link which I was hoping would include capabilities for retrieving folder contents, but when I fetch it I always get a 403 error.
* Using OSLC Service Discover, I can find the top-level Query Capability for my project (just named "Query Capability"). When executed, this returns every single artefact, including requirements and modules - great. However, if I add an oslc.where query parameter to try and limit results to my module, I always get 400 error.
* Using the REST API [https://jazz.net/wiki/bin/view/Main/DNGReportableRestAPI] I've tried to filter for resources by type (both by type name and type id). This always returns zero results (even though I can filter by resource ID, find the module, and confirm the type name and ID I'm supplying are both correct).
* Using the REST API, it is supposed to allow "artifact format" queries, e.g. "/rm/publish/modules" - this doesn't work for any artifact format listed for me, they all result in 400 error.
* I looked at how DOORS NG lists modules in the browser, and it appears to just POST a document to "/rm/views?execute=true" to get its results. I copied its payload & headers and tried to do the same, but I always get 400 error.
So I've tried at least 5 different ways to find my module by name, and none of them seem to work the way the documentation describes they should.
What am I doing wrong?
N.B. I am logging in first using j_security_check, and can succesfully download and upload various objects, and our projects are using configuration control and I'm setting the oslc.context query parameter accordingly (and I've verified this works and I can get different versions of an object by specifying different contexts).
3 answers
Further persistence has resolved at least one route (although I still don't understand many of the other behaviours above).
Using the top level project's query capability called "Query Capability", oslc.where does work for some things but ONLY if I provide an oslc.prefix for them - even common namespaces like dcterms.
This is odd in a couple of ways.
First the OSLC spec suggests common prefixes should be pre-defined by the server (and indeed ETM allows dcterms without prefix)
Second - the actual object when downloaded doesn't have a "dcterms:title" or "dcterms:identifier", but instead an "rrm:title" and "rrm:identifier".
For both rrm / dcterms, not providing the prefix results in 400 error. For rrm, providing the prefix causes the server to accept the query, but ignore the WHERE statement and just provide ALL (~112k) results. For dcterms, providing the prefix causes the server to correctly filter the results even though the property is really "rrm:title" not "dcterms:title"!
So I'm still confused, but may have a potential way forward. The worry is relying on confusing or unexplained behaviours doesn't sound like a robust way to make my tools fully generic and roll them out to many users.
Comments
OSLC Query will be able to find modules by name. Basically the oslc.where is the query and if it references any prefixes these must be specified in oslc.prefix. To get more attributes specify the ones you want using oslc.select - or for all attributes use * - again prefixes mentioned iun the select list must be in the oslc.prefix.
You MUST specify the configuration (in query parameter oslc_config.context for a gc or vvc.configuration for a local configuraiton or header Configuration-Context) you want to query otherwise you'll get results from the default configuration.
You must make sure the URL query parameter values are url-encoded.
You must provide header OSLC-Core-Version: 2.0
You must provide header Accept: applicaiton/rdf+xml
There is more discussion of query in this old article (note this was written before configurations were implemented in DOORS Next) https://jazz.net/library/article/1197
This landing page leads to all the public APIs https://jazz.net/wiki/bin/view/Deployment/ELMProductAPILanding
A query url for modules in a specified configuraiton might look like:
- https://jazz.ibm.com:9443/rm/views?oslc.query=true&projectURL=https%3A//jazz.ibm.com%3A9443/rm/process/project-areas/_JQCH8JibEfC81KA6oULdRQ&oslc.prefix=rdf%3D%3Chttp%3A//www.w3.org/1999/02/22-rdf-syntax-ns%23%3E%2Cdcterms%3D%3Chttp%3A//purl.org/dc/terms/%3E&oslc.where=rdf%3Atype%3D%3Chttp%3A//open-services.net/ns/rm%23RequirementCollection%3E%20and%20dcterms%3Atitle%3D%22Requirements%20for%20Reuse%22&oslc.select=%2A&oslc.paging=false&vvc.configuration=https%3A//jazz.ibm.com%3A9443/rm/cm/stream/_JiDsEJibEfC81KA6oULdRQ
This is fully encoded - de-encoded and split into parameters this looks like:
- https://jazz.ibm.com:9443/rm/views?oslc.query=true&projectURL=https://jazz.ibm.com:9443/rm/process/project-areas/_JQCH8JibEfC81KA6oULdRQ
- oslc.prefix=rdf=<http://www.w3.org/1999/02/22-rdf-syntax-ns#>,dcterms=<http://purl.org/dc/terms/>
- oslc.where=rdf:type=<http://open-services.net/ns/rm#RequirementCollection> and dcterms:title="Requirements for Reuse"
- oslc.select=*
- oslc.paging=false
- vvc.configuration=https://jazz.ibm.com:9443/rm/cm/stream/_JiDsEJibEfC81KA6oULdRQ
Also:
* Using OSLC Service Discover, I can find the top-level Query Capability for my project (just named "Query Capability"). When executed, this returns every single artefact, including requirements and modules - great. However, if I add an oslc.where query parameter to try and limit results to my module, I always get 400 error.
Seems like a reasonable default to return all if you don't specify oslc.where.
If you're getting a 400 when you add oslc.where there's something wrong with the syntax (or url encoding) of your oslc.where parameter value or the othher parts of the URL
* I looked at how DOORS NG lists modules in the browser, and it appears to just POST a document to "/rm/views?execute=true" to get its results. I copied its payload & headers and tried to do the same, but I always get 400 error.
That's probably a private API - use the OSLC Query Capability. Modules (and collections) can be found using oslc.where=rdf:type=<http://open-services.net/ns/rm#RequirementCollection> (with the value suitably URL-encoded, of course)