How to request a list of artifacts in a component using Python REST API?
I asked an earlier question about querying DNG to find the artifacts for a project, and realized I needed to include a specific OSLC request in my query. However I am still having issues getting any information other than the folder names/structure. I receive a 403: Forbidden error with every request I make, however I am fairly certain the authentication is working as I'm able to see these folders as well as some other protected information. I have also tried using a single session object to make the requests, with no change in results. Here is an example of a request I'm making to see the artifacts inside a component:
self.headers = {'Content-type': 'application/x-www-form-urlencoded', 'OSLC-Core-Version': '2.0', 'Accept': 'application/rdf+xml'}
info = {'oslc.select':'*' , 'oslc.where':'dcterms:identifier=123456' , 'oslc.prefix':'dcterms=https://ServerName/dc/terms/'}
encoded = urlencode(info, quote_via=quote_plus)baseURL = 'https://ServerName/rm/views?oslc.query=true&projectURL=https%3A%2F%2FServerName%2Frm%2Fprocess%2Fproject-areas%2F_1ipDAGUlEemksZVbt6XKiw&'
resp4_URL = baseURL + encoded
resp4 = s.get(resp4_URL,cookies=jar,verify=False,headers=self.headers)
And here is the response I'm receiving:
<Response [403]>
<rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
<rdf:Description>
<err:detailedMessage rdf:datatype="http://www.w3.org/2001/XMLSchema#string"
>CRRRS1602W The operation is forbidden</err:detailedMessage>
<err:errorMessage rdf:datatype="http://www.w3.org/2001/XMLSchema#string"
>Forbidden</err:errorMessage>
<err:errorStatus rdf:datatype="http://www.w3.org/2001/XMLSchema#long"
>403</err:errorStatus>
</rdf:Description>
</rdf:RDF>
Any help is greatly appreciated!
Accepted answer
Hi Allison
1. Using a single requests session is definitely the way to go as far as I'm concerned - once your session is authenticated you won't have to do anything with cookies, i.e. less code, less opportunity for mistakes.
2. For my test server (6.0.6.1 iFix013) on an opt-out project (i.e. configuration management not enabled) I get a query base like this:
https://jazz.ibm.com:9443/rm/views?oslc.query=true&projectURL=https%3A%2F%2Fjazz.ibm.com%3A9443%2Frm%2Fprocess%2Fproject-areas%2F__NDNJ0NcLEeqXpuBdEolY7wNOTE this doesn't end in &, where yours seems to - you need to confirm the full URL you are using is well-formed
3. A full query URL looks like this
https://jazz.ibm.com:9443/rm/views?oslc.pageSize=200&oslc.paging=true&oslc.prefix=dcterms%3D%3Chttp%3A%2F%2Fpurl.org%2Fdc%2Fterms%2F%3E&oslc.query=true&oslc.select=%2A&oslc.where=dcterms%3Aidentifier%3D3949&projectURL=https%3A%2F%2Fjazz.ibm.com%3A9443%2Frm%2Fprocess%2Fproject-areas%2F_NDNJ0NcLEeqXpuBdEolY7w
URL-decoded and spread out for readability out this has the following base URL and parameters (bolding for parameter values which have to be encoded):
https://jazz.ibm.com:9443/rm/views
?oslc.pageSize=200
&oslc.paging=true
&oslc.prefix=dcterms=<http://purl.org/dc/terms/>
&oslc.query=true
&oslc.select=*
&oslc.where=dcterms:identifier=3949
&projectURL=https://jazz.ibm.com:9443/rm/process/project-areas/NDNJ0NcLEeqXpuBdEolY7w
(apologies if the projectURL appears slightly different from the previous URL, this forum does something helpful/irritating with underscore characters, there's one at the start of the final part of projectURL)
I get two results (one is the core artifact, one is a module artifact)
NOTE the identifier must _not be surrounded by quotes - I tried this and got no results (but no 403)
NOTE the format of the value for oslc.prefix dcterms=<http://purl.org/dc/terms/>
The oslc.paging and oslc.pageSize are optional, and the server doesn't have to comply but I put them in anyway because it gives the opportunity when testing to speed things up by cutting short the query by setting the page size to e.g. 10 and stopping after the first page ;-)
Rather than using string concatenation, my python code takes the base query URL and uses urllib.parse.urlparse to split out the existing parameters, adds part 4 (the existing parameters from the base URL) to a dictionary of the specific query parameters then uses urllib.parse.urlunparse to correctly combine them back, this does the encoding at the same time :-) But as long as the final URL is well-formed that's really all that's needed.
4. Headers
My successful query doesn't have a Content-Type header, has the OSLC-Core-Version: 2.0 and Accept: application/rdf+xml but also has a vvc.configuration: https://jazz.ibm.com:9443/rm/cm/stream/_NMu5RtcLEeqXpuBdEolY7w (this is for an opt-out project, it doesn't give a 403 and results are fine if I don't provide this, but definitely would be required to get correct results for a configuration-managed project because it determines the component+configuration results come from)
Comparing to where you are in your question, I'd check:
1. What does your full query URL look like - are the parameters valid (first one with ? subsequent with &, that's a very easy mistake to make), is the format of each parameter correct (in particular dcterms), is it correctly encoded - you should be able to use this URL with a browser client like Postman, with the correct headers, and get results.
2. Check does your user in fact have permission to access the project?
Comments
4 other answers
Check baseURL. It should be the queryBase for the project area. Follow rootservices to rmServiceProviders to get the ServiceProviderCatalog, lookup the ServiceProvider for your project area, and get its Services XML document. The queryBase should be in there.
Are you including a cookie store with your connection so that the authentication tokens are included in subsequent requests? Here's some sample code that gives some hints...
self.base_url = server_url
self.userid = user
self.password = password
self.spc = None
self.sp = None
self.ownerMap = dict()
# Disable SSL so that we can read self-assigned certificates
self.http = httplib2.Http(".cache", disable_ssl_certificate_validation=True)
self.http.follow_redirects = True
self.headers = {'Content-type': 'application/rdf+xml'}
protectedResource = "/whoami"
# Create an authentication challenge by attempting to access a protected resource
resp, content = self.http.request( self.base_url + protectedResource, 'GET', headers=self.headers)
if (resp.status == 404):
# the JRS server does not support whoami
protectedResource = ''
resp, content = self.http.request( self.base_url + protectedResource, 'GET', headers=self.headers)
if ('x-com-ibm-team-repository-web-auth-msg' in resp and resp['x-com-ibm-team-repository-web-auth-msg'] == 'authrequired'):
# JEE Forms authentication
resp, content = self.http.request( self.base_url + protectedResource, 'GET', headers=self.headers)
self.headers['Cookie'] = resp['set-cookie']
self.headers['Content-type'] = 'application/x-www-form-urlencoded'
# now we can start the authentication via j_security_check page
resp, content = self.http.request(self.base_url+'/j_security_check' , 'POST', headers=self.headers,
body=urllib.parse.urlencode({'j_username': user, 'j_password': password}))
# Successful login will set the LtpaToken2 required for subsequent authenticated access
self.headers['Cookie'] = resp['set-cookie']
else:
# Try sending the credentials for OpenIDConnect
self.http.add_credentials(user, password)
# check authentication was successful, if not throw exception
resp, content = self.http.request( self.base_url + protectedResource, 'GET', headers=self.headers)
assert (resp.status == 200), f"Login failed with status: {resp.status}"
Comments
dcterms:identifier is of type xsd:string in the OSLC specification, and the OSLC query specification says strings have to be quoted in order to distinguish them from booleans, and numbers. So quoting the dcterms:identifier="330" should work and does for me. It also works without the quotes, but that is not following the OSLC standard.
Comments
Comments
Ian Barnard
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Dec 08 '20, 6:37 a.m.