Why using Perl LWP REST query returning 406 response on RTC V4.0.5
I am using a Perl program using LWP to access the RTC REST interface to query work items and get data from them.
The program works fine against our current production server running RTC V3.0.1.2 on AIX. We have a test server which was built from a recent backup of our production server and then upgraded to V4.0.5. The query is not working on the test server; returning error 406 (Not Acceptable). Running the exact query from firefox against the same test server works fine. The query is: https://test_server_name.com:9443/jazz/oslc/contexts/_NmIe8D8qEd6jzOcwVhPAgw/workitems.json?oslc_cm.pagesize=1000&oslc_cm.properties=dc:identifier&oslc_cm.query=dc:modified>="2013-11-30T23:59:59.000000Z" and dc:modified<="2014-02-06T08:17:24.000000Z" The Perl LWP request is: $response_TEST = $userAgent_TEST->request(GET "$HOST_TEST/oslc/contexts/_NmIe8D8qEd6jzOcwVhPAgw/workitems.json?oslc_cm.pagesize=1000&oslc_cm.properties=dc:identifier&oslc_cm.query=dc:modified>=\"$start_date\" and dc:modified<=\"$end_date\""); The Perl LWP request for the content of a work item works fine from both the production and test server: $response = $userAgent->request(GET "$HOST_TEST/oslc/workitems/$WIName.json?oslc_cm.properties=$item"); Any ideas as to why a query that work on the production server or from firefox with the test server does not work from Perl with the test server? Thanks...
showing 5 of 10
show 5 more comments
|
2 answers
Hello Rene,
I agree with Donald.
Please refer to https://jazz.net/wiki/bin/view/Main/WorkItemAPIsForOSLCCM20
and add appropriate headers
Thanks,
Eric
|
It is working now on both V4.0.5 and V4.0.6. I needed to add an HTTP header "Accept: application/json". This was NOT needed on a V3.0.1 server.
|
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.
Comments
Are you using BASIC or FORM authenticator? If you share your code, it may be easier for other to see what might go wrong.
Also, LWP is not a browser so it can behave differently to a browser.
Purely based on your code shown above and the returned HTTP code, I would suggest you replace "workitems.json" with "workitems" and add an HTTP header "Accept: application/json" to request a JSON response.
I did investigate more and it get weirder. Let's start with the code:
! /usr/bin/perl
use strict;
use DBI;
use lib ("/home/matteau/JSON-2.53/lib"); # So we can find json
use JSON; # imports encode_json, decode_json, to_json and from_json.
For LWP (https requests to RTC)
use LWP::UserAgent;
use HTTP::Request::Common;
use HTTP::Cookies;
use HTTP::Headers;
Usefull for printing structures for debugging
use Data::Dumper;
my $pw = "asdaa"; # not the real one :-)
my $RTC_USERID = "matteau";
my ($response, $response_string);
my $HOST_PROD = "https://PROD.com:9443/jazz";
my $HOST_TEST = "https://TEST.com:9443/jazz";
my $HOST = $HOST_TEST;
Need cookies for HTTPS
my $userAgent = LWP::UserAgent->new(agent => 'perl get');
$userAgent->cookie_jar({});
Authenticate
$response = $userAgent->request(POST "$HOST/authenticated/j_security_check?j_username=$RTC_USERID&j_password=$pw");
Looks like we need a code of 302 followed by checking the location
field for the https negotiation
if($response->code == 302) {
if($response->header('location') =~ m,/authfailed$,) {
print "Login failed\n";
# print Dumper($response);
die "ERROR: RTC authentication failed ";
} else {
print "Login sucessful\n";
}
} else {
print "j_security_check failure response : $response->as_string\n";
die "ERROR: RTC: j_security_check failure ";
}
Get WIs in a range
FAIL
$response = $userAgent->request(GET "$HOST/oslc/contexts/_NmIe8D8qEd6jzOcwVhPAgw/workitems.json?oslc_cm.pagesize=1000&oslc_cm.properties=dc:identifier&oslc_cm.query=dc:modified>=\"2013-11-30T00:01:59.000000Z\" and dc:modified<=\"2013-12-02T15:33:29.873500Z\"");
PASS
$response = $userAgent->request(GET "$HOST/oslc/contexts/_NmIe8D8qEd6jzOcwVhPAgw/workitems.json?oslc_cm.pagesize=1000&oslc_cm.properties=dc:identifier&oslc_cm.query=dc:modified>=\"2013-11-30T00:01:59.000000Z\" and dc:modified<=\"2013-12-02T15:33:29.873499Z\"");
if($response->is_success) {
print "success\n";
$response_string = $response->content;
} else {
my $junk = $response->code;
print "code\t\t=>$junk<=\n";
print "\n\n\n";
die "TEST FATAL ERROR in response from RTC";
}
my $base = decode_json $response_string;
my $ret = $base->{"oslc_cm:totalCount"};
RTC only return 1000 results, need shorter date range if it is exceeded
if ($ret > 1000) {
print "INFORMATION: $ret items available but only 1000 returned\n";
} else {
print "$ret items returned\n";
}
exit;
Look at the pass and fail example. The pass one returns 100 WI. The fail one return a code of 406. The only difference between the queries is the end time; it pass at .873499Z and fails at .873500 (the smallest increment possible to the end time). I doubt that 1000 work items got modified in the mean time...
Any idea?
Thanks...
One point that may be significant; the PASS query returns 100 items. Is it possible that I am hitting a limit as described in this document: https://jazz.net/forum/questions/140852/paged-oslc-query-result-limited-to-100?
That was not a problem in RTC V3.0.1 but may be a new limit introduced in 4.0.x but returning error 406 seems drastic...
I doubt that. You can just change the pagesize to 100 in the failing URL to verify that. I modified you code and ran it against my own 4.0.5 system, but got an HTTP 408 error "the time allowed for the login process has been exceeded". So I cannot verify whether .873500Z works or not at the moment.
Have you made any progress on this? I've got pas the authentication part and verified your script. No errors were found. The interesting bit is, there is no SimpleQuery service provider in my own CLM 4.0.5 enviroment, and as such the "oslc_cm.query" does not do anything at all. Given this strange thing, my environment may be not the right one to reproduce your problem at all.
Donald,
Changing the oslc_cm.pagesize to 100 in my code does not change anything, still get the 406 error. Changing oslc_cm.pageSize (note difference in case) to 100 make the program pass with 101 items returned.
So it looks like that for RTC V4.0.5, any queries returning more than 100 items return error 406 if pageSize if above 100. On V302, it accepted (I think) a pagesize of 1000 and returned a successful response with the first 1000 results but with a oslc_cm:results tags with the total number of records found so I could find that I received a truncated set and take appropriate action (in my case, narrow the time range and re-run).
We checked the system's workitem query settings on both systems and they are identical...
I had some trouble with passing on the oslc_cm.pageSize parameter to the query, or whatever parameters in the oslc_cm namespace. The only pagination that I can use is oslc.pageSize. This is in my Linux environment with Perl and LWP though. It seems that Perl is doing something funny and messes up the redirectURL.
Using RESTClient with Firefox, everything works as expected.
In summary, I cannot reproduce your problem, even with more than 100 work items.