Jazz Forum Welcome to the Jazz Community Forum Connect and collaborate with IBM Engineering experts and users

Why using Perl LWP REST query returning 406 response on RTC V4.0.5

I am using a Perl program using LWP to access the RTC REST interface to query work items and get data from them.

The program works fine against our current production server running RTC V3.0.1.2 on AIX. We have a test server which was built from a recent backup of our production server and then upgraded to V4.0.5. The query is not working on the test server; returning error 406 (Not Acceptable).

Running the exact query from firefox against the same test server works fine.

The query is: https://test_server_name.com:9443/jazz/oslc/contexts/_NmIe8D8qEd6jzOcwVhPAgw/workitems.json?oslc_cm.pagesize=1000&oslc_cm.properties=dc:identifier&oslc_cm.query=dc:modified>="2013-11-30T23:59:59.000000Z" and dc:modified<="2014-02-06T08:17:24.000000Z"

The Perl LWP request is:

$response_TEST = $userAgent_TEST->request(GET "$HOST_TEST/oslc/contexts/_NmIe8D8qEd6jzOcwVhPAgw/workitems.json?oslc_cm.pagesize=1000&oslc_cm.properties=dc:identifier&oslc_cm.query=dc:modified>=\"$start_date\" and dc:modified<=\"$end_date\"");

The Perl LWP request for the content of a work item works fine from both the production and test server:

$response = $userAgent->request(GET "$HOST_TEST/oslc/workitems/$WIName.json?oslc_cm.properties=$item");

Any ideas as to why a query that work on the production server or from firefox with the test server does not work from Perl with the test server?

Thanks...

0 votes

Comments

Are you using BASIC or FORM authenticator? If you share your code, it may be easier for other to see what might go wrong.
Also, LWP is not a browser so it can behave differently to a browser.
Purely based on your code shown above and the returned HTTP code, I would suggest you replace "workitems.json" with "workitems" and add an HTTP header "Accept: application/json" to request a JSON response.

I did investigate more and it get weirder. Let's start with the code:

! /usr/bin/perl



use strict;
use DBI;

use lib ("/home/matteau/JSON-2.53/lib"); # So we can find json
use JSON; # imports encode_json, decode_json, to_json and from_json.

For LWP (https requests to RTC)


use LWP::UserAgent;
use HTTP::Request::Common;
use HTTP::Cookies;
use HTTP::Headers;

Usefull for printing structures for debugging


use Data::Dumper;

my $pw = "asdaa"; # not the real one :-)

my $RTC_USERID = "matteau";



my ($response, $response_string);
my $HOST_PROD = "https://PROD.com:9443/jazz";
my $HOST_TEST = "https://TEST.com:9443/jazz";

my $HOST = $HOST_TEST;

Need cookies for HTTPS


my $userAgent = LWP::UserAgent->new(agent => 'perl get');
$userAgent->cookie_jar({});



Authenticate


$response = $userAgent->request(POST "$HOST/authenticated/j_security_check?j_username=$RTC_USERID&j_password=$pw");

Looks like we need a code of 302 followed by checking the location


field for the https negotiation


if($response->code == 302) {
  if($response->header('location') =~ m,/authfailed$,) {
    print "Login failed\n";
    #      print Dumper($response);
    die "ERROR: RTC authentication failed ";
  } else {
    print "Login sucessful\n";
  }
} else {
  print "j_security_check failure response : $response->as_string\n";
  die "ERROR: RTC: j_security_check failure ";
}


Get WIs in a range



FAIL


$response = $userAgent->request(GET "$HOST/oslc/contexts/_NmIe8D8qEd6jzOcwVhPAgw/workitems.json?oslc_cm.pagesize=1000&oslc_cm.properties=dc:identifier&oslc_cm.query=dc:modified>=\"2013-11-30T00:01:59.000000Z\" and dc:modified<=\"2013-12-02T15:33:29.873500Z\"");



PASS


$response = $userAgent->request(GET "$HOST/oslc/contexts/_NmIe8D8qEd6jzOcwVhPAgw/workitems.json?oslc_cm.pagesize=1000&oslc_cm.properties=dc:identifier&oslc_cm.query=dc:modified>=\"2013-11-30T00:01:59.000000Z\" and dc:modified<=\"2013-12-02T15:33:29.873499Z\"");



if($response->is_success) {
  print "success\n";
  $response_string = $response->content;
} else {

  my $junk = $response->code;
  print "code\t\t=>$junk<=\n";
  print "\n\n\n";
 
  die "TEST FATAL ERROR in response from RTC";
}

my $base = decode_json $response_string;

my $ret = $base->{"oslc_cm:totalCount"};

RTC only return 1000 results, need shorter date range if it is exceeded


if ($ret > 1000) {
  print "INFORMATION: $ret items available but only 1000 returned\n";
} else {
  print "$ret items returned\n";
}

exit;



Look at the pass and fail example. The pass one returns 100 WI. The fail one return a code of 406. The only difference between the queries is the end time; it pass at .873499Z and fails at .873500 (the smallest increment possible to the end time). I doubt that 1000 work items got modified in the mean time...



Any idea?

Thanks...

One point that may be significant; the PASS query returns 100 items. Is it possible that I am hitting a limit as described in this document: https://jazz.net/forum/questions/140852/paged-oslc-query-result-limited-to-100?

That was not a problem in RTC V3.0.1 but may be a new limit introduced in 4.0.x but returning error 406 seems drastic...

I doubt that. You can just change the pagesize to 100 in the failing URL to verify that. I modified you code and ran it against my own 4.0.5 system, but got an HTTP 408 error "the time allowed for the login process has been exceeded". So I cannot verify whether .873500Z works or not at the moment.

Have you made any progress on this? I've got pas the authentication part and verified your script. No errors were found. The interesting bit is, there is no SimpleQuery service provider in my own CLM 4.0.5 enviroment, and as such the "oslc_cm.query" does not do anything at all. Given this strange thing, my environment may be not the right one to reproduce your problem at all.

Donald,

Changing the oslc_cm.pagesize to 100 in my code does not change anything, still get the 406 error. Changing oslc_cm.pageSize (note difference in case) to 100 make the program pass with 101 items returned.

So it looks like that for RTC V4.0.5, any queries returning more than 100 items return error 406 if pageSize if above 100. On V302, it accepted (I think) a pagesize of 1000 and returned a successful response with the first 1000 results but with a oslc_cm:results tags with the total number of records found so I could find that I received a truncated set and take appropriate action (in my case, narrow the time range and re-run).

We checked the system's workitem query settings on both systems and they are identical...

I had some trouble with passing on the oslc_cm.pageSize parameter to the query, or whatever parameters in the oslc_cm namespace. The only pagination that I can use is oslc.pageSize. This is in my Linux environment with Perl and LWP though. It seems that Perl is doing something funny and messes up the redirectURL.
Using RESTClient with Firefox, everything works as expected.
In summary, I cannot reproduce your problem, even with more than 100 work items.

showing 5 of 10 show 5 more comments


2 answers

Permanent link
 Hello Rene,
I agree with Donald.
Please refer to https://jazz.net/wiki/bin/view/Main/WorkItemAPIsForOSLCCM20
and add appropriate headers

Thanks,
Eric

0 votes


Permanent link
It is working now on both V4.0.5 and V4.0.6. I needed to add an HTTP header "Accept: application/json". This was NOT needed on a V3.0.1 server.

0 votes

Your answer

Register or log in to post your answer.

Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.

Search context
Follow this question

By Email: 

Once you sign in you will be able to subscribe for any updates here.

By RSS:

Answers
Answers and Comments
Question details

Question asked: Feb 06 '14, 8:42 a.m.

Question was seen: 7,220 times

Last updated: Mar 14 '14, 2:52 p.m.

Confirmation Cancel Confirm