It's all about the answers!

Ask a question

Why using Perl LWP REST query returning 406 response on RTC V4.0.5


Rene Matteau (1111) | asked Feb 06 '14, 8:42 a.m.
I am using a Perl program using LWP to access the RTC REST interface to query work items and get data from them.

The program works fine against our current production server running RTC V3.0.1.2 on AIX. We have a test server which was built from a recent backup of our production server and then upgraded to V4.0.5. The query is not working on the test server; returning error 406 (Not Acceptable).

Running the exact query from firefox against the same test server works fine.

The query is: https://test_server_name.com:9443/jazz/oslc/contexts/_NmIe8D8qEd6jzOcwVhPAgw/workitems.json?oslc_cm.pagesize=1000&oslc_cm.properties=dc:identifier&oslc_cm.query=dc:modified>="2013-11-30T23:59:59.000000Z" and dc:modified<="2014-02-06T08:17:24.000000Z"

The Perl LWP request is:

$response_TEST = $userAgent_TEST->request(GET "$HOST_TEST/oslc/contexts/_NmIe8D8qEd6jzOcwVhPAgw/workitems.json?oslc_cm.pagesize=1000&oslc_cm.properties=dc:identifier&oslc_cm.query=dc:modified>=\"$start_date\" and dc:modified<=\"$end_date\"");

The Perl LWP request for the content of a work item works fine from both the production and test server:

$response = $userAgent->request(GET "$HOST_TEST/oslc/workitems/$WIName.json?oslc_cm.properties=$item");

Any ideas as to why a query that work on the production server or from firefox with the test server does not work from Perl with the test server?

Thanks...


Comments
Donald Nong commented Feb 10 '14, 1:04 a.m.

Are you using BASIC or FORM authenticator? If you share your code, it may be easier for other to see what might go wrong.
Also, LWP is not a browser so it can behave differently to a browser.
Purely based on your code shown above and the returned HTTP code, I would suggest you replace "workitems.json" with "workitems" and add an HTTP header "Accept: application/json" to request a JSON response.


Rene Matteau commented Feb 11 '14, 10:57 p.m.

I did investigate more and it get weirder. Let's start with the code:

! /usr/bin/perl



use strict;
use DBI;

use lib ("/home/matteau/JSON-2.53/lib"); # So we can find json
use JSON; # imports encode_json, decode_json, to_json and from_json.

For LWP (https requests to RTC)


use LWP::UserAgent;
use HTTP::Request::Common;
use HTTP::Cookies;
use HTTP::Headers;

Usefull for printing structures for debugging


use Data::Dumper;

my $pw = "asdaa"; # not the real one :-)

my $RTC_USERID = "matteau";



my ($response, $response_string);
my $HOST_PROD = "https://PROD.com:9443/jazz";
my $HOST_TEST = "https://TEST.com:9443/jazz";

my $HOST = $HOST_TEST;

Need cookies for HTTPS


my $userAgent = LWP::UserAgent->new(agent => 'perl get');
$userAgent->cookie_jar({});




Rene Matteau commented Feb 11 '14, 10:58 p.m.

Authenticate


$response = $userAgent->request(POST "$HOST/authenticated/j_security_check?j_username=$RTC_USERID&j_password=$pw");

Looks like we need a code of 302 followed by checking the location


field for the https negotiation


if($response->code == 302) {
  if($response->header('location') =~ m,/authfailed$,) {
    print "Login failed\n";
    #      print Dumper($response);
    die "ERROR: RTC authentication failed ";
  } else {
    print "Login sucessful\n";
  }
} else {
  print "j_security_check failure response : $response->as_string\n";
  die "ERROR: RTC: j_security_check failure ";
}



Rene Matteau commented Feb 11 '14, 10:59 p.m.

Get WIs in a range



FAIL


$response = $userAgent->request(GET "$HOST/oslc/contexts/_NmIe8D8qEd6jzOcwVhPAgw/workitems.json?oslc_cm.pagesize=1000&oslc_cm.properties=dc:identifier&oslc_cm.query=dc:modified>=\"2013-11-30T00:01:59.000000Z\" and dc:modified<=\"2013-12-02T15:33:29.873500Z\"");



PASS


$response = $userAgent->request(GET "$HOST/oslc/contexts/_NmIe8D8qEd6jzOcwVhPAgw/workitems.json?oslc_cm.pagesize=1000&oslc_cm.properties=dc:identifier&oslc_cm.query=dc:modified>=\"2013-11-30T00:01:59.000000Z\" and dc:modified<=\"2013-12-02T15:33:29.873499Z\"");




Rene Matteau commented Feb 11 '14, 10:59 p.m.

if($response->is_success) {
  print "success\n";
  $response_string = $response->content;
} else {

  my $junk = $response->code;
  print "code\t\t=>$junk<=\n";
  print "\n\n\n";
 
  die "TEST FATAL ERROR in response from RTC";
}

my $base = decode_json $response_string;

my $ret = $base->{"oslc_cm:totalCount"};

RTC only return 1000 results, need shorter date range if it is exceeded


if ($ret > 1000) {
  print "INFORMATION: $ret items available but only 1000 returned\n";
} else {
  print "$ret items returned\n";
}

exit;



Look at the pass and fail example. The pass one returns 100 WI. The fail one return a code of 406. The only difference between the queries is the end time; it pass at .873499Z and fails at .873500 (the smallest increment possible to the end time). I doubt that 1000 work items got modified in the mean time...



Any idea?

Thanks...


Rene Matteau commented Feb 12 '14, 11:19 a.m.

One point that may be significant; the PASS query returns 100 items. Is it possible that I am hitting a limit as described in this document: https://jazz.net/forum/questions/140852/paged-oslc-query-result-limited-to-100?

That was not a problem in RTC V3.0.1 but may be a new limit introduced in 4.0.x but returning error 406 seems drastic...


Donald Nong commented Feb 13 '14, 12:05 a.m.

I doubt that. You can just change the pagesize to 100 in the failing URL to verify that. I modified you code and ran it against my own 4.0.5 system, but got an HTTP 408 error "the time allowed for the login process has been exceeded". So I cannot verify whether .873500Z works or not at the moment.


Donald Nong commented Feb 18 '14, 1:27 a.m.

Have you made any progress on this? I've got pas the authentication part and verified your script. No errors were found. The interesting bit is, there is no SimpleQuery service provider in my own CLM 4.0.5 enviroment, and as such the "oslc_cm.query" does not do anything at all. Given this strange thing, my environment may be not the right one to reproduce your problem at all.


Rene Matteau commented Feb 21 '14, 1:53 p.m.

Donald,

Changing the oslc_cm.pagesize to 100 in my code does not change anything, still get the 406 error. Changing oslc_cm.pageSize (note difference in case) to 100 make the program pass with 101 items returned.

So it looks like that for RTC V4.0.5, any queries returning more than 100 items return error 406 if pageSize if above 100. On V302, it accepted (I think) a pagesize of 1000 and returned a successful response with the first 1000 results but with a oslc_cm:results tags with the total number of records found so I could find that I received a truncated set and take appropriate action (in my case, narrow the time range and re-run).

We checked the system's workitem query settings on both systems and they are identical...


Donald Nong commented Feb 25 '14, 1:36 a.m.

I had some trouble with passing on the oslc_cm.pageSize parameter to the query, or whatever parameters in the oslc_cm namespace. The only pagination that I can use is oslc.pageSize. This is in my Linux environment with Perl and LWP though. It seems that Perl is doing something funny and messes up the redirectURL.
Using RESTClient with Firefox, everything works as expected.
In summary, I cannot reproduce your problem, even with more than 100 work items.

showing 5 of 10 show 5 more comments

2 answers



permanent link
Eric Jodet (6.3k5111120) | answered Feb 10 '14, 4:55 a.m.
JAZZ DEVELOPER
 Hello Rene,
I agree with Donald.
Please refer to https://jazz.net/wiki/bin/view/Main/WorkItemAPIsForOSLCCM20
and add appropriate headers

Thanks,
Eric

permanent link
Rene Matteau (1111) | answered Mar 14 '14, 2:52 p.m.
It is working now on both V4.0.5 and V4.0.6. I needed to add an HTTP header "Accept: application/json". This was NOT needed on a V3.0.1 server.

Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.