Jazz Forum Welcome to the Jazz Community Forum Connect and collaborate with IBM Engineering experts and users

How to increase query cache size for EWM when using rest API of ALM using python .

Questions
Tags
Users
Badges
Akash Kaanholkar
(20●3●3●6) Mar 23 '23, 3:20 a.m.
I have connected to ALM server using API .
Just for some project purpose I want to collect few data for that I have written down the queries and used that in the python script..
But I am getting exception with 500 status code with explanation that 'Query has expired.'

Now after digging it up I get to know  that this happens when our servers query cache size is less.

So can you please tell me something that how can I increase the Query cache size using API for python.

0 votes

Comments
Ian Barnard
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Mar 23 '23, 4:57 a.m.
The link you give doesn't work. Are these OSLC Query calls you're making? To which application? What version? What ifix? Please edit this info into your questions and put it in all future questions when you create the question. 

David Honey
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Mar 23 '23, 7:25 a.m.
I came across a similar issue with a clustered EWM. The issue was because I was not using session affinity. The query initially runs on one node in the cluster. The results are paged, and may be stored in memory on that specific node. If, when you fetch the next page that request goes to a different node, that node does not have ny information about the cached query results because they only apply to a specific node. So in a clustered deployment, you have to remember to use session affinity. However, as Ian points out, your question is missing some basic information, such as which product, which release and iFix, details of the REST call you are making such as URI, headers, request body, and details of the response including headers and response body. Please read https://jazz.net/forum/questions/203755/how-should-i-ask-a-question-in-the-forum-if-i-want-to-receive-useful-answers.

Ralph Schoon
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Mar 23 '23, 10:36 a.m.
Note also that the server does not care or know if there is some API used. Any changes to server settings would be in the Advanced properties and for the whole server. This would require to be done by your administrators. 

3 answers
4,328 views
0 votes

3 answers

Permanent link
Akash Kaanholkar
(20●3●3●6) Mar 31 '23, 3:13 a.m.
Thanks to all of three for giving your suggestion .

I was not aware of this details last time .
I am using IBM ALM  (specifically EWM ) of version 7.0.2 SR1 having ifix : "iFix017"

We have developed 1 Tool for our internal use where we estimate story created on users name .
For this we have used "OSLC service " of ALM  to get the data with the help of REST API .

I am using the API is "https://'host'/ccm/oslc/queries/'Query_Id'/rtc_cm:results"

So now the issue  is sometimes I am getting all the result set  of this query and max time it is throwing ""500 Internal server error".

Specifically -->
 "<oslc_cm:error>
<oslc_cm:status>500</oslc_cm:status>
<oslc_cm:message>Query _iryl-c7NEe2x1fNfnKYFUQ expired.</oslc_cm:message>
</oslc_cm:error>"

Means the queries which are having result set less than 100 is executing fine and we are getting the result set as well  but the issue come in picture when the result set is greater than 100.
For the queries greater than 100 it is giving this above shown error and then it is leading to abortion of the same query (not giving result set even though it has loaded that 1st 100 result set but when it goes for 2nd 100  with the link given there only with "oslc_cm:next" it is throwing  "query expired"  and entire query then will get aborted not giving any single result).

Meanwhile I got to know about  the WI which was discussing on the same issue . And in that same issue these peoples had fixed that issue as well with new build.
To see these link you first need to login then you can see this .

So now my question is this query expiration issue is due to our code/logic or it is due to server settings and its model ?
If it is due to our code logic then can you please suggest something so that we will get out of this issue . And if it by server end then please give some solution over this  because we are so much struggling  with this error..

And if possible then can you please provide us that new API created about which was discussed above in the WI  I have provided link for .

And as discussed by @David Honey , same it is happening with me , but as I discussed earlier the tool has built  for our internal use on the Django Framework , So i never encountered with clustered Deployment  and session affinity.  If possible then can you please describe about this session affinity and how to use it .If I get any documentation of it or  code related to this then it will be very great @David

Thanks in advanced to all of you.

0 votes

Comments
Akash Kaanholkar
Mar 31 '23, 3:17 a.m.
So I am requesting you all if possible go through above question and please get me out of this issue. Thank you to all

Ian Barnard
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Mar 31 '23, 3:46 a.m.
Did you try increasing the query cache size as per the technote you link to? https://www.ibm.com/support/pages/increase-query-cache-size-prevent-query-expired-issue

Ralph Schoon
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Mar 31 '23, 3:40 a.m.
I have written up my experience here: https://rsjazz.wordpress.com/2022/03/01/ewm-rest-api-to-access-existing-work-item-queries/  Please be aware that just because the URI contains OLSC, this is NOT an OSLC based API as far as I am aware. This is an EWM legacy API. My experience is that it is very sensitive to the headers used. See my blog post above.   Have you actually read the work item you provide? It is for RTC 2.0. It is from 2009. It is pretty likely that whatever you see is not related to that work item. I have myself not experienced this behavior.

David Honey
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Apr 04 '23, 10:19 a.m.
If you are not using a clustered EWM deployment, then session affinity won't be relevant.

Permanent link
Ralph Schoon
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER (64.6k●3●36●49) edited
Mar 31 '23, 4:29 a.m.

 As mentioned above, this is NOT OSLC, just to set the context. 

I would suggest to open a case with support so that the data can be collected and looked at. 
This is EWM/RTC specific. The server has advanced properties that can be adjusted. E.g. the advanced server properties contains 

com.ibm.team.repository.service.internal.QueryService that has properties that can be adjusted e.g. Query Cache Size.

I have however no idea if this is involved and I think it is not a good idea to touch these, if there is no suggestion from support to do so.

0 votes

Comments
Ralph Schoon
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Mar 31 '23, 4:28 a.m.
If you don't want to use support, I would also suggest to have your admins fillow https://www.ibm.com/support/pages/increase-query-cache-size-prevent-query-expired-issue which is actually the setting I mention above. 

Ralph Schoon
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Mar 31 '23, 5:10 a.m.
Your administrators have to do that. And the application does not care who uses the API with what language/framework.

Permanent link
Akash Kaanholkar
(20●3●3●6) Mar 31 '23, 6:09 a.m.
Thank you Ian . But I didn't make changes in query cache size . I am feeling to do this first and analyze the  behaviour.

And thank you so much to  Ralph for giving this much of information .
I initially thought it is a REST API that I am using here . But Thanks a lot for correcting me and introducing this blog to me  you have return.
I will read this blog thoroughly  so that I can get the right API to query the work Items to get what I want without caching and pagination.

Mainly , I am getting the result set the only issue is that it is causing  pagination of the result set  thats why it throwing an error.

Just I got to know how can I avoid pagination and this query caching then easily I will get entire result set .

Thank again to both of you I will go through blog  but still if anyone will knows that how to avoid pagination of result set of query then it will be very good to me .

Thank again

0 votes

Comments
Ralph Schoon
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Mar 31 '23, 6:24 a.m.
I think it is paramount to understand the different concepts and how they relate. Please be aware that  REST is an architectural style for using HTTP  OSLC is a standard that defines an API/protocol that is also REST based EWM supports OSLC CM and additional APIS such as the Reportable REST API and the stored query API that are also REST based, but not part of OSLC I know, it gets confusing quickly.  

Akash Kaanholkar
Mar 31 '23, 6:21 a.m.
Ok  Ralph . Ya its too much confusing , but need to find out some solution. I will find out solution and will post here for future reference. Thank you so much again for this help Ralph

David Honey
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Apr 04 '23, 10:21 a.m.
I would recommend against disabling pagination. Pagination is there for good reasons: It limits the size of the HTTP response It improves the scalability of the solution The correct solution is to have correct client code that consumes the pages of the query result.

Akash Kaanholkar
Apr 04 '23, 11:52 a.m.
Yes , you are   right David . I personally feel that the pagination is good for us . But as per my requirements , I will get the result set of query more than 500 . Though I have written a code correctly still it is throwing  error  "query has expired" because of server I think. Because I have verified my code multiple times .  So keeping my requirements in mind right now I just want to collect entire result set  and use that further. That's why I need to ignore that pagination. So I request you if you know how to disable pagination then please tell me. Anything , like any attribute-value pair or any code or any reference will help me a lot  So in any case you know please let me know about disabling pagination. Thanks in advance.

Ralph Schoon
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Apr 05 '23, 1:33 a.m.
I find it quite telling how often the term requirement is incorrectly used here in the forum. Your requirement is to be able to run the query. Even that is an implementation detail. You need the Data. Running the query without pagination is an implementation detail and not a requirement. Running a query without pagination, is not suggested and can cause the same over-usage of resources that the query cache is supposed to limit. If you have looked at the query specification however, it is up to you how big the pages are. It is also your responsibility to let your administrators know that you might overuse the system and bring it down and they should monitor while you crank the page size up. Or you talk to the admins about the query cache size and discuss with them to monitor  the systems resource usage and performance while they increase the cache, until all works as desired or it becomes apparent the system resources need to be increased to keep up. Finally you, the admins and all the other stakeholder should be interested in a stable system that provides what is needed.

Your answer

Register or log in to post your answer.

Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.

Ask a question
Guidelines
FAQ
Search context
Follow this question

By Email: 

Once you sign in you will be able to subscribe for any updates here.

By RSS:

Answers
Answers and Comments
Question details
× 11,109

Question asked: Mar 23 '23, 3:20 a.m.

Question was seen: 4,328 times

Last updated: Apr 05 '23, 1:33 a.m.

Confirmation Cancel Confirm