How to increase query cache size for EWM when using rest API of ALM using python .
I have connected to ALM server using API .
Just for some project purpose I want to collect few data for that I have written down the queries and used that in the python script..
But I am getting exception with 500 status code with explanation that 'Query has expired.'
Now after digging it up I get to know that this happens when our servers query cache size is less.
So can you please tell me something that how can I increase the Query cache size using API for python.
For you to get reference from , visit here: " https://www.ibm.com/support/pages/increase-query-cache-size-prevent-query-expired-issue"
3 answers
Thanks to all of three for giving your suggestion .
I was not aware of this details last time .
I am using IBM ALM (specifically EWM ) of version 7.0.2 SR1 having ifix : "iFix017"
We have developed 1 Tool for our internal use where we estimate story created on users name .
For this we have used "OSLC service " of ALM to get the data with the help of REST API .
I am using the API is "https://'host'/ccm/oslc/queries/'Query_Id'/rtc_cm:results"
So now the issue is sometimes I am getting all the result set of this query and max time it is throwing ""500 Internal server error".
Specifically -->
"<oslc_cm:error>
<oslc_cm:status>500</oslc_cm:status>
<oslc_cm:message>Query _iryl-c7NEe2x1fNfnKYFUQ expired.</oslc_cm:message>
</oslc_cm:error>"
<oslc_cm:status>500</oslc_cm:status>
<oslc_cm:message>Query _iryl-c7NEe2x1fNfnKYFUQ expired.</oslc_cm:message>
</oslc_cm:error>"
Means the queries which are having result set less than 100 is executing fine and we are getting the result set as well but the issue come in picture when the result set is greater than 100.
For the queries greater than 100 it is giving this above shown error and then it is leading to abortion of the same query (not giving result set even though it has loaded that 1st 100 result set but when it goes for 2nd 100 with the link given there only with "oslc_cm:next" it is throwing "query expired" and entire query then will get aborted not giving any single result).
Meanwhile I got to know about the WI which was discussing on the same issue . And in that same issue these peoples had fixed that issue as well with new build.
You can see that here : https://jazz.net/jazz/web/projects/Jazz%20Foundation?_ga=2.267364936.71288068.1680150979-1478019675.1679977389#action=com.ibm.team.workitem.viewWorkItem&id=76944
which I get from https://www.ibm.com/support/pages/increase-query-cache-size-prevent-query-expired-issue
To see these link you first need to login then you can see this .
So now my question is this query expiration issue is due to our code/logic or it is due to server settings and its model ?
If it is due to our code logic then can you please suggest something so that we will get out of this issue . And if it by server end then please give some solution over this because we are so much struggling with this error..
And if possible then can you please provide us that new API created about which was discussed above in the WI I have provided link for .
And as discussed by @David Honey , same it is happening with me , but as I discussed earlier the tool has built for our internal use on the Django Framework , So i never encountered with clustered Deployment and session affinity. If possible then can you please describe about this session affinity and how to use it .If I get any documentation of it or code related to this then it will be very great @David
Thanks in advanced to all of you.
Comments
So I am requesting you all if possible go through above question and please get me out of this issue.
Thank you to all
Did you try increasing the query cache size as per the technote you link to? https://www.ibm.com/support/pages/increase-query-cache-size-prevent-query-expired-issue
I have written up my experience here: https://rsjazz.wordpress.com/2022/03/01/ewm-rest-api-to-access-existing-work-item-queries/
Please be aware that just because the URI contains OLSC, this is NOT an OSLC based API as far as I am aware. This is an EWM legacy API. My experience is that it is very sensitive to the headers used. See my blog post above.
Have you actually read the work item you provide? It is for RTC 2.0. It is from 2009. It is pretty likely that whatever you see is not related to that work item.
I have myself not experienced this behavior.
Have you actually read the work item you provide? It is for RTC 2.0. It is from 2009. It is pretty likely that whatever you see is not related to that work item.
I have myself not experienced this behavior.
If you are not using a clustered EWM deployment, then session affinity won't be relevant.
As mentioned above, this is NOT OSLC, just to set the context.
I would suggest to open a case with support so that the data can be collected and looked at.
This is EWM/RTC specific. The server has advanced properties that can be adjusted. E.g. the advanced server properties contains
com.ibm.team.repository.service.internal.QueryService that has properties that can be adjusted e.g. Query Cache Size.
I have however no idea if this is involved and I think it is not a good idea to touch these, if there is no suggestion from support to do so.
Comments
If you don't want to use support, I would also suggest to have your admins fillow https://www.ibm.com/support/pages/increase-query-cache-size-prevent-query-expired-issue which is actually the setting I mention above.
Your administrators have to do that. And the application does not care who uses the API with what language/framework.
Thank you Ian . But I didn't make changes in query cache size . I am feeling to do this first and analyze the behaviour.
And thank you so much to Ralph for giving this much of information .
I initially thought it is a REST API that I am using here . But Thanks a lot for correcting me and introducing this blog to me you have return.
I will read this blog thoroughly so that I can get the right API to query the work Items to get what I want without caching and pagination.
Mainly , I am getting the result set the only issue is that it is causing pagination of the result set thats why it throwing an error.
Just I got to know how can I avoid pagination and this query caching then easily I will get entire result set .
Thank again to both of you I will go through blog but still if anyone will knows that how to avoid pagination of result set of query then it will be very good to me .
Thank again
Comments
I think it is paramount to understand the different concepts and how they relate. Please be aware that
- REST is an architectural style for using HTTP
- OSLC is a standard that defines an API/protocol that is also REST based
Ok Ralph .
Ya its too much confusing , but need to find out some solution.
I will find out solution and will post here for future reference.
Thank you so much again for this help Ralph
I would recommend against disabling pagination. Pagination is there for good reasons:
- It limits the size of the HTTP response
- It improves the scalability of the solution
The correct solution is to have correct client code that consumes the pages of the query result.
Yes , you are right David .
I personally feel that the pagination is good for us .
But as per my requirements , I will get the result set of query more than 500 .
Though I have written a code correctly still it is throwing error "query has expired" because of server I think. Because I have verified my code multiple times .
So keeping my requirements in mind right now I just want to collect entire result set and use that further. That's why I need to ignore that pagination.
So I request you if you know how to disable pagination then please tell me.
Anything , like any attribute-value pair or any code or any reference will help me a lot
So in any case you know please let me know about disabling pagination.
Thanks in advance.
I find it quite telling how often the term requirement is incorrectly used here in the forum. Your requirement is to be able to run the query. Even that is an implementation detail. You need the Data.
Running the query without pagination is an implementation detail and not a requirement.
Running a query without pagination, is not suggested and can cause the same over-usage of resources that the query cache is supposed to limit. If you have looked at the query specification however, it is up to you how big the pages are.
It is also your responsibility to let your administrators know that you might overuse the system and bring it down and they should monitor while you crank the page size up.
Or you talk to the admins about the query cache size and discuss with them to monitor the systems resource usage and performance while they increase the cache, until all works as desired or it becomes apparent the system resources need to be increased to keep up.
Running the query without pagination is an implementation detail and not a requirement.
Running a query without pagination, is not suggested and can cause the same over-usage of resources that the query cache is supposed to limit. If you have looked at the query specification however, it is up to you how big the pages are.
It is also your responsibility to let your administrators know that you might overuse the system and bring it down and they should monitor while you crank the page size up.
Or you talk to the admins about the query cache size and discuss with them to monitor the systems resource usage and performance while they increase the cache, until all works as desired or it becomes apparent the system resources need to be increased to keep up.
Finally you, the admins and all the other stakeholder should be interested in a stable system that provides what is needed.
Comments
Ian Barnard
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Mar 23 '23, 4:57 a.m.The link you give doesn't work.
David Honey
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Mar 23 '23, 7:25 a.m.I came across a similar issue with a clustered EWM. The issue was because I was not using session affinity. The query initially runs on one node in the cluster. The results are paged, and may be stored in memory on that specific node. If, when you fetch the next page that request goes to a different node, that node does not have ny information about the cached query results because they only apply to a specific node. So in a clustered deployment, you have to remember to use session affinity.
However, as Ian points out, your question is missing some basic information, such as which product, which release and iFix, details of the REST call you are making such as URI, headers, request body, and details of the response including headers and response body. Please read https://jazz.net/forum/questions/203755/how-should-i-ask-a-question-in-the-forum-if-i-want-to-receive-useful-answers.
Ralph Schoon
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Mar 23 '23, 10:36 a.m.Note also that the server does not care or know if there is some API used. Any changes to server settings would be in the Advanced properties and for the whole server. This would require to be done by your administrators.