Unable to get cookies in the response header on hitting the authorization URL

However this approach of explicit initial authentication has the significant disadvantage that when authentication expires you won't get re-authenticated, you'll just get operations failing.

A better approach is authenticate only when the server indicates authentication is required, i.e. try do each GET/POST operation that your application wants to do always using a single session (i.e. so cookies received from the server are automatically provided).

This description is for FORM authentication:

1. Ensure your http client is always automatically following redirects and is a session that you always use, so cookies are propagated
2. Do the operation (GET/POST/PUT etc.) that your application needs to do
3. Check the response; if it is 200 AND there's a header X-com-ibm-team-repository-web-auth-msg: authrequired then you ignore any content and do the authentication GET as you describe (using the same session, of course)
4. If the response to the auth GET returns 200 with a cookie JAZZ_AUTH_TOKEN then you have authenticated successfully. If not successful then give up now no point continuing because you aren't authenticated
5. If the original operation was GET then automatically following the redirects means the response from the authentication GET is the data you requested. If the original operation was POST/PUT you'll have to repeat that with the original data.

Above approach means your client will (re-)authenticate completely automatically without your client code worrying about authentication timeout.

Here's an example of some Python code that does the above sequence for repeated GET operations (code for PUT/POST would add a redo of the original operation once authentication is successful), in this case to access the Reportable REST API of DOORS Next, but the same principle will work for the OSLC APIs. Also note this code is simplified as indicated in the comments to only apply to DOORS Next which uses jts for authentication, and this is hardcoded.

import requests

import urllib3

import xml.etree.ElementTree as ET

# Disable the InsecureRequestWarning so we can quietly control SSL certificate validation

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

# details of test env

host = "https://jazz.ibm.com:9443"

username,password= "USERNAME","PASSWORD"

def get_with_optional_login(s,url, username, password):

    '''

    This function tries to perform a GET on url - and checks the result,

    If authentication is needed it does this using provided username/password, and then returns the data

    ONLY WORKS FOR FORM AUTHENTICATION!

    '''

    # execute the request - NOTE verify (of SSL cert) is set to False because test server using self-signed cert

    response = s.get(url,verify=False )

    response.raise_for_status()

    # The response is "good" (i.e. 200) but if auth is needed there is a specific header+value

    # (in which case the response is not the data requested, but the data will be returned by authenticating)

    if response.headers.get('X-com-ibm-team-repository-web-auth-msg',"") == 'authrequired':

        print("Auth required")

        # Need to construct the authentication URL

        # A more sophisticated and general-purpose method would use urllib.parse/unparse

        #  to split the URL, modify it using the authentication path specified in the JazzFormAuth cookie

        #  and reassemble. That would work for any context root e.g. /jts1

        #  This simple code which is only for RM hardcodes the auth URL to assume /jts

        auth_url = f"{host}/jts/j_security_check?j_username={username}&j_password={password}"

        

        # Do the authentication GET - if successful, this automatically redirects to GET from the original URL - i.e. the ressults requested are retrieved                                   

        # this happens because the server puts client-specific infomation into cookies - i.e. what URL was initially accessed which needed authentication

        # (e.g. in this code /rm/publish/resources/) - into the cookies on the session - not literally, but the server remembers the details

        # and following successful auth the server redirects to that URL, i.e. the data is retrieved

        # NOTE authentication on POST would need more careful handling - because http protocol specifies that redirects on a POST if followed

        # automatically must end with a GET - so for POST needing auth this code would have to explicitly retry the POST

        # but this code doesn't do POST, and note in general you will always do a GET on a protected resource (and therefore be authenticated)

        # before doing any POST

        # but note you might be unlucky and auth token expires just before a POST and then get occasional http failures because unauthenticated

        # (bad idea to equate unlikely with impossible)

        print(f"Authenticating to {auth_url}")

        response = s.get(auth_url)

        response.raise_for_status()

        # check the auth was successful - is there a JAZZ_AUTH_TOKEN?

        if 'JAZZ_AUTH_TOKEN' not in response.cookies:

            raise Exception ( f"Authentication failed using {username=} {password=}!" )

        print("Auth successful")

    return response

# Create the session used for all requests to the server

s = requests.session()

# setup a proxy to allow watching the query/results

# the Reportable Rest query url (this also works in your browser)

# advise DON'T USE THIS URL ON YOUR PRODUCTION SERVER because it may add a lot of load!

# because it tries to retrieve all artifacts!

url = f"{host}/rm/publish/resources/"

# limit to three pages just in case you ignore the comment above

# about not running it on production

pagelimit = 3

# retrieve all the results - works for one or many pages

while url is not None and pagelimit > 0:

    response = get_with_optional_login(s,url , username, password)

    xmlresult = ET.fromstring(response.content)

    # do something with this page of results...

    # ....

    print( ".",end='',flush=True )

    # find the href for the next page of results

    url = xmlresult.get("href",None)

    pagelimit -= 1

        

print( "\nFinished" )