It's all about the answers!

Ask a question

How to authenticate using Python for RTC REST API


Carlos Ferreira (91123) | asked Dec 28 '11, 8:18 p.m.
JAZZ DEVELOPER
Hi I have been trying to use RTC REST reporting interfaces using Python v2.7.1 URLLib2 to access this REST Reporting URL using the following code. RTC v3.0.1.1 on Windows with Tomcat v5.5

def rest_call( uid, pw, url):


print( "Entered rest_call" )
print( "url passed in was %s" %url)
password_manager = urllib2.HTTPPasswordMgrWithDefaultRealm()
password_manager.add_password(
None, url, uid, pw
)
auth_handler = urllib2.HTTPBasicAuthHandler(password_manager)
opener = urllib2.build_opener(auth_handler)
urllib2.install_opener(opener)
try:
data = urllib2.urlopen(url).read()
except urllib2.HTTPError, e:
print( "HTTP error: %d" % e.code )
print( "HTTP error: %d" % e.reason )
print( "HTTP error: %s" % e.headers )
except urllib2.URLError, e:
print( "Network error: %s" % e.reason.args[1])
print("Here is what came back from the rest_call %s " %data)
return (data)

I was able to get this to work on Jazz.net/hub but when I try doing this on my own local RTC Tomcat Based server it doesn't work.

I keep getting html back instead of the XML that I get when I try the same url in the browser. Any ideas? Since I am not getting an authentication error I don't think that is the problem.

URL being used is http://localhost:9080/ccm/rpt/repository/workitem?

Thanks for the help.

Carlos Ferreira

Other unanswered questions on Authenticated use of RTC REST interfaces :
https://jazz.net/forums/viewtopic.php?t=18749
https://jazz.net/forums/viewtopic.php?t=16162


Here is what comes back using above code:
<Licensed>

<html>
<head>
<meta>
<meta>
<title></title>

<link>
<link>

<style>
#net-jazz-ajax-NoScriptMessage {
width: 100%;
color: #D0D0D0;
font-size: 2em;
text-align: center;
position: absolute;
top: 1%;
z-index: 999;
}
</style>
</head>
<body>
<noscript><div>Javascript is either disabled or not available in your Browser</div></noscript>
<div>Loading...</div>
<div></div>
<script>
djConfig = {
isDebug: false,
usePlainJson: true,
baseUrl: "/ccm/web/dojo/",
locale: "en-us",
localizationComplete: true
};
/*null*/
net = {jazz: {ajax: {}}};
net.jazz.ajax._contextRoot = "/ccm";
net.jazz.ajax._webuiPrefix = "/web/";
</script>
<script></script>
<script>
/* <CDATA> */
</script>
<script>
/* <CDATA> */
</script>
</body>
</html>

10 answers



permanent link
sam detweiler (12.4k6180201) | answered Dec 29 '11, 8:06 a.m.
I don't use CURL, but maybe my salesforce.com code can help..

see https://jazz.net/forums/viewtopic.php?t=12745

Sam

permanent link
Sean G Wilbur (8722421) | answered Jan 01 '12, 8:49 p.m.
JAZZ DEVELOPER
Carlos,

You need to explicitly set the content-type and accept headers on your request to "application/x-oslc-cm-change-request+xml" or "application/x-oslc-cm-change-request+json".

-Sean

permanent link
Carlos Ferreira (91123) | answered Jan 04 '12, 9:43 a.m.
JAZZ DEVELOPER
Here is what I had to do to ultimately get it to work.
1. Set the authentication mechanism to basic authentication on the Tomcat server per the previously mentioned Jazz.net technote.
2. Set user name and password in the Headers and base64 encode it.
3. Set a header to fake that the request was coming from a browser.
4. Store the cookies so for subsequent requests credentials / cookie id was passed in.

Here is the method implemented that this worked using Python.
def rest_call_basic_security(feed, encoded_url):

    COOKIEFILE = 'cookies.lwp'

# the path and filename to save your cookies in

cj = None
ClientCookie = None
cookielib = None

# Let's see if cookielib is available
try:
import cookielib
except ImportError:
# If importing cookielib fails
# let's try ClientCookie
try:
import ClientCookie
except ImportError:
# ClientCookie isn't available either
urlopen = urllib2.urlopen
Request = urllib2.Request
else:
# imported ClientCookie
urlopen = ClientCookie.urlopen
Request = ClientCookie.Request
cj = ClientCookie.LWPCookieJar()

else:
# importing cookielib worked
urlopen = urllib2.urlopen
Request = urllib2.Request
cj = cookielib.LWPCookieJar()
# This is a subclass of FileCookieJar
# that has useful load and save methods

if cj is not None:
# we successfully imported
# one of the two cookie handling modules

if os.path.isfile(COOKIEFILE):
# if we have a cookie file already saved
# then load the cookies into the Cookie Jar
cj.load(COOKIEFILE)

# Now we need to get our Cookie Jar
# installed in the opener;
# for fetching URLs
if cookielib is not None:
# if we use cookielib
# then we get the HTTPCookieProcessor
# and install the opener in urllib2
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
urllib2.install_opener(opener)

else:
# if we use ClientCookie
# then we get the HTTPCookieProcessor
# and install the opener in ClientCookie
opener = ClientCookie.build_opener(ClientCookie.HTTPCookieProcessor(cj))
ClientCookie.install_opener(opener)

txdata = None
# if we were making a POST type request,
# we could encode a dictionary of values here,
# using urllib.urlencode(somedict)

txheaders = {'User-agent' : 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'}
# fake a user agent, some websites (like google) don't like automated exploration


try:
req = Request(encoded_url, txdata, txheaders)
# create a request object encoding the userid and password and adding them to the headers.
base64string = base64.encodestring(
'%s:%s' % (feed.ds_uid, feed.ds_pw))[:-1]
authheader = "Basic %s" % base64string
req.add_header("Authorization", authheader)

handle = urlopen(req)
# and open it to return a handle on the url

data = handle.read()
#XML Document returned from the RTC REST Service.
print "Data returned %s" %data
return (data)

except IOError, e:
print 'We failed to open "%s".' % encoded_url
if hasattr(e, 'code'):
print 'We failed with error code - %s.' % e.code
elif hasattr(e, 'reason'):
print "The error object has the following 'reason' attribute :"
print e.reason
print "This usually means the server doesn't exist, is down, or we don't have an internet connection."

# TODO store the status as part of the feed status last time it was run instead of sys exit
# HTTP Error 401 means wrong user id or password was provided.
sys.exit()

else:
print 'Here are the headers of the page :'
print handle.info()
# handle.read() returns the page
# handle.geturl() returns the true url of the page fetched
# (in case urlopen has followed any redirects, which it sometimes does)

print
if cj is None:
print "We don't have a cookie library available - sorry."
print "I can't show you any cookies."
else:
print 'These are the cookies we have received so far :'
for index, cookie in enumerate(cj):
print index, ' : ', cookie
cj.save(COOKIEFILE)
# save the cookies again

permanent link
Carlos Ferreira (91123) | answered Jan 04 '12, 9:44 a.m.
JAZZ DEVELOPER
Thanks, that helped me go down the path of trying to figure out what the right headers to set and uid/pw encryption.

permanent link
Carlos Ferreira (91123) | answered Jan 04 '12, 9:45 a.m.
JAZZ DEVELOPER
Hi Sean, I am using the straight RTC repository reporting REST API. I wasn't using the OSLC CM REST api so I didn't have to do this step to get it to work.

permanent link
Nick Edgar (6.5k711) | answered Jan 05 '12, 7:17 a.m.
JAZZ DEVELOPER
If you've configured the server for basic HTTP authentication, then the code that uses basic auth in your first example should work fine, and you should not need to worry about cookies.

If it's using form-based auth (the default), then you have to follow the form-based auth protocol, and do need to track cookies. More info on the latter approach can be found in this article:
http://www.ibm.com/developerworks/rational/library/10/programmatic-authentication-and-certificate-handling-for-rational-team-concert-2-0/index.html?ca=drs-
(found via searching for 'authentication' at https://jazz.net/library)

permanent link
Carlos Ferreira (91123) | answered Jan 11 '12, 9:59 a.m.
JAZZ DEVELOPER
Thanks Nick

permanent link
sam detweiler (12.4k6180201) | answered Mar 19 '12, 7:48 p.m.
I had some code that I wrote to connect to RTC from saleforce.com using the OLC discovery, and forms logon, to rtc 2.0.0.2 and it worked fine..

but not I can't make it work anymore, to 3.0.1.1

I get a 200 from the post to the j_security_check, but don't get the redirect back to the url that forced the logon.. (get the catalog)

any ideas?

Sam

permanent link
sam detweiler (12.4k6180201) | answered Mar 20 '12, 12:17 p.m.
here are the steps I go thru

read the root services document

10:47:30.123 (123386000)|CALLOUT_REQUEST||System.HttpRequest
10:47:30.546 (546649000)|CALLOUT_RESPONSE||System.HttpResponse

parse the rootservices doc and get the workitem catalog

10:47:30.551 (551161000)|CALLOUT_REQUEST||System.HttpRequest
10:47:30.712 (712911000)|CALLOUT_RESPONSE||System.HttpResponse

302 is 'moved', and the 'location' header contains the new url.
so, lets go there

10:47:30.715 (715888000)|CALLOUT_REQUEST||System.HttpRequest
10:47:30.847 (847604000)|CALLOUT_RESPONSE||System.HttpResponse

according to the referenced doc, I now need to 'post' back to j_security_check, the username and password of the user..
    The next step is to construct a form to pass to the WebSphere Application Server j_security_check. To do this, use the Name Value Pair classes of the Apache HttpClient:


List<NameValuePair> authFormParams = new ArrayList<NameValuePair>();
authFormParams.add(new BasicNameValuePair("j_username", "myUsername"));
authFormParams.add(new BasicNameValuePair("j_password", "myPassword"));


Now, construct a URLEncodedFormEntity. This entity is composed of a list of URL-encoded pairs. Here, you use it to pass the user name and password to the Rational Team Concert server in an HTTP Post request:

UrlEncodedFormEntity entity = new UrlEncodedFormEntity(authFormParams, "UTF-8");
HttpPost httpPostAuth = new HttpPost(
"https://myRtcServer:9443/jazz/authenticated/j_security_check");
httpPostAuth.setEntity(entity);


but note guide is old, using the 'jazz' context.

so, (a value/pair set).. Salesforce http doesn't provide this class.

j_username=uuuuuuu,j_password=pppppppp

and the post back

10:47:30.850 (850732000)|CALLOUT_REQUEST||System.HttpRequestMethod=POST]
10:47:31.097 (1097240000)|CALLOUT_RESPONSE||System.HttpResponseStatusCode=200]

and we get these header keys back

X-com-ibm-team-repository-web-auth-msg, Date, Content-Length, Via, X-Cache-Lookup, Content-Type, X-Cache, Server, Proxy-Connection

and the X-com-ibm-team-repository-web-auth-msg header value is 'authfailed'

but I don't see any indication of why..

and I also don't know how to cause the post successful authentication to redirect back to the original site request

permanent link
sam detweiler (12.4k6180201) | answered Mar 21 '12, 11:15 a.m.
maybe someone can help me out here..

there is one thing I didn't mention..

salesforce will not talk to a https site without an official ssl certificate.
so I am hacking the code during dev & test by replacing the https and ssl port with the non-secure version (and have configed the server to not require ssl).. but I can't change the home url.. which is set to server:9643

SO, if I send an auth request to the NON-secure port, will is always fail?

Your answer


Register or to post your answer.