It's all about the answers!

Ask a question
QuestionsTagsUsersBadges
Follow this question
Question details

×42,788

question asked: Jul 14 '11, 1:41 p.m.
question was seen: 5,676 times
last updated: Jul 14 '11, 1:41 p.m.

Related questions

FAQ - How this Forum works

lscm command fails randomly on AIX

0
Donald Mason (51●5●5) | asked Jul 14 '11, 1:41 p.m.
Problem:

I have a build shop that uses AIX 5.3.0.0 on a 2 processor Power PC, 3GB memory,

I am getting inconsistent intermittent failures using the lscm command on AIX.

I need help debuging and resolving this issue so that I can finish an evaluation of RTC.
Setup of the AIX client:

I have followed suggestions on the RTC forums to use the linux client on AIX. (A sun howto)

I am connecting to RTC 3.0 servers, so I used the RTC-Client-Linux-3.0iFix1.zip file.

* I unzipped the RTC-Client-Linux-3.0iFix1.zip file under /home/dons/RTC-Client-Linux-3.0iFix1/ the directory
* mv /home/dons/RTC-Client-Linux-3.0iFix1/jazz/client/eclipse/jdk /home/dons/RTC-Client-Linux-3.0iFix1/jazz/client/eclipse/jdk.orig
* cd /home/dons/RTC-Client-Linux-3.0iFix1/jazz/client/eclipse
* unziped an AIX java.zip

/home/dons/RTC-Client-Linux-3.0iFix1/jazz/client/eclipse/jdk/jre/bin/java -version gives: 

                  java version "1.6.0"

                  Java(TM) SE Runtime Environment (build pap3260sr9-20101125_01(SR9))

                  IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 AIX ppc-32 jvmap3260sr9-20101124_69295 (JIT enabled, AOT enabled)

                  J9VM - 20101124_069295

                  JIT  - r9_20101028_17488ifx2

                  GC   - 20101027_AA)

                  JCL  - 20101119_01


* mv /home/dons/RTC-Client-Linux-3.0iFix1/jazz/client/eclipse/java /home/dons/RTC-Client-Linux-3.0iFix1/jazz/client/eclipse/jdk
* cp /home/dons/RTC-Client-Linux-3.0iFix1/jazz/scmtools/eclipse/lscm /home/dons/RTC-Client-Linux-3.0iFix1/jazz/scmtools/eclipse/lscm.orig
* changed /home/dons/RTC-Client-Linux-3.0iFix1/jazz/scmtools/eclipse/lscm as follows:

CHANGED: SCM_DAEMON_PATH="${PRGPATH}/scm"

TO: SCM_DAEMON_PATH="${PRGPATH}/scm.sh"

Problem Recreate Scenario

This works every time without error (other than slowness):

* /home/dons/RTC-Client-Linux-3.0iFix1/jazz/scmtools/eclipse/scm.sh login -r https://SERVERNAME.com:9443/jazz/ -u dons -P password

This command works, usually the first 3 or 4 times, but then starts failing randomly about 75 percent of the time:

* /home/dons/RTC-Client-Linux-3.0iFix1/jazz/scmtools/eclipse/lscm login -r https://SERVERNAME.com:9443/jazz/ -u dons -P password

When the command fails, it always gives the following error: 

        java.net.ConnectException: A remote host did not respond within the timeout period.

                 at java.net.PlainSocketImpl.socketConnect(Native Method)

                 at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:363)

                 at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:225)

                 at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:212)

                 at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:377)

                 at java.net.Socket.connect(Socket.java:539)

                 at java.net.Socket.connect(Socket.java:488)

                 at java.net.Socket.(Socket.java:385)

                 at java.net.Socket.(Socket.java:199)

                 at com.ibm.team.filesystem.cli.minimal.client.HttpSwitchingClient.negotiate(HttpSwitchingClient.java:88)

                 at com.ibm.team.filesystem.cli.minimal.client.FrontEndClient.run(FrontEndClient.java:345)

                 at com.ibm.team.filesystem.cli.minimal.client.FrontEndClient.run(FrontEndClient.java:144)

                 at com.ibm.team.filesystem.cli.minimal.client.FrontEndClient.main(FrontEndClient.java:281)


More Information:

This same RTC-Client-Linux-3.0iFix1 package used with the included jdk and unchanged commands on a RHEL AS v3 u4 Linux box (3.20Ghz Intel Xeon, 2048MB memory) work fine and fast.

Both machines are on the same network.

The failure is happening the same way on all 3 AIX servers I have tried.

The AIX servers are functioning fine for all other services (DB2, http, and many other programs).

I have searched with Google and found some tidbits, but none have helped.

* http://www.ibm.com/developerworks/forums/thread.jspa?threadID=271715&tstart=-1

I tried this out, AIXTHREAD_SCOPE to "S" but it had no effect on my results.
* Most others talk about a hard failure where nothing else works.

2 answers

Most liked answers ↑|Newest answers|Oldest answers

0
permanent link
Jeff Care (1.0k●38●33) | answered Jul 16 '11, 9:11 a.m.
It would (potentially) be helpful if we could get some additional debugging info out of the client about what exactly is going on.

Does the client produce any logs? Is there any way to change/augment what gets logged?
0
permanent link
Donald Mason (51●5●5) | answered Jul 18 '11, 2:08 p.m.
I am still looking for any insights on how to debug this problem. I have done a few
things in the last few days, but I am no closer to a solution.


Here a Couple of questions I recieved outside of this post:
1) As I understand, if you run 'lscm login' subcommand continuously (I am calling out 'login' just for this discussion), the subcommand fails with the below mentioned error. Once you get this error, does it work for further runs of the login subcommand? Or it always fails once you have encountered this error?
It will still work randomly after a failure.
2) Are you running the lscm subcommands from within the same sandbox? The reason I ask is because whenever you run the lscm commands it will launch a daemon process and registers the current working directory with that daemon process. If you run again from some other directory, it will launch a new daemon process.
I am not working with any sandbox. I am working from one command line and I believe it is only one daemon, when it starts one.
3) Is the daemon still alive? You could check this by running... 'scm list daemon'. It will list the description, port number and probably you could also confirm by running netstat with the port number.

We have additional logging for the daemon process but I don't think it will help the cause because it fails at the very first step of connecting to the daemon process. You could try launching the daemon manually and then execute the subcommands.... scm daemon start -vmargs -Ddaemon.log.file=/log.txt. The 'daemon start' has other options such as --port and --connection-timeout that could be set. The log files will be located at /,jazz-scm/scratch/{Number} directory. The config_dir is usually users home directory unless specifically set using --config option.

If the command starts off, there is a daemon. I ran another command that fails and the daemon is there still, I run another command that passes and the same daemon is still there.

I have gotten the debug file, though it did not work for me exactly as you had it listed.
Since I am on AIX, I had to edit the scm.sh file and add the
-Ddaemon.log.file=/home/dons/debuglog.txt to the java line itself. This did produce the file in my home directory.

It looks like the log gets 3 lines for every successful command and nothing is logged when the command fails.
I have given all the commands invoked and after each command, I have cated the log file to show the contents.

The following contents of the debuglog.txt file are only added to on a successful command: 


               cat /home/dons/RTC-Client-Linux-3.0iFix1/jazz/scmtools/eclipse/debuglog.txt



                   # Time - Type - Port - Description

                   1310999434964 - CONNECT - 38586 - Client connected

                   1310999435007 - REQUEST - 38586 - GET /com.ibm.team.filesystem.cli.client.internal.daemon.CommandLine Client

                   1310999447240 - DISCONNECT - 0 - Client disconnected



It was also suggested that I try using the same jre that was shipped with the RTC AIX server.
I downloaded RTC 3.0.1 Server install image for AIX , and used it's jre, but the results did not change.

Your answer

Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.