"Personal dashboards" shows this server is offline or unreachable

Kot T. (1.5k●1●12●19) | asked May 31 '13, 3:12 p.m.
JAZZ DEVELOPER

Migrated a 3011 server from one machine to another, then upgraded it to 4.0.2. After the upgrade, all applications seem to communicate OK. However, when clicking on the Home icon, the Personal Dashboards section shows "this server is offline or unreachable". No dashboard can be loaded.

The following error is found in jts.log:

2013-05-30 11:59:28,689 [ http-bio-9443-exec-32] ERROR m.dashboard.service.internal.DashboardQueryService - Error handling POST on /jts/dashboard-query
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:140)

dashboard error

5 answers

Most liked answers|Newest answers|Oldest answers ↑

permanent link

Curtis d'Entremont (1.3k●3) | answered May 31 '13, 3:19 p.m.
FORUM MODERATOR / JAZZ DEVELOPER

Hi Kot,

The dashboard (and other JFS consumers that use the HTTP API) communicate with other services via local HTTP requests. This means it's important that the server be able to make a local HTTP request to its public URL, otherwise dashboards won't work.

The error message indicates that the local request from the dashboard query service to the JFS query service timed out. The most common cause for this is firewalls - if the server happens to go out on the network while trying to make a request to itself, it can get blocked by a firewall, resulting in error message like this.

A good test to do is try to ping the server's public URL from the server (make it ping itself). If it's unable to do so, then there is something blocking the request.

I recommend adding an entry in /etc/hosts to resolve the server's public URL to a local IP so that the local requests does not go on the network at all. This will not only prevent such problems, but will also make the server faster.

0	permanent link Kot T. (1.5k●1●12●19) \| answered May 31 '13, 4:19 p.m. JAZZ DEVELOPER Hi Curtis, Are there any other possible causes? The user confirmed that the entry is already in the hosts file and that he can ping the URL from the server itself.

permanent link

Curtis d'Entremont (1.3k●3) | answered May 31 '13, 4:49 p.m.
FORUM MODERATOR / JAZZ DEVELOPER

Can you confirm that the hosts file has an IP like 127.0.0.1 and not the server's external IP? You may be able to use other tools like traceroute to see where the request is going.

Also note that firewalls may block only some ports, so try something like "telnet <hostname> <port>", where hostname is the host in the public URL, and port is the HTTPS port of the JTS (default 9443). See whether it can open a connection, or if it just gets stuck at "Trying <ip>..."

Last note, DNS lookups are cached indefinitely on the Jazz servers (I'm not sure if this is a Java thing or just the client we're using), so if you've changed the way the public URL resolves to an IP (like editing the /etc/hosts file) without restarting the JVM, then it's probably still going to the old one.

permanent link

Michael Thai (11●2●4) | answered May 31 '13, 5:46 p.m.
edited May 31 '13, 5:47 p.m.

Hi Curtis,

I have both 127.0.0.1 and the server IP in the host file. I listed my Jazz URL under the server IP. I tried putting it under 127.0.0.1 and got the same results. I restarted the JVM each time.

I was able to telnet on the server and open a connection.

Also, traceroute returns 1 row with the server IP.

Comments

Curtis d'Entremont commented May 31 '13, 6:00 p.m.

FORUM MODERATOR / JAZZ DEVELOPER

Might there be something on the machine itself either preventing the request from going out from the JVM, or perhaps certain connections from certain programs, or as certain users? I'm imagining perhaps a local firewall or security suite on the machine itself. Is there a proxy configured or anything like that?

Also worth clarifying that it hits the configured public URL of the server, which may be different from the machine's actual hostname. You can find the configured public URL in the server status at /jts/admin.

There definitely seems to be something blocking the request if that's the exception you're getting every time. I recommend scanning the logs looking for more clues - does this error happen every time you try to load the page (maybe there is another underlying problem)?

permanent link

Michael Thai (11●2●4) | answered Jun 04 '13, 12:05 a.m.

I know the server does not have internet. There could be something blocking. Any suggestions on how to verify?

Is there a reason why the 3.0.1.1 dashboards were working before the 4.0.2 upgrade?

I am going to try a clean 4.0.2 install to see if I can reproduce the issue to eliminate the upgrade as the issue.

Comments

Curtis d'Entremont commented Jun 04 '13, 11:25 a.m.

FORUM MODERATOR / JAZZ DEVELOPER

I can't think of any reason why 3.x would behave differently than 4.0.2 here. It's possible that it's the keys, but I would have expected to see a different error in the logs if that were the case. Are you still seeing the "java.net.SocketTimeoutException: Read timed out" in the logs every time this happens? That one definitely hints at a network issue, can't think of any other obvious reason.

Michael Thai commented Jun 04 '13, 11:34 a.m.

The 4.0.2 clean install dashboards are appearing. I think we can eliminate the server as the issue. Could it be the inbound and outbound keys?

Kot T. commented Jun 04 '13, 12:15 p.m.

JAZZ DEVELOPER

I wonder if this has something to do with the fact that this server has gone thru server rename. Could it be that it is trying to reach out to the old URL and since it does not have internet access, it times out?

If we ping the server by its original URL, does it returns the IP of the machine itself?

Michael Thai commented Jun 04 '13, 12:22 p.m.

I tried it the upgrade without the rename yesterday and the problem still existed.

Both the old URL and new are in the host file so they return the IP of the server.

It's all about the answers!

Related questions

"Personal dashboards" shows this server is offline or unreachable

5 answers

Your answer