Jazz Forum Welcome to the Jazz Community Forum Connect and collaborate with IBM Engineering experts and users

Caching Proxy Solution Squid

Hello together,

currently I'm facing a problem getting the caching proxy solution with squid running.
I know this article Using content caching proxies for Jazz SCM and tried to setup a squid server running on RHEL 5.8
My Jazz server is running on Windows Server 2008 RC2 and is working on a WAS 7.
On the same server I have a reverse proxy running so that the URL of the RTC is https://jazzdev.rsint.net/ccm

After setting up the Squid solution described in the article above we could connect to it using cURL on the same machine. When using the Jazz SCM or the Eclipse or Browser on another windows client we where not able to connect to the Jazz server using the proxy.

We search for any issues and extended the configuration of squid for the parts we found on the www.
Our configuration is this now:
debug_options ALL,1 33,2 28,9 44,3
cache_replacement_policy heap GDSF
memory_replacement_policy heap GDSF
cache_dir aufs /usr/local/squid/var/cache 6000 256 256
cache_mem 2000 MB
cache_store_log none
cache_peer jazzdev.rsint.net parent 443 0 no-query originserver name=httpsAccel ssl login=PROXYPASS sslflags=DONT_VERIFY_PEER
cache_peer_access httpsAccel allow all
coredump_dir /usr/local/squid/var/cache
http_access allow all
http_port 3128 transparent
https_port 3129 transparent cert=/usr/local/squid/etc/server.pem accel key=/usr/local/squid/etc/privkey.pem
refresh_pattern . 0 20% 4320
cachemgr_passwd disable all
maximum_object_size 1024 MB
maximum_object_size_in_memory 16 MB
buffered_logs on
visible_hostname t-wsg01.rsint.net
access_log /usr/local/squid/var/logs/access.log squid
#
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
#
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
#
#http_access allow manager localhost
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

Now I can connect to the proxy and can download files using Eclipse.
The Jazz scm refuses the connection with a "forbidden" error message.

But my main problem is: Even when loading files using Eclipse the caching proxy is caching .... nothing ....

Anybody any ideas why the cache is not getting filled ?

Greetings,
Simon

0 votes

Comments

 Can you post a sample of your access log?  It is saying "TCP_MISS" ?

Hi John,

yes, very much TCP_MISS messages:


Greetings,
Simon

I don't see any lines in that output which say IVersionedContentService ... can you grep/show the part of the log where that service is being accessed?   

Hi John, there are now such entries inside the log...

For clarity : The lines with IVersionedContentService say TCP_MISS ?  

Hi John, sorry there was a char too much inside my answer ...
I meant "there are no such entries inside the log".

I just see TCP_MISS entries ... nothing else.

At the moment I connect the client using the proxy entries inside network configuration.
In eclipse e.g. I connect directly to the jazz server and have the network configured to use a proxy. Is this wrong?

Running squid -v prints:
rbroot@t-wsg01# sbin/squid -v
Squid Cache: Version 3.0.STABLE26
configure options:  '-prefix=/usr/local/squid' '--with-pthreads' '--enable-storeio=ufs,aufs' '--enable-removal-policies=lru,heap' '--enable-ssl' '--with-large-files' 'CFLAGS=-O2 -pipe -m64 -march=core2 -fomit-frame-pointer -s' 'LDFLAGS=-m64 -s -Wl,-O1' 'CXXFLAGS=-O2 -pipe -m64 -march=core2 -fomit-frame-pointer -s'

showing 5 of 6 show 1 more comments


5 answers

Permanent link
hm.. here is my squid.conf
<code>
cache_replacement_policy heap GDSF
memory_replacement_policy heap GDSF
cache_dir aufs /usr/local/squid/var/cache 10240 256 256
cache_mem 1024 MB
cache_store_log stdio:/usr/local/squid/var/logs/cachelog
cache_peer 192.168.2.106 parent 9743 0 no-query originserver name=httpsAccel ssl login=PROXYPASS sslflags=DONT_VERIFY_PEER
cache_peer 192.168.2.106 parent 9843 0 no-query originserver name=httpsAccel1 ssl login=PROXYPASS sslflags=DONT_VERIFY_PEER
cache_peer 192.168.2.106 parent 9643 0 no-query originserver name=httpsAccel2 ssl login=PROXYPASS sslflags=DONT_VERIFY_PEER
https_port 9743 cert=/usr/local/squid/etc/server.pem accel key=/usr/local/squid/etc/privkey.pem vhost
https_port 9843 cert=/usr/local/squid/etc/server.pem accel key=/usr/local/squid/etc/privkey.pem vhost
https_port 9643 cert=/usr/local/squid/etc/server.pem accel key=/usr/local/squid/etc/privkey.pem vhost
acl ccm1 port 9743
acl ccm2 port 9843
acl ccm3 port 9643
#cache_peer_access httpsAccel allow ccm1
#cache_peer_access httpsAccel deny  ccm2 ccm3
#cache_peer_access httpsAccel allow all
#cache_peer_access httpsAccel1 allow ccm2
#cache_peer_access httpsAccel1 deny  ccm1 ccm3
#cache_peer_access httpsAccel2 allow ccm3
#cache_peer_access httpsAccel2 deny  ccm1 ccm2

coredump_dir /usr/local/squid/var/cache
http_access allow ccm1
#http_access allow ccm2
#http_access allow ccm3

refresh_pattern .              0       20%     40320
cachemgr_passwd disable all
maximum_object_size 5120 MB
maximum_object_size_in_memory 16 MB
buffered_logs on
visible_hostname 192.168.2.32

</code>


1 vote


Permanent link
can u comment out your ACLs for a test?

I don't have any on mine, and can connect just fine from my build machines

0 votes

Comments

Hi Sam, even after outcommenting those statements there are only "TCP_MISS" inside the logfile.


Permanent link
Oh, there was also a bug introduced in squid v 3. I had to fall back to 2.8 I think
my build server is talking to the proxy as if it was the RTC server..

https://192.168.2.32:9743

0 votes

Comments

This means you take the URL of your proxy at the client and the proxy rewrites this to the address of the jazz server - right?

This means inside of Eclipse you don't need the proxy configuration you just connect directly to the proxy?

correct.. do not use the Eclipse proxy config..

all apps talk directly to the squid proxy as if it IS the RTC server


Permanent link
correct.. the proxy IS the RTC server from all clients.  we do not use the formal 'passthru proxy' configuration.

on my dev system, I changed the DNS entry for the RTC server to use the IP address of the squid server and the clients didn't have to do anything.

the squid url rewrite cannot change the structure of the URL.. Eclipse is sensitive to that..
so
https://server:port/context

must remain
https://realserver/context,

it cannot change to
https://realserver
or
https://realserver/context/context2

0 votes

Comments

Hi Sam,

thanks for the answer, but as I understood this article from IBM correctly, this should normally not be needed.
You describe a scenario where everybody is using the same proxy, as the proxy DNS name is the old application DNS name. But in our environment we need more than one proxy - one proxy for each location. This is why I would prefer using this solution without a "DNS rename".

Next is - shouldn't it be possible to use the proxy cache with an own address which is beeing used in the "https_proxy" environment variable? Means that I still connect to the application server but the indication weather I use a caching proxy or not should be only through this variable or the network settings inside of Eclipse.

As I understand the article and the how IBM talks about it, this way should be possible and in our case this would be the most preferred way.

Greetings,
Simon


Permanent link
the article says

Using a Reverse Accelerator Proxy

 You can also configure squid or apache httpd (with the mod_cache module) to run as a accelerator proxy server against your Jazz RTC Team Server.  The strategy here is that instead of referencing the Jazz RTC Team Server URI from your client, you would connect instead to a proxy server which forwards all requests onto the Jazz RTC Team Server. 

Note: For best results in using a reverse accelerator proxy, clients should start new eclipse, command line or visual studio sandboxes.  There can be issues in switching between different repository URIs pointing to the same repository from within the clients. 

Setting up Squid with HTTP as Accelerator Proxy



there is also talk of the forward proxy, at the beginning, but not much. The accelerator reverse caching proxy works.. we will have a number of them thruout our network. some near the servers in our central site, and one at some of our remote locations.  the proxy servers will also talk to each other as siblings (cache peers) 

this is a typical multi-proxy design used for web technology for years.

the design is twin purposed: 1. remove as much redundant traffic from the server as possible, and 2. for remote locations, provide as close to local lan speeds as possible.

For builds this reduces the loading time from hours to minutes. slightly slower that the original local lan setup the users had before moving to the central admin structure

the DNS server at the locations (local or remote) will direct the users application (eclipse, Windows, or Browser, or SCM tools) to the caching proxy nearest them, to provide the fastest speeds possible.

0 votes

Your answer

Register or log in to post your answer.

Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.

Search context
Follow this question

By Email: 

Once you sign in you will be able to subscribe for any updates here.

By RSS:

Answers
Answers and Comments
Question details
× 1,203
× 457
× 27
× 8

Question asked: Apr 05 '13, 9:56 a.m.

Question was seen: 10,170 times

Last updated: May 09 '13, 11:38 a.m.

Confirmation Cancel Confirm