I am not writing something very much ground-breaking, but since friends do ask me how to use their slow internet lines, here is the solution: Web Proxy with Cache and DNS Caching.
You might find exhaustive articles all over the net, this one is which I made from my experience. Getting them to work is so so simple. It isn’t a rocket science as you might be thinking.
You basically need Squid to act as Web Server Proxy and Bind9 as DNS Caching. The work of Bind9 and squid is not limited just for caching, but you might want to know that Bind powers much of world’s DNS servers. Let’s start
You first need squid and squid-common packages for squid and bind9 and dnsutils for bind.
sudo apt-get install squid bind9
squid-common and dnsutils are dependencies of the the two, so you don’t need to specify
BIND
Just open the file named.conf.options in the folder /etc/bind and change the following section
// forwarders {
// 0.0.0.0;
// }
to
forwarders {
208.67.222.222;
208.67.220.220;
};
Here the two IP addresses are those of the DNS servers (OpenDNS in this case). You can even use the IP address of your router if the latter is configured for DNS.
Now restart bind daemon
sudo /etc/init.d/bind9 restart
Bind can also be used for more than just DNS caching. It can be used for Primary Master Server, Secondary Master Server or mix of all of them. The slightly geekish HOWTO is here. It is written for Ubuntu, but nearly same for all other distros.
Squid
When you installed squid, it has been auto-configured as a caching proxy server. You just need to point your browser’s proxy settings to it. The best way is to set up Network Proxy so that every application can use it.
System > Preferences > Network Proxy
GNOME Proxy Settings
208.67.222.222;
Squid uses port no 3128 as default. You can change it later.
If you want Squid just to surf the internet, then can just change the proxy settings of Firefox.
Edit > Preferences > Advanced > Network
Choose the Settings Button, set up the proxy settings similar to that shown inthe above figure.
You can check the official docs at squid for configuring squid or use this simplified doc at Ubuntu Wiki. Again, it works on all the distros.
Using squid you can customized error pages, shut down network for specified duration of time, block websites and many more.
You might also be interested in DNSMasq. I think it’s a bit lighter weight than Bind and easier to set-up.
I also thought of using it, but put it off the list after reading this
https://help.ubuntu.com/community/Dnsmasq
It says, that
Any workaround for it? I don’t want to uninstall NetworkManager. Not because I hate doing things manually, but because when things can be done using NetworkManager, then why not use it?
Yes, it interferes and really badly at that. Remember how I had problems connecting to wifi? So much that I had to uninstall network manager.
NetworkManager is an unusual piece of software!! When it works, it works perfectly. When it refuses, even lord can’t make it budge.
I do remember your situation of wifi. You spent hours connecting to wifi. Sometimes it was hilarious.
“Just open the file named.conf.options and change the following section”—this file is located under /etc/bind.
nice and small walkthrough..
thanks for it!
Just slipped out of my mind. Thanks for informing. Updated the post!
Just immigrate to USA if u need fast internet connection (hilarious and dumb solution I know but sorry I had to say it.)
But yes if you want real super-duper ultra-fast connection, move to korea or japan. When cable internet connection was just invented and getting ready to be rolled out to the masses, I was highly impressed with Tokyo’s basic DSL connection which was churning out an impressive 300 kb/s at all times.
In India, we also have fast internet connections, but it is not widely available. Plus, laptop is meant to be portable and you don’t know that the next place you visit is going to have a decent sane internet connection.
The other face of this post is that when you can save bandwidth, then why shouldn’t you? If everyone cuts down this over heard, I think we would have more bandwidth in this world.
check your squid logs. what is your cache hit rate? i bet it is nearly 0%. this is because your browser already caches whatever content it can
squid only pays off as a central proxy in a larger network environment where many users go through the same proxy. in a home environment it is only an additional overhead on your system, but it is no smarter than the browser cache.
Jack,
There is another angle. I use more than one browser – Epiphany and Opera. Squid helps in this case.
I can feel the improved speed, not a drastic one, but still there is improvement. The time delay has been minimised between I hit Enter and loading starts.
I use sarg and analysed the logs. The logs were so exhaustive that I closed it. You know a better method to analyse them?
You could try something like:
grep TCP_MISS /var/log/squid/access.log | wc -l
and
grep TCP_HIT /var/log/squid/access.log | wc -l
The 2 commands will give you the cache misses and hits, so you can see how many times squid actually saved you a fetch from the internet.
I would be interested to see if you actually get a good result. I tried many different tweaks over many weeks and the result was always the same. This was a few years ago now, so maybe something has changed.
$ grep TCP_MISS /var/log/squid/access.log | wc -l
17989
$ grep TCP_HIT /var/log/squid/access.log | wc -l
674
Not very encouraging!
Doesn’t look like something really big chances have been made since you last tried.
I think the speed gain is probably due to DNS caching rather than Squid.
If I set it to do more aggressive caching, then the browsing experience may be hampered as it might fetch the older page even when the page has actually been changed.
This problem might be somewhat like I found in Drupal. I set aggressive caching in Admin Panel and always the older page was fetched.
Excellent site, keep up the good work