Cache Youtube videos using Squid in Ubuntu Linux

It is well known fact that nowadays more and more ISPs in the world use Ubuntu driven servers as proxies, domain name servers, routers etc. Squid is one the major open source web caching proxy software for Ubuntu and Linux as a whole. Term ‘caching‘ means a way to store Internet objects on locally deployed proxy server for sake of reducing bandwidth consumption and access time to popular web content. You may read more about squid caching software at wikipedia or squid-cache.org.

Youtube videos can also be cached and squid web caching proxy under Ubuntu is reasonable choice for this purpose. It’s common practice when one user downloads some very popular youtube video and then shares its URL to other users withing the same organization. Caching of such youtube video using squid will definitely save Internet connection bandwidth and traffic as every time users download that video it will be fetched from local web cache rather than from youtube servers.

Use Synaptic package manager to install squid package, here is the line to get it installed using terminal:

sudo aptitude install squid

(please notice that youtube caching requires squid of at least 2.7STABLE6 version).

Once installed you should edit /etc/squid/squid.conf (main configuration of squid proxy) to apply configuration necessary to cache video from youtube. Below is an example of such configuration file for squid-2.7STABLE9 so you are welcome to open squid.conf with your favorite text file editor and copy/paste below config into it.

1. Open terminal and type the following commands to get started:

sudo echo -n > /etc/squid/squid.conf

(this cleans default squid.conf configuration file)

sudo gedit /etc/squid/squid.conf

(paste below configuration example and save changes)

http_port 3128
access_log none
coredump_dir none
cache_dir ufs /var/spool/squid/ 10000 16 256

acl all src all
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
acl ssl_ports port 443
acl safe_ports port 80 21 443 1025-65535
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !safe_ports
http_access deny CONNECT !ssl_ports
http_access allow all
icp_access deny all
acl QUERY2 urlpath_regex get_video\? videoplayback\? videodownload\?
cache allow QUERY2
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
acl youtube dstdomain .youtube.com
cache allow youtube
acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9]
upgrade_http0.9 deny shoutcast
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache

refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern -i \.flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private
refresh_pattern (get_video\?|videoplayback\?|videodownload\?) 5259487 99999999% 5259487 override-expire ignore-reload negative-ttl=0
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 0% 4320

quick_abort_min -1 KB
maximum_object_size 4 GB
minimum_object_size 512 bytes

acl store_rewrite_list urlpath_regex \/(get_video\?|videodownload\?|videoplayback.*id)
storeurl_access allow store_rewrite_list
storeurl_access deny all

storeurl_rewrite_program /etc/squid/storeurl.pl
storeurl_rewrite_children 1
storeurl_rewrite_concurrency 10

2. Create /etc/squid/storeurl.pl file:

sudo gedit /etc/squid/storeurl.pl

(paste below configuration example and then save changes)

#!/usr/bin/perl

$|=1;
while (<>) {
@X = split;
$x = $X[0];
$_ = $X[1];

if (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?\&(itag=[0-9]*).*?\&(id=[a-zA-Z0-9]*)/) {
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/" . $2 . "&" . $3 . "\n";

} else {
print $x . $_ . "\n";
}
}

3. Apply execution bit to /etc/squid/storeurl.pl, here is terminal command for this:

sudo chmod +x /etc/squid/storeurl.pl

4. Now it’s time to start squid daemon:

sudo service squid start

In order to check it squid was started successfully type the following command:

sudo netstat -lnp | grep 3128

It should show one line in case of success. You can also point Ubuntu browser to 127.0.0.1:3128 as a proxy and try loading some web page like www.ubuntuka.com which could also set as a start page there :-)