Installing thummer 0.01 on Ubuntu 8.04 LTS Server

Posted on Wed 14 January 2009 in thummer • Tagged with apache, cutycapt, django, gecko, thummer, ubuntu, webkit, xvfb

thummer is a website-snapshot and thumbnailing utility I am working on, built using django. Read my previous thummer blog post to find out more about the project.

Assumptions

In this article, I am assuming that we are installing thummer on an existing Ubuntu server (8.04 or later) with Apache already installed, and that you are comfortable with editing apt's sources.list file.

Step 1: Install A "Fake" X-Server

If you do not have a desktop environment installed on your server (probably a good thing!), then you will need to install xvfb, which provides a virtual screen so that we can capture the rendered output of the website:

Xvfb provides an X server that can run on machines with no display hardware and no physical input devices. It emulates a dumb framebuffer using virtual memory.
sudo apt-get install xvfb

Step 2: Install CutyCapt

We now need to install CutyCapt (webkit rendering).

First, we need to install the cutycapt dependencies and build tools:

sudo apt-get install build-essential

For hardy only: We also need some packages from hardy-backports - add the following lines to /etc/apt/sources.list.

deb http://archive.ubuntu.com/ubuntu hardy-backports main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu hardy-backports main restricted universe multiverse``

Now install the required qt4 packages:

sudo apt-get update
sudo apt-get install libqt4-dev libqt4-webkit libqt4-svg

It's now probably a good idea to remove (or comment out) the hardy-backports lines from your sources.list file.

Download cutycapt source, and expand the tarball:

cd ~/
wget http://cutycapt.svn.sourceforge.net/viewvc/cutycapt/CutyCapt.tar.gz\?view\=tar
tar -xvvzf CutyCapt.tar.gz\?view\=tar

Compile cutycapt:

cd CutyCapt
qmake
make

Step 3: Install Django

First install some packages we'll need later:

sudo apt-get install python-django python-imaging python-pysqlite2 libapache2-mod-python

For hardy only: The version of django hardy ships with is too old for thummer. Install Jaunty's version of python-django from the updates repository by downloading the .deb file from Launchpad (remember to keep an eye out for future security updates):

cd ~/
wget http://launchpadlibrarian.net/17665378/python-django_1.0-1ubuntu1_all.deb
sudo dpkg -i python-django_1.0-1ubuntu1_all.deb

Step 4: Install & Configure thummer

Download thummer, expand the tarball, and move the extracted files to /var/www:

cd ~/
wget http://launchpad.net/thummer/trunk/0.01/+download/thummer_0.01.tar.gz
tar -xvvzf thummer_0.01.tar.gz
sudo mv thummer_0.01 /var/www/thummer

Make sure subversion is installed:

sudo apt-get install subversion

Download the required site-package, sorl-thumbnails:

sudo svn checkout http://sorl-thumbnail.googlecode.com/svn/trunk/sorl /usr/local/lib/python2.5/site-packages/sorl``

Create the database, and initial admin user:

cd /var/www/thummer/thummer
./manage.py syncdb

Set owner so that the apache process can write to the database:

sudo chown -R www-data /var/www/thummer/database

Set owner so that the apache process can write to the media directory:

sudo chown -R www-data /var/www/thummer/media

Update the django website settings:

sudo nano ./settings.py

Make sure that "XVFB = True", specify the full path to the database for DATABASE_NAME.

Ensure the full path to the CutyCapt binary is correct.

Step 5: Configure Apache

Ensure that the python module for Apache is enabled:

sudo a2enmod mod_python

Create a virtual-host site configuration file:

sudo nano /etc/apache2/sites-available/thummer

Paste in the following configuration - replace "thummer.domainname.com" with your desired domain name:

<VirtualHost *>

  ServerName thummer.domainname.com
  DocumentRoot /var/www/thummer/thummer

  <Directory "/var/www/thummer/thummer">
    AllowOverride All
    Order Allow,Deny
    Allow from All

    SetHandler python-program
    PythonHandler django.core.handlers.modpython
    SetEnv DJANGO_SETTINGS_MODULE thummer.settings
    PythonPath "['/var/www/thummer'] + sys.path"
  </Directory>

  # Static Media Content
  Alias /media /var/www/thummer/media
  <Location "/media">
    SetHandler None
    Order Allow,Deny
    Allow from All
  </Location>

  Alias /admin-media /usr/share/python-support/python-django/django/contrib/admin/media
  <Location "/admin-media">
    SetHandler None
    Order Allow,Deny
    Allow from All
  </Location>

</VirtualHost>

Enable the site, and restart apache:

sudo a2ensite thummer
sudo /etc/init.d/apache2 restart

Bam! Thats It! Enjoy, and let me know how it goes!

Usage

Just use the following URL syntax to reference the image (e.g. in img elements):

http://thummer.domainname.com/[width]/[height]/[crop]/http://url-to-capture/

the value of [crop] can either be 0 or 1, where 0 = scale & fit, and 1 = scale & crop.

e.g. to generate a 300x300 pixel cropped thumbnail of the BBC News website:

http://thummer.domainname.com/300/300/1/http://news.bbc.co.uk/

<img src="http://thummer.domainname.com/300/300/1/http://news.bbc.co.uk/" alt="BBC News website thumbnail" />

And here is the result:

BBC News website thumbnail

Remember you can access the admin interface by going to http://thummer.domainname.com/admin where you can delete snapshots - so that they are regenerated next time they are requested.


Website Thumbnail Generator Web App - thummer

Posted on Sat 10 January 2009 in thummer • Tagged with cutycapt, django, thummer, ubuntu, xvfb

BBC News Website Thumbnail

Over Christmas I began learning django and python. I always find it better to learn new languages when you are applying what you read to a project that you are actually interested in, and think might come in useful. Well, after finding this great article on website thumbnail generation using cutycapt and django I decided to make a small web service / web app which would generate and serve website thumbnail screenshots of varying sizes.

I've always had difficulty with the "free" web thumbnail services out there - they usually limit the number of requests and/or thumbnail sizes as well as using watermarks or requiring link backs. And this is understandable - running this kind of service en-mass is resource hungry, and those servers aren't going to pay for themselves.

The idea of thummer is to allow people to set up their own thumbnail service on their own server, allowing them to generate snapshots of websites at any size with no restrictions.

Unfortunately because of some dependencies (CutyCapt requires qt4 and an X server) setting it up isn't quite as easy as I'd hoped, but isn't very difficult (lots of steps - but not tricky).

I'll post some instructions in the near future for Ubuntu 8.04 LTS Server.

I have now posted the (rather lengthy) installation instructions for Ubuntu 8.04 LTS.

I have set up a project page on Launchpad for thummer, so any bugs and questions can be tracked. The current release can be downloaded from Launchpad here.

A demo I set up is available here (but may disappear in the future if it starts getting hammered!).

This is my first django/python project, so any feedback is more than welcome!