Choose to download a file depending on its modification date

This program is an example of how to check the date that a file was last modified using a HEAD request to a web server to get the modification date.

#!/home/ben/software/install/bin/perl
use warnings;
use strict;
use autodie;
use LWP::UserAgent;

# Download the "radkfile" and "kradfile" files, if they are newer than
# the versions we currently possess.

my $base_url = 'ftp://ftp.monash.edu.au/pub/nihongo';
my $agent = LWP::UserAgent->new ();

download ('radkfile');
download ('kradfile');
download ('kanjidic');

sub download
{
    my ($file) = @_;
    my $url = "$base_url/$file.gz";
    if (-f $file) {
        # There is a local file, so first compare the dates of the remote
        # file and the local file, and only download the remote file if it
        # is newer.
        print "Local file '$file' exists.\n";
        my $local_date = mdate ($file);
        print "Local date: $local_date.\n";

        my $response = $agent->head($url);
        # Check for errors
        my $remote_date = $response->last_modified;
        print "Remote date: $remote_date.\n";
        if ($local_date < $remote_date) {
            print "Remote file is newer, downloading.\n";
            $response = $agent->get ($url, ":content_file" => "$file.gz");
            # Check for errors
            gunzip ($file);
        }
        else {
            print "Remote file is older, not downloading.\n";
        }
    } else {
        # There is no local file, so just download it.
        print "Local file '$file' does not exist.\n";
        my $response = $agent->get ($url, ":content_file" => "$file.gz");
        # Check for errors
        gunzip ($file);
    }
}

# Decompress a .gz file.

sub gunzip
{
    my ($file) = @_;
    if (-f $file) {
        unlink $file;
    }
    system ("gzip -d $file.gz") == 0 
        or die "gzip failed.\n";
}

# Given a file name, return its modification date.

sub mdate
{
    my ($filename) = @_;
    if (!-e $filename) {
        die "reference file '$filename' not found";
    }
    my @stat = stat ($filename);
    if (@stat == 0) {
        die "'stat' failed for '$filename': $@";
    }
    return $stat[9];
}

When you run this, it produces output something like the following:

Local file 'radkfile' exists.
Local date: 1272339648.
Remote date: 1278932808.
Remote file is newer, downloading.
Local file 'kradfile' exists.
Local date: 1298512093.
Remote date: 1278932808.
Remote file is older, not downloading.

The above numbers are dates in Unix epoch form.


Copyright © Ben Bullock 2009-2012. All rights reserved. For comments, questions, and corrections, please email Ben Bullock (ben.bullock@lemoda.net) / Privacy / Disclaimer