Choose to download a file depending on its modification date

This program is an example of how to check the date that a file was last modified using a HEAD request to a web server to get the modification date.

#!/home/ben/software/install/bin/perl
use warnings;
use strict;
use LWP::UserAgent;
use FindBin;
use File::Compare;
use Getopt::Long;
use MyDownload 'mydownload';

GetOptions (
    "force" => \my $force,
    "help" => \my $help,
);

if ($help) {
    usage ();
    exit ();
}

# Download the "radkfile" and "kradfile" files, if they are newer than
# the versions we currently possess.

my $base_url = 'ftp://ftp.edrdg.org/pub/Nihongo';
my $agent = LWP::UserAgent->new ();
my $tempdir = "$FindBin::Bin/download-temp";

download ('radkfile', $force);
download ('kradfile', $force);
download ('kanjidic', $force);
system ("chdir $FindBin::Bin;/usr/local/bin/git add .;/usr/local/bin/git commit -a -m \"data update\"");

sub download
{
    my ($file, $force) = @_;
    mydownload (
        agent => $agent,
        force => $force,
        url => "$base_url/$file.gz",
        file => "$file.gz",
        tempdir => $tempdir,
    );
    gunzip ("$file.gz");
}

# Decompress a .gz file.

sub gunzip
{
    my ($file) = @_;
    system ("gzip --keep -d -f $file") == 0 
        or die "gzip failed.\n";
}

# Given a file name, return its modification date.

sub mdate
{
    my ($filename) = @_;
    if (!-e $filename) {
        die "reference file '$filename' not found";
    }
    my @stat = stat ($filename);
    if (@stat == 0) {
        die "'stat' failed for '$filename': $@";
    }
    return $stat[9];
}

sub usage
{
    print <<EOF;
This script downloads EDRDG (Jim Breen)'s kradfile, kanjidic, and
radkfile automatically.

Options:
--force    Force a download, ignoring update times
--help     Print this message.
EOF
}

When you run this, it produces output something like the following:

Local file 'radkfile' exists.
Local date: 1272339648.
Remote date: 1278932808.
Remote file is newer, downloading.
Local file 'kradfile' exists.
Local date: 1298512093.
Remote date: 1278932808.
Remote file is older, not downloading.

The above numbers are dates in Unix epoch form.


Copyright © Ben Bullock 2009-2017. All rights reserved. For comments, questions, and corrections, please email Ben Bullock (benkasminbullock@gmail.com). / Privacy / Disclaimer