Time-Stamping Usage (GNU Wget 1.21.1-dirty Manual)
5.1 Time-Stamping Usage
The usage of time-stamping is simple. Say you would like to download a file so that it keeps its date of modification.
wget -S http://www.gnu.ai.mit.edu/
ls -l shows that the time stamp on the local file equals the state of the
Last-Modified header, as returned by the server. As you can see, the time-stamping info is preserved locally, even without ‘
-N’ (at least for HTTP).
Several days later, you would like Wget to check if the remote file has changed, and download it if it has.
wget -N http://www.gnu.ai.mit.edu/
Wget will ask the server for the last-modified date. If the local file has the same timestamp as the server, or a newer one, the remote file will not be re-fetched. However, if the remote file is more recent, Wget will proceed to fetch it.
The same goes for FTP. For example:
(The quotes around that URL are to prevent the shell from trying to interpret the ‘
After download, a local directory listing will show that the timestamps match those on the remote server. Reissuing the command with ‘
-N’ will make Wget re-fetch only the files that have been modified since the last download.
If you wished to mirror the GNU archive every week, you would use a command like the following, weekly:
wget --timestamping -r ftp://ftp.gnu.org/pub/gnu/
Note that time-stamping will only work for files for which the server gives a timestamp. For HTTP, this depends on getting a
Last-Modified header. For FTP, this depends on getting a directory listing with dates in a format that Wget can parse (see FTP Time-Stamping Internals).