linux poison RSS
linux poison Email

Download entire website using Wget for offline viewing on Linux

GNU Wget is a free utility for non-interactive download of files from the Web.  It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies.

Wget is non-interactive, meaning that it can work in the background, while the user is not logged on.  This allows you to start a retrieval and disconnect from the system, letting Wget finish the work.  By contrast, most of the Web browsers require constant user's presence, which can be a  great hindrance when transferring a lot of data.

Wget can follow links in HTML and XHTML pages and create local versions of remote web sites, fully recreating the directory structure of the original site.  This is sometimes referred to as "recursive downloading."  While doing that, Wget respects the Robot Exclusion Standard (robots.txt).  Wget can be instructed to convert the links in downloaded HTML files to the local files for offline viewing.

Wget has been designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved.  If the server supports regetting, it will instruct the server to continue the download from where it left off.

Most of the Linux distribution comes with Wget installed, so you don't have to do anything to install Wget

Using Wget to download entire website:
Create directory where you are planing to store the website content: mkdir /home/nikesh/linuxpoison
use following command to download the website:
wget -r -Nc -mk http://linuxpoison.blogspot.com/
-r  Turn on recursive retrieving
-N  Turn on time-stamping
-m  Create a mirror
-k  Convert the link

After completion all content will get downloaded into your directory for  offline viewing.


3 comments:

Sathishkumar said...

Thanks ! Its helped me

Martial Tchuente said...

Great!!!
Thanks for sharing this.
I tried it and I must confess that wget is really a very powerful command, and its beauty resides in its simplicity.
Have a nice day guys...
From a Linux newbie.

Rajdeep said...

-N is for time stamping. what is the 'c' after -N for?

Post a Comment

Related Posts with Thumbnails