How to read and download a webpage in Java

Problem Description

How to read and download a webpage?

Solution

Following example shows how to read and download a webpage URL() constructer of net.URL class.

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.InputStreamReader;
import java.net.URL;

public class Main {
   public static void main(String[] args) throws Exception {
      URL url = new URL("http://www.google.com");
      BufferedReader reader = new BufferedReader(new InputStreamReader(url.openStream()));
      BufferedWriter writer = new BufferedWriter(new FileWriter("data.html"));
      String line;
      
      while ((line = reader.readLine()) != null) {
         System.out.println(line);
         writer.write(line);
         writer.newLine();
      }
      reader.close();
      writer.close();
   }
}

Result

The above code sample will produce the following result.

Welcome to Java Tutorial
	Here we have plenty of examples for you!
	
   Come and Explore Java!

The following is an another example to read and download a webpage.

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;

import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;

public class NewClass {
   public static void main(String[] args) {
      URL url;
      InputStream is = null;
      BufferedReader br;
      String line;
      try {
         url = new URL("https://www.tutorialspoint.com/javaexamples/net_singleuser.htm");
         is = url.openStream();  // throws an IOException
         br = new BufferedReader(new InputStreamReader(is));
         
         while ((line = br.readLine()) != null) {
            System.out.println(line);
         } 
      } catch (MalformedURLException mue) {
         mue.printStackTrace();
      } catch (IOException ioe) {
         ioe.printStackTrace();
      } 
      finally {
         try {
            if (is != null) is.close();
         } catch (IOException ioe) {} 
      } 
   }
}

Result

The above code sample will produce the following result.

<!DOCTYPE html>
<!--[if IE 8]><html class="ie ie8"> <![endif]-->
<!--[if IE 9]><html class="ie ie9"> <![endif]-->
<!--[if gt IE 9]><!-->	<html> <!--<![endif]-->
<head>
<!-- Basic -->
<meta charset="utf-8">
<title>Java Examples - Socket to a single client</title>
<meta name="description" content="Java Examples Socket to a single client : A beginner's tutorial containing complete knowledge of Java Syntax Object Oriented Language, Methods, Overriding, Inheritance, Polymorphism, Interfaces, Packages, Collections, Networking, Multithreading, Generics, Multimedia, Serialization, GUI." />
<meta name="keywords" content="Java, Tutorials, Learning, Beginners, Basics, Object Oriented Language, Methods, Overriding, Inheritance, Polymorphism, Interfaces, Packages, Collections, Networking, Multithreading, Generics, Multimedia, Serialization, GUI." />
<base href="https://www.tutorialspoint.com/" />
<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon" />
<meta name="viewport" content="width=device-width,initial-scale=1.0,user-scalable=yes">
<meta property="og:locale" content="en_US" />
<meta property="og:type" content="website" />
<meta property="fb:app_id" content="471319149685276" />
<meta property="og:site_name" content="www.tutorialspoint.com" />
<meta name="robots" content="index, follow"/>
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black">
<meta name="author" content="tutorialspoint.com">
<script type="text/javascript" src="/theme/js/script-min-v4.js"></script>

java_networking.htm