We ve now investigated remote operating classes that encapsulate communication protocols and work like local classes; now we ll have a short look at some Web access classes of the .NET Framework. Three classes are of particular interest: System.Net.WebRequest, System.Net.WebResponse, and System.Net.WebProxy.

As mentioned in the introductory sections, the abstract classes WebRequest and WebResponse are general APIs to underlying protocol handlers like an HTTP handler. Your goal is to develop a small application that makes a request via an HTTP proxy to a Web server and receives the response. You could use such an application as a base for a Web browser or a crawler, for instance.

A crawler is an application that walks through the Web along the links in the HTML documents to track the structure behind. Crawlers are used by search engines to create a searchable database of documents. A search request to a search engine means that a query to the database of the engine is made. A crawler can also be useful for intranets to determine the structure, for example, for creating index pages.

General Usage of Needed .NET Classes

Today many companies use proxies to channelize and control outgoing requests from the company to the Web (see Figure 5.108).

Figure 5.108: Clients Access the Internet via Proxy

So, you can first define the parameters for the proxy to give them to the requesting class. This class then makes the request and receives the results page:

// create...