C# download web page

December 10, 2009 in .net, Agile, C#, Linux, Networking, code, internet by orn | No comments

I’m sure you had the need sometime to download the content of a web page to be able to analyze it or work with the content. Here is a code snippet just for you, we will use simple http GET for the specified URL.

public static string getWebPage(string psUrl)
{
    WebResponse result = null;
    string sRet = string.Empty;

    try
    {
        WebRequest req = WebRequest.Create(psUrl);
        req.Method = “GET”;
        req.Timeout = 3 * 1000;  // 3 secs
        // Explicit no caching, usually this is the default
        req.CachePolicy = new RequestCachePolicy(RequestCacheLevel.BypassCache);

        // Has to process the results if the responding service is spitting it out
        result = req.GetResponse();
        Stream ReceiveStream = result.GetResponseStream();
        Encoding encode = System.Text.Encoding.GetEncoding(“utf-8″);
        StreamReader sr = new StreamReader(ReceiveStream, encode);
        // in case the caller is interested
        sRet = sr.ReadToEnd();
    }
    finally
    {
        if (result != null) result.Close();
    }

    return sRet;
}

Tags: , , , ,