I am implementing a webcrawler using httpwebrequest object and streamreader.readtoend method.
My code is running smoothly but I suffer with performance of streamreader.readtoend() method, this method takes several seconds to execute.
I have already tried solutions mentioned on below links but not succeded.
http://forums.asp.net/t/1762719.aspx/1?Why+is+StreamReader+ReadToEnd+so+slow+
http://stackoverflow.com/questions/901323/c-sharp-httpwebresponse-streamreader-very-slow
Could you please let me know why this method takes so much time to process? and is there any alternative to speed my code excution time?
Your help is highly appreciated.
Regards,
Mangesh