HttpClient GetStringAsync для веб-сайта Best Buy ⇐ C#

[Гость]

I'm working on a web scraping project, and I'm just trying to retrieve the full HTML of Best Buy products, but I'm not receiving any HTML response. Here's the code.

var url = "https://www.bestbuy.com/site/computer-c ... acture~AMD"; var httpClient = new HttpClient(); var html = httpClient.GetStringAsync(url); Console.WriteLine(html.Result); So, this is all I'm doing, and Best Buy is the only website that times out and doesn't return HTML. I've tried this getting it for Amazon, eBay, and just other random websites, and I'm able to receive their HTML. Why is Best Buy timing out?

Could it be something with their robots.txt file? Do I need a specific User Agent? I'm not sure.

Any help is appreciated. Thanks!


Источник: https://stackoverflow.com/questions/668 ... ys-website