Is it possible to configure HttpClient in C# to consistently retrieve website meta data without being blocked?
I’ve set up an HttpClient to retrieve meta data from websites to build a preview when a url is posted in a user message to my website. This works with links to most sites but a significant number do not return the data expected. For example https://www.opendemocracy.net/en/uk/ – both Facebook and Twitter retrieve the meta … Read more