On webmaster tools I was presented with the scanning error on wp-content/themes/Newspaper/ Googlebot could not access the contents of the URL because the server encountered an internal error while trying to process the request. These errors are generally due to the server itself and not to the request. how can I fix it? Thank you
Hi,
I found this useful guide about what to do in this case
– https://www.rebelytics.com/crawl-errors-google-search-console/
I believe you should disallow access to the wp-content folder either through htacess or robots, it seems that is the general recommendation. Some useful resources here
– https://productforums.google.com/forum/#!msg/webmasters/SK2BsweaF-0/UuZrsK3Fq00J
– https://yoast.com/ultimate-guide-robots-txt/
Thanks
