Netlify HTTP 5xx error reports from search engines

I have a couple of sites hosted on Netlify, and during last few months I started getting HTTP 5xx error reports (mainly 502 Bad Gateway) and slow response reports from search engine bots.

I was able to reproduce the issue locally with CURL when setting User-Agent to Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots).

Again, the problem appeared during last few months, previously I never seen this issue on Netlify.

I contacted Netlify support team (luckily they reply really fast) and they suggested me to disable Prerendering feature in deploy settings.

After disabling Prerendering in deploy settings, the issue dissapeared. I believe it was enabled automatically for all sites for some reason, however the feature is still in Beta. I turned it off for all my sites, since they are prerendered at build time in my case.

Now no more annoying HTTP 5xx error reports in Google Search Console and Yandex Webmaster Tools.