Do you know that generally these bot safety providers will serve Google and different serps noindex directives? Google’s John Mueller mentioned this on X, saying, “Generally there’s bot-protection (or a login, interstitial, and many others) triggering that has a noindex on it.”
He additionally added that it will be higher for them to serve 503 server standing codes. “Higher could be to make use of 503 for server-side blocks like bot-protection,” he wrote.
Ultimately, the individual he replied to was having a distinct difficulty inflicting the noindex, it was not bot safety. He mentioned it was “some shenanigans with NextJS doing consumer aspect rendering as an alternative of server rendering.” “The HTML had a noindex tag, which was eliminated after rendering on the browser. The HTML proven on GSC is the one AFTER JavaScript rendering, which is why I couldn’t discover it and thought it was a GSC bug,” he added.
This was the error he was seeing:
Listed here are these posts:
wtf is happening right here? why is Google not indexing this one?
They are saying there is a “noindex” tag, however there is not one. robots.txt seems good too.
Anybody? #search engine optimization assist pic.twitter.com/DYzJsLQ4Yk
— Guilherme (@goenning) December 26, 2023
Thanks John, finally discovered it due to @HamilcDev
Some shenanigans with NextJS doing consumer aspect rendering as an alternative of server rendering
The HTML had a noindex tag, which was eliminated after rendering on the browser. The HTML proven on GSC is the one AFTER JavaScript rendering,… https://t.co/NffU7suVLM
— Guilherme (@goenning) December 26, 2023
Discussion board dialogue at X.