AdSense, WordPress sites and the dreaded "crawler errors"

sumi

Rest in Peace 1980-2020
Premium Feather Member
8 Years
Jun 28, 2011
39,162
27,209
1,302
Just leaving this here for those, who, like me, are seriously NOT geek-minded and completely bewildered by messages on your AdSense account status saying things like this:

"Our crawler was unable to access the pages listed in the ’Blocked URLs’ column to determine the content and display ads. Click the ’plus’ icon beside each domain to see an expanded list of affected URLs. When our crawler can’t access your content we won’t show ads, resulting in lower revenue and coverage. Follow the link in the ’Error’ column to learn how to correct these errors."

scratching_head-gif.1055365


The "Error column" was very helpful, all it had to say was "No Data"

Google "Help" didn't help much either, telling me to: Give access to our crawler in your robots.txt file

scratching_head-gif.1055365


I didn't remove their access. I don't even know WHERE to commit this mistake.

About 30 minutes of frantic searches, reading, cussing, back and forth between 4 tabs, Google searches, troubleshooting and reading even more bewildering "Help" pages I thought of something….

WordPress sites have an annoying habit of "losing" ad codes…
ll-1-gif.555237


O.K. check site theme… No ad code! :rolleyes: That might explain everything?!?

Solution:

1. Find the ad code. 2. Go into your site's themes. 3. Paste the code just below the first <head> and… Success! The ads are showing up again. Google is "crawling" my site again and me?

oregonian_winesmiley-gif.556998


So, if you have a WordPress site and see no ads showing where you previously had ads and Google gives you the above verdict. Don't panic, before you go searching high and low for errors and blocked URL's and txt files and did I mention panic? Check if your ad codes are where you last left them!

I really hope this helps someone. I did not find this solution in the troubleshooting guides and pages I did look at, so I thought I'd leave it here for anyone that needs it :frow
 
Wow, look at you being all techie and such!!!

:bow

Great job figuring it out!

:thumbsup


So, a big red-flag is that the ad code goes "missing". That should definitely not be happening!

The only thing I can think is if you're updating the style/theme of the site, or the WP site itself, and this is somehow overwriting your edits with the default ones?
 
The only thing I can think is if you're updating the style/theme of the site, or the WP site itself, and this is somehow overwriting your edits with the default ones?
Rob, now you're giving credit where it's not due lol I have no idea how to do that! :lol: Seriously though, when I did a Google troubleshoot a few weeks ago when my ads went AWOL, one of the items on the check list was "Is it a WordPress site?" or similar, so they are known to do that apparently. Now that I think about it though… WP now and then sends me an email telling me THEY updated something-or-another on there, so I'll make a note to ad check every time they do that and see if they caused the ad code to disappear.
 
I've had updates (to the core of the software or to the themes / styles) overwrite customization.

On BYC I had that happen with smilies. We'd update the site and it replaced our custom smileys with the default ones. I ended up having to change the storage path for the files so they were separate from the default smileys.
 
That's interesting and good to know! I thought once I've put something somewhere it would stay there. Curiously no-one that I can recall mentioned the codes going missing issue when I signed up for ads and watched tutorials on the topic :hmm And slightly frustratingly, there are no bells and whistles on your AdSense account page when you check in saying "Hey, thanks for leaving?!?"
 

New posts New threads Active threads

Back
Top Bottom