Google Indexing Errors - Major Issue Sitewide

Hey Steve,

Sorry for letting my frustration shine through. My arguments do hold water. If the site was just saddled with low search engine rankings, that would be one thing. But the site is ending up in the Omitted Results. Let me shed some light on my process...

• I was asked by Windmasters to assist in getting their site into the rankings, as it did not show up anywhere, and they had heard of my success with Warehouse Toasmasters
• We went through most all of the things that one should do to rank high in the search engines, including all of what has been suggested here. I won't go into detail of all those things right now.
• I added the Windmaster's site to my Google Webmaster Tools Console (Now called Google Search Tools) and submitted the pages for crawling. I did the same with my Bing/Yahoo Webmaster Tools. Everyone administering a website should become familiar with these tools if they aren't already.
• We waited for a couple of weeks to let things settle in. The new site showed up under some search criteria (Site Name), but was nowhere to be found under the critical search criteria - Alameda Toastmasters (City Name). One of the members actually went through the 17 pages of results and clicked on the Omitted Results link at the bottom. Voila, there was the missing Search Engine Listing. But why was it omitted? Google started omitting pages when Spammers figured out how to create thousands of meaningless pages to flood the search engines.
• A review of possible culprits shortlisted Content Blocked From The Crawler. This led me to look further into my Google Search Tools - and there was the glaring error. Think about it. The biggest search engine just told us that they could not index our pages properly. That all they see is the basic header info and a statement to enable Java Script - if we have a lot of these pages out there, they look like spam to the search engine. Additionally, Google explicitly listed the inability to completely crawl a site as reason for omitting the site from the results.
• But why do some sites not have an issue. To understand this, we would have to know a few things. Was the Robot.txt file changed anytime in the past year or so? (perhaps it was changed to reduce the stress on the servers from search engine crawlers) - Is our site compliant with Google's ever changing algorithms? (the tools tell us that we are not) - What are the ages of the individual Toastmaster sites. (Sites that have been around a long time will have been in the search engine indexes using Googles older algorithm, and are more likely to have plenty of backlinks from other websites.)
• The Google Search Tools tell me we have an issue with not just one site, but all of our sites. I would love someone to tell my why I am wrong. Someone to tell me why we should ignore the results of the tools created by Google, the world largest search engine. I am not asking for anything to be changed to "my specification" or jump through hoops - certainly if it is not for the good of all involved. I have about 30 years experience as an IT professional. I do my research and try not to jump to conclusions. I don't think that I am crying wolf...

- - Worth A Quick Read - -
Here is the link from Google explaining the situation affecting all of us:
support.google.com/webmasters/answer/6153277

All the public email addresses in your windmasters home page are an issue. My SEO tools confirm this. The warehouse website does not have public email addresses on its home page. It is a key difference, and my SEO tools like the warehouse home page better than the windmasters home page.

The lack of alt text on your home page images should be addressed.

You should create/submit a site map,

We have over 10,000 clubs using the same system running the same code on the same server. If we were having a system wide issue as you seem to indicate, we would be having a lot more people complaining than just you.

Lastly, while we have periodically updated our robots.txt file periodically to stay ahead of badly behaved bots that drag down system performance, robots.txt indicates that we are not currently blocking the js or css folders.

Cool. It looks like someone agreed that it was a system wide issue. The Robots.txt has been revised. I will have Google re-crawl the site and see if it resolves the issue.

The updated Robots.txt entry for Google:


How it now renders for the GoogleBot (note that the images directory is still blocked):

SteveTheTechie wrote: All the public email addresses in your windmasters home page are an issue.

Steve,

Thanks for pointing this out. As a practice, I never put an email address on a webpage - that is why formmail was created. I volunteered to assist getting their site into Google hopefully get a decent ranking. I was not aware that email addresses were a big hit in the algorithm. So we definitely need to explore other options for them.

The reason they felt the need to put the email addresses there is because the meetings are held in a secured building. Any guest will need to contact them in advance to arrange to get proper security authorization (vs Warehouse TM, where we never know who might pop in).

So my question, as a sub-topic - perhaps better as a new topic: Can we link to some contact form within the text of our Home Page?
Something like: "Visit Us - But Please Contact Us In Advance By Clicking HERE"

Thanks
Brian

Per the notes on the Contact Us Form tab of Email and Contact Forms module:

NOTE: To link directly to your "Contact Us" form, use http://brian.toastmastersclubs.org?contactus

I recommend just configuring the Contact Us Form as needed.