~~~ Please read before posting. ~~~

Important: We need your Club Number at a minimum, and as many details as possible.
For further info please read This page before posting.

Robots.txt Unreachable – Club #38 Website Indexing Issue

  • Club38
  • Club38's Avatar Topic Author
  • Offline
  • New Member
  • New Member
  • Posts: 5
  • Thanks: 0

Robots.txt Unreachable – Club #38 Website Indexing Issue

2 months 1 week ago
#95797
 

Please Log in or Create an account to join the conversation.

  • Club38
  • Club38's Avatar Topic Author
  • Offline
  • New Member
  • New Member
  • Posts: 5
  • Thanks: 0

Re: Google cannot index our club site (robots.txt unreachable)

2 months 1 week ago
#95798
Hello,Our club site 38.toastmastersclubs.org/ is not being indexed properly by Google. In Search Console, when we request indexing, it shows an error saying robots.txt is unreachable.The site loads fine in a browser, and I can open robots.txt manually, but Googlebot seems unable to reach it.Could you please check if:
  • robots.txt is consistently reachable by Googlebot,
  • there’s any hosting-level block or crawl-delay that may cause timeouts,
  • or if a misconfiguration is preventing indexing.
Other Toastmasters club sites appear to work, so I want to confirm if this is specific to ours.Thank you,
Lisa
First Canadian Toastmasters Club #38

Please Log in or Create an account to join the conversation.

  • Brian
  • Brian's Avatar
  • Offline
  • Administrator
  • Administrator
  • Posts: 10775
  • Thanks: 3614

Re: Robots.txt Unreachable – Club #38 Website Indexing Issue

2 months 1 week ago
#95799
You will have to give us more information. Which of the thousands of search bots is giving this message?
What were you doing when you got this message?

Please Log in or Create an account to join the conversation.

  • Club38
  • Club38's Avatar Topic Author
  • Offline
  • New Member
  • New Member
  • Posts: 5
  • Thanks: 0

Re: Robots.txt Unreachable – Club #38 Website Indexing Issue

2 months 1 week ago
#95808
Here's my error from Google search console
 

Please Log in or Create an account to join the conversation.

  • Club38
  • Club38's Avatar Topic Author
  • Offline
  • New Member
  • New Member
  • Posts: 5
  • Thanks: 0

Re: Robots.txt Unreachable – Club #38 Website Indexing Issue

2 months 1 week ago
#95815
Hi,

Google Search Console is showing a site-wide indexing issue for our club site:
38.toastmastersclubs.org Googlebot cannot access the robots.txt file, and as a result, the entire site is blocked from indexing.

Here's what I’ve confirmed:
  • The site loads fine for users
  • Google reports: “Page fetch failed: robots.txt unreachable”
  • The sitemap ( 38.toastmastersclubs.org/sitemap.xml ) redirects to the homepage
  • Manual indexing fails due to this robots.txt error
We’re no longer showing up in Google searches, which impacts visibility for potential members.Can you confirm if this is a known issue and whether a fix is in progress?Thanks,
Lisa

Please Log in or Create an account to join the conversation.

  • Brian
  • Brian's Avatar
  • Offline
  • Administrator
  • Administrator
  • Posts: 10775
  • Thanks: 3614

Re: Robots.txt Unreachable – Club #38 Website Indexing Issue

2 months 1 week ago
#95824
Please try this again. Several google IP were banned. Those bans have been removed.
The following user(s) said Thank You: Club38

Please Log in or Create an account to join the conversation.

Moderators: BrianHeniPamrhtaylor3marc33NotLiableNSBjgavinLcala305peterb323DebbieT
Time to create page: 0.183 seconds