~~~ Please read before posting. ~~~
Important: We need your Club Number at a minimum, and as many details as possible.
For further info please read
This page
before posting.
Adding custom pages which can be found by Google
- chrissimmonsFUTC
-
Topic Author
- Offline
- New Member
-
- Posts: 12
- Thanks: 1
Adding custom pages which can be found by Google
3 years 10 months ago
Hi
What's the best way to add a Custom Web page, so that Google will index it?
I ask this because it seems to me, the method I have used (which I think is pretty standard), prevents the page from being found.
Months ago, I added a Custom Web page to our FreeToastHost site ( flindersuni.toastmastersclubs.org ), called "Club News". It's public, has keyworks, is in the site menu and frequently updated. It's also linked to from other pages. But Google never indexes it, despite repeated requests.
I think the reason is, that all pages have a robots meta tag which explicitly asks search engines to ignore all links:<meta name="robots" content="index,nofollow,noimageindex" />
I can imagine good reasons for doing this for some links (e.g. the generic links to TMI content, which are already well-known).
But for custom pages, this gets in the way of promoting the club website content.
Is there a way of making sure that custom pages don't get caught by this meta tag?
Or is this "by design" for some reason?
developers.google.com/search/docs/advanc...alify-outbound-links
thanks
Chris Simmons
What's the best way to add a Custom Web page, so that Google will index it?
I ask this because it seems to me, the method I have used (which I think is pretty standard), prevents the page from being found.
Months ago, I added a Custom Web page to our FreeToastHost site ( flindersuni.toastmastersclubs.org ), called "Club News". It's public, has keyworks, is in the site menu and frequently updated. It's also linked to from other pages. But Google never indexes it, despite repeated requests.
I think the reason is, that all pages have a robots meta tag which explicitly asks search engines to ignore all links:<meta name="robots" content="index,nofollow,noimageindex" />
I can imagine good reasons for doing this for some links (e.g. the generic links to TMI content, which are already well-known).
But for custom pages, this gets in the way of promoting the club website content.
Is there a way of making sure that custom pages don't get caught by this meta tag?
Or is this "by design" for some reason?
developers.google.com/search/docs/advanc...alify-outbound-links
thanks
Chris Simmons
The topic has been locked.
- SteveTheTechie
-
- Offline
- Administrator
-
- Posts: 11526
- Thanks: 3050
Re: Adding custom pages which can be found by Google
3 years 10 months ago - 3 years 10 months ago
Well, it is true that given we only have one donated server for the entire system to handle all 12,000+ websites, we do place a number of restrictions on what we will allow bots to do and not do, in order to prevent them from using up too much server processing power. However, it is a bit of a subjective balancing act on our part regarding on what restrictions we put in place--we have access to overall performance data for the system, so we are certainly better positioned than individual users or club admins. (That is part of our role in maintaining the system to keep it viable and performant.)
Let me take a look into why I had put those specifics in place for bots with that meta tag. Keep in mind though, that my conclusion may well be that we are not going to change it. Because of the nature of how resource constrained the FTH system is, there are just some things we will not support.
Let me take a look into why I had put those specifics in place for bots with that meta tag. Keep in mind though, that my conclusion may well be that we are not going to change it. Because of the nature of how resource constrained the FTH system is, there are just some things we will not support.
Last edit: 3 years 10 months ago by SteveTheTechie.
The following user(s) said Thank You: chrissimmonsFUTC
The topic has been locked.
- chrissimmonsFUTC
-
Topic Author
- Offline
- New Member
-
- Posts: 12
- Thanks: 1
Re: Adding custom pages which can be found by Google
3 years 10 months ago
The team must be doing some real magic, to be able to support all this on one server!
The need to keep consistency must be very high. And keeping the right crawlers crawling no more than necessary takes work.
(I can relate to the situation: at the University where I was in charge of the main website, we were running for years on just two servers. It took quite a bit of careful tuning and monitoring to keep it going. And everyone wanted it to run *their way*!).
The need to keep consistency must be very high. And keeping the right crawlers crawling no more than necessary takes work.
(I can relate to the situation: at the University where I was in charge of the main website, we were running for years on just two servers. It took quite a bit of careful tuning and monitoring to keep it going. And everyone wanted it to run *their way*!).
The topic has been locked.
Moderators: Brian, Pam, rhtaylor3, marc33, NotLiable, jgavin, Lcala305, Jane Atkinson, peterb323
Time to create page: 0.380 seconds
Copyright © 2025 FreeToastHost 3 Support. All Rights Reserved.