Important: when posting, please provide your Club Number at a minimum, and as many details as possible.
For further info, please read This page before posting.

Regarding the club websites

  • SteveTheTechie
  • SteveTheTechie's Avatar
  • Offline
  • Emeritus
  • Emeritus
  • Posts: 11492
  • Thanks: 3057

Re: Regarding the club websites

9 years 5 months ago
#58911
Ok, i took out:
Code:
User-agent: * Disallow: / Crawl-delay: 60

Thanks for the heads-up! B) Given that the last changed file date was 7/6/2016, I think it was changed around the officer changeover time as previously noted.

There is still room for some discussion about whether we should be disallowing js folders or not. I know that Google wants to index javascript... I am just a bit uncertain about it as far as potential system performance impact. It is a single server system running a MySQL database for 11,000+ clubs/districts. We do a number of things to keep it running smoothly, but it does not take that much to max it out.
The topic has been locked.
  • GeorgeMarshall
  • GeorgeMarshall's Avatar
  • FreeToastHost Ambassador
  • FreeToastHost Ambassador
  • Thanks: 0

Re: Regarding the club websites

9 years 5 months ago
#58912
Is there really any reason why bots should be allowed in index JS? Is there any club-specific public data there?
The topic has been locked.
  • SteveTheTechie
  • SteveTheTechie's Avatar
  • Offline
  • Emeritus
  • Emeritus
  • Posts: 11492
  • Thanks: 3057

Re: Regarding the club websites

9 years 5 months ago - 9 years 5 months ago
#58913
GeorgeMarshall wrote: Is there really any reason why bots should be allowed in index JS? Is there any club-specific public data there?

No.

The following info looks relevant and interesting though:
searchengineland.com/tested-googlebot-cr...heres-learned-220157

The key thing is that the website template is filled server side not via Javascript. Could do some template filling on AJAX calls in the future, but nothing now. (do a little of that in a few places now but not much)

However, there are some Javascript redirects, as mentioned in the article.

How much of this is important?... not sure.
Last edit: 9 years 5 months ago by SteveTheTechie.
The following user(s) said Thank You: GeorgeMarshall
The topic has been locked.
Moderators: BrianHeniGeorgeMarshallPamrhtaylor3marc33NotLiableNSBjgavinLcala305peterb323DebbieT
Time to create page: 0.140 seconds