FreeToastHost recently had a very difficult month or so.

The sites seemed at times to be down more than they were up. One item would get fixed, we'd hope that it was going to be plain sailing from then on, and then there would be another episode where sites were either terribly slow or impossible to access at all.

That made it very difficult for clubs to add new members, create and distribute agendas, and update their home pages. To those of you who were affected by all this, thank you for being patient and sticking with us.

What was happening?

In a nutshell, there was far more traffic than any ordinary webserver knows how to handle. Even larger platforms can struggle at times, and FTH is definitely not one of them.

Despite being a smaller system, FTH is able to run about 11,000 club and district sites. This is an impressive use of resources, and for the most part it has worked well.

The trouble is that there are nearly 11,000 separate club and district sites and they need to be inspected (or indexed) separately by the search engines. This is so that you get useful results when you use Google or others to look for something. In recent times, AI bots have joined the mix. Ever wondered where the AI systems get all their info from? They go around the internet, reading what everyone else has done.

This might not be a problem if they all behaved themselves. But they don't. Instead of taking a quick look a couple of times a month (or even a week), they hammer away multiple times a day.

You can have 1.5 million hits a day (spread over the 11,000 sites, but still the one server), which makes it totally unsurprising that ordinary Toastmasters are struggling to get anything done.

This would be bad enough on its own. But there are others who cause us grief. They are commonly known as "script kiddies", and they are the shallow end, if you like, of the online hackers' pool. They get their name because they get hold of scripts (sets of computer instructions) from various sources and then set them up to attack other sites. They are looking for weaknesses in software programs, mostly PHP applications (which FTH doesn't use, by the way).

They can be extraordinarily persistent. They've been seen to launch over 30,000 attacks against a single small home site. Where a normal human might give up, these automated items just keep going. More sophisticated are the DDoS (distributed denial of service) attacks, which are huge floods of mostly junk packets designed to overload a server. The reason why they pick on particular targets is often unknown.

Was it just us having problems?

No. Other sites much bigger than us have had problems, though for other reasons as well. Amazon AWS, a large and very popular provider of hosting for smaller sites, was down for some hours on 20 October and caused major disruption. (FTH uses Amazon AWS.) Even Cloudflare, regarded by some as the last word in protection from unwanted traffic, is not immune. It was out for a short time on 19 November and, again, caused major havoc.

The internet is definitely more unfriendly than it used to be. Some smaller sites have even given it up as not worthwhile and simply closed down.

FTH doesn't intend to. We want to keep on providing free club sites for Toastmasters, and this is what we've been doing.

What we've been doing about it

This was fairly complex and initially we didn't know how much would be needed. We ended up doing a number of things to get sites to behave.

The first item was the site visitor counter. We realised that this was slower than it needed to be. This didn't matter much if only a few people were visiting, but if there were a lot of them, it would slow down the sites a lot. Some database and disk storage tuning was done at the same time.

This helped, but it wasn't the whole story.

We needed to keep out bad guys, of which there are many; plus good, or at least OK, guys doing bad things that placed a terrific load on the server.

In desperation, most of them were blocked altogether; even Google's search engine was. It's not the best approach, because then searching for a site isn't going to find up-to-date information, but we considered it was preferable to not being able to use the site at all.

Finally, a new, shiny script was developed to block various bad networks, people trying to find stuff they don't need and so on. This meant that some search engine spiders were allowed access again, though they need to be kept to reasonable limits.

Most importantly, regular reports are being run so that we can see what's working and what needs to be changed.

it's been a good deal of work, and some frustration along the way, but it seems that things are working. The server load has been reduced by 298%+.

For now it is looking remarkably better, and will continue to improve.