If you run a WordPress site, no doubt you’ve dealt with bots. They’re seemingly everywhere in the online world, and not every one of them is bad. But when it’s time to stop bots in WordPress, some approaches work better than others.
Bad bots are more than a nuisance. They can disrupt the functionality of your WordPress site, slow down workflow, and drive away users. Fortunately, when it’s time to block bots, WordPress gives us several practical solutions.
In this guide, we’ll discuss what bots are (the good and the bad), how to block the bad ones, and how to keep them from unwittingly crawling your WordPress site. To block bots vulnerable WordPress sites invite in, take a few minutes now to read this guide. By the end, you’ll have the solutions you need to block bots in WordPress. Let’s take a look.
What Is a Bot?
As you probably already figured out, the term bot is short for “robot.” Sometimes, people refer to the bots we’re discussing today as internet bots.
In a nutshell, a bot is a constructed computer program that operates as an independent agent for a person or larger computer program. Often, bots are used to simulate the activity of people and may do so quite elaborately.
Typically, a bot is employed to automate tasks. This means that a bot will continue to run without any need for continued aid or instruction from a person.
WordPress site owners, as well as other individuals and organizations, use bots to take the place of repetitive tasks that a person would need to perform without the aid of a bot. And the truth is, good bots are quite a bit faster than people at performing these types of often-mundane tasks (I’m not a bot and I did write this article, I promise!).
How Exactly Do Bots Work?
In most instances, a bot operates over a network. When bots are made to communicate with one another, they’ll use different services to do so, such as IRC (Internet Relay Chat), direct messaging, or other interfaces such as Twitterbots.
Generally speaking, over half of all internet traffic is actually bots that are interacting with various web pages, talking directly with users, scanning for specific content, or performing other mundane tasks.
A bot is “constructed” from different algorithm sets that aid them in performing the tasks that are designated to do. Bots can handle such tasks as talking with people (the most sophisticated ones attempt to mimic true human behavior) and gathering website content from around the internet.
But the fact is that there are many different bot types, some good and some bad, that are designed in different ways, to accomplish a huge array of different agendas.
One common example of a bot you’ve probably interacted with are chatbots. These bots operate using one of several different methods.
Chatbots that are rule-based interact with human users by providing prompts that are pre-defined for a user to choose from. Chatbots that are intellectually independent make use of machine learning to learn and understand human input and respond to known keywords.
AI (artificial intelligence) chatbots combine the characteristics of intellectually independent bots and rule-based bots. These sophisticated bots use natural language processing, pattern matching and natural language generation tools to replicate human interaction in very realistic ways.
People and organizations that use bots will typically utilize bot management software that includes tools which ain in protecting from malicious bots while managing the good ones.
Typically, these bot manages are included within a web app security platform. Bot managers are used to allow good bots to properly function while blocking bad bots that could do harm to software systems.
The bot manager then takes suspect or bad traffic from bots and directs it away from a website. Some of the more basic features of bot management include CAPTCHAs and IP rate limiting, which limits the number of requests that come from an identical address.
8 Common Types of Bots
There are many different bot types, each of which have their own unique tasks and agendas.
Some of the most common bots include:
- Chatbots – as discussed, these are bots that simulate online conversation and interact as a person does. Did you know that prior to the internet, one of the first chatbots was named Eliza? This was a program that acted like a psychotherapist and answered questions with additional questions.
- Shopbots – this is a program that scans the internet on a user’s behalf. Its job is to locate the lowest cost for any product, item or service that a user is looking for. Bots such as OpenSesame observe user website navigation patterns and customize the site for each individual user.
- Social bots – these bots operate on Facebook, Twitter, and other social media platforms.
- Knowbots – a knowbot is a program that works to collect user knowledge by visiting websites to retrieve specific information that meets criteria that’s been pre-determined by the knowbot programmer.
- Crawlers and spiders (sometimes referred to as web crawlers) – spiders and crawlers are used to access websites with the purpose of gathering their content for search engine indexes.
- Web scraping crawlers – these are similar to other crawlers. However, they’re used to harvest data and extract other content that’s relevant.
- Transactional bots – they’re used for completing transactions on behalf of their human programmer.
- Monitoring bots – as the name implies, monitoring bots monitor the overall health of a computer system or website.
Remember, bots are also classified into bad bots and good bots. In other words, there are some bots that will not do any harm to your system or your WordPress site. However, there are others that pose real threats and could do substantial harm to your WordPress site if they’re not blocked.
That’s when it becomes critical to learn how to stop bot traffic WordPress can’t stop on its own.
Bot Examples and Uses
Good bots are used in the field of customer service, as well as entertainment, search functionality and scheduling. The use of bots in these areas brings different benefits and advantages.
As an example, in the customer service field, a bot can be available 24 hours per day, seven days per week to answer common questions and give basic assistance. This helps free up customer service staff so they can focus on more complex issues that require human interaction.
These programs are sometimes referred to as virtual agents, or virtual representatives. Andrette and Red are the names of two of the pioneering customer service bots that could be programmed to answer detailed questions from people who are seeking answers about a product or service.
Additional services that use bots are:
- Instant messaging apps, like WhatsApp, Slack and Facebook Messenger
- News apps, such as The New York Times, to display breaking news
- Rideshare apps like Lyft, where users request rides and can direct message their assigned driver prior to arrival
- Services that schedule meetings, like X.ai
Of course, these don’t even begin to scratch the service of what bots are used for in technology and business.
Good Bots vs. Bad Bots
While there are bots that serve very positive purposes for people and businesses, there are also malicious bots that automate actions that lead to hacking and cybercrimes.
Some of the most common malicious, or bad bots, include:
- DDoS or DoS bots that use an extreme amount of bots to overload server resources and halt services from working
- Spambots that promote unsolicited commercial content with the intent of driving traffic to a different website
- Hacker bots that attack the infrastructure of a website and distribute malware
Some additional types of malicious bots include email harvesters, malicious web crawlers, brute force password cracking, and credential stuffing bots.
To end these malicious bots, it’s important to use a bot manager and beef up your WordPress site security.
More on that in a bit.
The Advantages and Disadvantages Of Bots
As with other areas of technology, there are many advantages that come with employing the use of bots on your WordPress site.
Of course, there are also some disadvantages.
Advantages of bot use include:
- They perform repetitive tasks fast than people can
- Bots save human time for direct client and customer interaction
- They’re available at all times of the day and night
- You can reach a lot of people very quickly
- Website UX (user experience) can be drastically improved
On the other end of the spectrum, some of the disadvantages to bots are:
- Bots can be malicious if they’re programmed to do harm
- They cannot be programmed to perform many more complex and specific tasks
- Bots often misunderstand users
- They’re constantly used for spam
With that said, it’s time to block bots WordPress allows to enter into your online space.
How Do I Block Bad Bots In WordPress?
Learning how to stop bot traffic in WordPress begins with understanding that a bad bot is simply one that hits your WordPress site and offers no benefit to you as the site owner.
Bad bots consume a lot of server resources. This is especially true if they continually hit your wp-login page or other areas of your site.
By blocking them, you won’t need to deal with as much server stress. You’ll also be able to potentially save on hosting costs, your bandwidth, and even speed your site up.
Here’s how to get started:
1. Get the iThemes Security plugin
By using the iThemes Security plugin, you’ll get a WordPress security log that starts collecting the security events on your website.
Download iThemes Security now
WordPress security logs have several benefits in your overall security strategy, allowing you to:
- Identity and stop malicious behavior.
- Spot activity that can alert you of a breach.
- Assess how much damage was done.
- Aide in the repair of a hacked site.
If your site does get hacked, you will want to have the best information to aid in a quick investigation and recovery.
2. Identify the Bad Bots in Your WordPress Security Logs
Take a few minutes to observe your WordPress security log. Look to see if you notice suspicious or malicious bots that repetitively hit your site.
Create a list to record hostnames or IPs, which is displayed in the WordPress security log. Google each of the hostnames to see if other site developers have reported them as bad bots.
Keep in mind that you’ll need to do some research if you want to ensure a bot is a spam bot. Bots like Googlebot are legitimate and don’t need to be blocked. However, keep a close eye on suspicious ones and write them in your list.
The cool thing is the iThemes Security Pro takes your WordPress security logs and turns them into a real-time dashboard. From this view, you can see even more information.
After compiling the list of all hostnames for spam bots, there are a couple of different options for getting them blocked. Remember that bad bots are constantly evolving. This means that it’s a good idea to use a software solution that gets updated on a regular basis.
4. Ban Bots with iThemes Security
The iThemes Security Pro plugin has a great way to easily block bots by hostname with the Ban Users feature.
In this section of the plugin, you can add the bot IPs to the Ban Hosts and Ban User Agents sections. Here are a few more helpful settings:
- Default Ban List – When enabled, iThemes Security will use the hackrepair.com’s blocklist to ban known bad actors from your website.
- Ban Lists – When enabled, iThemes Security will be able to add IPs to the blocklist.
- Ban Hosts – IPs in this list will not be allowed to access your website. The ban list will show both IPs banned by iThemes Security and IPs manually added by you.
- Limit Banned IPs in Server Configuration Files – Limiting the number of IPs blocked by the Server Configuration Files (.htaccess and nginx.conf) will help reduce the risk of a server timeout when updating the configuration file.
- Ban User Agents – User agents in this list will not be allowed to access your website.
5. Limit the Number Of Login Attempts
By lowering the number of login attempts you allow your site users, you’ll immediately lock out the users and bots that have repeatedly entered invalid login criteria on the wp-login page.
This is an effective way to block out the spambots that excessively hit your site.
The iThemes Security Pro Local Brute Force Protection feature keeps track of invalid login attempts made by a host or IP address and a username. Once an IP or username has made too many consecutive invalid login attempts, they will get locked out and will be prevented from making any more attempts for a set period of time.
To get started using the Local Brute Force Protection feature, enable it on the main page of the iThemes Security Pro settings page.
Blocking Bad Bots In WordPress Will Make Your Life Easier
If you’ve been a WordPress site owner for any period of time, you’ve almost certainly dealt with bad bots attacking your site.
With these simple tricks, coupled with the best WordPress backup plugin if things ever get sticky (and we all know they do), you’ll set yourself up for a more secure future as a website owner.
Get the bonus content: A Guide to WordPress Security
Keep reading the article at WordPress News and Updates from iThemes – iThemes. The article was originally written by Kristen Wright on 2021-04-13 18:22:34.
The article was hand-picked and curated for you by the Editorial Team of WP Archives.