Why Does Site or Server Load Increase?

Why server load increases

There's only one good reason why the load on your hosting server starts increasing–the rest are bad. I'll look at how and why they all happen.

I was often contacted by site owners who had a problem with high server load. This common condition is first noticed when an owner gets a warning message from their hosting company. Such messages can be precursors to the blocking of the site, and it can happen to almost any site owner or webmaster.

This article covers the different reasons why the load on a site or server might be increasing, and what can be done about it.

Webmasters usually find out about excess load from their hosters. Hosting companies regulate and control resource usage for each hosting plan. If they run operating systems such as CloudLinux OS, they can set numbers for allowable CPU and memory usage for each account, using figures such as the percentage of total CPU power, or elapsed CPU time. 

Examples of charts used to detect patterns in CPU (left) and I/O (right).

The hoster usually wants to distribute CPU resources evenly across all client sites on a server. (If someone's hosting account uses 90% of the system's resources, that leaves only 10% for other clients.) In these cases, the account owner will get a warning, notifying them of their overuse and a reminder of their quotas. Systematic violations may cause the account to be blocked, to avoid service interruptions for other users on the same server.

Let's be clear. This is not an attempt to upgrade clients to more expensive tariffs, as some webmasters tend to think. It is hosting companies doing their best to make sure that all clients can run their businesses smoothly, without the interruptions caused by a few renegade sites.

I'll now go through what can cause excessive server load patterns, considering them from the perspective of external versus internal factors.

External factors

This means anything that does not depend on the configuration of the hosting platform, the operation of scripts or the administration of the site. In other words, external requests to the site from various services, bots or other sites. There are multiple possibilities.

1. Scanning for vulnerabilities, sensitive files, and admin panels
Any public site whose pages are indexed in a search engine can become a target for hackers and bots. Hackers will scan it in the search for hackable security holes. It is impossible to stop but you can defend against it.

Intensive site queries consume a lot of CPU resources, especially if they are using the POST method. Scanning a site with an external scanner can therefore show up as an increase in system load.

If an attacker finds a vulnerability or a variant for hacking while scanning, they are more likely to download malicious code, or try to perform destructive actions.

When scanning stops, the site will continue to work as normal, and system load will return to its normal value. That is, until the next scan.

2. Brute-force attack on site admin panel
This is one of the most popular attacks. The method involves obtaining administrative access by repeatedly trying popular combinations of usernames and passwords. A hacker bot will use a dictionary with the top (say) 1000 most popular combinations (e.g. admin/admin, admin/123456, etc).

This process of enumeration increases load on the system, because the panel's administrative page is constantly accessed, and requests are made using the resource-intensive POST method.

(For more about the problem of weak passwords, see here.)

3. Mass registration of users or bulk sending of spam via unprotected feedback forms
Often, sites will have user feedback or registration forms with weak bot protection mechanisms such as a "CAPTCHA" of the type "prove you are not a bot". Once the site is infiltrated by spammers, there is no protection from an "http flood"-type of attack. This is how it starts: the mass registration of users with spam profiles, and the sending of mail through the form.

The hosting platform CPU gets loaded and starts spamming. The hosting company disconnects the email service or blocks the site completely.

Almost all simple defense mechanisms are easily circumvented by modern bots. You must use something more effective such as Google's reCAPTCHA or the CAPTCHA built into the Imunify360 firewall.

4. Indexing a site by search bots

Sometimes, with a sufficiently large search index (e.g. when many pages are accessed by a Google search index), the re-indexing process can take a long time and create a heavy load on the server. Even if there are only a dozen pages on your site, you can also face a similar problem, such as, for example, the site was hacked and there are 50,000 pages of search results on it. It can also happen when the search index was spammed by a competitor that took advantage of errors in the work of the scripts of your site. There are many possibilities.

5. Grabbing and scraping content (site cloning)

Owners of unique content should be concerned with the problem of their content being downloaded from their site (so-called 'scraping and grabbing'). This can be done with special bots that load the pages of the site, copy all text and pictures, then create clone sites. If the scanning of your site is continuous, and there are many pages on the site, a significant load will be suffered by the hosting server.

6. Importing data (feeds, downloading of e-commerce data)

Often, e-commerce resources use a mechanism for exchanging data with external services. For example, a list of items can be downloaded from online stores, and news sites can regularly export news feeds, etc. If the content is dynamic, each such request will create a high load on the server.

7. Using pictures or links to your site

One of the less obvious causes of server load is through linking to or using an image on a much more popular site. This is the so-called slashdot effect, when the site does not cope with the flow of visitors from a more popular source.

Another situation is when someone posts a picture on a blog (for example, in the comments box). This is downloaded by each visitor and creates a load on your hosting server. Serious problems occur if the picture is generated through the action of a script (for example, when being scaled with timthumb or phpthumb).

8. Attacks on other sites (for example, vulnerable xmlrpc.php file)

Sites that contain vulnerabilities are often used by hackers to conduct attacks on other resources. Sometimes, an attacker doesn't even need to hack the site. Owners of the latest versions of WordPress, for example, may see this problem when the xmlrpc.php file is compromised. In this case, your site will act as an intermediate link, and the work of the site's scripts creates a heavy load on the server.

9. DDoS attack

If a DDoS (Distributed Denial of Service) attack happens, the site will fail unless special technical approaches are used, such as using a CDN or a DDoS traffic-cleaning service.

Detecting a DDoS attack is difficult. Because of the large load on the server, each hoster may act differently, offering DDoS attack protection services, moving the site to another server, or completely blocking and disconnecting the site. To protect against such an attack, you must have a prearranged solution, so that when a problem arises, it can be quickly solved.

10. Increase in number of visitors

The best reason for the growth of server load is a natural, organic increase in visitor numbers. This tells you it's time to scale-up the site and think about optimizing your scripts for increased traffic.

To find the cause of such an externally-caused load, you can analyze the logs of the web server, with special applications or a combination of commands in the SSH console.

A good list of the things to start looking for is:

  • top 20 requests that use POST in the results of the analysis;
  • top 20 requests by the method GET/HEAD;
  • top 20 IP addresses by number of hits;
  • top 20 referring pages by number of hits.

Together, these will reveal the source and type of traffic, the entry points to the site, and the scripts that are called most often, as most likely they are causing a high load.

To reduce the load for external attacks or intensive requests, in most cases it is enough to enable HTTP flood protection (for example, a classic "splash screen"). Or you can connect the site to DDoS traffic cleaning services that will block dangerous or especially active requests, skipping legitimate ones. In addition, static content, such as pictures, scripts and styles, will not be delivered from your site, but from CDN servers. This will also significantly reduce the load.

You can try to connect a caching plugin in the CMS, or a caching service on the hosting, but in the case of external factors affecting the load, this may not help.

Internal factors


Internal factors include everything that affects site performance at the level of scripts and settings, i.e., anything that can be controlled by the webmaster or site owner.

  1. Suboptimal scripts and a growing database
    It is possible that a visit to a start page, or just showing search results, can seriously load a server. This might be because of a badly designed web application architecture, or bad programming by inexperienced developers. The growth in the size of any underlying database with each site update, for example, because of an increase in e-commerce store items, will slow it down more and more, increasing the load on the hosting server. Individual pages of a site, each page with many content blocks, can send dozens of requests to the database, repeatedly performing the same operations on files, and sometimes even blocking the work of other elements of the site. This problem is common in online stores running old versions of Joomla with the VirtueMart plugin.

  2. Infecting a site with viruses
    Hacking and infecting a site with malicious scripts is a common cause of load growth. It is caused by virus activity, itself the result of malicious code fragments inserted into legitimate scripts. The load time and operation of processes suffer, and connections between scripts to external resources can time-out.

  3. External legitimate connections to third-party resources
    Few system administrators remember to take into account the load created by connections to external sources of information. Examples of these are widgets, weather and currency rate tickers, and news feeds. Data downloaded from elsewhere is rarely cached, so every time the page is opened, a script connects and downloads content from another server. If, for some reason, the external source stops responding, the download speed of the main site can be affected.

  4. Errors in the operation of scripts
    When running scripts, errors can occur that are invisible to visitors, but are visible in web server or PHP logs. If the site has high visitor numbers, or there are many errors, the load on the server can increase. Errors appear when the site is switched to a more recent version of PHP, one incompatible with existing scripts. Conflicts can arise between the new CMS core and old versions of the plugins, especially when only some, not all, site components are updated.

To analyze a high-load problem caused by internal factors, it is necessary to scan the site for malicious code. If none is detected, script profiling can be performed using the xhprof or xdebug modules.

High load caused by malicious code activity should be rectified with a site cleanup, and the installation of measures to protect against repeated hacking. Site treatment and protection is best done by information security professionals, not web developers.

If the cause of problems is the website architecture, the site must be optimized with the help of an experienced web developer. An alternative solution is to install a caching plugin, which can reduce the consumption of CPU resources (i.e. the load on hosting) and speed-up the site.

Conclusion


There's one more property of processor load I would like to mention, and that is its duration. High load might be both a short-term peak on the chart during the day, or a constant growth over a long period. A one-time spike on the CPU consumption graph is nothing to worry about—it is practically invisible, does not affect the availability of the site and does not interfere with your hosting neighbors. However, if the chart crawls at the same level for a long time, or for several days it hits or exceeds the limit of the CPU load, then action is needed. What do you do in such cases? The answer is to audit your sites as described above, checking for both external and internal factors that may be causing problems.

. . .

Imunify360 is a complete security solution for web hosters. To learn more, visit Imunify360.com. 

DON’T GET BLACKLISTED ON BLACK FRIDAY
Patterns of thought: the psychology of weak passwo...
 

Comments

No comments made yet. Be the first to submit a comment
Already Registered? Login Here
Guest
Wednesday, 19 December 2018

Captcha Image