I really don’t like this look. Doesn’t the following URL look much better?

I think yes. So here is tutorial how you can beautify your links using Apache and mod_rewrite.

The Test Script

We assume that mod_rewrite is loaded in your Apache config and generate a simple PHP script called “test.php”:

<?php
echo "Parameter 1: " . $_REQUEST['param1'] . "<br />";
echo "Parameter 2: " . $_REQUEST['param2'] . "<br />";
echo "Parameter 3: " . $_REQUEST['param3'] . "<br />";
?>

This script gets the parameters param1, param2, etc. which have been given to the PHP page and prints them out. To check it we put the URL into the browser:

http://technitip.net/test.php?param1=p1&param2=p2&param3=p3

And get the result:

Parameter 1: p1
Parameter 2: p2
Parameter 3: p3

Simple Rewrite

Fine, passing parameters to our PHP script is working but still looking ugly. Now we generate a .htaccess file in the root directory of your web directory:

RewriteEngine On
RewriteBase /
RewriteRule ^(test)/?(.*)$ test.php [L,QSA,NC]

What will happen? test.php will be redirected to test and all parameters will be parsed like before, we can test it:

First Parameter Rewrite

Nice, but not yet what we really want. So we put another line into the .htaccess (before our first RewriteRule, not after! It’s important):

RewriteRule ^(test)/([a-z0-9]+)(/[^/]+)?/?(.*)$ test.php?param1=$2 [L,QSA,NC]
RewriteRule ^(test)/?(.*)$ test.php [L,QSA,NC]


For every parameter we need a new expression /([a-z0-9]+)(/[^/]+)?/

This will convert the line /test/p1 into test.php?param1=p1. Note that only the characters a-z und 0-9 are allowed as parameters. For the first parameter $2 is used.

Second Parameter

We repeat this step for every further parameter we need to hand over, $4 is used for the second parameter (not $3!):

RewriteRule ^(test)/([a-z0-9]+)(/[^/]+)?/([a-z0-9]+)(/[^/]+)?/?(.*)$ test.php?param1=$2&param2=$4 [L,QSA,NC]
RewriteRule ^(test)/([a-z0-9]+)(/[^/]+)?/?(.*)$ test.php?param1=$2 [L,QSA,NC]
RewriteRule ^(test)/?(.*)$ test.php [L,QSA,NC]


Three Parameters

Or with 3 parameters. Note: Since the .htaccess is parsed from top to bottom the rewrite rule with the highest number of parameters must be located at the top of the .htaccess. $6 is used for the third parameter!

RewriteRule ^(test)/([a-z0-9]+)(/[^/]+)?/([a-z0-9]+)(/[^/]+)?/([a-z0-9]+)(/[^/]+)?/?(.*)$ test.php?param1=$2&param2=$4&param3=$6 [L,QSA,NC]
RewriteRule ^(test)/([a-z0-9]+)(/[^/]+)?/([a-z0-9]+)(/[^/]+)?/?(.*)$ test.php?param1=$2&param2=$4 [L,QSA,NC]
RewriteRule ^(test)/([a-z0-9]+)(/[^/]+)?/?(.*)$ test.php?param1=$2 [L,QSA,NC]
RewriteRule ^(test)/?(.*)$ test.php [L,QSA,NC]


We test again with 3 parameters:

Parameter 1: p1
Parameter 2: p2
Parameter 3: p3

Looks better, indeed.

Hint

Be sure you have enabled “Options FollowSymLinks” or Options “SymLinksIfOwnerMatch” in your Apache or virtual host config. Otherwise mod rewrite will not work!

Basics

But there are a few things which could help to improve your performance. On my configuration I’ve added skip-name-resolve in my.cnf:

[mysqld]
skip-name-resolve

Now the MySQL server will not do any DNS name resolves. Please note that permissions based on hostnames will no longer work!

For security reasons I would also recommend binding only to the localhost address. Using this the MySQL server is not reachable on the Ethernet interface. In earlier version of MySQL this was usually done with the skip-networking parameter.

[mysqld]
bind-address    = 127.0.0.1

And I’ve added skip-external-locking and skip-locking:

[mysqld]

skip-external-locking
skip-locking

This could also improve your performance.

Search Length

For text search queries with a allowed length of 2 characters add this line:

ft_min_word_len="2"

Otherwise default minimum length is 3 characters, which means that a search query “select * like ‘%AB%’” will never return any data.

Cache Settings

Caching seems to be a very important thing when optimizing your server. My feeling is that the standard configuration of a MySQL server is not really designed for machines with 1 GByte memory or above.

Below is my configuration, with a few cache settings increased.

[mysqld]
query_cache_limit    =  1M
query_cache_size     = 32M
thread_cache_size    =  8
join_buffer          =  1M
table_cache          = 16M
sort_buffer_size     =  2M
read_buffer_size     =  2M
read_rnd_buffer_size =  2M
tmp_table_size       = 48M
max_allowed_packet   = 16M
thread_stack         = 128K
key_buffer_size      = 32M

This configuration works on a machine with 1 GByte RAM and several databases running. Basically I recommend to increase the cache size settings. And memory helps almost always.

Applies to MySQL version: 5.0.32

Running PHP scripts with MySQL queries on a web server may need lots of resources and let get the load of your server high. This is especially the case when you will get kind of attacked by scripts which call your PHP scripts very often in a very short time.

The idea

Having a simple PHP class which can be easily included in every PHP script on your server and helps to avoid getting your CPU load very high caused by too much MySQL requests e.g. from scripts scanning your server.

The solution

Somewhere (I don’t remember where) I’ve discovered a simple PHP flood protection class.  I’ve used this as base for my adjustments which does good work me.

How it works

The class stores the IP address and current time of every request into a database. During the next request the stored IP address and time is check and counted. After a given time and number of requests within this time the script will report a flood violation. Now the  IP address will be blocked for a given time, e.g. 10 minutes. After this time the entry will be deleted from the database to keep the contents of the floodprotection table small.

The implementation

In this example the ADOdb database abstraction library for PHP was used. I personally like this library since it offers a good possibility to analyze e.g. time intensive SQL queries and other nice features.

First of all you need a new table in a database, here we assume you already have a database setup:

CREATE TABLE `floodprotection` (
  `IP` char(32) NOT NULL default '',
  `TIME` char(20) NOT NULL default '',
  `COUNT` bigint(20) NOT NULL default '0',
  `URL` varchar(255) NOT NULL,
  `LOCK` enum('true','false') NOT NULL default 'false',
  PRIMARY KEY  (`IP`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

Ok, now generate a sample PHP script, let’s say a index.php:

<?php
include("include/config.php");

echo "Flood or no flood, this is the question.";
?>

Just a regular PHP script which includes a extra config.php script at the top. Next let’s have a look at this config.php script which can be included in the same way in every PHP script on your server:

<?php
/***********************************************************
 config.php - global include script.

 2008 - technitip.net
 ***********************************************************/

$config = array();
$config['BASE_DIR'] = '/path/to/my/webpages';

require_once($config['BASE_DIR'].'/include/adodb/adodb.inc.php');
require_once($config['BASE_DIR'].'/include/floodprotection.php');

$DBTYPE     = 'mysql';
$DBHOST     = 'localhost';
$DBUSER     = 'myuser';
$DBPASSWORD = 'mypassword';
$DBNAME     = 'mydatabase';

// open db connection
$conn = &ADONewConnection($DBTYPE);
$conn->PConnect($DBHOST, $DBUSER, $DBPASSWORD, $DBNAME);

// get the called page name (full path withour host)
$self = $_SERVER[PHP_SELF];

// define pages which should be ignored
$ignore_pages = array(
  "/ignore.php",
  "/other_ignore.php" );

// ignore flooding for certain pages
if ( !in_array( $self, $ignore_pages ))
{
  // create instantce of flood protection class
  $protect = new flood_protection();

  // check the current requet with remote address
  if($protect->check_request(getenv('REMOTE_ADDR')))
  {
    // dispay error page in case of flooding and exit
    header("Location: error.html");
    exit;
  }
}
?>

It opens a database connection and calls the flood protection class. Also here it’s possible to define PHP pages which should be ignored from the flood detection. If a flood is detected a error page “error.html” will be displayed.

And finally the floodprotection.php class. Within this class the repeat values for a valid flood as well as the time can be defined. Also a e-mail can be given to which a detected flood will be reported with some debug information. Important referrers like Google are ignored, they never should be blocked!

Please note that there is absolutely no warranty or support for this script.

<?php
/***********************************************************
 floodprotection.php - a simple floodprotection class.

 2008 - technitip.net
 ***********************************************************/

class flood_protection
{
  // Number of secounds between a request
  var $secs = 1.5;
  // Number of any URL repeats within $secs to cause flood
  var $flood = 20;
  // Number of same URL repeats within $secs to cause flood
  var $repeat = 10;
  // Number of secounds to keep the user blocked
  var $keep_secs = 600;
  // e-eail address to send debug information
  var $email_to = "mailto@mydomain.com";
  // e-eail adress which appears as from
  var $email_from = "flood@mydomain.com";
  // e-eail subject
  var $email_subj = "Flood";

  // internal variables
  var $url = "";
  var $lock = false;

  // add user ip address to database
  function register_user($ip)
  {
    // insert ip and currnt time into database
    $sql = 'INSERT INTO `floodprotection`
            (`IP`,`TIME`,`COUNT`, `URL`)
            VALUES(\'' . mysql_real_escape_string( $ip ) . '\',
                   \'' . $this->microtime_float() . '\',
                   0,
                   \'' . $this->url . '\') ';

    $result = mysql_query($sql);

    if(!$result)
    {
      return false;
    }
    return true;
  }

  // returns exact time
  function microtime_float()
  {
    list($usec, $sec) = explode(" ", microtime());
    return ((float)$usec + (float)$sec);
  }

  // check to see if the user is flooding
  function check_request($ip)
  {
    $this -> url = mysql_real_escape_string($_SERVER['REQUEST_URI']);

    // find out if the user is in the db or not
    if($this -> user_in_db($ip))
    {
      // if yes check if there flooding
      $return = $this->user_flooding($ip);
      // update there last request
      $this->update_user($ip);
      // remove old users
      $this->remove_old_users();

      // return if there is flooding or not
      return $return;
    }
    else
    {
      // if not add user to db
      $this->register_user($ip);
      // remove expired users
      $this->remove_old_users();

      // return false because user is not in db
      return false;
    }
  }

  // checks if user is already in db or not
  function user_in_db($ip)
  {
    // query db to see if user is in db
    $sql = 'SELECT `TIME`,`LOCK`
            FROM `floodprotection`
            WHERE `IP` = \''. mysql_real_escape_string( $ip ) . '\'
            LIMIT 1';

    $result = mysql_query($sql);

    // if more than 0 records are returned user is in db
    if(mysql_num_rows($result) > 0)
    {
      $row=mysql_fetch_assoc($result);
      if ( $row['LOCK'] == "true" )
        $this->lock = true;

      return true;
    }

    // otherwise return false
    return false;
  }

  function user_flooding($ip)
  {
    // don't check flood protection for important search robots
    if ( (strstr($_SERVER[HTTP_USER_AGET] ,' googlebot' )) ||
         (strstr($_SERVER[HTTP_USER_AGENT], 'Googlebot')) ||
         (strstr($_SERVER[HTTP_USER_AGENT], 'Mediapartners-Google')) ||
         (strstr($_SERVER[HTTP_USER_AGENT], 'eBay Relevance Ad Crawler')) ||
         (strstr($_SERVER[HTTP_USER_AGENT], 'Yahoo! Slurp;' )))
     return false;

    // check if user is already locked
    if ( $this->lock == true )
      return true;

    // query db to see if there is flooding
    $result = mysql_query('
      SELECT `TIME`,`COUNT`,`URL`,`LOCK`
      FROM `floodprotection`
      WHERE `IP` = \''. mysql_real_escape_string( $ip ) . '\'
      AND `TIME` >= ' . ($this->microtime_float() - $this->secs) . '
      LIMIT 1');

    if(mysql_num_rows($result) > 0)
    {
      // if more than 0 records are returned there is flooding
      $row=mysql_fetch_assoc($result);

      $s = "";
      $s = $s . "\n" . "Self:  " . $_SERVER[PHP_SELF];
      $s = $s . "\n" . "Ref:   " . $_SERVER[HTTP_REFERER];
      $s = $s . "\n" . "Query: " . $_SERVER[QUERY_STRING];
      $s = $s . "\n" . "Uri:   " . $this->url;
      $s = $s . "\n" . "UriDB: " . $row['URL'];
      $s = $s . "\n" . "Agent: " . $_SERVER[HTTP_USER_AGENT];
      $s = $s . "\n" . "IP:    " . getenv('REMOTE_ADDR');
      $s = $s . "\n" . "CPU:   " . exec('uptime');
      $s = $s . "\n" . "COUNT: " . $row['COUNT'] . "/" . $this->flood;

      // user is already locked
      if ( $row['LOCK'] == "true" )
      {
        mail( $email_to, "Lock", $s, "From: " . $email_from );

        return true;
      }

      // to many requests in a certain time => flood detected
      if ( $row['COUNT'] > $this->flood )
      {
        $sql = 'UPDATE `floodprotection`
                SET `LOCK`=\'true\'
                WHERE `IP` = \''. mysql_real_escape_string( $ip ) . '\'';
        mysql_query($sql);

        mail( $email_to, "Flood", $s, "From: " . $email_from );

        return true;
      }
      // to many requests in a certain time => check for same URL
      else if ( $row['COUNT'] > $this->repeat )
      {
        // check if same URL has been accessed in a certain time
        if ( !strcmp( $row['URL'], $this->url))
        {
          $sql = 'UPDATE `floodprotection`
                  SET `LOCK`=\'true\'
                  WHERE `IP` = \''. mysql_real_escape_string( $ip ) . '\'';

          mysql_query($sql);

          mail( $email_to, "Flood Repeat", $s, "From: " . $email_from );

          return true;
        }
        else
          return false;
      }
      else
      {
        return false;
      }
    }
    $sql = 'UPDATE `floodprotection`
            SET `COUNT`=\'0\'
            WHERE `IP` = \''. mysql_real_escape_string( $ip ) . '\'';

    mysql_query($sql);

    // otherwise return false
    return false;
  }

  function update_user($ip)
  {
    $sql = 'UPDATE `floodprotection`
            SET `TIME` = \'' . $this->microtime_float() . '\',
                `COUNT`=`COUNT`+1,
                `URL`=\'' . $this->url . '\'
            WHERE `IP` = \''. mysql_real_escape_string( $ip ) . '\'';

    // query db to update the user last request
    $result = mysql_query($sql);
  }

  function remove_old_users()
  {
    // query db to remove all the old users
    mysql_query('
      DELETE FROM `floodprotection`
      WHERE `TIME` <= \'' . ($this->microtime_float() - $this->keep_secs) . '\'');
  }
}
?>

• Download the source
• Unpack the source and change to the unpacked directory
• run “phpize”
• run “./configure”
• run “make”
• run “make install”
• run ‘”mkdir /tmp/eaccelerator”
• run “chmod 0777 /tmp/eaccelerator”
• edit “/etc/php5/apache2/php.ini” (e.g. on Debian Etch)

[eAccelerator]
extension="eaccelerator.so"
eaccelerator.shm_size="128"
eaccelerator.cache_dir="/tmp/eaccelerator/"
eaccelerator.enable="1"
eaccelerator.optimizer="1"
eaccelerator.check_mtime="1"
eaccelerator.debug="0"
eaccelerator.filter=""
eaccelerator.shm_max="0"
eaccelerator.shm_ttl="0"
eaccelerator.shm_prune_period="0"
eaccelerator.shm_only="0"
eaccelerator.compress="1"
eaccelerator.compress_level="9"

• restart apache “/etc/init.d/apache2 restart”

In my example the shm_size has been set to 128 MByte because the server hosts serveral domains with PHP pages.

Working with MySQL version 5.0.32 and PHP version 5.2.0 on Debian Etch.

Script is designed to run from command line “php optimize.php” maybe from a cron job. Be careful: avoid running this script during your server “rush hour”, it may slow down your server.

<?php
/***********************************************************
 optimimze.php - optimizes all databases and tables of the
                 given mysql host.

 2008 - technitip.net
 ***********************************************************/

$mysqlhost = "localhost"; // enter MySQL host
$mysqluser = "user";      // enter MySQL user
$mysqlpwd  = "password";  // enter password

###########################################

$connection = mysql_connect($mysqlhost, $mysqluser, $mysqlpwd);
if (mysql_error())
{
  echo "Could not connect to database server! " . mysql_error() . "\n";
  exit;
}

$db_list = mysql_list_dbs();
$i = 0;
$cnt = mysql_num_rows($db_list);

while ($i < $cnt)
{
  $db = mysql_db_name($db_list, $i);
  ###########################################
  mysql_select_db($db, $connection);
  $result = mysql_list_tables($db);

  while ($row = mysql_fetch_row($result))
  {
    if ( $db == "information_schema" )
      continue;

    echo $db . " : `" . $row[0] . "`";

    $sql = "OPTIMIZE TABLE `".$row[0]."`";
    $erg = mysql_query($sql, $connection) or die(mysql_error());
    $data= mysql_fetch_array($erg, MYSQL_ASSOC);

    if($data)
    {
      echo " - " . $data['Msg_text'] . "\n";
    }
  }
  ###########################################
  $i++;
}

?>

This script is intended to be called periodically (e.g. every day) from cron:

2 2       * * *   root /root/scripts/mysql_backup.sh

#/bin/sh
now=`date "+%Y-%m-%d"`
user="mysql_user"
password="mysql_password"
path="/home/backup/"

cd $path

mysqldump -u $user -p$password  --all-databases | gzip -c > backup_all_$now.sql.gz

# delete files older than 3 days
find . -name "*.gz" -type f -mtime +3 -exec rm {} ";"