January 10th, 2009 Category: Linux Server
A real nice and easy to use tool for measuring the network throughput on Linux/Unix as well as Windows is NETIO. You only need to download the binaries from the NETIO homepage, store on your server or clients and execute them.
The example below shows a sample measurement between to servers connected with a 100 MBit ethernet card and located at two different providers.
On one machine NETIO is started as server:
machine1:~/netio/bin# ./linux-i386 -s
NETIO - Network Throughput Benchmark, Version 1.26
(C) 1997-2005 Kai Uwe Rommel
TCP server listening.
UDP server listening.
TCP connection established ...
Receiving from client, packet size 1k ...
Sending to client, packet size 1k ...
Receiving from client, packet size 2k ...
Sending to client, packet size 2k ...
Receiving from client, packet size 4k ...
Sending to client, packet size 4k ...
Receiving from client, packet size 8k ...
Sending to client, packet size 8k ...
Receiving from client, packet size 16k ...
Sending to client, packet size 16k ...
Receiving from client, packet size 32k ...
Sending to client, packet size 32k ...
Done.
TCP server listening.
On the other machine NETIO is called e.g. with -t option for TCP and the name or IP of the other machine:
machine2:~/netio/bin# ./linux-i386 -t machine1.mydomain.de
NETIO - Network Throughput Benchmark, Version 1.26
(C) 1997-2005 Kai Uwe Rommel
TCP connection established.
Packet size 1k bytes: 7309 KByte/s Tx, 7645 KByte/s Rx.
Packet size 2k bytes: 7276 KByte/s Tx, 7724 KByte/s Rx.
Packet size 4k bytes: 7291 KByte/s Tx, 7737 KByte/s Rx.
Packet size 8k bytes: 7276 KByte/s Tx, 7723 KByte/s Rx.
Packet size 16k bytes: 7286 KByte/s Tx, 7719 KByte/s Rx.
Packet size 32k bytes: 7291 KByte/s Tx, 7725 KByte/s Rx.
Done.
We take a rough average of the result can multiply it by 8: 7,5 * 8 = 60 MBit/s. This is a acceptable value for a connection between two servers located at different providers within Germany.
For sure a useful tool! Also check Network Bandwidth Performance Measurement with Iperf
Written on January 10, 2009 | Posted in
Linux Server |
1 Comment
January 9th, 2009 Category: PHP/MySQL
The problem
Running PHP scripts with MySQL queries on a web server may need lots of resources and let get the load of your server high. This is especially the case when you will get kind of attacked by scripts which call your PHP scripts very often in a very short time.
The idea
Having a simple PHP class which can be easily included in every PHP script on your server and helps to avoid getting your CPU load very high caused by too much MySQL requests e.g. from scripts scanning your server.
The solution
Somewhere (I don’t remember where) I’ve discovered a simple PHP flood protection class. I’ve used this as base for my adjustments which does good work me.
How it works
The class stores the IP address and current time of every request into a database. During the next request the stored IP address and time is check and counted. After a given time and number of requests within this time the script will report a flood violation. Now the IP address will be blocked for a given time, e.g. 10 minutes. After this time the entry will be deleted from the database to keep the contents of the floodprotection table small.
The implementation
In this example the ADOdb database abstraction library for PHP was used. I personally like this library since it offers a good possibility to analyze e.g. time intensive SQL queries and other nice features.
First of all you need a new table in a database, here we assume you already have a database setup:
CREATE TABLE `floodprotection` (
`IP` char(32) NOT NULL default '',
`TIME` char(20) NOT NULL default '',
`COUNT` bigint(20) NOT NULL default '0',
`URL` varchar(255) NOT NULL,
`LOCK` enum('true','false') NOT NULL default 'false',
PRIMARY KEY (`IP`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Ok, now generate a sample PHP script, let’s say a index.php:
<?php
include("include/config.php");
echo "Flood or no flood, this is the question.";
?>
Just a regular PHP script which includes a extra config.php script at the top. Next let’s have a look at this config.php script which can be included in the same way in every PHP script on your server:
<?php
/***********************************************************
config.php - global include script.
2008 - technitip.net
***********************************************************/
$config = array();
$config['BASE_DIR'] = '/path/to/my/webpages';
require_once($config['BASE_DIR'].'/include/adodb/adodb.inc.php');
require_once($config['BASE_DIR'].'/include/floodprotection.php');
$DBTYPE = 'mysql';
$DBHOST = 'localhost';
$DBUSER = 'myuser';
$DBPASSWORD = 'mypassword';
$DBNAME = 'mydatabase';
// open db connection
$conn = &ADONewConnection($DBTYPE);
$conn->PConnect($DBHOST, $DBUSER, $DBPASSWORD, $DBNAME);
// get the called page name (full path withour host)
$self = $_SERVER[PHP_SELF];
// define pages which should be ignored
$ignore_pages = array(
"/ignore.php",
"/other_ignore.php" );
// ignore flooding for certain pages
if ( !in_array( $self, $ignore_pages ))
{
// create instantce of flood protection class
$protect = new flood_protection();
// check the current requet with remote address
if($protect->check_request(getenv('REMOTE_ADDR')))
{
// dispay error page in case of flooding and exit
header("Location: error.html");
exit;
}
}
?>
It opens a database connection and calls the flood protection class. Also here it’s possible to define PHP pages which should be ignored from the flood detection. If a flood is detected a error page “error.html” will be displayed.
And finally the floodprotection.php class. Within this class the repeat values for a valid flood as well as the time can be defined. Also a e-mail can be given to which a detected flood will be reported with some debug information. Important referrers like Google are ignored, they never should be blocked!
Please note that there is absolutely no warranty or support for this script.
<?php
/***********************************************************
floodprotection.php - a simple floodprotection class.
2008 - technitip.net
***********************************************************/
class flood_protection
{
// Number of secounds between a request
var $secs = 1.5;
// Number of any URL repeats within $secs to cause flood
var $flood = 20;
// Number of same URL repeats within $secs to cause flood
var $repeat = 10;
// Number of secounds to keep the user blocked
var $keep_secs = 600;
// e-eail address to send debug information
var $email_to = "mailto@mydomain.com";
// e-eail adress which appears as from
var $email_from = "flood@mydomain.com";
// e-eail subject
var $email_subj = "Flood";
// internal variables
var $url = "";
var $lock = false;
// add user ip address to database
function register_user($ip)
{
// insert ip and currnt time into database
$sql = 'INSERT INTO `floodprotection`
(`IP`,`TIME`,`COUNT`, `URL`)
VALUES(\'' . mysql_real_escape_string( $ip ) . '\',
\'' . $this->microtime_float() . '\',
0,
\'' . $this->url . '\') ';
$result = mysql_query($sql);
if(!$result)
{
return false;
}
return true;
}
// returns exact time
function microtime_float()
{
list($usec, $sec) = explode(" ", microtime());
return ((float)$usec + (float)$sec);
}
// check to see if the user is flooding
function check_request($ip)
{
$this -> url = mysql_real_escape_string($_SERVER['REQUEST_URI']);
// find out if the user is in the db or not
if($this -> user_in_db($ip))
{
// if yes check if there flooding
$return = $this->user_flooding($ip);
// update there last request
$this->update_user($ip);
// remove old users
$this->remove_old_users();
// return if there is flooding or not
return $return;
}
else
{
// if not add user to db
$this->register_user($ip);
// remove expired users
$this->remove_old_users();
// return false because user is not in db
return false;
}
}
// checks if user is already in db or not
function user_in_db($ip)
{
// query db to see if user is in db
$sql = 'SELECT `TIME`,`LOCK`
FROM `floodprotection`
WHERE `IP` = \''. mysql_real_escape_string( $ip ) . '\'
LIMIT 1';
$result = mysql_query($sql);
// if more than 0 records are returned user is in db
if(mysql_num_rows($result) > 0)
{
$row=mysql_fetch_assoc($result);
if ( $row['LOCK'] == "true" )
$this->lock = true;
return true;
}
// otherwise return false
return false;
}
function user_flooding($ip)
{
// don't check flood protection for important search robots
if ( (strstr($_SERVER[HTTP_USER_AGET] ,' googlebot' )) ||
(strstr($_SERVER[HTTP_USER_AGENT], 'Googlebot')) ||
(strstr($_SERVER[HTTP_USER_AGENT], 'Mediapartners-Google')) ||
(strstr($_SERVER[HTTP_USER_AGENT], 'eBay Relevance Ad Crawler')) ||
(strstr($_SERVER[HTTP_USER_AGENT], 'Yahoo! Slurp;' )))
return false;
// check if user is already locked
if ( $this->lock == true )
return true;
// query db to see if there is flooding
$result = mysql_query('
SELECT `TIME`,`COUNT`,`URL`,`LOCK`
FROM `floodprotection`
WHERE `IP` = \''. mysql_real_escape_string( $ip ) . '\'
AND `TIME` >= ' . ($this->microtime_float() - $this->secs) . '
LIMIT 1');
if(mysql_num_rows($result) > 0)
{
// if more than 0 records are returned there is flooding
$row=mysql_fetch_assoc($result);
$s = "";
$s = $s . "\n" . "Self: " . $_SERVER[PHP_SELF];
$s = $s . "\n" . "Ref: " . $_SERVER[HTTP_REFERER];
$s = $s . "\n" . "Query: " . $_SERVER[QUERY_STRING];
$s = $s . "\n" . "Uri: " . $this->url;
$s = $s . "\n" . "UriDB: " . $row['URL'];
$s = $s . "\n" . "Agent: " . $_SERVER[HTTP_USER_AGENT];
$s = $s . "\n" . "IP: " . getenv('REMOTE_ADDR');
$s = $s . "\n" . "CPU: " . exec('uptime');
$s = $s . "\n" . "COUNT: " . $row['COUNT'] . "/" . $this->flood;
// user is already locked
if ( $row['LOCK'] == "true" )
{
mail( $email_to, "Lock", $s, "From: " . $email_from );
return true;
}
// to many requests in a certain time => flood detected
if ( $row['COUNT'] > $this->flood )
{
$sql = 'UPDATE `floodprotection`
SET `LOCK`=\'true\'
WHERE `IP` = \''. mysql_real_escape_string( $ip ) . '\'';
mysql_query($sql);
mail( $email_to, "Flood", $s, "From: " . $email_from );
return true;
}
// to many requests in a certain time => check for same URL
else if ( $row['COUNT'] > $this->repeat )
{
// check if same URL has been accessed in a certain time
if ( !strcmp( $row['URL'], $this->url))
{
$sql = 'UPDATE `floodprotection`
SET `LOCK`=\'true\'
WHERE `IP` = \''. mysql_real_escape_string( $ip ) . '\'';
mysql_query($sql);
mail( $email_to, "Flood Repeat", $s, "From: " . $email_from );
return true;
}
else
return false;
}
else
{
return false;
}
}
$sql = 'UPDATE `floodprotection`
SET `COUNT`=\'0\'
WHERE `IP` = \''. mysql_real_escape_string( $ip ) . '\'';
mysql_query($sql);
// otherwise return false
return false;
}
function update_user($ip)
{
$sql = 'UPDATE `floodprotection`
SET `TIME` = \'' . $this->microtime_float() . '\',
`COUNT`=`COUNT`+1,
`URL`=\'' . $this->url . '\'
WHERE `IP` = \''. mysql_real_escape_string( $ip ) . '\'';
// query db to update the user last request
$result = mysql_query($sql);
}
function remove_old_users()
{
// query db to remove all the old users
mysql_query('
DELETE FROM `floodprotection`
WHERE `TIME` <= \'' . ($this->microtime_float() - $this->keep_secs) . '\'');
}
}
?>
Written on January 9, 2009 | Posted in
PHP/MySQL |
2 Comments
January 6th, 2009 Category: General
For personal use there is a simple free setup builder GkSetup available. It’s suitable for standard setups. So if you are looking for one give this one a try.
The FREE Edition offers:
- Uninstaller
- Start Menu Links
- Readme/License Texts
- Multiple Setup Types
- OCX, OLE/COM Server registration
- Registry/INI-File modifications
- Multilanguage support
- and much more
Written on January 6, 2009 | Posted in
General |
Leave a comment
January 6th, 2009 Category: General
A new version of Hot Babe for Windows has been released!
- Now with setup
- Option for auto start on system boot
- Options are saved in registry
- Options are adjustable in tray icon menu
Download at hotbabe.planlos.org.
Read more about: Hot Babe on TechniTip.Net.
Written on January 6, 2009 | Posted in
General |
Leave a comment
January 4th, 2009 Category: General, Linux
File transfers with resume are also possible for secure SFTP connections with WinSCP. A nice free SFTP, FTP and SCP client for Windows. At the WinSCP homepage more information regarding resume can be found.
For secure file transfers SFTP protocol allows resume. WinSCP definitly a nice and free tool!
See also: FTP with resume
Written on January 4, 2009 | Posted in
General,
Linux |
Leave a comment
