You are viewing william_os4y

Tue, Dec. 30th, 2008, 12:11 pm
Fapws3-0.1 versus Fapws2-0.3: First comparison

As promised, you can see here under results of performance tests I've made.

Machine: AMD Athlon(tm) 64 X2 Dual Core Processor 4800+ A
OS: Linux 2.6.27-ARCH #1 SMP PREEMPT
Benchmark tool: ApacheBench, Version 2.3 ($Revision: 655654 $)
Python: Python 2.6.1
Libev: libev-3.49
Methodology: run webserver and ApacheBench ("ab" in short) on the same host; 
but I use "nice -n20" for "ab". Wait enough time between each run to assure all "waiting" 
connections  will be effectively closed (+- 1 minute on my machine). Execute each 
tests 5 times and take one "ab" report which is representative (to avoid abnormal good/bad 
results). 


Observations

(results are in the annexes):
Errors: we don't see it in the results here bellow, but neither Fapws2 neither Fapws3 have 
generated errors.
Requests per seconds: Fapws3 is a bit better(10%) than Fapws2 for low number of concurrent 
requests. Above 500 concurrent requests Fapws2 is a bit better (5%).
mean: Fapws3 has a better mean than Fapws2 for all tests.
median: Fapws3 is better for all tests
Standard deviation: Both are equal until 100 concurrent requests. Above Fapws2 is better. 


To summarize I would say that, thanks to libev, Fapws3 is not too bad.


As comparison, I've added results of cherrypy (threaded). By the way Charrypy is not bad 
at all ;-). We can see that, as expected (cherrypy is fully written with python), 
that number of requests per seconds are much lower than Fapws. We also see the impact of 
a threaded system on the Standard Deviation. With such a big standard deviation, user's 
perception can be quite various. 



William


Annexes


Fapws3-0.1Fapws2-0.3
Server Software:        fapws3/0.1
Document Path:          /test
Document Length:        13 bytes
Concurrency Level:      10
Complete requests:      10000        
Requests per second:    5565.03 [#/sec] (mean)
Connection Times (ms)
              min  mean[+/-sd] median   max
Total:          1    2   0.7      2      17
Server Software:        fapws2/0.3
Document Path:          /hello
Document Length:        13 bytes
Concurrency Level:      10
Complete requests:      10000        
Requests per second:    4964.60 [#/sec] (mean)
Connection Times (ms)
              min  mean[+/-sd] median   max
Total:          1    2   0.0      2       3
Server Software:        fapws3/0.1
Document Path:          /test
Document Length:        13 bytes
Concurrency Level:      100
Complete requests:      10000        
Requests per second:    5798.24 [#/sec] (mean)
Connection Times (ms)
              min  mean[+/-sd] median   max
Total:          1    3   4.4      2     212
Server Software:        fapws2/0.3
Document Path:          /hello
Document Length:        13 bytes
Concurrency Level:      100
Complete requests:      10000        
Requests per second:    4847.79 [#/sec] (mean)
Connection Times (ms)
              min  mean[+/-sd] median   max
Total:         12   21   1.1     21      28
Server Software:        fapws3/0.1
Document Path:          /test
Document Length:        13 bytes
Concurrency Level:      500
Complete requests:      10000        
Requests per second:    4464.30 [#/sec] (mean)
Connection Times (ms)
              min  mean[+/-sd] median   max
Total:          2   20  36.1      3     368
Server Software:        fapws2/0.3
Document Path:          /hello
Document Length:        13 bytes
Concurrency Level:      500
Complete requests:      10000
Requests per second:    4818.79 [#/sec] (mean)
Connection Times (ms)
              min  mean[+/-sd] median   max
Total:         85  103   2.6    104     107
Server Software:        fapws3/0.1
Document Path:          /test
Document Length:        13 bytes
Concurrency Level:      1000
Complete requests:      10000        
Requests per second:    4300.18 [#/sec] (mean)
Connection Times (ms)
              min  mean[+/-sd] median   max
Total:         60  120  46.3     92     220
Server Software:        fapws2/0.3
Document Path:          /hello
Document Length:        13 bytes
Concurrency Level:      1000
Complete requests:      10000
Requests per second:    4594.43 [#/sec] (mean)
Connection Times (ms)
              min  mean[+/-sd] median   max
Total:         11  154  92.7    221     680



Charrypy-3.1.1
Server Software:        CherryPy/3.1.1
Document Path:          /
Document Length:        13 bytes
Concurrency Level:      10
Complete requests:      10000        
Requests per second:    1040.73 [#/sec] (mean)
Connection Times (ms)
              min  mean[+/-sd] median   max
Total:          1    5 105.4      3    9608
Server Software:        CherryPy/3.1.1
Document Path:          /
Document Length:        13 bytes
Concurrency Level:      100
Complete requests:      10000        
Requests per second:    1910.31 [#/sec] (mean)
Connection Times (ms)
              min  mean[+/-sd] median   max
Total:          0   26 268.2      3    4884
Server Software:        CherryPy/3.1.1
Document Path:          /
Document Length:        13 bytes
Concurrency Level:      500
Complete requests:      10000        
Requests per second:    1250.71 [#/sec] (mean)
Connection Times (ms)
              min  mean[+/-sd] median   max
Total:          1  228 659.8      5    6243
Server Software:        CherryPy/3.1.1
Document Path:          /
Document Length:        13 bytes
Concurrency Level:      1000
Complete requests:      10000
Requests per second:    1518.28 [#/sec] (mean)
Connection Times (ms)
              min  mean[+/-sd] median   max
Total:          1  275 865.1      5    6324

Sun, Dec. 28th, 2008, 02:29 pm
Fapws3-0.1 has just been released

Hello all,

After several months of hesitations, work, trials and tests, I'm very happy to announce the first release of Fapws3.

Fapws3 is a full re-write of Fapws2 but based on libev.

Indeed, since the API changes in libevent (>1.4.0) we were forced to modify the libevent sources before compiling it, just for the usage of Fapws2. This was not really comfortable.
Moreover, libev is incredibly very well documented.

Fapws2 will stay open just for bug fixing, but my future work will now be focused on Fapws3.

This has been successfully tested on Linux 2.6.27 and FreeBSD-6.1 with libev-3.49.

Benchmark between Fapws2 and Fapws3 is on going and will be published as soon as possible.

And because the number of people willing to discuss about Fapws is increasing, I've setup a dedicated mailing-list: http://groups.google.com/group/fapws
Feel free to join ;-).



William


ANNEXES:
--------

source repository for version 0.1
Download fapws3-0.1 as a tarball
Git repository: http://github.com/william-os4y/fapws3/tree/master
Demo site running wsgi_wiki with Fapws3: http://fapws3.vmrt.homelinux.com/

Mon, Nov. 3rd, 2008, 02:14 pm
my contact details

Since the crash of my server: opensource4you.com, I've spent lot of time to recover my files from few old backups.

I've not yet find back my mailing-lists for fapws, but interested people can contact me at my new gmail account: william dot os4y at gmail.com

Sorry for disturbance.

William

Sun, Nov. 2nd, 2008, 08:46 am
FAPWS2 on github

Thanks to Piranha, I've discovered a git repository provider: github.

The new fapws2 git repository is now: http://github.com/william-os4y/fapws2/tree/master.

For interested people, I've got several good feedback about FAPWS2. Some of them have internet websites runnning without any problems. Some have even reported an uptime of +2 months without any issue.
You are always free to send me your websites running with fapws2, if they are public, I'll be happy to talk about them ;-).

My current work with FAPWS is to facilitate the integration with libevent. Indeed, for fapws2, I'm using a different methodology to parse URL than libevent does. This force me to modify the code of libevent (evhttp_handle_request); which is not a so good solution.

William

Tue, Sep. 2nd, 2008, 05:22 pm
Year the reasons why the server as desapeard

I've stored all emails from the sivit.fr support team into one Yahoo folder. Even the emails send into the Yahoo Spam.

As you can see under suddenly, all of the sivit's communications were filtered out by Yahoo as Spam.
Indeed, they have used wrong email format with the fromaddress. That's why yahoo complains.



On that suddent email problem, they don't give any answers.

They claim that I've been informed on time about the issue and the associated risks. But, between the first mail and crash of the machine, we hae only 10 hours. Which are the sleeping hours.

This was, thus, rather impossible to treat them accordingly.

More over, they never explained why the server has been destroyed.
And for sure, they have never give any excuses.

This is just my, bad, story with them.
You should not be generalised, because, maybe, some customers are happy with their quality of service.

Up to everyone to decide if they are trustable.

Tue, Sep. 2nd, 2008, 04:30 am
Repository is down

After a Technical intervention of sivit.fr, the dedicated server has been removed.
The server opensource4you.com is, thus, no more available.

I'm no more trusting them and will surely no more use their services.

I'll keep you informed when a new repository will be setup.

William

Tue, Apr. 15th, 2008, 01:53 pm
using fapws2 as a proxy

On current multi core machine, I have the idea to run several fapws2 instances for complex websites. That way I will have several fapws2 server dealing each a part of the web application.

To stay with fapws2, that means that one instance will be a "front end" web server. He will querying the others.

But what is the best way to implement such proxy mechanism (by only using libevent, no threads) ?
People having knowledge, references of document exaplaining it, ... are welcome ;).

William

Tue, Apr. 15th, 2008, 01:32 pm
fapws2 with libevent-1.4.3-stable

I've just tried compiling fapws2 with the brand new libevent library. With all the changes that they have made inside libevent, I was suprised that fapws2 is running without major changes ;).

Patch for libevent-1.4:
Remove "static" for the procedure called "evhttp_handle_request" in http.c.
FAPWS2 over-writte this procedure, thus it cannot be static ;(.
I've provided my evhttp_handle_request (with backward compatibility), but the libevent development team has not include it ;(.




On the most important feature of fapws2: performance, and based on my simple "hello world" sample, I see a small improvement: 6%. (633#/sec with libevent-1.4.3 against 594#/sec with libevent-1.3e).

Additional tests must be made with this new library but the starting point is promising ;).

William

git repository
Tar file here


Annexe


Patch on the libevent-1.4.3 sources
[libevent]$ diff http.c http_orig.c
1936c1936
< void
---
> static void



with fapws2 based on libevent-1.4.3
[fapws2]# ab -n 1000 -c10 http://127.0.0.1:8080/hello
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Finished 1000 requests


Server Software:        fapws2/0.3
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /hello
Document Length:        13 bytes

Concurrency Level:      10
Time taken for tests:   1.579206 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      105000 bytes
HTML transferred:       13000 bytes
Requests per second:    633.23 [#/sec] (mean)
Time per request:       15.792 [ms] (mean)
Time per request:       1.579 [ms] (mean, across all concurrent requests)
Transfer rate:          64.59 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       3
Processing:     1   15   9.5     12      48
Waiting:        0   14   9.4     12      48
Total:          3   15   9.5     12      48

Percentage of the requests served within a certain time (ms)
  50%     12
  66%     13
  75%     14
  80%     15
  90%     38
  95%     42
  98%     45
  99%     46
 100%     48 (longest request)

with fapws2 based on libevent-1.3e
[root@myhost fapws2]# ab -n 1000 -c10 http://127.0.0.1:8080/hello
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Finished 1000 requests


Server Software:        fapws2/0.2
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /hello
Document Length:        13 bytes

Concurrency Level:      10
Time taken for tests:   1.683275 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      105000 bytes
HTML transferred:       13000 bytes
Requests per second:    594.08 [#/sec] (mean)
Time per request:       16.833 [ms] (mean)
Time per request:       1.683 [ms] (mean, across all concurrent requests)
Transfer rate:          60.60 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       2
Processing:     1   16   9.5     13      58
Waiting:        0   15   9.4     13      55
Total:          3   16   9.5     13      58

Percentage of the requests served within a certain time (ms)
  50%     13
  66%     14
  75%     16
  80%     19
  90%     29
  95%     42
  98%     44
  99%     48
 100%     58 (longest request)

Mon, Nov. 12th, 2007, 07:20 pm
Fapws2-0.2 ready for tests

I've just released the 0.2 version of FAPWS2 (Fast Asynchronous Python WebServer) which is WSGI compliant.

With this release, several bug fixes has been done (binaries are better working, cookies, ...).
Now, FAPWS2 can now serve Django application (GET and POST), the admin page is working too ;-).

Please not that FAPWS2 has been developed based on libevent-1.3x releases (on linux, but has been reported to work on Mac-OSX too).

TGZ: http://www.opensource4you.com/downloads/fapws2-0.2.tgz
Git repository: http://www.opensource4you.com/cgi-bin/gitweb.cgi?p=fapws2;a=summary

Your feedbacks, comments, tests results, ... are very welcome.

William

Thu, Nov. 8th, 2007, 09:01 am
fapws2 (based on evhttp api of libevent) is working ;-)

Nearly 1 month after my last post, I've got a working version of fapws using the http api of libevent.

Thanks to a wonderfull book call "C in a nutshell", the python API documentation and to the google code search engine, I've been able to write the required code; but also to track memory leaks, debug, ...

Thus, definitively YES, writing python API in C is not that difficult (for a python developer who don't know C)

Concerning our WSGI webserver, it can serve +1100#/sec on my Athlon-XP 1.2Ghz. It can resist to massive C10k attacks and it can serve Django applications (look into the samples).

It was really exiting and funny to write this peace of code and I would thanks all people who provide me help, feedbacks and productive remarks. I would also specially thanks Lateef J. who has guide me (more than the others) even concerning stupid questions I have had.

Because of a better integration with C, this new fapws (called fapws2, see later) is +-40% quicker than fapws for the simple hello_world.
For Django application, it runs 30% quicker than CherryPy 3.0.2 (thread base).
(I'm not against cherrypy, which is a great webserver. But Cherrypy is, for me, the best one, thus the reference.).
At least, Fapws2 prove that select based webserver can be a good alternative to thread base one.

The goal was to show the feasibility and the capabilities, lot of works still need to be done on this peace of code. Do not use it others than for tests.
The prove-of-concept is finished, now, based on the results, a project can start. In that context help of everyone is welcome: for testing, coding, documentation, and ... to find a better name (sounds that fapws is too close to fap which is not related to serious IT matters ;-) ).

Source code is here: http://www.opensource4you.com/cgi-bin/gitweb.cgi?p=fapws2;a=summary
tgz is here: http://www.opensource4you.com/downloads/


William (william _:at: opensource4you :dot:_ com)

PS:
Like for my previous post, prove (apache benchmark) will come soon.

(added on November, 10th)
ANNEXE
----------------
FAPWS2 performance on "hello world":
[root@myhost]# nice -n20 ab -n1000 -c10  http://127.0.0.1:8080/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Finished 1000 requests


Server Software:        fapws2/0.1
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /
Document Length:        13 bytes

Concurrency Level:      10
Time taken for tests:   1.313237 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      105000 bytes
HTML transferred:       13000 bytes
Requests per second:    761.48 [#/sec] (mean)
Time per request:       13.132 [ms] (mean)
Time per request:       1.313 [ms] (mean, across all concurrent requests)
Transfer rate:          77.67 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       2
Processing:     1   11  13.0      8     107
Waiting:        0   10  12.3      7     107
Total:          2   11  13.0      8     107

Percentage of the requests served within a certain time (ms)
  50%      8
  66%      8
  75%      8
  80%      9
  90%     33
  95%     40
  98%     41
  99%    106
 100%    107 (longest request)


FAPWS perofrmance on "Hello world"
[root@myhost]# nice -n20 ab -n1000 -c10  http://127.0.0.1:8080/hello
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Finished 1000 requests


Server Software:        fapws/0.7
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /hello
Document Length:        11 bytes

Concurrency Level:      10
Time taken for tests:   1.909155 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      112000 bytes
HTML transferred:       11000 bytes
Requests per second:    523.79 [#/sec] (mean)
Time per request:       19.092 [ms] (mean)
Time per request:       1.909 [ms] (mean, across all concurrent requests)
Transfer rate:          57.09 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        2    8  11.0      6     108
Processing:     3   10   9.5      8     110
Waiting:        0    6   8.9      5     108
Total:         13   18  13.3     14     112

Percentage of the requests served within a certain time (ms)
  50%     14
  66%     14
  75%     15
  80%     16
  90%     43
  95%     47
  98%     49
  99%    112
 100%    112 (longest request)


On this example, FAPWS2 is 45% quicker than FAPWS.

skipped back 10