william_os4y ([info]william_os4y) wrote,
@ 2007-10-08 18:35:00
Previous Entry  Add to memories!  Tell a Friend!  Next Entry
Libevent has pure http API
It's now long time since my last post.

This surely does not means that I'm stopping FAPWS, at contrary I'm more and more convinced that one of the beauty of Python reside in his gluing capabilities.
This observation is probably not new to lot of people, but for my personal case it's better now than never ;-), no ?

After that small philosophical intro, please go back to our main interest: integrate libevent within the python world.
What's great with libevent is that since several release, they have introduced several HTTP low level APIs: send_reply, send_error, ...
I've tried to use them thanks to pyrex, ctype and swig.

With every of those tools I'm facing issues, problems, ... to get a webserver working correctly.
Because I want FAPWS to be light, efficient and stable, I don't like the idea to add layers on which I don't have too much control. Hoooooo, I'm already hearing some voices ;-). The point is not linked with those tools, but more to my competencies to be able to have them working correctly. Indeed, when you are new to C (which is my case) and try to understand how to use swig, trust me, this is not piece of cake.
Thus I'll use the simple, low level Python API for this project. This will be a way for me to learn C and C API of Python.


Will it be easy to learn C from Python ?

We will see.
Any how, any helps are welcome ;-).





Here after, you can see a minimalist webserver serving the famous "Hello World!":
import ctypes
import sys

libevent=ctypes.cdll.LoadLibrary('libevent.so')

def root_handler(req, *arg):
    #print "got request:", req
    libevent.evbuffer_new.restype=ctypes.c_void_p
    #Build a buffer and put page content into it
    buf=libevent.evbuffer_new()
    libevent.evbuffer_add_printf.argtypes=[ctypes.c_void_p, ctypes.c_char_p]
    libevent.evbuffer_add_printf(buf,"Hello world!\n\n")
    #send the buffer with a return code of 200
    libevent.evhttp_send_reply.argtypes=[ctypes.c_void_p, ctypes.c_int, ctypes.c_char_p, ctypes.c_void_p]
    libevent.evhttp_send_reply(req, 200, "OK", buf)
    return 1


    
def main():
    #We just initialise and give the server address and port
    libevent.event_init()
    libevent.evhttp_start.argtypes=[ctypes.c_char_p, ctypes.c_short]
    libevent.evhttp_start.restype=ctypes.c_void_p
    http=libevent.evhttp_start('0.0.0.0',8080)
    #we lik the call to root_handler at a request of "/"
    FUNC=ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_void_p)
    libevent.evhttp_set_cb.argtypes=[ctypes.c_void_p, ctypes.c_char_p, ctypes.c_void_p]
    libevent.evhttp_set_cb(http,"/",FUNC(root_handler), None)
    #we loop and wait events ;-)
    libevent.event_dispatch()
    #we close the loop
    print "end loop main" 
    libevent.evhttp_free.argtypes=[ctypes.c_void_p]
    libevent.evhttp_free(http)

if __name__=="__main__":
    main()

Note: I don't know exsactly why, but the server does not stop when you hit "-c". You must kill it ;-(.

Amazingly, I'm able to get 1700#/sec on my 1.2Ghz Athlon.



Annexes:

[myhost ~]$ ab -n2000 -c10 http://127.0.0.1:8080/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)

Server Software:
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /
Document Length:        14 bytes

Concurrency Level:      10
Time taken for tests:   1.170227 seconds
Complete requests:      2000
Failed requests:        0
Write errors:           0
Total transferred:      196000 bytes
HTML transferred:       28000 bytes
Requests per second:    1709.07 [#/sec] (mean)
Time per request:       5.851 [ms] (mean)
Time per request:       0.585 [ms] (mean, across all concurrent requests)
Transfer rate:          163.22 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       3
Processing:     2    5   2.5      4      15
Waiting:        1    5   2.6      4      15
Total:          2    5   2.5      4      15

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      7
  80%      7
  90%      9
  95%     11
  98%     13
  99%     14
 100%     15 (longest request)
[myhost ~]$             



(Post a new comment)

Very intereasting
(Anonymous)
2007-10-08 05:58 pm UTC (link)
This was really cool. I had to use libevent=ctypes.cdll.LoadLibrary('libevent-1.3b.1.0.3.dylib') because I am running on a mac book pro. But the performance I got was closer 8K req/sec. I would love to see a version of FAWPS using this :). WSGI server support that was async would be really nice.
Lateef

PS: Below is my ab report

[myhost -]$ ab -n2000 -c100 http://127.0.0.1:8080/
This is ApacheBench, Version 1.3d <$Revision: 1.73 $> apache-1.3
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 200 requests
Completed 400 requests
Completed 600 requests
Completed 800 requests
Completed 1000 requests
Completed 1200 requests
Completed 1400 requests
Completed 1600 requests
Completed 1800 requests
Finished 2000 requests
Server Software:
Server Hostname: 127.0.0.1
Server Port: 8080

Document Path: /
Document Length: 14 bytes

Concurrency Level: 100
Time taken for tests: 0.306 seconds
Complete requests: 2000
Failed requests: 22
(Connect: 15, Length: 7, Exceptions: 0)
Broken pipe errors: 0
Total transferred: 194824 bytes
HTML transferred: 27832 bytes
Requests per second: 6535.95 [#/sec] (mean)
Time per request: 15.30 [ms] (mean)
Time per request: 0.15 [ms] (mean, across all concurrent requests)
Transfer rate: 636.68 [Kbytes/sec] received

Connnection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.4 0 4
Processing: 0 2 1.0 2 8
Waiting: 0 2 1.1 2 8
Total: 0 2 1.3 2 11

Percentage of the requests served within a certain time (ms)
50% 2
66% 2
75% 2
80% 3
90% 4
95% 5
98% 9
99% 9
100% 11 (last request)

(Reply to this)

signal handling in libevent
(Anonymous)
2007-11-04 05:55 pm UTC (link)
You wrote:
> Note: I don't know exsactly why, but the server does not stop when you hit "-c". You must kill it ;-(.

Well, it's because libevent catches signals.

For libevent-python you should add handler using command:
libevent.createSignalHandler(signal.SIGINT, callback_interrupt).addToLoop()

See example on my blog:
http://majek4.blogspot.com/2007/11/libevent-under-python.html

I would give link to libevent-python, but the project seems to be down and unmaintained.

majek

(Reply to this)


[info]aleksandyr
2008-02-13 08:17 am UTC (link)
So this is by far the best libevent resource I've found. :)

I'm having trouble getting anything, though: according to the libevent docs, req should be a struct pointer that has all of the request information --- none of my arguments are getting through (arg is always empty in the callback.)

Is there something I'm missing?

(Reply to this) (Thread)


[info]william_os4y
2008-02-13 04:32 pm UTC (link)
you can send your code to my mailbox (william -at- opensource4you -dot- com, I'll check it.

(Reply to this) (Parent)


(Anonymous)
2008-08-04 08:15 pm UTC (link)
It's too bad you only tried Pyrex and not Cython. Would have been a lot easier.

(Reply to this)


Create an Account
Forgot your login?
Login w/ OpenID
English • Español • Deutsch • Русский…