benchmarking / perfomance measuring

[slomo]

ideas:

ab
http://tsung.erlang-projects.org/

[mren]

siege :)

[mren]

I benchmarked the current version (b2a3b40) with 100 current connection and a small database (petra.osm)

siege http://localhost:8080/node -c 100


Transactions:               3308 hits
Availability:               100.00 %
Elapsed time:               20.48 secs
Data transferred:           4.15 MB
Response time:              0.08 secs
Transaction rate:           161.52 trans/sec
Throughput:                 0.20 MB/sec
Concurrency:                13.36
Successful transactions:    3308
Failed transactions:        0
Longest transaction:        1.05
Shortest transaction:       0.04

With 100 concurrent connections we have a success rate of 100%.
The average connection takes under 0.1 seconds.

Currently the process can't handle more 1000 concurrent connections.

siege http://localhost:8080/node -c 1000

Transactions:             716 hits
Availability:             30.85 %
Elapsed time:             10.82 secs
Data transferred:         0.90 MB
Response time:            1.58 secs
Transaction rate:         66.17 trans/sec
Throughput:               0.08 MB/sec
Concurrency:              104.79
Successful transactions:  716
Failed transactions:      1605
Longest transaction:      3.01
Shortest transaction:     0.00

[mren]

what are the requirements for benchmarking?

my suggestion is to compare our implementation with jxapi