Perf. Test of Phoenix(Elixir), Phalcon(PHP 7) and Go
Being an avid learner and a tech perfectionist for more than a decade I like to run various performance tests. And although I agree that all tests are subjective and skewed, sometimes there are grains of truth. I’m involved in several big PHP projects(see Homediary.com) on my day-to-day job where we use Phalcon - a clean, super fast framework and I also learn Elixir and Go. PHP got a serious performance boost with the recent release of PHP 7. Hearing only awesome stories about Phoenix/Elixir performance and with Go on the rise I decided to look for any existing projects measuring the performance of the above mentioned web frameworks.
Luckily, there was PETE, which stands for PErformance TEst of different web frameworks: phoenix (elixir), slim (PHP), phalcon (PHP). It is a simple single page web app showing a list of images. Nevertheless it consists of a controller, a view and a template: the View/Controller part of the MVC stack. That’s exactly what can be evaluated from the performance point of view. The project didn’t have a Go version of the app so I had to code one. I have submitted a pull request to have it included in the original repo.
I did my own test on three m4.xlarge AWS EC2 instances: one instance was used for wrk, another instance was used for nginx/Php-fpm/phalcon setup and the last instance was used for Elixir/Erlang stack. I reused the PHP instance to test Go version of the app by stopping nginx and php-fpm and launching the Go app.
I repeated every wrk launch several times to ensure consistency in numbers. The results are quite surprising. I expected to see a tight fight between Phoenix/Elixir and Go/stdlib but instead the Go version has been surpassed by Phalcon/PHP! In fact, the Go version of the gallery app is the slowest among the three.
Phalcon 3.0.2 with PHP 7
wrk -t4 -c100 -d60s --timeout 2000 http://35.162.148.251/gallery
Running 1m test @ http://35.162.148.251/gallery
4 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 14.00ms 5.40ms 227.06ms 90.31%
Req/Sec 1.82k 58.01 2.06k 74.92%
435522 requests in 1.00m, 772.52MB read
Requests/sec: 7255.99
Transfer/sec: 12.87MB
Phoenix 1.2.0 with Erlang/OTP 19
wrk -t4 -c100 -d60s --timeout 2000 http://35.160.152.71/gallery
Running 1m test @ http://35.160.152.71/gallery
4 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 5.80ms 4.30ms 228.66ms 74.56%
Req/Sec 4.56k 546.57 9.26k 74.86%
1091298 requests in 1.00m, 2.05GB read
Requests/sec: 18158.02
Transfer/sec: 34.95MB
Go 1.7.3
wrk -t4 -c100 -d60s --timeout 2000 http://35.162.148.251/gallery
Running 1m test @ http://35.162.148.251/gallery
4 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 19.65ms 19.01ms 223.41ms 85.76%
Req/Sec 1.57k 144.09 2.13k 68.25%
375624 requests in 1.00m, 651.61MB read
Requests/sec: 6256.53
Transfer/sec: 10.85MB
It is easy to see that Phalcon serves 7255 requests/sec but that pales in comparison with Phoenix’s 18158 requests/sec (served with only 5.8ms avg latency!!). The Go version Go is the slowest with only 6256 requests/sec despite Go being a compiled language.
Update
Performance of the Go version can be improved by almost 5x by moving template parsing outside of the handler function so that templates are only parsed once during app initialization. I’m not 100% sure but I think that’s how other frameworks deal with templates too, so that should be fair comparison. The updated results for Go look like this:
wrk -t4 -c100 -d60s --timeout 2000 http://35.162.251.83/gallery
Running 1m test @ http://35.162.251.83/gallery
4 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 4.49ms 3.66ms 218.68ms 84.18%
Req/Sec 5.93k 407.07 23.34k 95.59%
1416365 requests in 1.00m, 2.40GB read
Requests/sec: 23567.17
Transfer/sec: 40.88MB
This makes Go/stdlib the winner, Elixir/Phoenix the runner up (23% fewer requests than Go) and PHP/Phalcon taking the last place.
Update 2
My smart readers told me net/http can’t do connection pooling and is not a very fast HTTP server in general and advised to use valyala/fasthttp. Well, the Go version of the app with fasthttp has just blown my socks off with a whopping 28816 requests/sec, a 22% increase!
wrk -t4 -c100 -d60s --timeout 2000 http://35.163.173.123/gallery
Running 1m test @ http://35.163.173.123/gallery
4 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 3.74ms 3.43ms 211.54ms 90.18%
Req/Sec 7.24k 237.32 7.90k 68.79%
1729095 requests in 1.00m, 2.96GB read
Requests/sec: 28816.34
Transfer/sec: 50.46MB