Developers Club geek daily blog

1 year, 8 months ago
How to select a programming language?

Such question gave command of the Mail.Ru Mail before writing of the next service. A main objective of such choice — high performance of development process within selected languages/technologies. What influences this indicator?
  • Performance;
  • Existence of debugging tools and profiling;
  • The big community allowing to find quickly answers to questions;
  • Existence of the stable libraries and modules necessary for development of web applications;
  • The number of developers in the market;
  • Possibility of development in modern IDE;
  • Threshold of entry in language.

Besides, developers welcomed briefness and expressiveness of language. Laconicism, certainly, also influences efficiency of development as lack of kilogram weights on probability of success of the marathoner.

Basic data


Applicants


As many server mikrotask quite often are born in client part of mail, the first applicant is, of course, Node.js with its native JavaScript and V8 from Google.

After discussion and proceeding from preferences in command other participants of tender were defined: Scala, Go and Rust.

As performance test it was offered to write the simple HTTP server which receives from the general service of template making of HTML and gives to the client. Such task is dictated by the current realities of work of mail — all template making of client part happens on V8 by means of fest shablonizator.

When testing it became clear that all applicants work approximately with identical performance in such setting — everything rested against V8 performance. However implementation of a task was not superfluous — development on each of languages allowed to make considerable part of subjective appraisals which anyway could influence the final choice.

So, we have two scenarios. The first is just a greeting on root URL:
GET / HTTP/1.1
Host: service.host

HTTP/1.1 200 OK

Hello World!

The second — the client's greeting on his name transferred to ways of URL:
GET /greeting/user HTTP/1.1
Host: service.host

HTTP/1.1 200 OK

Hello, user

Environment


All tests were carried out on the VirtualBox virtual computer.

Host, MacBook Pro:
  • 2,6 GHz Intel Core i5 (dual core);
  • CPU Cache L1: 32 KB, L2: 256 KB, L3: 3 MB;
  • 8 GB 1600 MHz DDR3.

VM:
  • 4 GB RAM;
  • VT-x/aMD-v, PAE/NX, KVM.

Software:
  • CentOS 6.7 64bit;
  • Go 1.5.1;
  • Rustc 1.4.0;
  • Scala 2.11.7, sbt 0.13.9;
  • Java 1.8.0_65;
  • Node 5.1.1;
  • Node 0.12.7;
  • nginx 1.8.0;
  • wrk 4.0.0.

In addition to standard modules, in examples on Rust hyper, on Scala — spray was used. In Go and Node.js only native packets / modules were used.

Instruments of measurement


Performance of services was tested by means of the following tools:

In this article benchmarks of wrk and ab are considered.

Results


Performance


wrk

Data of the five-minute test, from 1000 connections and 50 flows are given below:
wrk -d300s -c1000 -t50 --timeout 2s http://service.host
Label Average Latency, ms Request, #/sec
Go 104,83 36 191,37
Rust 0,02906 32 564,13
Scala 57,74 17 182,40
Node 5.1.1 69,37 14 005,12
Node 0.12.7 86,68 11 125,37

wrk -d300s -c1000 -t50 --timeout 2s http://service.host/greeting/hello
Label Average Latency, ms Request, #/sec
Go 105,62 33 196,64
Rust 0,03207 29 623,02
Scala 55,8 17 531,83
Node 5.1.1 71,29 13 620,48
Node 0.12.7 90,29 10 681,11

So looking good, but, unfortunately, improbable digits in results of Average Latency at Rust testify to one feature which is present at the hyper module. All the matter is that parameter - with in wrk speaks about the number of connections which wrk will open on everyone rub and will not close, i.e. keep-alive of connections. Hyper works with keep-alive it is not absolutely expected — time, two.

Moreover, if to display through a Lua-script distribution of requests on the treda sent to wrk we will see what sends all requests only one rubs.

For the interested of Rust it is also worth noting that these features led here to what.

Therefore that the test was reliable, it was decided to carry out the similar test, having put before the nginx service which will hold connections with wrk and to proxy them in the necessary service:
upstream u_go {
    server 127.0.0.1:4002;
    keepalive 1000;
}

server {
        listen 80;
        server_name go;
        access_log off;

        tcp_nopush on;
        tcp_nodelay on;

        keepalive_timeout 300;
        keepalive_requests 10000;

        gzip off;
        gzip_vary off;

        location / {
                proxy_pass http://u_go;
        }
}

wrk -d300s -c1000 -t50 --timeout 2s http://nginx.host/service
Label Average Latency, ms Request, #/sec
Rust 155,36 9 196,32
Go 145,24 7 333,06
Scala 233,69 2 513,95
Node 5.1.1 207,82 2 422,44
Node 0.12.7 209,5 2 410,54

wrk -d300s -c1000 -t50 --timeout 2s http://nginx.host/service/greeting/hello
Label Average Latency, ms Request, #/sec
Rust 154,95 9 039,73
Go 147,87 7 427,47
Node 5.1.1 199,17 2 470,53
Node 0.12.7 177,34 2 363,39
Scala 262,19 2 218,22

Apparently from results, overhead with nginx is considerable, but in our case we are interested in performance of services which are on an equal footing, irrespective of nginx delay.

ab

The utility from Apache ab, unlike wrk, does not hold keep-alive of connections therefore nginx is not useful to us here. Let's try to execute 50 000 requests in 10 seconds, with 256 possible parallel queries.
ab -n50000 -c256 -t10 http://service.host/
Label Completed requests, # Time per request, ms Request, #/sec
Go 50 000,00 22,04 11 616,03
Rust 32 730,00 78,22 3 272,98
Node 5.1.1 30 069,00 85,14 3 006,82
Node 0.12.7 27 103,00 94,46 2 710,22
Scala 16 691,00 153,74 1 665,17

ab -n50000 -c256 -t10 http://service.host/greeting/hello
Label Completed requests, # Time per request, ms Request, #/sec
Go 50 000,00 21,88 11 697,82
Rust 49 878,00 51,42 4 978,66
Node 5.1.1 30 333,00 84,40 3 033,29
Node 0.12.7 27 610,00 92,72 2 760,99
Scala 27 178,00 94,34 2 713,59

It should be noted that some "warming up" because of possible optimization of JVM which occur during application operating time is characteristic of a Scala-application.

Apparently, without nginx hyper in Rust still badly copes even without keep-alive of connections. And only who managed to process 50 000 requests in 10 seconds there was Go.

Source code


Node.js
var cluster = require('cluster');
var numCPUs = require('os').cpus().length;
var http = require("http");
var debug = require("debug")("lite");
var workers = [];
var server;

cluster.on('fork', function(worker) {
    workers.push(worker);

    worker.on('online', function() {
        debug("worker %d is online!", worker.process.pid);
    });

    worker.on('exit', function(code, signal) {
        debug("worker %d died", worker.process.pid);
    });

    worker.on('error', function(err) {
        debug("worker %d error: %s", worker.process.pid, err);
    });

    worker.on('disconnect', function() {
        workers.splice(workers.indexOf(worker), 1);
        debug("worker %d disconnected", worker.process.pid);
    });
});

if (cluster.isMaster) {
    debug("Starting pure node.js cluster");

    ['SIGINT', 'SIGTERM'].forEach(function(signal) {
        process.on(signal, function() {
            debug("master got signal %s", signal);
            process.exit(1);
        });
    });

    for (var i = 0; i < numCPUs; i++) {
        cluster.fork();
    }
} else {
    server = http.createServer();

    server.on('listening', function() {
        debug("Listening %o", server._connectionKey);
    });

    var greetingRe = new RegExp("^\/greeting\/([a-z]+)$", "i");
    server.on('request', function(req, res) {
        var match;

        switch (req.url) {
            case "/": {
                res.statusCode = 200;
                res.statusMessage = 'OK';
                res.write("Hello World!");
                break;
            }

            default: {
                match = greetingRe.exec(req.url);
                res.statusCode = 200;
                res.statusMessage = 'OK';
                res.write("Hello, " + match[1]);    
            }
        }

        res.end();
    });

    server.listen(8080, "127.0.0.1");
}

Go
package main

import (
    "fmt"
    "net/http"
    "regexp"
)

func main() {
    reg := regexp.MustCompile("^/greeting/([a-z]+)$")
    http.ListenAndServe(":8080", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        switch r.URL.Path {
        case "/":
            fmt.Fprint(w, "Hello World!")
        default:
            fmt.Fprintf(w, "Hello, %s", reg.FindStringSubmatch(r.URL.Path)[1])
        }
    }))
}

Rust
extern crate hyper;
extern crate regex;

use std::io::Write;
use regex::{Regex, Captures};

use hyper::Server;
use hyper::server::{Request, Response};
use hyper::net::Fresh;
use hyper::uri::RequestUri::{AbsolutePath};

fn handler(req: Request, res: Response<Fresh>) {
    let greeting_re = Regex::new(r"^/greeting/([a-z]+)$").unwrap();

    match req.uri {
        AbsolutePath(ref path) => match (&req.method;, &path;[..]) {
            (&hyper;::Get, "/") => {
                hello(&req;, res);
            },
            _ => {
                greet(&req;, res, greeting_re.captures(path).unwrap());
            }
        },
        _ => {
            not_found(&req;, res);
        }
    };
}

fn hello(_: &Request;, res: Response<Fresh>) {
    let mut r = res.start().unwrap();
    r.write_all(b"Hello World!").unwrap();
    r.end().unwrap();
}

fn greet(_: &Request;, res: Response<Fresh>, cap: Captures) {
    let mut r = res.start().unwrap();
    r.write_all(format!("Hello, {}", cap.at(1).unwrap()).as_bytes()).unwrap();
    r.end().unwrap();
}

fn not_found(_: &Request;, mut res: Response<Fresh>) {
    *res.status_mut() = hyper::NotFound;
    let mut r = res.start().unwrap();
    r.write_all(b"Not Found\n").unwrap();
}

fn main() {
    let _ = Server::http("127.0.0.1:8080").unwrap().handle(handler);
}

Scala
package lite

import akka.actor.{ActorSystem, Props}
import akka.io.IO
import spray.can.Http
import akka.pattern.ask
import akka.util.Timeout
import scala.concurrent.duration._
import akka.actor.Actor
import spray.routing._
import spray.http._
import MediaTypes._
import org.json4s.JsonAST._

object Boot extends App {
  implicit val system = ActorSystem("on-spray-can")
  val service = system.actorOf(Props[LiteActor], "demo-service")
  implicit val timeout = Timeout(5.seconds)
  IO(Http) ? Http.Bind(service, interface = "localhost", port = 8080)
}

class LiteActor extends Actor with LiteService {
  def actorRefFactory = context
  def receive = runRoute(route)
}

trait LiteService extends HttpService {
  val route =
    path("greeting" / Segment) { user =>
      get {
        respondWithMediaType(`text/html`) {
          complete("Hello, " + user)
        }
      }
    } ~
    path("") {
      get {
        respondWithMediaType(`text/html`) {
          complete("Hello World!")
        }
      }
    }
}


Generalization


Let's provide the criteria of success in a type of the table defined at the beginning of article. All applicants have means of a debag and profiling therefore the corresponding columns in the table are absent.

Label Performance Rate0 Community size1 Packages count IDE Support Developers5
Go 100,00% 12 759 104 3832 + 315
Rust 89,23% 3 391 3 582 +4 21
Scala 52,81% 44 844 172 5933 + 407
Node 5.1.1 41,03% 102 328 215 916 + 654
Node 0.12.7 32,18% 102 328 215 916 + 654

0 Performance was considered on the basis of the five-minute wrk tests without nginx, in the RPS parameter.
1 The size of community was evaluated on an indirect sign — quantity of questions with the corresponding tag on StackOverflow.
2 Quantity of packets, indexed on godoc.org.
3 Very much approximately — search in the languages Java, Scala on github.com.
4 Under many favourite Idea there is no plug-in still.
5 According to hh.ru.

Visually can tell about the sizes of community here such diagrams of quantity of questions on tags in a day:

Go

How to select a programming language?

Rust

How to select a programming language?

Scala

How to select a programming language?

Node.js

How to select a programming language?

For comparison, PHP:

How to select a programming language?

Outputs


Understanding that performance benchmarks — a thing rather unsteady and ungrateful, it is difficult to draw some unambiguous conclusions only on the basis of such tests. Certainly, everything is dictated by type of a task which needs to be solved requirements to indicators of the program and other nuances of an environment.

In our case on set of the criteria defined above and, anyway, subjective views we selected Go.

Content of subjective appraisals was intentionally lowered in this article not to do next nabros and not to provoke holivar. Especially as if such estimates were not considered, then by the criteria stated above the result would remain the same.

How to select a programming language?

This article is a translation of the original post at habrahabr.ru/post/273341/
If you have any questions regarding the material covered in the article above, please, contact the original author of the post.
If you have any complaints about this article or you want this article to be deleted, please, drop an email here: sysmagazine.com@gmail.com.

We believe that the knowledge, which is available at the most popular Russian IT blog habrahabr.ru, should be accessed by everyone, even though it is poorly translated.
Shared knowledge makes the world better.
Best wishes.

comments powered by Disqus