142 lines
6.0 KiB
Plaintext
142 lines
6.0 KiB
Plaintext
EventMachine now supports epoll, bringing large increases in performance and scalability to Ruby programs.
|
|
|
|
Epoll(7) is a alternative mechanism for multiplexed I/O that is available in Linux 2.6 kernels.
|
|
It features significantly greater performance than the standard select(2) mechanism, when used in
|
|
applications that require very large numbers of open I/O descriptors.
|
|
|
|
EventMachine has always used select(2) because its behavior is well standardized and broadly supported.
|
|
But select becomes unreasonably slow when a program has a
|
|
very large number of file descriptors or sockets. Ruby's version of select hardcodes a limit
|
|
of 1024 descriptors per process, but heavily loaded processes will start to show performance
|
|
degradation even after only a few hundred descriptors are in use.
|
|
|
|
Epoll is an extended version of the poll(2) call, and it solves the problems with select. Programs
|
|
based on epoll can easily scale past Ruby's 1024-descriptor limit, potentially to tens of thousands
|
|
of connectors, with no significant impact on performance.
|
|
|
|
(Another alternative which is very similar to epoll in principle is kqueue, supplied on BSD and its
|
|
variants.)
|
|
|
|
|
|
|
|
This note shows you how to use epoll in your programs.
|
|
|
|
=== Compiling EventMachine to use epoll.
|
|
|
|
You don't have to do anything to get epoll support in EventMachine.
|
|
When you compile EventMachine on a platform that supports epoll, EM will
|
|
automatically generate a Makefile that includes epoll. (At this writing, this will only work
|
|
on Linux 2.6 kernels.) If you compile EM on a platform without epoll, then epoll support will
|
|
be omitted from the Makefile, and EM will work just as it always has.
|
|
|
|
=== Using epoll in your programs.
|
|
|
|
First, you need to tell EventMachine to use epoll instead of select (but see below, as this requirement
|
|
will be removed in a future EventMachine version). Second, you need to prepare your program to use
|
|
more than 1024 descriptors, an operation that generally requires superuser privileges. Third, you will probably
|
|
want your process to drop the superuser privileges after you increase your process's descriptor limit.
|
|
|
|
=== Using EventMachine#epoll
|
|
|
|
Call the method EventMachine#epoll anytime before you call EventMachine#run, and your program will
|
|
automatically use epoll, if available. It's safe to call EventMachine#epoll on any platform because
|
|
it compiles to a no-op on platforms that don't support epoll.
|
|
|
|
require 'rubygems'
|
|
require 'eventmachine'
|
|
|
|
EM.epoll
|
|
EM.run {
|
|
...
|
|
}
|
|
|
|
|
|
EventMachine#epoll was included in this initial release only to avoid changing the behavior of existing
|
|
programs. However, it's expected that a future release of EM will convert EventMachine#epoll to a no-op,
|
|
and run epoll by default on platforms that support it.
|
|
|
|
=== Using EventMachine#set_descriptor_table_size
|
|
|
|
In Linux (as in every Unix-like platform), every process has a internal table that determines the maximum
|
|
number of file and socket descriptors you may have open at any given time. The size of this table is
|
|
generally fixed at 1024, although it may be increased within certain system-defined hard and soft limits.
|
|
|
|
If you want your EventMachine program to support more than 1024 total descriptors, you must use
|
|
EventMachine#set_descriptor_table_size, as follows:
|
|
|
|
require 'rubygems'
|
|
require 'eventmachine'
|
|
|
|
new_size = EM.set_descriptor_table_size( 60000 )
|
|
$>.puts "New descriptor-table size is #{new_size}"
|
|
|
|
EM.run {
|
|
...
|
|
}
|
|
|
|
If successful, this example will increase the maximum number of descriptors that epoll can use to 60,000.
|
|
Call EventMachine#set_descriptor_table_size without an argument at any time to find out the current
|
|
size of the descriptor table.
|
|
|
|
Using EventMachine#set_descriptor_table_size ONLY affects the number of descriptors that can be used
|
|
by epoll. It has no useful effect on platforms that don't support epoll, and it does NOT increase the
|
|
number of descriptors that Ruby's own I/O functions can use.
|
|
|
|
#set_descriptor_table_size can fail if your process is not running as superuser, or if you try to set a
|
|
table size that exceeds the hard limits imposed by your system. In the latter case, try a smaller number.
|
|
|
|
|
|
=== Using EventMachine#set_effective_user
|
|
|
|
In general, you must run your program with elevated or superuser privileges if you want to increase
|
|
your descriptor-table size beyond 1024 descriptors. This is easy enough to verify. Try running the
|
|
sample program given above, that increases the descriptor limit to 60,000. You will probably find that
|
|
the table size will not be increased if you don't run your program as root or with elevated privileges.
|
|
|
|
But of course network servers, especially long-running ones, should not run with elevated privileges.
|
|
You will want to drop superuser privileges as soon as possible after initialization. To do this,
|
|
use EventMachine#set_effective_user:
|
|
|
|
require 'rubygems'
|
|
require 'eventmachine'
|
|
|
|
# (Here, program is running as superuser)
|
|
|
|
EM.set_descriptor_table_size( 60000 )
|
|
EM.set_effective_user( "nobody" )
|
|
# (Here, program is running as nobody)
|
|
|
|
EM.run {
|
|
...
|
|
}
|
|
|
|
Of course, you will need to replace "nobody" in the example with the name of an unprivileged user
|
|
that is valid on your system. What if you want to drop privileges after opening a server socket
|
|
on a privileged (low-numbered) port? Easy, just call #set_effective_user after opening your sockets:
|
|
|
|
require 'rubygems'
|
|
require 'eventmachine'
|
|
|
|
# (Here, program is running as superuser)
|
|
|
|
EM.set_descriptor_table_size( 60000 )
|
|
|
|
EM.run {
|
|
EM.start_server( "0.0.0.0", 80, MyHttpServer )
|
|
EM.start_server( "0.0.0.0", 443, MyEncryptedHttpServer )
|
|
|
|
EM.set_effective_user( "nobody" )
|
|
# (Here, program is running as nobody)
|
|
|
|
...
|
|
}
|
|
|
|
|
|
Because EventMachine#set_effective_user is used to enforce security
|
|
requirements, it has no nonfatal errors. If you try to set a nonexistent or invalid effective user,
|
|
#set_effective_user will abort your program, rather than continue to run with elevated privileges.
|
|
|
|
EventMachine#set_effective_user is a silent no-op on platforms that don't support it, such as Windows.
|
|
|
|
|