Interrupted system calls

On POSIX systems, signals are common. Code calling system calls must be prepared to handle them. Examples of signals:

• The most common signal is SIGINT, the signal sent when CTRL+c is pressed. By default, Python raises a KeyboardInterrupt exception when this signal is received.
• When running subprocesses, the SIGCHLD signal is sent when a child process exits.
• Resizing the terminal sends the SIGWINCH signal to the applications running in the terminal.
• Putting the application in background (ex: press CTRL-z and then type the bg command) sends the SIGCONT signal.

Writing a C signal handler is difficult: only “async-signal-safe” functions can be called (for example, printf() and malloc() are not async-signal safe), and there are issues with reentrancy. Therefore, when a signal is received by a process during the execution of a system call, the system call can fail with the EINTR error to give the program an opportunity to handle the signal without the restriction on signal-safe functions.

This behaviour is system-dependent: on certain systems, using the SA_RESTART flag, some system calls are retried automatically instead of failing with EINTR. Regardless, Python’s signal.signal() function clears the SA_RESTART flag when setting the signal handler: all system calls will probably fail with EINTR in Python.

Since receiving a signal is a non-exceptional occurrence, robust POSIX code must be prepared to handle EINTR (which, in most cases, means retry in a loop in the hope that the call eventually succeeds). Without special support from Python, this can make application code much more verbose than it needs to be.

Status in Python 3.4

In Python 3.4, handling the InterruptedError exception (EINTR’s dedicated exception class) is duplicated at every call site on a case-by-case basis. Only a few Python modules actually handle this exception, and fixes usually took several years to cover a whole module. Example of code retrying file.read() on InterruptedError:

while True:
    try:
        data = file.read(size)
        break
    except InterruptedError:
        continue

List of Python modules in the standard library which handle InterruptedError:

• asyncio
• asyncore
• io, _pyio
• multiprocessing
• selectors
• socket
• socketserver
• subprocess

Other programming languages like Perl, Java and Go retry system calls failing with EINTR at a lower level, so that libraries and applications needn’t bother.

Use Case 1: Don’t Bother With Signals

In most cases, you don’t want to be interrupted by signals and you don’t expect to get InterruptedError exceptions. For example, do you really want to write such complex code for a “Hello World” example?

while True:
    try:
        print("Hello World")
        break
    except InterruptedError:
        continue

InterruptedError can happen in unexpected places. For example, os.close() and FileIO.close() may raise InterruptedError: see the article close() and EINTR.

The Python issues related to EINTR section below gives examples of bugs caused by EINTR.

The expectation in this use case is that Python hides the InterruptedError and retries system calls automatically.

Use Case 2: Be notified of signals as soon as possible

Sometimes yet, you expect some signals and you want to handle them as soon as possible. For example, you may want to immediately quit a program using the CTRL+c keyboard shortcut.

Besides, some signals are not interesting and should not disrupt the application. There are two options to interrupt an application on only some signals:

• Set up a custom signal handler which raises an exception, such as KeyboardInterrupt for SIGINT.
• Use a I/O multiplexing function like select() together with Python’s signal wakeup file descriptor: see the function signal.set_wakeup_fd().

The expectation in this use case is for the Python signal handler to be executed timely, and the system call to fail if the handler raised an exception – otherwise restart.

Modified functions

Example of standard library functions that need to be modified to comply with this PEP:

• open(), os.open(), io.open()
• functions of the faulthandler module
• os functions:
  • os.fchdir()
  • os.fchmod()
  • os.fchown()
  • os.fdatasync()
  • os.fstat()
  • os.fstatvfs()
  • os.fsync()
  • os.ftruncate()
  • os.mkfifo()
  • os.mknod()
  • os.posix_fadvise()
  • os.posix_fallocate()
  • os.pread()
  • os.pwrite()
  • os.read()
  • os.readv()
  • os.sendfile()
  • os.wait3()
  • os.wait4()
  • os.wait()
  • os.waitid()
  • os.waitpid()
  • os.write()
  • os.writev()
  • special cases: os.close() and os.dup2() now ignore EINTR error, the syscall is not retried
• select.select(), select.poll.poll(), select.epoll.poll(), select.kqueue.control(), select.devpoll.poll()
• socket.socket() methods:
  • accept()
  • connect() (except for non-blocking sockets)
  • recv()
  • recvfrom()
  • recvmsg()
  • send()
  • sendall()
  • sendmsg()
  • sendto()
• signal.sigtimedwait(), signal.sigwaitinfo()
• time.sleep()

(Note: the selector module already retries on InterruptedError, but it doesn’t recompute the timeout yet)

os.close, close() methods and os.dup2() are a special case: they will ignore EINTR instead of retrying. The reason is complex but involves behaviour under Linux and the fact that the file descriptor may really be closed even if EINTR is returned. See articles:

• Returning EINTR from close()
• (LKML) Re: [patch 7/7] uml: retry host close() on EINTR
• close() and EINTR

The socket.socket.connect() method does not retry connect() for non-blocking sockets if it is interrupted by a signal (fails with EINTR). The connection runs asynchronously in background. The caller is responsible to wait until the socket becomes writable (ex: using select.select()) and then call socket.socket.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR) to check if the connection succeeded (getsockopt() returns 0) or failed.

InterruptedError handling

Since interrupted system calls are automatically retried, the InterruptedError exception should not occur anymore when calling those system calls. Therefore, manual handling of InterruptedError as described in Status in Python 3.4 can be removed, which will simplify standard library code.

Wakeup file descriptor

Since Python 3.3, signal.set_wakeup_fd() writes the signal number into the file descriptor, whereas it only wrote a null byte before. It becomes possible to distinguish between signals using the wakeup file descriptor.

Linux has a signalfd() system call which provides more information on each signal. For example, it’s possible to know the pid and uid who sent the signal. This function is not exposed in Python yet (see issue 12304).

On Unix, the asyncio module uses the wakeup file descriptor to wake up its event loop.

Multithreading

A C signal handler can be called from any thread, but Python signal handlers will always be called in the main Python thread.

Python’s C API provides the PyErr_SetInterrupt() function which calls the SIGINT signal handler in order to interrupt the main Python thread.

Signals on Windows

Links