PyPy powers up Python 3.5

Pythonistas will be happy to hear that new versions of PyPy, the runtime for the Python language that provides just-in-time compilation and a major speed boost, will now support versions 2.7 and 3.5 of Python, including the new language syntax features found in version 3.5.

Even more valuable to many developers: New versions of PyPy will have better support for popular packages that use C extensions, such as Numpy, in the Python 2.7-compatible version of the runtime.

How do you like me now?

PyPy’s big draw is that by merely replacing the stock CPython interpreter with PyPy, it speeds up Python applications—sometimes by an order of magnitude or more. But PyPy has also lagged behind the mainline version of Python in the language features it supports, meaning it could be a drop-in replacement for apps that don’t use cutting-edge features.

Development on the Python 3 branch of PyPy (PyPy3) has historically been behind the Python 2 branch. That changed last year when the Mozilla Foundation awarded PyPy’s developers $200,000 as part of its Open Source Initiative to further PyPy3 development. The investment seems to have paid off; PyPy3 now supports some of Python’s most recent innovations.

One key area where PyPy lagged was support for Python 3’s recent changes to the language’s syntax that make it easier to perform asynchronous programming. Those changes were made to the language in late 2015, and PyPy3 now supports that syntax at last. Likewise, asyncio, a module added in Python 3.4 to provide asynchronous behaviors (per the async/await syntax added to Python 3.5), works in the latest PyPy3 editions and provides a sizable performance boost according to PyPy’s benchmarks.

A few features from Python 3.6—such as f-strings, a simplified option for in-place string formatting—have even been backported into PyPy3’s current version. F-strings is not a universally beloved feature, but the fact it was added to the current PyPy3 edition is a sign that other features from future versions of Python could be back-adopted if needed.

Wanna C some tricks?

PyPy2, the Python 2.x-compatible branch of PyPy, hasn’t been stagnant. It now supports Python 2.7.13, the latest bug fix version of the language. But the most notable changes with PyPy2 are around support for C extensions for Python, which allows Python applications to use modules written in pure C for speed.

C extensions for Python don’t work with PyPy as-is. To get around this, PyPy uses an API compatibility layer called cpyext. The latest improvements to cpyext now allow many more C extensions to run as-is, including widely used math-and-stats packages like NumPy and Pandas.

That’s great news for those shops that have Python 2.x-only scripts written with those packages in mind. The bad news is that cpyext is something of a bottleneck, so a C extension that ran well in “vanilla” Python may be slower in PyPy, depending on the number of calls between PyPy and the C extension.

One solution is to port the underlying packages to PyPy. NumPyPy, for instance, is a fork of NumPy designed to take advantage of PyPy’s native methodologies for integrating with C code. But ports like NumPyPy stand in contrast to PyPy’s general philosophy of being a drop-in replacement for Python, so further work on cpyext seems like the best long-term strategy.

Starting in 2020, Python 2.x will no longer be supported. But many people still use Python 2.x applications, and PyPy—the first versions of which centered on 2.x—could become one of the few ways for them to receive support for that branch of Python, once official support dries up.