Diversity amongst Python interpreters

We now have multiple Python interpreters and let's see what is in store.

Whenever we mention Python, we are mostly talking about the reference implementation CPython. CPython is a great project, but it is important to take stock of other implementations which are good alternatives and have some advantages. I only list the active projects below, as of 2019:

  • PyPy or rpython: nearly complete compliance with Python 3.5 and 3.6 standard library, faster
  • RustPython: the newest kid in the block. Can be compiled to WebAssembly. Potentially we might get Rust's memory management as garbage collectors.
  • GraalPython: based on Java GraalVM with a goal to support Python's standard library and scientific ecosystem.
  • Jython: embeds Java in Python 2.7
  • IronPython: embeds .NET frameworks in Python 2.7
  • MicroPython and CircuitPython: Python runtime and compiler for microcontrollers

To summarize there are three motivations behind these projects. To make Python:

  • easier to embed in other environments (browser, microcontrollers)
  • integrated with other languages
  • improve performance (removing GIL, adding JIT, faster eval)

There were some discussion on reducing1, reusing2, reorganising3, the standard libraries and even rewriting the standard library as pure-Python modules4. It is nice that such conversations are happenning as they can move the code base and sister projects forward by getting rid of baggage, not doing wheel reinventing and not breaking compatibility.

If there is anything I would like to see, it is more it is...

Performance!

Note that it is possible to get really good performance with CPython by writing extensions using Pythran, Numba etc. Let's see if how much faster can these alternate implementations be, based on a naive benchmark, loop hundred million times, and do nothing.

N = 100_000_000
for i in range(N):
    pass

and a slightly more efficient looping

from itertools import repeat

N = 100_000_000
for _ in repeat(None, N):
    pass

CPython

In [5]:
%%bash
time python -c '
N = 100_000_000
for i in range(N):
    pass
'
real	0m3.232s
user	0m3.231s
sys	0m0.000s
In [6]:
%%bash
time python -c '
from itertools import repeat

N = 100_000_000
for _ in repeat(None, N):
    pass
'
real	0m2.233s
user	0m2.233s
sys	0m0.000s

PyPy

In [7]:
%%bash
time pypy3 -c '
N = 100_000_000
for i in range(N):
    pass
'
real	0m0.433s
user	0m0.183s
sys	0m0.020s
In [8]:
%%bash
time pypy3 -c '
from itertools import repeat

N = 100_000_000
for _ in repeat(None, N):
    pass
'
real	0m0.255s
user	0m0.209s
sys	0m0.021s

RustPython

In [18]:
%%bash
time rustpython -c '
N = 1000000
for i in range(N):
    pass
'
real	0m3.308s
user	0m3.297s
sys	0m0.010s
In [19]:
%%bash
time rustpython -c '
from itertools import repeat

N = 1000000
for _ in repeat(None, N):
    pass
'
real	0m12.876s
user	0m12.865s
sys	0m0.010s

RustPython is surpisingly slow at the moment, so we don't do 100 million iterations and only a million instead.

GraalPython

In [27]:
%%bash
time graalpython -c '
N = 100_000_000
for i in range(N):
    pass
'
Please note: This Python implementation is in the very early stages, and can run little more than basic benchmarks at this point.

real	0m6.894s
user	0m6.829s
sys	0m0.190s
In [28]:
%%bash
time graalpython -c '
from itertools import repeat

N = 100_000_000
for _ in repeat(None, N):
    pass
'
Please note: This Python implementation is in the very early stages, and can run little more than basic benchmarks at this point.

real	0m5.960s
user	0m5.909s
sys	0m0.200s

Final comments

And the winners are...

  1. PyPy
  2. CPython
  3. GraalPython
  4. RustPython

I would love to use PyPy as my daily driver, but the only reason I couldn't do it is because I almost never manage to get packages like numpy working. Although PyPy claims otherwise.

Some eyecandy...

It is also interesting how the prompt looks :)

In [22]:
!python
Python 3.7.4 (default, Jul 16 2019, 07:12:58) 
[GCC 9.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyboardInterrupt
>>> 
In [23]:
!pypy3
Python 3.6.1 (784b254d669919c872a505b807db8462b6140973, May 09 2019, 13:17:30)
[PyPy 7.1.1-beta0 with GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
Jedi is not installed, falling back to readline
And now for something completely different: ``PyPy is an exciting technology
that lets you to write fast, portable, multi-platform interpreters with less
effort''
=
>>>> 
>
KeyboardInterrupt
=
>>>> 
>=
>>>> 
In [24]:
!rustpython
Welcome to the magnificent Rust Python 0.1.0 interpreter 😱 🖖
>>>>> 
^C

>>>>> 
In [25]:
!graalpython
Python 3.7.3 (Sat Jul 13 09:46:34 UTC 2019)
[GraalVM CE, Java 1.8.0_222] on linux
Type "help", "copyright", "credits" or "license" for more information.
Please note: This Python implementation is in the very early stages, and can run little more than basic benchmarks at this point.
>>> 
>>> 

Versions

I have used the latest and greatest releases. For future reference:

In [26]:
%%bash
python -V
pypy3 -V
rustpython -V
graalpython -V
Python 3.7.4
Python 3.6.1 (784b254d669919c872a505b807db8462b6140973, May 09 2019, 13:17:30)
[PyPy 7.1.1-beta0 with GCC 8.3.0]
RustPython 0.1.0
Python 3.7.3 (GraalVM CE Native 19.1.1)

You can download this notebook, or see a static view on nbviewer.