Shed Skin restricted-Python-to-C++ transpiler 0.9.13

I have just released version 0.9.13 of Shed Skin, a restricted-python to C++ transpiler. Shed Skin allows one to effectively convert (or transpile) pure Python code to highly optimized machine code. This comes at the cost though of having to conform to a seriously restricted subset of Python features/libraries. Programs currently also cannot be too large, although it is possible to generate extension modules, that can be used in larger programs.

The following screenshot is of a DOOM engine, that becomes more than 30 times faster after transpilation. The engine is compiled with Shed Skin, then imported in a larger program that uses Pygame for the UI (see also a before-after video).

Version 0.9.13 is the first version that was heavily improved by the use of AI (more specifically, Claude - thanks to Shakeeb for starting this). It was able to spot (and fix) many bugs, especially in the C++ backend, but also test gaps. It also helped to add support for many missing features in Python 3.15. We are now actually pretty close to full compatibility there, at least with regards to the supported modules and everything that is compatible with Shed Skin (no os.walk yet, for example, as it uses heterogenous tuples of length > 2, something we may fix for 0.9.14).

The use of AI should make it much easier to contribute as well. For example, if your program doesn't work, AI can minimize it and produce a useful bug report to submit to the project. It can even start to try and fix the problem, create new tests and so on. Or otherwise look for gaps/bugs to work on.

Shed Skin restricted-Python-to-C++ compiler v0.9.11 released

I have just released version 0.9.11 of Shed Skin, a restricted-Python-to-C++ compiler. Most importantly, it adds support for Python 3.14. It also adds support for many 3.x features that were not yet implemented, in addition to basic support for the base64 module. It also optimizes a few more common code patterns.

Paul Boddie was able to add support for libpcre2, and in the process updated conan to version 2. Thanks to Shakeeb and now Paul, Shed Skin has had first-class Windows support for the last few releases.

A new release is often triggered by a nice new example. In this case I found an advanced/educational 3d renderer by Benny Bobaganoosh, and rewrote it from Java to Python. In ~500 lines of code, it renders an .obj file with perspective-correct texture mapping and so on, clipping, lighting.. It becomes about 13 times faster after compilation (in other words, it goes from about 2 to about 30 FPS).

For the full list of changes in the release, please see the release notes.

Something I have noticed while working on this release is that small object allocations seem to have become faster under Linux, to the degree that programs that would become _slower_ after compilation because of excessive small-object allocation, are now usually _faster_ again, at least on my system. This motivated me to measure the speedup for all 84 example programs at the moment versus cpython 3.13. While it's still all over the place, I was happy to see a median speedup of 12 times, and an average of 20 times.

I would very much appreciate more feedback on/assistance with the project. There is always enough low-hanging fruit to help with! See for example the current list of issues for 0.9.12. But just testing random things, finding interesting new example programs, cleaning up parts of the code and such are also much appreciated.

Shed Skin restricted-Python-to-C++ compiler 0.9.9

I have just released version 0.9.9 of Shed Skin, a restricted-Python-to-C++ compiler. It comes with many changes under the hood to improve the code base. For example, Shakeeb has started adding type annotations to Shed Skin itself, whereas I have fixed many C++ compiler warnings. We have also replaced the old CPython-based dict and set implementations with STL unordered_map and unordered_set. While this may cost some performance for especially small toy programs, it really helps with maintainability.

There was also a type inference improvement, which hasn't happened in a while since it wasn't needed. It was however needed to add a cool new example called "pycsg", which shows how one can perform CSG (constructive solid geometry) using binary space partitioning (BSP). After some investigation I decided to make the type inference slightly less "optimistic", which should make it scale a lot better, at the cost of being slower overall. In the end this new example runs about 15 times faster after compilation on my system (not 15%.. 15 times!).

Most notably on the outside, Shed Skin now comes with --floatXX/--intXXX options, which allow you to specify the desired float/int precision. Unfortunately under windows, --int128 does not yet work, since C++ has not yet standardized 128-bit integers.. The relatively new othello2 and collatz examples now require --int64 and --int128, respectively, to function properly.

For the full list of changes, please see the release notes.

I would like to take this opportunity to (once again) invite others to help out in Shed Skin development. There is always enough to do, also low-hanging fruit, on both the Python and the C++ side of things. I at least think it is rather fun to work on.. :) Let me know if you'd like to contribute but aren't sure what you could bring to the table.

Of course just sending in feedback or new example programs (especially if they fail!) can be very motivating as well to keep improving things.

LEGO Othello board

I have a weird bucket list. One remaining item was to create a full-size, playable Othello board, with ridges to avoid discs from moving around. I finally decided to try and mimic an existing board. The result looks pretty close to the original:

Unfortunately I was unable to figure out how to create smooth (and yet flat) discs, but they are at least quite playable. Suggestions for improvements are welcome. I'd also be happy to share more information or even a parts list anyone would like to recreate it.

Shed Skin restricted-Python-to-C++ compiler 0.9.7

I have just released version 0.9.7 of Shed Skin, a restricted-Python-to-C++ compiler. The highlight of this release is native Windows support, by moving to CMake (and Conan). Shakeeb Alireza came up with this idea and did most of the implementation. So to build something on Windows, one doesn't need a weird emulation layer like MingW anymore, just Visual Studio build tools, CMake and Conan (popular tools that many will already have installed). He also made sure that most things work under OSX. Here is a screenshot of an (also) new DOOM-engine example running under windows:

With the recent Python 3 and now native Windows support (plus an ever growing Python community), I'm hoping the project will see more usage and feedback/contributions over the coming years. Please consider joining the project, as there is always enough to do!

Besides other contributions from Folkert van Heusden and Humhue, the release comes with many other improvements, such as support for relative imports, the walrus operator and struct.pack_{from, into}. See here for the full release notes.

Fast DOOM WAD renderer in 999 lines of Python

For the longest time, I've wanted to re-implement the original DOOM engine in Python, and compile it with Shedskin to get reasonable performance. So when I finally ran across a pretty small engine written in Java, by Leonardo Ono, I decided to convert his version to Python.

I left out some optimizations (most notably "visplanes"), for readability and in order to keep everything under 1000 lines of code (while following PEP8). So it could run quite a bit faster still. This is including a WAD loader and interactive Pygame wrapper, that makes it almost feel like you are playing DOOM.

It's interesting how using Python (using just CPython) seems to be about as fast as using assembler back in the day.

This video shows the day-and-night difference in FPS after compilation using Shedskin.

The source code can be found in the Shedskin examples directory.

I may revisit the code now and then, to try and improve readability by adding comments and/or refactoring parts. Patches in this direction very much welcome of course.

Shed Skin restricted-Python-to-C++ compiler 0.9.6

I recently decided to sit down and finally port Shed Skin, an experimental restricted-Python-to-C++ compiler in the works since 2005 or so, to Python3. Three painful months and a total diff of 50k lines later, everything now works with Python3 (Shed Skin itself, and all tests and examples..)

This does not mean that every Python3 feature is supported, but what was there now at least works fine with Python3.. For example, unicode is still restricted to 1-byte characters, and there is no support (yet) for nice new features such as f-strings. Python2 support has been dropped with the new release, and subsequent releases should add support for various new Python3 features.

The following people have contributed along the way:

• Johan Kristensen (large patch for moving from compiler.ast to ast, still on Python2)
• Shakeeb Alireza (extension module support for Python3, ported many examples, improved OSX support, and various code cleanups)
• Folkert van Heusden (some fixes on the C++ side, move to c++17)
• Jeremie Roquet, Thomas Spura, Paul Boddie and others who kept doing maintenance on the project in my absence

I started work on the port after realizing that Shed Skin was being removed from distributions, especially Debian, as it was still tied to Python2, and I really want to keep Shed Skin in a working state if possible.

So what about the future of the project? Not really sure, but I'm happy to sit down again in a few months to prepare a new release. Any feedback on what to support or improve would be very welcome!

For fun, here are screenshots of some of the Shed Skin example programs (in total 75 working example programs can be found on the github site):