• Mon. Dec 23rd, 2024

Cython tutorial: How to speed up Python

Byadmin

Dec 5, 2024



Python’s profiler, cProfile, generates reports that show which functions take up the most time in a given Python program. By default, Cython code doesn’t show up in those reports. However, you can enable profiling on Cython code by inserting a compiler directive at the top of the .pyx file with functions you want to include in the profiling:

# cython: profile=True

You can also enable line-by-line tracing on the C code generated by Cython. Doing this imposes a lot of overhead, so it’s turned off by default. (Profiling generally imposes overhead, so be sure to toggle it off for code that is being shipped into production.)

Cython can also generate code reports that indicate how much of a given .pyx file is being converted to C, and how much of it remains Python code. The setup.py file from our example specified this with the annotate=True statement in the cythonize() function.

Delete the .c files generated in the project and re-run the setup.py script to recompile everything. When you’re done, you should see an HTML file in the same directory that shares the name of your .pyx file—in this case, num.html. Open the HTML file and you’ll see the parts of your code that are still dependent on Python highlighted in yellow. You can click on the yellow areas to see the underlying C code generated by Cython.

A Cython code annotation report. The yellow highlights indicate parts of the code that still depend on the Python runtime.
IDG

In this case, the def f function is still highlighted in yellow, despite having its variables explicitly typed. Why? Because it’s not annotated explicitly as a Cython function. Cython assumes any function not annotated with @cython.cfunc is just a Python function, and not convertible to pure C.

You can fix this by editing the def f declaration to read:

@cython.cfunc
def f(x: cython.double) -> cython.double:

Save and recompile the file, and reload the report. You should now see the def f function is no longer highlighted in yellow; it’s pure C.

The revised Cython function, now pure C, generates no highlights.
IDG

Note that if you have profiling enabled as described above, even “pure” C functions will show some highlighting, because they have been decorated with trace code that calls back into Python’s internals.

Also note that the division operation in line 12 is also highlighted; this is because Cython automatically inserts tests for division by zero, and raises a Python exception if that’s the case. You can disable this with the cdivision compiler directive, or by way of a decorator on the function (@cython.cdivision(True)).

Cython resources

Now that you have some idea of how Cython integrates with an existing Python app, the next step is to put it to work in your own applications, and to make use of other resources out there:



Source link