Python builds¶
See also Analysis of CPython binary assembly and C compilers.
Release build¶
In release mode, Python is built with -O3 -DNDEBUG
compiler flags; defining
NDEBUG
macro removes assertions.
LTO¶
Use gcc -flto
Maybe also: -fuse-linker-plugin -ffat-lto-objects -flto-partition=none
.
Check if Python was built with LTO:
$ python3 -c "import sysconfig; print('lto' in (sysconfig.get_config_var('PY_CFLAGS') + sysconfig.get_config_var('PY_CFLAGS_NODIST')))"
True
PGO¶
- Build Python with
gcc -fprofile-generate
- Run the test suite:
make run_profile_task
usesPROFILE_TASK
variable of Makefile. configure uses./python -m test --pgo
by default. List of test used with--pgo
: see libregrtest/pgo.py. - Rebuild Python with
gcc -fprofile-use
Check if Python was built with PGO:
$ python3 -c "import sysconfig; print('-fprofile-use' in (sysconfig.get_config_var('PY_CFLAGS') + sysconfig.get_config_var('PY_CFLAGS_NODIST')))"
True
PGO + LTO¶
Use PGO and add LTO flags.
Debug build¶
- Fedora:
dnf install python3-debug
to get thepython3-debug
program: run it instead of runningpython3
. ./configure --with-pydebug
- Python built using
-Og
if available, or-O0
otherwise. - Define the
Py_DEBUG
macro which enables many runtime checks to ease debug.
Python built in debug mode is very effecient to detect bugs in C extensions.
Effects of the Debug Mode:
- Build Python with assertions (don’t set
NDEBUG
macro):assert(...);
_PyObject_ASSERT(...);
- etc.
- Install debug hooks on memory allocators to detect buffer overflow and other memory errors.
- Add runtime checks, code surroundeded by
#ifdef Py_DEBUG
and#endif
. - Unicode and int objects are created with their memory filled with a pattern to help detecting uninitialized bytes.
- Many functions ensure that are not called with an exception raised, since
they can clear or replace the current exception:
assert(!_PyErr_Occurred(tstate));
.
Checks in the GC:
- The GC
visit_decref()
function checks if an object was freed using_PyObject_IsFreed()
. This function is used internally during a GC collection on all Python objects tracked by the GC to break reference cycles. PyObject_GC_Track()
checks if the object was freed using_PyObject_IsFreed()
.- GC
gc_decref()
function checks if the GC reference count is valid.
Check if Python is built in debug mode¶
Py_DEBUG adds sys.gettotalrefcount()
function:
$ python3 -c "import sys; print('Debug build?', hasattr(sys, 'gettotalrefcount'))"
Debug build? False
$ python3-debug -c "import sys; print('Debug build?', hasattr(sys, 'gettotalrefcount'))"
Debug build? True
or: Check sysconfig.get_config_var('Py_DEBUG')
value:
$ python3 -c "import sysconfig; print('Debug build?', bool(sysconfig.get_config_var('Py_DEBUG')))"
Debug build? False
$ python3-debug -c "import sysconfig; print('Debug build?', bool(sysconfig.get_config_var('Py_DEBUG')))"
Debug build? True
Extra debug¶
Objects/dictobject.c
: define DEBUG_PYDICT macro to add_PyDict_CheckConsistency()
checks to debug corrupted dictionaries.Objects/obmalloc.c
: define PYMEM_DEBUG_SERIALNO to store a serial number in allocated memory blocks.