As this is a PEP, there is no real code to create, unlike in previous chapters. What we will do is cover the details of the proposal and how they are intended to be implemented:
- Two new APIs would be added to the gc module:
- The gc.set_mode(mode) API configures the garbage-collection mode between serial and threaded. If it is currently set to threaded, but the setting is switched to serial, the function waits for the garbage collection thread to complete before changing.
- The gc.get_mode() API returns the current mode of operation.
- The collection mode can be switched between the two options, so it is recommended that it be set at the beginning of a program, or when child processes are created.
- The actual implementation happens through adding the flag gc_is_threaded to the gc module; internally, a thread lock is added, to prevent multiple garbage collection instances from running simultaneously.
- In addition, two private functions, threading._ensure_dummy_thread(name) and threading._remove_dummy_thread(thread), are added to the threading module. The former creates a thread with the provided name, whereas the latter removes the thread from the module's internal state.
These functions allow the current thread to provide the name of the garbage collection thread when called within a finalization callback.
- Pseudocode is provided, demonstrating how the actual code would be implemented in the gc Python module as C code:
-
- callback_collect.txt simply enhances the current function by running garbage collection, up to the current object generation:
def collect_with_callback(generation):
"""
Collect up to the given *generation*.
"""
# Same code as currently
# (see collect_with_callback() in gcmodule.c)
-
- collect_gens.txt is much the same, as it doesn't modify the existing functionality. It is designed to collect all objects, as determined by the heuristic algorithm:
def collect_generations():
"""
Collect as many generations as desired
by the heuristic.
"""
# Same code as currently
# (see collect_generations() in gcmodule.c)
-
- lock_collect.txt demonstrates how garbage collection will be handled in a thread-safe manner; that is, the thread is locked during collection:
def lock_and_collect(generation=-1):
"""
Perform a collection with thread safety.
"""
me = PyThreadState_GET()
if gc_mutex.owner == me:
# reentrant GC collection request, bail out
return
Py_BEGIN_ALLOW_THREADS
gc_mutex.lock.acquire()
Py_END_ALLOW_THREADS
gc_mutex.owner = me
try:
if generation >= 0:
return collect_with_callback(generation)
else:
return collect_generations()
finally:
gc_mutex.owner = NULL
gc_mutex.lock.release()
-
- sched_gc.txt ensures that garbage collection is in the threaded mode, and then requests the collection of resources, when available:
def schedule_gc_request():
"""
Ask the GC thread to run an implicit collection.
"""
assert gc_is_threaded == True
# Note this is extremely fast
# if a collection is already requested
if gc_thread.collection_requested == False:
gc_thread.collection_requested = True
gc_thread.wakeup.release()
-
- implicit_gc.txt doesn't modify the existing code. It simply calls for collection if the heuristic algorithm determines it is necessary:
def is_implicit_gc_desired():
"""
Whether an implicit GC run is currently desired based
on allocation stats. Return a generation number,
or -1 if none desired.
"""
# Same heuristic as currently
# (see _PyObject_GC_Alloc in gcmodule.c)
-
- gc_malloc.txt allocates the memory resources to support a garbage collection object:
def PyGC_Malloc():
"""
Allocate a GC-enabled object.
"""
# Update allocation statistics (same code
# as currently, omitted for brevity)
if is_implicit_gc_desired():
if gc_is_threaded:
schedule_gc_request()
else:
lock_and_collect()
# Go ahead with allocation (same code as currently,
# omitted for brievity)
-
- gc_thread.txt spawns the garbage collection thread when called for:
def gc_thread(interp_state):
"""
Dedicated loop for threaded GC.
"""
# Init Python thread state
# (omitted, see t_bootstrap in _threadmodule.c)
# Optional: init thread in Python threading module,
# for better introspection
me = threading._ensure_dummy_thread(name="GC thread")
while gc_is_threaded == True:
Py_BEGIN_ALLOW_THREADS
gc_thread.wakeup.acquire()
Py_END_ALLOW_THREADS
if gc_thread.collection_requested != 0:
gc_thread.collection_requested = 0
lock_and_collect(generation=-1)
threading._remove_dummy_thread(me)
# Signal we're exiting
gc_thread.done.release()
# Free Python thread state (omitted)
-
- gc_set_mode.txt actually sets the garbage collection mode, between serial and threaded:
def gc.set_mode(mode):
"""
Set current GC mode.
This is a process-global setting.
"""
if mode == "threaded":
if not gc_is_threaded == False:
# Launch thread
gc_thread.done.acquire(block=False)
# should not fail
gc_is_threaded = True
PyThread_start_new_thread(gc_thread)
elif mode == "serial":
if gc_is_threaded == True:
# Wake up thread, asking it to end
gc_is_threaded = False
gc_thread.wakeup.release()
# Wait for thread exit
Py_BEGIN_ALLOW_THREADS
gc_thread.done.acquire()
Py_END_ALLOW_THREADS
gc_thread.done.release()
else:
raise ValueError("unsupported mode %r" %
(mode,))
-
- gc_get_mode.txt is a getter function that simply reports whether the garbage collector is threaded or serial:
def gc.get_mode(mode):
"""
Get current GC mode.
"""
return "threaded" if gc_is_threaded else "serial"
-
- gc_collect.txt represents a simple function that locks the thread and calls for garbage collection of the current object generation:
def gc.collect(generation=2):
"""
Schedule collection of the given generation
and wait for it to finish.
"""
return lock_and_collect(generation)
Again, all of the preceding code is just pseudocode, representing how the C code would be implemented in the Python interpreter. It is not production code, and any attempt to use it as-is will fail.