Metaprogramming & Runtime Mechanics
Inspect CPython runtime protocols. Build custom attribute descriptors (`__get__`, `__set__`), control instance memory allocation (`__new__` vs. `__init__`), enforce immutable hashability contracts, and navigate C3 Linearization with parameterless `super()`.
Senior Systems Studio: Metaprogramming & Runtime Mechanics
Select any senior topic below to inspect field validation descriptors, test primitive memory allocation (__new__), verify equality and hashing contracts, and analyze dynamic C3 MRO execution.
Core Principles of Metaprogramming & Runtime Mechanics
- Descriptor Protocol: Data descriptors (`__get__`, `__set__`) take precedence over instance `__dict__` entries, serving as the foundational engine behind `@property`, ORMs, and Pydantic validators.
- Allocation vs Initialization: `__new__` allocates a raw instance in RAM; `__init__` configures state. Subclassing immutable primitives (`str`, `int`) requires `__new__` because instance value is locked at allocation.
- Equality & Hashability Contract: Defining `__eq__` sets `__hash__ = None` by default. Hashability requires defining `__hash__()` backed strictly by immutable fields; mutable state must never participate in hashing.
- Cooperative MRO & `super()`: C3 Linearization dictates method resolution sequence. Parameterless `super()` invokes the next class in the runtime MRO chain, enabling cooperative mixins without hardcoding parent names.
1. The Attribute Lookup Engine (`user.name`) & Descriptors
Accessing an attribute on an instance triggers a deterministic 5-step fallback chain in CPython's PyObject_GenericGetAttr:
- Step 1 — Search Class MRO for Data Descriptor: If attribute is a Data Descriptor (defines
__get__AND__set__or__delete__), calldescriptor.__get__()immediately. - Step 2 — Search Instance Dict: Look up
instance.__dict__['name']. If found, return instance value. - Step 3 — Search Class MRO for Non-Data Descriptor: If attribute implements ONLY
__get__(e.g. methods/functions), calldescriptor.__get__(). - Step 4 — Search Class MRO for Standard Class Attribute: If found in
Class.__dict__['name'], return static class attribute. - Step 5 — Fallback to
__getattr__: If steps 1–4 fail, invokeinstance.__getattr__('name'). (RaisesAttributeErrorif not implemented).
__getattribute__ intercepts every attribute access unconditionally before the chain starts. Overriding it requires delegating to super().__getattribute__ to avoid infinite recursion.
class BoundedNumber:
"""Data descriptor validating numeric bounds at runtime."""
def __init__(self, min_val: float, max_val: float):
self.min_val = min_val
self.max_val = max_val
def __set_name__(self, owner, name):
self._name = f"_{name}"
def __get__(self, instance, owner):
if instance is None: return self
return getattr(instance, self._name, None)
def __set__(self, instance, value: float):
if not (self.min_val <= value <= self.max_val):
raise ValueError(f"Value {value} outside range [{self.min_val}, {self.max_val}]")
setattr(instance, self._name, value)
class Metric:
cpu_usage = BoundedNumber(0.0, 100.0)2. `__new__` vs. `__init__` Allocation Mechanics
`__new__` is the actual instance allocator returning a new object; `__init__` configures state. Subclassing immutable primitives (`str`, `int`, `tuple`) or writing singletons requires intercepting `__new__`.
class UppercaseString(str):
"""Subclassing immutable str via __new__ allocation."""
def __new__(cls, value: str):
# Value must be transformed prior to C-level allocation
return super().__new__(cls, value.upper())
code = UppercaseString("production_key")
print(code) # "PRODUCTION_KEY"3. Identity and Equality (`__repr__`, `__eq__`, `__hash__`)
Overriding `__eq__` without `__hash__` automatically sets `__hash__ = None`, rendering instances unhashable. Mutable state must never participate in hashing algorithms.
class FrozenRecord:
def __init__(self, record_id: str, code: int):
self._record_id = record_id
self._code = code
def __eq__(self, other):
if not isinstance(other, FrozenRecord): return NotImplemented
return self._record_id == other._record_id and self._code == other._code
def __hash__(self):
# Must hash strictly immutable tuple state
return hash((self._record_id, self._code))4. C3 Linearization & Diamond Inheritance (`super()`)
Python calculates Method Resolution Order using C3 Linearization algorithm:
The merge algorithm guarantees 3 properties:
- Children before parents: A derived class is evaluated before any of its base classes.
- Header Order: Left-to-right order declared in the class header is preserved.
- Monotonicity: Ancestor evaluation order remains consistent across all derived subclasses.
class A:
def hello(self): print("A")
class B(A):
def hello(self): print("B")
class C(A):
def hello(self): print("C")
class D(B, C):
pass
# D.__mro__ == (D, B, C, A, object)
# Executing D().hello() prints "B".
# If B.hello is removed, it prints "C" (never "A" first).