Python is the most popular language for backend engineering, data science, and ML interviews. This guide covers the Python-specific questions that distinguish senior engineers from juniors — from generators and decorators to the GIL and memory management. These are the questions you will face at Google, Meta, Stripe, and any company using Python in production.
Generators and Iterators
A generator is a function that yields values lazily (one at a time) instead of returning a list. def fibonacci(): a, b = 0, 1; while True: yield a; a, b = b, a + b. Each call to next() resumes execution from the last yield. Memory efficient: a generator producing 1 billion Fibonacci numbers uses O(1) memory (only the current state). A list would use O(N) memory. Generator expressions: (x**2 for x in range(10)) — lazy equivalent of [x**2 for x in range(10)]. Use generators for: processing large files line by line, streaming data pipelines, and infinite sequences. Iterator protocol: any object implementing __iter__() and __next__() is an iterator. Generators automatically implement this protocol. The for loop calls iter() to get an iterator, then calls next() repeatedly until StopIteration. yield from: delegates to a sub-generator. def flatten(nested): for item in nested: if isinstance(item, list): yield from flatten(item) else: yield item. Cleaner than a for loop with yield inside. send(): send a value into a generator. value = yield result. The caller calls gen.send(value), which resumes the generator with the sent value. Used in coroutine patterns (pre-asyncio). Interview question: “What is the difference between a generator and a list comprehension?” Answer: memory (lazy vs eager), reusability (generators are single-use), and syntax (parentheses vs brackets).
Decorators
A decorator wraps a function to add behavior without modifying its code. @timer def slow_function(): … is equivalent to slow_function = timer(slow_function). A decorator takes a function, returns a new function that calls the original with added logic. Common patterns: (1) Timing: def timer(func): def wrapper(*args, **kwargs): start = time.time(); result = func(*args, **kwargs); print(f”{func.__name__} took {time.time()-start:.2f}s”); return result; return wrapper. (2) Caching (memoization): @functools.lru_cache(maxsize=128) caches function results by arguments. Automatic memoization for pure functions. (3) Retry: def retry(max_retries=3): def decorator(func): def wrapper(*args, **kwargs): for i in range(max_retries): try: return func(*args, **kwargs) except Exception: if i == max_retries – 1: raise; return wrapper; return decorator. (4) Authentication: @login_required checks if the user is authenticated before calling the view function. @functools.wraps(func): preserve the original function name, docstring, and attributes. Without it, the decorated function loses its identity (func.__name__ becomes “wrapper”). Class decorators: a class with __call__ can be a decorator. Useful when the decorator needs to maintain state (count calls, cache results). Decorators with arguments: require an extra layer of nesting — the outer function takes the arguments, returns the actual decorator. @retry(max_retries=5) -> retry(max_retries=5) returns a decorator, which wraps the function.
Async/Await and Concurrency
Python concurrency model: (1) asyncio (async/await) — cooperative multitasking on a single thread. Coroutines yield control at await points. Best for: I/O-bound tasks (HTTP requests, database queries, file I/O). A web server handling 10,000 concurrent connections uses one thread. (2) threading — OS-level threads. Limited by the GIL (Global Interpreter Lock): only one thread executes Python bytecode at a time. Threads are useful for: I/O-bound tasks (the GIL is released during I/O operations) but NOT for CPU-bound tasks (threads do not parallelize CPU work). (3) multiprocessing — separate processes with separate GILs. True parallelism for CPU-bound tasks. Each process has its own memory space. Communication via pipes, queues, or shared memory. async/await basics: async def fetch_data(): response = await httpx.get(url); return response.json(). The await keyword suspends the coroutine until the I/O completes. Other coroutines run during the suspension. asyncio.gather(): run multiple coroutines concurrently. results = await asyncio.gather(fetch_a(), fetch_b(), fetch_c()). All three requests run concurrently on a single thread. Interview question: “When would you use asyncio vs threading vs multiprocessing?” Answer: asyncio for many concurrent I/O operations (web servers, API clients). Threading for legacy code with I/O waits. Multiprocessing for CPU-bound computation (data processing, ML inference).
The GIL (Global Interpreter Lock)
The GIL is a mutex that allows only one thread to execute Python bytecode at a time in CPython. Why it exists: CPython memory management (reference counting) is not thread-safe. The GIL simplifies the interpreter implementation and makes single-threaded code faster. Impact: multithreaded Python programs cannot achieve true CPU parallelism on multiple cores. Two threads doing CPU-intensive work take roughly the same time as one thread (the GIL serializes them). Workarounds: (1) multiprocessing — each process has its own GIL. True parallelism. (2) C extensions — numpy, pandas, and scikit-learn release the GIL during C-level computation. Multithreaded numpy operations ARE parallel. (3) Free-threaded Python (PEP 703) — Python 3.13+ includes an experimental build without the GIL (–disable-gil). This is a major change being rolled out gradually. (4) Alternative interpreters — PyPy (JIT-compiled, still has a GIL but faster), Jython (Java-based, no GIL), and GraalPy (GraalVM-based). When the GIL does NOT matter: I/O-bound tasks (the GIL is released during I/O), C extension heavy code (numpy releases the GIL), and asyncio (single-threaded by design). Interview tip: knowing the GIL, its impact, and workarounds is a common senior Python interview question. Mention free-threaded Python 3.13 to show you follow recent developments.
Type Hints, Dataclasses, and Modern Python
Type hints (PEP 484+): def greet(name: str) -> str: return f”Hello, {name}”. Types are not enforced at runtime — they are for: static analysis (mypy, pyright), IDE autocompletion, and documentation. Modern type features: Optional[str] = str | None (Python 3.10+), list[int] instead of List[int] (Python 3.9+), TypeVar and Generic for generic types, and Protocol for structural subtyping (duck typing with type checking). Dataclasses (PEP 557): @dataclass class Point: x: float; y: float. Auto-generates: __init__, __repr__, __eq__, and optionally __hash__, __lt__ (with order=True). Replaces boilerplate class definitions. Pydantic: like dataclasses but with runtime validation. class User(BaseModel): name: str; age: int = Field(gt=0). Invalid data raises ValidationError. Standard for API input validation (FastAPI uses Pydantic). match/case (Python 3.10+): structural pattern matching. match command: case [“quit”]: exit(); case [“go”, direction]: move(direction); case _: print(“unknown”). More powerful than switch/case — matches on structure, not just values. Walrus operator (:=, Python 3.8+): if (n := len(data)) > 10: print(f”Too long: {n}”). Assigns and uses in one expression. Interview question: “What are the benefits of type hints in Python?” Answer: catches bugs at development time (mypy), improves IDE experience (autocomplete, refactoring), serves as documentation, and enables tools like Pydantic for runtime validation.
Memory Management and Performance
Python memory management: (1) Reference counting — each object has a counter tracking references to it. When the count drops to 0, the object is freed immediately. This is deterministic and fast for most objects. (2) Cyclic garbage collector — reference counting cannot handle cycles (A references B, B references A, both have count > 0 but are unreachable). The cyclic GC periodically scans for unreachable cycles and frees them. Three generations: generation 0 (new objects, collected frequently), generation 1 (survived one collection), generation 2 (long-lived, collected rarely). Performance tips: (1) Use built-in data structures — list, dict, set are implemented in C and highly optimized. Custom Python classes are slower. (2) Use list comprehensions over loops — [x**2 for x in range(1000)] is faster than appending in a loop (the comprehension runs in C). (3) Use generators for large datasets — avoid materializing large lists in memory. (4) Use __slots__ — class MyClass: __slots__ = [“x”, “y”]. Prevents __dict__ creation per instance. Saves 40-50% memory for classes with many instances. (5) Use numpy for numerical computation — vectorized operations in C are 10-100x faster than Python loops. (6) Profile before optimizing — cProfile for CPU profiling, tracemalloc for memory profiling. Optimize the bottleneck, not random code. Common memory leak: circular references with __del__ methods (the cyclic GC cannot collect objects with custom __del__ in certain cases — improved in Python 3.4+). Also: global caches that grow unboundedly, and unclosed resources (files, connections).
Drill the patterns next:
