2.3 Package Hierarchy and Navigation

Package Hierarchy and Navigation

As applications grow to encompass dozens or hundreds of modules, placing all files in a single flat directory becomes unmanageable. Python addresses this complexity through packages—a hierarchical directory structuring mechanism that groups related modules under common namespaces.

---

1. What is a Python Package?

In Python, a package is essentially a filesystem directory that contains one or more module files (.py) and typically a special initialization file named __init__.py.

Just as modules prevent global variable collisions between functions, packages prevent module name collisions between different libraries (e.g., your project's utils.py will not conflict with a third-party library's utils.py because they reside in different package namespaces: myproject.utils vs requests.utils).

ecommerce/                  # Top-level Package
├── __init__.py             # Package initialization file
├── inventory/              # Subpackage
│   ├── __init__.py
│   └── warehouse.py        # Module: ecommerce.inventory.warehouse
├── orders/                 # Subpackage
│   ├── __init__.py
│   └── processing.py       # Module: ecommerce.orders.processing
└── payments/               # Subpackage
    ├── __init__.py
    └── gateway.py          # Module: ecommerce.payments.gateway

The Role of __init__.py

The presence of __init__.py serves several vital functions in Python:

1. Package Identifier: Historically required to mark a directory as a regular Python package rather than a plain directory (since Python 3.3, PEP 420 introduced implicit namespace packages, but __init__.py remains standard for regular packages).
2. Initialization Code: Whenever any module inside the package is imported, the package's __init__.py file is executed automatically first.
3. Public API Aggregation: __init__.py can import key functions and classes from submodules so callers do not need to know the internal directory structure.
4. Wildcard Export Control: Defining __all__ inside __init__.py dictates which submodules are imported when a user runs from package import *.

# File: ecommerce/__init__.py
print("[ecommerce] Initializing ecommerce package")

# Expose subpackage shortcuts directly at the top level
from .payments.gateway import process_credit_card
from .inventory.warehouse import check_stock

__all__ = ['process_credit_card', 'check_stock']

With this __init__.py in place, consumers can simply write:

import ecommerce
ecommerce.process_credit_card(150.00)

---

2. Package Navigation and Dotted Notation

Python uses dotted notation (pkg.subpkg.module) to traverse nested package hierarchies.

Syntax Variations for Packages

# 1. Full Module Import
import ecommerce.payments.gateway
ecommerce.payments.gateway.process_credit_card(99.95)

# 2. Module Import with Alias
import ecommerce.payments.gateway as gw
gw.process_credit_card(99.95)

# 3. Submodule Import from Package
from ecommerce.payments import gateway
gateway.process_credit_card(99.95)

# 4. Direct Entity Import
from ecommerce.payments.gateway import process_credit_card
process_credit_card(99.95)

---

3. The Module Search Path: sys.path

How does Python locate modules and packages when an import statement is executed? It queries the search path stored in sys.path—a standard Python list of filesystem directory paths.

Strict Search Resolution Order

When an import statement is evaluated, Python searches the directories listed in sys.path in strict sequential order from index 0 upwards:

1. sys.path[0] (Script Directory): The directory containing the top-level script used to invoke the interpreter (or current working directory if running in an interactive REPL).
2. PYTHONPATH Directories: Any directory paths defined in the operating system environment variable PYTHONPATH.
3. Standard Library Directories: The official installation directory containing Python's built-in and standard library modules (e.g., math, os, sys, random).
4. Site-Packages Directories: The third-party installation directory where pip installs external packages.

If the target module or package is not found in any of these directories, Python raises a ModuleNotFoundError.

import sys

print("=== Python Module Search Path ===")
for idx, directory in enumerate(sys.path):
    print(f"[{idx}] {directory}")

Modifying sys.path Dynamically at Runtime

Because sys.path is a mutable Python list, programs can dynamically modify the search path at runtime before invoking import:

import sys

# Add a custom directory to the end of the search path
sys.path.append('/opt/shared_libraries')

# Or insert at index 0 to give custom modules highest priority
sys.path.insert(0, '/home/user/custom_plugins')

# Now modules inside /home/user/custom_plugins can be imported
import custom_auth

---

4. Absolute vs. Relative Imports

Within packages, Python supports two distinct import strategies: absolute imports and relative imports.

Absolute Imports

An absolute import specifies the complete path from the project's root package or an entry in sys.path:

# Inside ecommerce/orders/processing.py
from ecommerce.payments.gateway import process_credit_card
from ecommerce.inventory.warehouse import check_stock

• Pros: Explicit, unambiguous, and recommended by PEP 8.
• Cons: Renaming the top-level package requires updating import statements throughout all files.

Relative Imports

Relative imports specify the target module relative to the location of the current module within the package hierarchy using leading dot (.) syntax:

• Single dot (.): The current package directory.
• Double dot (..): The parent package directory.
• Triple dot (...): The grandparent package directory.

# Inside ecommerce/orders/processing.py (located in ecommerce/orders/)

# Import from the SAME subpackage (ecommerce/orders/)
from . import invoice_generator

# Import from a SIBLING subpackage (ecommerce/payments/)
from ..payments.gateway import process_credit_card

# Import from another SIBLING subpackage (ecommerce/inventory/)
from ..inventory.warehouse import check_stock

The Relative Import Execution Constraint

CRITICAL PCAP EXAM TRAP: Relative imports rely strictly on the module's __name__ and __package__ attributes to resolve parent hierarchy.

If you attempt to run a submodule containing relative imports directly as a script (e.g., python ecommerce/orders/processing.py), Python assigns __name__ = '__main__' and leaves __package__ = None.

When Python encounters from ..payments import gateway, it cannot determine the parent package and immediately raises: ImportError: attempted relative import with no known parent package

To execute a package submodule as an entry point, use Python's -m (module) flag from the root project directory:

python -m ecommerce.orders.processing