Skip to content

Split monolithic Python packages into per-subpackage components#12

Merged
ethenotethan merged 1 commit intomainfrom
feat/python-subpackage-components
Apr 21, 2026
Merged

Split monolithic Python packages into per-subpackage components#12
ethenotethan merged 1 commit intomainfrom
feat/python-subpackage-components

Conversation

@ethenotethan
Copy link
Copy Markdown
Collaborator

Summary

The Python discovery plugin previously emitted exactly one Component per pyproject.toml, even when the source tree contained many logically independent sub-packages. For monolithic Python codebases (the common case — PEP 621 has no workspace manifest like Cargo/npm), this hid the internal dependency structure that Flashlight's multi-agent analysis is designed to exploit.

This PR mirrors the Go plugin's single-module-monorepo approach: when a project has a flat- or src-layout source tree with ≥2 substantial top-level sub-packages, emit a Component per sub-package in addition to the root, with internal_dependencies derived from AST-based import analysis (handles both absolute from pkg.sub import X and relative from ..sub import X forms).

Classification per sub-package uses import-statement regex with word boundaries to avoid false positives (e.g., "dashboard" in a docstring was previously matching the "dash" frontend framework marker).

Changes

  • agent/discovery/languages/python.py — add _find_source_package, _discover_subpackages, _scan_python_imports (with relative-import resolution), and _classify_subpackage; tighten classifier to use word-boundary matching on import statements.
  • tests/discovery/test_python_plugin.py — 17 new tests covering flat/src layouts, relative imports, hyphenated project names, skip rules for tests/private dirs, min-sub-package threshold, classification, and the docstring-false-positive regression.

Validation

Validated end-to-end against jundot/omlx via GLM-5.1 on OpenRouter: 12 components discovered across 6 dependency-depth levels, correctly classifying the FastAPI-based api as SERVICE and everything else as LIBRARY.

Test plan

  • pytest tests/discovery/test_python_plugin.py — 17 new tests plus existing coverage.
  • End-to-end discovery run against a real monorepo confirms per-subpackage components and correct internal_dependencies edges.

The Python discovery plugin previously emitted exactly one Component per
pyproject.toml, even when the source tree contained many logically
independent sub-packages. For monolithic Python codebases (the common
case — PEP 621 has no workspace manifest like Cargo/npm), this hid the
internal dependency structure that Flashlight's multi-agent analysis is
designed to exploit.

Mirror the Go plugin's single-module-monorepo approach: when a project
has a flat- or src-layout source tree with ≥2 substantial top-level
sub-packages, emit a Component per sub-package in addition to the root,
with internal_dependencies derived from AST-based import analysis
(handles both absolute `from pkg.sub import X` and relative
`from ..sub import X` forms).

Classification per sub-package uses import-statement regex with word
boundaries to avoid false positives (e.g., "dashboard" in a docstring
was previously matching the "dash" frontend framework marker).

Validated end-to-end against jundot/omlx via GLM-5.1 on OpenRouter:
12 components discovered across 6 dependency-depth levels, correctly
classifying the FastAPI-based `api` as SERVICE and everything else as
LIBRARY.

- agent/discovery/languages/python.py: add _find_source_package,
  _discover_subpackages, _scan_python_imports (with relative-import
  resolution), and _classify_subpackage; tighten classifier to use
  word-boundary matching on import statements
- tests/discovery/test_python_plugin.py: 17 new tests covering flat/src
  layouts, relative imports, hyphenated project names, skip rules for
  tests/private dirs, min-sub-package threshold, classification, and
  the docstring-false-positive regression
@ethenotethan ethenotethan merged commit 6d3a85f into main Apr 21, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants