ffx/SCRATCHPAD.md

Scratchpad

Goal

• Capture a compact, project-wide list of optimization candidates after a broad scan of the current FFX codebase, tooling, and requirements.

Settled

• The biggest near-term wins are in startup cost, repeated subprocess work, repeated database query patterns, and general repo hygiene.
• This list is intentionally optimization-oriented rather than bug-oriented. Some items below also improve correctness or maintainability, but they were selected because they can reduce runtime cost, operator friction, or iteration overhead.
• A first modern integration slice now exists under tests/integration/subtrack_mapping. Remaining test-suite cleanup is now mostly about migrating and shrinking the legacy harness surface under tests/legacy.
• The CLI root now lazy-loads heavy runtime dependencies so lightweight commands such as version, help, configure_workstation, and upgrade stay import-light.
• Shared CLI defaults for container/output tokens now live outside src/ffx/ffx_controller.py, and a focused unit test locks in the lazy-import contract.
• FileProperties now uses one cached ffprobe -show_format -show_streams -of json call per source file, and the combined payload was confirmed against the Dragonball asset to satisfy both previous probe call sites fully.
• Database startup now bootstraps schema only when required tables are actually missing, while version enforcement still runs on ordinary DB-backed context creation.
• Helper filename and rich-text utilities now use compiled raw regexes plus translate-based filename filtering, with unit coverage for TMDB suffix rewriting and Rich color stripping.
• Process resource limiting now has explicit disabled/default states in the CLI and requirements, and combined CPU-plus-niceness wrapping now executes as cpulimit -- nice -n ... <command> instead of a less explicit prefix chain.
• FFX logger setup now reuses named handlers, and fallback logger access no longer mutates handlers in ordinary constructors and helpers.
• The process wrapper now uses subprocess.run(...) with centralized command formatting plus stable timeout and missing-command error mapping.
• Active ORM controllers now use single-query accessors instead of paired count() plus first() lookups.
• Pattern matching now uses cached compiled regexes plus explicit duplicate-match errors, and pattern creation flows no longer persist zero-track patterns.

Focused Snapshot

• Highest-leverage application optimizations:

  • Revisit crop detection cost now that the probe path is consolidated.

• Highest-leverage repo and workflow optimizations:

  • Consolidate setup and upgrade tooling to reduce overlapping shell-script responsibilities.
  • Continue migrating the oversized legacy test/combinator surface into focused modern tests so it is easier to run, debug, and extend.

Optimization Candidates

1. Crop detection is always a full extra ffmpeg scan

• src/ffx/file_properties.py runs a dedicated ffmpeg -vf cropdetect pass for each file when crop detection is requested.
• Optimization:
  • Cache crop results for repeated runs on the same source.
  • Consider exposing shorter sampling windows or probe presets for large files.
• Expected value:
  • Lower latency on repeated experimentation.

2. Tooling overlap and naming drift

• There are still overlapping workstation-setup entrypoints across tools/configure_workstation.sh, tools/setup.sh, and newer CLI maintenance commands.
• Optimization:
  • Decide which scripts remain canonical.
  • Replace or remove legacy wrappers once equivalent CLI commands exist.
  • Keep CLI maintenance commands and shell wrappers aligned.
• Expected value:
  • Less operator confusion.
  • Fewer duplicated procedures to maintain.

3. Placeholder UI surfaces should either ship or disappear

• src/ffx/help_screen.py and src/ffx/settings_screen.py are placeholders.
• Optimization:
  • Either remove them from the active UI surface or complete them.
  • Avoid paying ongoing maintenance cost for unfinished navigation targets.
• Expected value:
  • Leaner interface.
  • Lower UX ambiguity.

4. Large Textual screens repeat configuration and controller loading

• Screens such as src/ffx/media_details_screen.py, src/ffx/pattern_details_screen.py, and src/ffx/show_details_screen.py repeat setup patterns and local metadata filtering extraction.
• Optimization:
  • Extract a shared screen base or helper for common config/controller/bootstrap logic.
  • Reduce repeated table refresh and repeated DB fetch code where possible.
• Expected value:
  • Lower maintenance overhead.
  • Easier UI iteration.

5. Several helper functions are unfinished or dead-weight

• src/ffx/helper.py contains permutateList(...): pass.
• There are many combinator and conversion placeholders across tests and migrations.
• Optimization:
  • Remove dead code, finish it, or isolate it behind a clearly dormant area.
  • Avoid carrying stubbed utility surface that looks reusable but is not.
• Expected value:
  • Smaller mental model.
  • Less time spent re-evaluating inactive paths.

6. Test suite shape is expensive to understand and likely expensive to run

• The project still carries a large legacy matrix of combinator files under tests/legacy, several placeholder pass implementations, and at least one suspicious filename with an embedded space: [tests/legacy/disposition_combinator_2_3 .py](/home/osgw/.local/src/codex/ffx/tests/legacy/disposition_combinator_2_3 .py).
• A first focused replacement slice now exists in tests/integration/subtrack_mapping/test_cli_bundle.py, so the remaining work is migration and consolidation rather than creating the modern test shape from scratch.
• Optimization:
  • Continue replacing broad combinator matrices with focused parametrized integration and unit tests.
  • Retire the bespoke legacy discovery and runner path once equivalent coverage exists.
  • Normalize file naming and test discovery conventions.
• Expected value:
  • Faster contributor onboarding.
  • Easier CI adoption later.

Open

• Should optimization work focus first on operator-perceived latency, internal maintainability, or correctness-risk cleanup that also has performance upside?
• Is the long-term supported model still “local Linux workstation plus Textual UI,” or should optimization decisions bias toward a more scriptable/headless CLI?

Gaps Right Now

• No explicit prioritization owner or milestone for the optimization backlog.
• No benchmark or timing harness exists for startup, probe, DB, or conversion orchestration overhead.
• Repo hygiene is still mixed with generated artifacts and some clearly unfinished files.

Next

1. Triage the list into quick wins, medium refactors, and long-horizon cleanup.
2. Tackle the cheapest high-impact items first:
   • crop detection sampling or caching pass.
3. Decide which setup and upgrade entrypoints stay canonical before adding more maintenance surface.

Delete When

• Delete this scratchpad once the optimization backlog is either converted into issues/work items or distilled into durable project guidance.