ffx/SCRATCHPAD.md

# Scratchpad

## Goal

- Capture a compact, project-wide list of optimization candidates after a broad scan of the current FFX codebase, tooling, and requirements.

## Settled

- The biggest near-term wins are in startup cost, repeated subprocess work, repeated database query patterns, and general repo hygiene.
- This list is intentionally optimization-oriented rather than bug-oriented. Some items below also improve correctness or maintainability, but they were selected because they can reduce runtime cost, operator friction, or iteration overhead.
- A first modern integration slice now exists under [`tests/integration/subtrack_mapping`](/home/osgw/.local/src/codex/ffx/tests/integration/subtrack_mapping). Remaining test-suite cleanup is now mostly about migrating and shrinking the legacy harness surface under [`tests/legacy`](/home/osgw/.local/src/codex/ffx/tests/legacy).
- The CLI root now lazy-loads heavy runtime dependencies so lightweight commands such as `version`, `help`, `configure_workstation`, and `upgrade` stay import-light.
- Shared CLI defaults for container/output tokens now live outside [`src/ffx/ffx_controller.py`](/home/osgw/.local/src/codex/ffx/src/ffx/ffx_controller.py), and a focused unit test locks in the lazy-import contract.
- `FileProperties` now uses one cached `ffprobe -show_format -show_streams -of json` call per source file, and the combined payload was confirmed against the Dragonball asset to satisfy both previous probe call sites fully.
- Database startup now bootstraps schema only when required tables are actually missing, while version enforcement still runs on ordinary DB-backed context creation.
- FFX logger setup now reuses named handlers, and fallback logger access no longer mutates handlers in ordinary constructors and helpers.
- The process wrapper now uses `subprocess.run(...)` with centralized command formatting plus stable timeout and missing-command error mapping.
- Active ORM controllers now use single-query accessors instead of paired `count()` plus `first()` lookups.
- Pattern matching now uses cached compiled regexes plus explicit duplicate-match errors, and pattern creation flows no longer persist zero-track patterns.

## Focused Snapshot

- Highest-leverage application optimizations:
  - Revisit crop detection cost now that the probe path is consolidated.

- Highest-leverage repo and workflow optimizations:
  - Consolidate setup and upgrade tooling to reduce overlapping shell-script responsibilities.
  - Continue migrating the oversized legacy test/combinator surface into focused modern tests so it is easier to run, debug, and extend.

## Optimization Candidates

1. Crop detection is always a full extra ffmpeg scan
- [`src/ffx/file_properties.py`](/home/osgw/.local/src/codex/ffx/src/ffx/file_properties.py) runs a dedicated `ffmpeg -vf cropdetect` pass for each file when crop detection is requested.
- Optimization:
  - Cache crop results for repeated runs on the same source.
  - Consider exposing shorter sampling windows or probe presets for large files.
- Expected value:
  - Lower latency on repeated experimentation.

2. Tooling overlap and naming drift
- There are still overlapping workstation-setup entrypoints across [`tools/configure_workstation.sh`](/home/osgw/.local/src/codex/ffx/tools/configure_workstation.sh), [`tools/setup.sh`](/home/osgw/.local/src/codex/ffx/tools/setup.sh), and newer CLI maintenance commands.
- Optimization:
  - Decide which scripts remain canonical.
  - Replace or remove legacy wrappers once equivalent CLI commands exist.
  - Keep CLI maintenance commands and shell wrappers aligned.
- Expected value:
  - Less operator confusion.
  - Fewer duplicated procedures to maintain.

3. Placeholder UI surfaces should either ship or disappear
- [`src/ffx/help_screen.py`](/home/osgw/.local/src/codex/ffx/src/ffx/help_screen.py) and [`src/ffx/settings_screen.py`](/home/osgw/.local/src/codex/ffx/src/ffx/settings_screen.py) are placeholders.
- Optimization:
  - Either remove them from the active UI surface or complete them.
  - Avoid paying ongoing maintenance cost for unfinished navigation targets.
- Expected value:
  - Leaner interface.
  - Lower UX ambiguity.

4. Large Textual screens repeat configuration and controller loading
- Screens such as [`src/ffx/media_details_screen.py`](/home/osgw/.local/src/codex/ffx/src/ffx/media_details_screen.py), [`src/ffx/pattern_details_screen.py`](/home/osgw/.local/src/codex/ffx/src/ffx/pattern_details_screen.py), and [`src/ffx/show_details_screen.py`](/home/osgw/.local/src/codex/ffx/src/ffx/show_details_screen.py) repeat setup patterns and local metadata filtering extraction.
- Optimization:
  - Extract a shared screen base or helper for common config/controller/bootstrap logic.
  - Reduce repeated table refresh and repeated DB fetch code where possible.
- Expected value:
  - Lower maintenance overhead.
  - Easier UI iteration.

5. Several helper functions are unfinished or dead-weight
- [`src/ffx/helper.py`](/home/osgw/.local/src/codex/ffx/src/ffx/helper.py) contains `permutateList(...): pass`.
- There are many combinator and conversion placeholders across tests and migrations.
- Optimization:
  - Remove dead code, finish it, or isolate it behind a clearly dormant area.
  - Avoid carrying stubbed utility surface that looks reusable but is not.
- Expected value:
  - Smaller mental model.
  - Less time spent re-evaluating inactive paths.

6. Test suite shape is expensive to understand and likely expensive to run
- The project still carries a large legacy matrix of combinator files under [`tests/legacy`](/home/osgw/.local/src/codex/ffx/tests/legacy), several placeholder `pass` implementations, and at least one suspicious filename with an embedded space: [`tests/legacy/disposition_combinator_2_3 .py`](/home/osgw/.local/src/codex/ffx/tests/legacy/disposition_combinator_2_3 .py).
- A first focused replacement slice now exists in [`tests/integration/subtrack_mapping/test_cli_bundle.py`](/home/osgw/.local/src/codex/ffx/tests/integration/subtrack_mapping/test_cli_bundle.py), so the remaining work is migration and consolidation rather than creating the modern test shape from scratch.
- Optimization:
  - Continue replacing broad combinator matrices with focused parametrized integration and unit tests.
  - Retire the bespoke legacy discovery and runner path once equivalent coverage exists.
  - Normalize file naming and test discovery conventions.
- Expected value:
  - Faster contributor onboarding.
  - Easier CI adoption later.

7. Process resource limiting semantics could be clearer
- [`src/ffx/process.py`](/home/osgw/.local/src/codex/ffx/src/ffx/process.py) prepends `nice` and `cpulimit` directly when values are set.
- Optimization:
  - Validate and document effective behavior for combined `nice` + `cpulimit`.
  - Consider explicit no-limit vs configured-limit states in the CLI and requirements.
- Expected value:
  - Fewer surprises in production-like runs.
  - Easier support for user-reported performance behavior.

8. Regex and string utility cleanup
- [`src/ffx/helper.py`](/home/osgw/.local/src/codex/ffx/src/ffx/helper.py) still has repeated string-replacement churn in filename/TMDB normalization helpers, and regex handling in helpers is easy to regress quietly.
- Optimization:
  - Keep regex literals raw and centralized where appropriate.
  - Review filename and TMDB substitution helpers for repeated string churn.
- Expected value:
  - Cleaner runtime output.
  - Less warning noise during dry-run maintenance commands.

## Open

- Should optimization work focus first on operator-perceived latency, internal maintainability, or correctness-risk cleanup that also has performance upside?
- Is the long-term supported model still “local Linux workstation plus Textual UI,” or should optimization decisions bias toward a more scriptable/headless CLI?

## Gaps Right Now

- No explicit prioritization owner or milestone for the optimization backlog.
- No benchmark or timing harness exists for startup, probe, DB, or conversion orchestration overhead.
- Repo hygiene is still mixed with generated artifacts and some clearly unfinished files.

## Next

1. Triage the list into quick wins, medium refactors, and long-horizon cleanup.
2. Tackle the cheapest high-impact items first:
   - crop detection sampling or caching pass.
3. Decide which setup and upgrade entrypoints stay canonical before adding more maintenance surface.

## Delete When

- Delete this scratchpad once the optimization backlog is either converted into issues/work items or distilled into durable project guidance.