diff --git a/README.md b/README.md index 5fa42fc..6eed68c 100644 --- a/README.md +++ b/README.md @@ -1,48 +1,135 @@ # FFX +FFX is a local CLI and Textual TUI for inspecting TV episode files, storing normalization rules in SQLite, and converting outputs into a predictable stream, metadata, and filename layout. + +## Requirements + +- Linux-like environment +- `python3` +- `ffmpeg` +- `ffprobe` +- `cpulimit` + ## Installation -per https: +FFX uses a two-step local setup flow. + +### 1. Install The Bundle + +This step creates or reuses the persistent bundle virtualenv in `~/.local/share/ffx.venv`, installs FFX into it, and ensures `ffx` is exposed through a shell alias. ```sh -pip install https:////ffx.git@ +bash tools/setup.sh ``` -per git: +If you also want the Python packages needed for the modern test suite: ```sh -pip install git+ssh://@//ffx.git@ +bash tools/setup.sh --with-tests ``` -## Version history +You can verify the bundle state without changing anything: -### 0.1.1 +```sh +bash tools/setup.sh --check +``` -Bugfixes, TMBD identify shows +### 2. Prepare System Dependencies And Local User Files -### 0.1.2 +This step installs or verifies workstation dependencies and seeds local config and data directories. It is the step wrapped by the CLI command `ffx configure_workstation`. -Bugfixes +Run it directly: -### 0.1.3 +```sh +bash tools/configure_workstation.sh +``` -Subtitle file imports +Or through the installed CLI: -### 0.2.0 +```sh +ffx configure_workstation +``` -Tests, Config-File +Check-only mode is available in both forms: -### 0.2.1 +```sh +bash tools/configure_workstation.sh --check +ffx configure_workstation --check +``` -Signature, Tags cleaning, Bugfixes, Refactoring +`tools/configure_workstation.sh` does not manage the bundle virtualenv. Python-side test packages belong to `tools/setup.sh --with-tests`. -### 0.2.2 +## Basic Usage -CLI-Overrides +Examples: + +```sh +ffx version +ffx inspect /path/to/episode.mkv +ffx convert /path/to/episode.mkv +ffx shows +``` + +## Modern Tests + +Install Python test packages first: + +```sh +bash tools/setup.sh --with-tests +``` + +Then run the modern automatically discovered test suite: + +```sh +./tools/test.sh +``` + +This runner uses `pytest` and intentionally excludes the legacy harness under `tests/legacy/`. + +## Default Local Paths + +- Config: `~/.local/etc/ffx.json` +- Database: `~/.local/var/ffx/ffx.db` +- Log file: `~/.local/var/log/ffx.log` +- Bundle venv: `~/.local/share/ffx.venv` + +## TMDB + +TMDB-backed metadata enrichment requires `TMDB_API_KEY` to be set in the environment. + +## Version History ### 0.2.3 -PyPi packaging -Templating output filename -Season shiftung -DB-Versionierung +- PyPI packaging +- output filename templating +- season shifting +- DB versioning + +### 0.2.2 + +- CLI overrides + +### 0.2.1 + +- signature handling +- tag cleanup +- bugfixes and refactoring + +### 0.2.0 + +- tests +- config file + +### 0.1.3 + +- subtitle file imports + +### 0.1.2 + +- bugfixes + +### 0.1.1 + +- bugfixes +- TMDB show identification diff --git a/SCRATCHPAD.md b/SCRATCHPAD.md index 5c4d5c2..ce4ba22 100644 --- a/SCRATCHPAD.md +++ b/SCRATCHPAD.md @@ -10,6 +10,7 @@ - This list is intentionally optimization-oriented rather than bug-oriented. Some items below also improve correctness or maintainability, but they were selected because they can reduce runtime cost, operator friction, or iteration overhead. - A first modern integration slice now exists under [`tests/integration/subtrack_mapping`](/home/osgw/.local/src/codex/ffx/tests/integration/subtrack_mapping). Remaining test-suite cleanup is now mostly about migrating and shrinking the legacy harness surface under [`tests/legacy`](/home/osgw/.local/src/codex/ffx/tests/legacy). - FFX logger setup now reuses named handlers, and fallback logger access no longer mutates handlers in ordinary constructors and helpers. +- The process wrapper now uses `subprocess.run(...)` with centralized command formatting plus stable timeout and missing-command error mapping. ## Focused Snapshot @@ -26,7 +27,7 @@ ## Optimization Candidates 1. CLI startup and import cost -- [`src/ffx/cli.py`](/home/osgw/.local/src/codex/ffx/src/ffx/cli.py) imports a large portion of the application at module import time, even for cheap commands such as `version`, `help`, `setup_dependencies`, and `upgrade`. +- [`src/ffx/cli.py`](/home/osgw/.local/src/codex/ffx/src/ffx/cli.py) imports a large portion of the application at module import time, even for cheap commands such as `version`, `help`, `configure_workstation`, and `upgrade`. - Optimization: - Move heavy imports into the commands that actually need them. - Keep the CLI root importable with only core stdlib and Click dependencies. @@ -70,18 +71,8 @@ - Expected value: - Lower latency on repeated experimentation. -6. Process wrapper lacks stronger execution controls -- [`src/ffx/process.py`](/home/osgw/.local/src/codex/ffx/src/ffx/process.py) uses `Popen(...).communicate()` without timeout handling, structured error mapping, or direct missing-command handling. -- Optimization: - - Add timeout support and clearer `FileNotFoundError` handling. - - Consider `subprocess.run(..., check=False, text=True)` where streaming is not required. - - Centralize return/error formatting. -- Expected value: - - Better failure diagnosis. - - Cleaner process management semantics. - -7. Tooling overlap and naming drift -- There are still overlapping prep and setup entrypoints across [`tools/prepare.sh`](/home/osgw/.local/src/codex/ffx/tools/prepare.sh), [`tools/setup.sh`](/home/osgw/.local/src/codex/ffx/tools/setup.sh), and newer CLI maintenance commands. +6. Tooling overlap and naming drift +- There are still overlapping workstation-setup entrypoints across [`tools/configure_workstation.sh`](/home/osgw/.local/src/codex/ffx/tools/configure_workstation.sh), [`tools/setup.sh`](/home/osgw/.local/src/codex/ffx/tools/setup.sh), and newer CLI maintenance commands. - Optimization: - Decide which scripts remain canonical. - Replace or remove legacy wrappers once equivalent CLI commands exist. @@ -90,7 +81,7 @@ - Less operator confusion. - Fewer duplicated procedures to maintain. -8. Placeholder UI surfaces should either ship or disappear +7. Placeholder UI surfaces should either ship or disappear - [`src/ffx/help_screen.py`](/home/osgw/.local/src/codex/ffx/src/ffx/help_screen.py) and [`src/ffx/settings_screen.py`](/home/osgw/.local/src/codex/ffx/src/ffx/settings_screen.py) are placeholders. - Optimization: - Either remove them from the active UI surface or complete them. @@ -99,7 +90,7 @@ - Leaner interface. - Lower UX ambiguity. -9. Large Textual screens repeat configuration and controller loading +8. Large Textual screens repeat configuration and controller loading - Screens such as [`src/ffx/media_details_screen.py`](/home/osgw/.local/src/codex/ffx/src/ffx/media_details_screen.py), [`src/ffx/pattern_details_screen.py`](/home/osgw/.local/src/codex/ffx/src/ffx/pattern_details_screen.py), and [`src/ffx/show_details_screen.py`](/home/osgw/.local/src/codex/ffx/src/ffx/show_details_screen.py) repeat setup patterns and local metadata filtering extraction. - Optimization: - Extract a shared screen base or helper for common config/controller/bootstrap logic. @@ -108,7 +99,7 @@ - Lower maintenance overhead. - Easier UI iteration. -10. Several helper functions are unfinished or dead-weight +9. Several helper functions are unfinished or dead-weight - [`src/ffx/helper.py`](/home/osgw/.local/src/codex/ffx/src/ffx/helper.py) contains `permutateList(...): pass`. - There are many combinator and conversion placeholders across tests and migrations. - Optimization: @@ -118,7 +109,7 @@ - Smaller mental model. - Less time spent re-evaluating inactive paths. -11. Test suite shape is expensive to understand and likely expensive to run +10. Test suite shape is expensive to understand and likely expensive to run - The project still carries a large legacy matrix of combinator files under [`tests/legacy`](/home/osgw/.local/src/codex/ffx/tests/legacy), several placeholder `pass` implementations, and at least one suspicious filename with an embedded space: [`tests/legacy/disposition_combinator_2_3 .py`](/home/osgw/.local/src/codex/ffx/tests/legacy/disposition_combinator_2_3 .py). - A first focused replacement slice now exists in [`tests/integration/subtrack_mapping/test_cli_bundle.py`](/home/osgw/.local/src/codex/ffx/tests/integration/subtrack_mapping/test_cli_bundle.py), so the remaining work is migration and consolidation rather than creating the modern test shape from scratch. - Optimization: @@ -129,7 +120,7 @@ - Faster contributor onboarding. - Easier CI adoption later. -12. Process resource limiting semantics could be clearer +11. Process resource limiting semantics could be clearer - [`src/ffx/process.py`](/home/osgw/.local/src/codex/ffx/src/ffx/process.py) prepends `nice` and `cpulimit` directly when values are set. - Optimization: - Validate and document effective behavior for combined `nice` + `cpulimit`. @@ -138,7 +129,7 @@ - Fewer surprises in production-like runs. - Easier support for user-reported performance behavior. -13. Import-time dependency coupling makes maintenance commands brittle +12. Import-time dependency coupling makes maintenance commands brittle - Even after recent CLI maintenance additions, the top-level CLI module still imports most application modules before Click dispatch. - Optimization: - Push imports for ORM, Textual, TMDB, ffmpeg helpers, and descriptors behind the commands that actually need them. @@ -146,7 +137,7 @@ - Maintenance commands such as setup and upgrade stay usable when optional runtime dependencies are broken. - Better separation between media runtime code and maintenance tooling. -14. Regex and string utility cleanup +13. Regex and string utility cleanup - [`src/ffx/helper.py`](/home/osgw/.local/src/codex/ffx/src/ffx/helper.py) still emits a `SyntaxWarning` for `RICH_COLOR_PATTERN`. - Optimization: - Convert regex literals to raw strings where appropriate. @@ -155,7 +146,7 @@ - Cleaner runtime output. - Less warning noise during dry-run maintenance commands. -15. Database startup always runs schema creation and version checks +14. Database startup always runs schema creation and version checks - [`src/ffx/database.py`](/home/osgw/.local/src/codex/ffx/src/ffx/database.py) runs `Base.metadata.create_all(...)` and version checks every time a DB-backed context is created. - Optimization: - Measure startup cost and consider separating bootstrapping from ordinary command execution. diff --git a/pyproject.toml b/pyproject.toml index 224d73f..da2dc02 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -45,6 +45,7 @@ ffx = "ffx.cli:ffx" [tool.pytest.ini_options] testpaths = ["tests"] python_files = ["test_*.py"] +norecursedirs = ["tests/legacy", "tests/support"] addopts = "-ra" markers = [ "integration: exercises the FFX bundle with real ffmpeg/ffprobe processes", diff --git a/requirements/project.md b/requirements/project.md index 574b2a7..d47b826 100644 --- a/requirements/project.md +++ b/requirements/project.md @@ -35,7 +35,11 @@ ## Functional Requirements -- The system shall provide a CLI entrypoint named `ffx` with commands for `convert`, `inspect`, `shows`, `unmux`, `cropdetect`, `version`, and `help`. +- The system shall provide a CLI entrypoint named `ffx` with commands for `convert`, `inspect`, `shows`, `unmux`, `cropdetect`, `configure_workstation`, `upgrade`, `version`, and `help`. +- The system shall support a two-step local installation and preparation flow: + - `tools/setup.sh` is the first step and shall own bundle virtualenv creation, package installation, shell alias exposure, and optional Python test-package installation. + - `tools/configure_workstation.sh` is the second step and shall own workstation dependency checks and installation plus local config and directory seeding. +- The CLI command `ffx configure_workstation` shall act as a wrapper for the second-step preparation flow in `tools/configure_workstation.sh`. - The system shall persist reusable normalization rules in SQLite for: - shows and show formatting digits, - regex-based filename patterns, @@ -65,7 +69,7 @@ - The system should stay understandable as a small local tool: controllers, descriptors, models, and screens should remain separate enough for contributors to trace a workflow end to end. - The system should produce predictable output for the same database rules, CLI overrides, and source files. - The system should preserve a lightweight operational footprint: local SQLite state, local log file, no mandatory background services. -- The system should be testable through the existing combinatorial CLI-oriented test harness and through isolated logic in descriptors and controllers. +- The system should be testable through modern automatically discovered tests and through remaining legacy harness coverage during migration. - The system should expose enough logging to diagnose failed probes, failed conversions, and rule mismatches without requiring a debugger. ## Constraints And Assumptions @@ -84,6 +88,9 @@ - Third-party dependencies: - `ffmpeg`, `ffprobe`, and `cpulimit`. - TMDB API access through `TMDB_API_KEY` for metadata enrichment. +- Installation assumptions: + - The Python-side bundle install step and optional Python test extras are managed by `tools/setup.sh`. + - The workstation-preparation step is managed separately by `tools/configure_workstation.sh` or `ffx configure_workstation`. ## Acceptance Scope diff --git a/src/ffx/cli.py b/src/ffx/cli.py index dfa559e..2d18395 100755 --- a/src/ffx/cli.py +++ b/src/ffx/cli.py @@ -58,7 +58,7 @@ def ffx(ctx, database_file, verbose, dry_run): ctx.obj = {} - if ctx.invoked_subcommand in ('setup_dependencies', 'upgrade'): + if ctx.invoked_subcommand in ('configure_workstation', 'upgrade'): ctx.obj['dry_run'] = dry_run ctx.obj['verbosity'] = verbose return @@ -104,8 +104,8 @@ def getRepoRootPath(): return os.path.dirname(os.path.dirname(os.path.dirname(currentFilePath))) -def getPrepareScriptPath(): - return os.path.join(getRepoRootPath(), 'tools', 'prepare.sh') +def getConfigureWorkstationScriptPath(): + return os.path.join(getRepoRootPath(), 'tools', 'configure_workstation.sh') def getBundleVenvDirectory(): @@ -120,22 +120,23 @@ def getBundleRepoPath(): return getRepoRootPath() -@ffx.command(name='setup_dependencies') +@ffx.command(name='configure_workstation') @click.pass_context -@click.option('--check', is_flag=True, default=False, help='Only verify dependency readiness') -@click.argument('prepare_args', nargs=-1, type=click.UNPROCESSED) -def setup_dependencies(ctx, check, prepare_args): - prepareScriptPath = getPrepareScriptPath() +@click.option('--check', is_flag=True, default=False, help='Only verify workstation-configuration readiness') +@click.argument('configure_args', nargs=-1, type=click.UNPROCESSED) +def configure_workstation(ctx, check, configure_args): + """Prepare workstation dependencies and local config after bundle install.""" + configureScriptPath = getConfigureWorkstationScriptPath() - if not os.path.isfile(prepareScriptPath): - raise click.ClickException(f"Preparation script not found at {prepareScriptPath}") + if not os.path.isfile(configureScriptPath): + raise click.ClickException(f"Workstation configuration script not found at {configureScriptPath}") - commandSequence = ['bash', prepareScriptPath] + commandSequence = ['bash', configureScriptPath] if check: commandSequence.append('--check') - commandSequence += list(prepare_args) + commandSequence += list(configure_args) if ctx.obj.get('dry_run', False): click.echo(' '.join(commandSequence)) diff --git a/src/ffx/ffx_controller.py b/src/ffx/ffx_controller.py index 131809e..f3241fc 100644 --- a/src/ffx/ffx_controller.py +++ b/src/ffx/ffx_controller.py @@ -54,6 +54,13 @@ class FfxController(): self.__logger: Logger = context['logger'] + def executeCommandSequence(self, commandSequence): + out, err, rc = executeProcess(commandSequence, context=self.__context) + if rc: + raise click.ClickException(f"Command resulted in error: rc={rc} error={err}") + return out, err, rc + + def generateAV1Tokens(self, quality, preset, subIndex : int = 0): return [f"-c:v:{int(subIndex)}", 'libsvtav1', @@ -288,9 +295,7 @@ class FfxController(): self.__logger.debug("FfxController.runJob(): Running command sequence") if not self.__context['dry_run']: - out, err, rc = executeProcess(commandSequence, context=self.__context) - if rc: - raise click.ClickException(f"Command resulted in error: rc={rc} error={err}") + self.executeCommandSequence(commandSequence) return if videoEncoder == VideoEncoder.AV1: @@ -320,7 +325,7 @@ class FfxController(): self.__logger.debug(f"FfxController.runJob(): Running command sequence") if not self.__context['dry_run']: - executeProcess(commandSequence, context = self.__context) + self.executeCommandSequence(commandSequence) if videoEncoder == VideoEncoder.H264: @@ -350,7 +355,7 @@ class FfxController(): self.__logger.debug(f"FfxController.runJob(): Running command sequence") if not self.__context['dry_run']: - executeProcess(commandSequence, context = self.__context) + self.executeCommandSequence(commandSequence) @@ -382,7 +387,7 @@ class FfxController(): self.__logger.debug(f"FfxController.runJob(): Running command sequence 1") if not self.__context['dry_run']: - executeProcess(commandSequence1, context = self.__context) + self.executeCommandSequence(commandSequence1) commandSequence2 = (commandTokens + self.__targetMediaDescriptor.getImportFileTokens() @@ -409,9 +414,7 @@ class FfxController(): self.__logger.debug(f"FfxController.runJob(): Running command sequence 2") if not self.__context['dry_run']: - out, err, rc = executeProcess(commandSequence2, context = self.__context) - if rc: - raise click.ClickException(f"Command resulted in error: rc={rc} error={err}") + self.executeCommandSequence(commandSequence2) @@ -436,4 +439,4 @@ class FfxController(): str(length), path] - out, err, rc = executeProcess(commandTokens, context = self.__context) + self.executeCommandSequence(commandTokens) diff --git a/src/ffx/process.py b/src/ffx/process.py index b2ab4c4..7db5492 100644 --- a/src/ffx/process.py +++ b/src/ffx/process.py @@ -1,19 +1,23 @@ +import shlex import subprocess -from typing import List +from typing import Iterable, List from .logging_utils import get_ffx_logger -def executeProcess(commandSequence: List[str], directory: str = None, context: dict = None): +COMMAND_TIMED_OUT_RETURN_CODE = 124 +COMMAND_NOT_FOUND_RETURN_CODE = 127 + + +def formatCommandSequence(commandSequence: Iterable[str]) -> str: + return shlex.join([str(token) for token in commandSequence]) + + +def getWrappedCommandSequence(commandSequence: List[str], context: dict = None) -> List[str]: """ niceness -20 bis +19 cpu_percent: 1 bis 99 """ - if context is None: - logger = get_ffx_logger() - else: - logger = context['logger'] - niceSequence = [] niceness = int((context or {}).get('resource_limits', {}).get('niceness', 99)) @@ -24,11 +28,72 @@ def executeProcess(commandSequence: List[str], directory: str = None, context: d if cpu_percent >= 1: niceSequence += ['cpulimit', '-l', str(cpu_percent), '--'] - niceCommand = niceSequence + commandSequence + return niceSequence + [str(token) for token in commandSequence] - logger.debug(f"executeProcess() command sequence: {' '.join(niceCommand)}") - process = subprocess.Popen(niceCommand, stdout=subprocess.PIPE, stderr=subprocess.PIPE, encoding='utf-8', cwd = directory) - output, error = process.communicate() - - return output, error, process.returncode +def getProcessTimeoutSeconds(context: dict = None, timeoutSeconds: float = None): + if timeoutSeconds is None: + timeoutSeconds = (context or {}).get('resource_limits', {}).get('timeout_seconds') + + if timeoutSeconds is None: + return None + + timeoutSeconds = float(timeoutSeconds) + + return timeoutSeconds if timeoutSeconds > 0 else None + + +def executeProcess( + commandSequence: List[str], + directory: str = None, + context: dict = None, + timeoutSeconds: float = None, +): + + logger = context['logger'] if context is not None and 'logger' in context else get_ffx_logger() + wrappedCommandSequence = getWrappedCommandSequence(commandSequence, context=context) + timeoutSeconds = getProcessTimeoutSeconds(context=context, timeoutSeconds=timeoutSeconds) + + logger.debug( + "executeProcess() cwd=%s timeout=%s command=%s", + directory or '.', + timeoutSeconds if timeoutSeconds is not None else 'none', + formatCommandSequence(wrappedCommandSequence), + ) + + try: + completed = subprocess.run( + wrappedCommandSequence, + capture_output=True, + text=True, + cwd=directory, + timeout=timeoutSeconds, + check=False, + ) + except FileNotFoundError as ex: + error = ( + "Command not found while running " + + f"{formatCommandSequence(wrappedCommandSequence)}: {ex.filename or ex}" + ) + logger.error(error) + return '', error, COMMAND_NOT_FOUND_RETURN_CODE + except subprocess.TimeoutExpired as ex: + stdout = ex.stdout or '' + stderr = ex.stderr or '' + error = ( + f"Command timed out after {timeoutSeconds} seconds while running " + + formatCommandSequence(wrappedCommandSequence) + ) + if stderr: + error = f"{error}\n{stderr}" + logger.error(error) + return stdout, error, COMMAND_TIMED_OUT_RETURN_CODE + + if completed.returncode != 0: + logger.warning( + "executeProcess() rc=%s command=%s", + completed.returncode, + formatCommandSequence(wrappedCommandSequence), + ) + + return completed.stdout, completed.stderr, completed.returncode diff --git a/tests/unit/test_process.py b/tests/unit/test_process.py new file mode 100644 index 0000000..a379444 --- /dev/null +++ b/tests/unit/test_process.py @@ -0,0 +1,52 @@ +from __future__ import annotations + +from pathlib import Path +import sys +import unittest + + +SRC_ROOT = Path(__file__).resolve().parents[2] / "src" + +if str(SRC_ROOT) not in sys.path: + sys.path.insert(0, str(SRC_ROOT)) + + +from ffx.process import ( # noqa: E402 + COMMAND_NOT_FOUND_RETURN_CODE, + COMMAND_TIMED_OUT_RETURN_CODE, + executeProcess, +) + + +class ProcessTests(unittest.TestCase): + def test_execute_process_returns_stdout_for_success(self): + out, err, rc = executeProcess( + [sys.executable, "-c", "print('hello from process')"] + ) + + self.assertEqual(0, rc) + self.assertEqual("", err) + self.assertEqual("hello from process\n", out) + + def test_execute_process_maps_missing_command_to_stable_error(self): + out, err, rc = executeProcess(["ffx-command-that-does-not-exist"]) + + self.assertEqual("", out) + self.assertEqual(COMMAND_NOT_FOUND_RETURN_CODE, rc) + self.assertIn("Command not found while running", err) + self.assertIn("ffx-command-that-does-not-exist", err) + + def test_execute_process_maps_timeout_to_stable_error(self): + out, err, rc = executeProcess( + [sys.executable, "-c", "import time; time.sleep(0.2)"], + timeoutSeconds=0.05, + ) + + self.assertEqual("", out) + self.assertEqual(COMMAND_TIMED_OUT_RETURN_CODE, rc) + self.assertIn("Command timed out", err) + self.assertIn(sys.executable, err) + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/prepare.sh b/tools/configure_workstation.sh similarity index 91% rename from tools/prepare.sh rename to tools/configure_workstation.sh index f3c49ed..7d302bc 100755 --- a/tools/prepare.sh +++ b/tools/configure_workstation.sh @@ -2,8 +2,6 @@ set -u -SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" - CONFIG_DIR="${FFX_CONFIG_DIR:-${HOME}/.local/etc}" CONFIG_FILE="${FFX_CONFIG_FILE:-${CONFIG_DIR}/ffx.json}" VAR_DIR="${FFX_VAR_DIR:-${HOME}/.local/var/ffx}" @@ -11,6 +9,7 @@ LOG_DIR="${FFX_LOG_DIR:-${HOME}/.local/var/log}" DATABASE_FILE="${FFX_DATABASE_FILE:-${VAR_DIR}/ffx.db}" CHECK_ONLY=0 +WITH_TESTS=0 MUTATIONS=0 INSTALL_FAILURES=0 @@ -33,12 +32,13 @@ fi usage() { cat < ${VENV_FFX} + - optionally install Python packages required for modern tests Options: - --check Report readiness only. Do not create or modify anything. - --help Show this help text. + --check Report readiness only. Do not create or modify anything. + --with-tests Also install and verify Python packages required for modern tests. + --help Show this help text. + +Notes: + - This is the first installation step. + - tools/configure_workstation.sh is the second step and configures system dependencies plus local user files. EOF } @@ -100,6 +107,10 @@ check_venv_ffx() { [ -x "${VENV_FFX}" ] } +check_venv_pytest() { + check_venv_dir && "${VENV_PYTHON}" -m pytest --version >/dev/null 2>&1 +} + check_bashrc_file() { [ -f "${BASHRC_FILE}" ] } @@ -136,6 +147,14 @@ detail_venv_ffx() { fi } +detail_venv_pytest() { + if check_venv_pytest; then + "${VENV_PYTHON}" -m pytest --version 2>/dev/null | head -n 1 + else + printf 'missing pytest in %s' "${VENV_DIR}" + fi +} + detail_bashrc_file() { if check_bashrc_file; then printf '%s' "${BASHRC_FILE}" @@ -186,6 +205,17 @@ print_status_report() { READINESS_FAILURES=$((READINESS_FAILURES + 1)) fi + if [ "${WITH_TESTS}" -eq 1 ]; then + echo + echo "Bundle test package status:" + if check_venv_pytest; then + report_component ok "bundle pytest" "$(detail_venv_pytest)" + else + report_component failed "bundle pytest" "$(detail_venv_pytest)" + READINESS_FAILURES=$((READINESS_FAILURES + 1)) + fi + fi + echo echo "Shell exposure status:" if check_bashrc_file; then @@ -220,11 +250,23 @@ ensure_bundle_venv() { return 1 fi - printf 'Installing FFX package into %s...\n' "${VENV_DIR}" - if ! "${VENV_PIP}" install --editable "${ROOT_DIR}"; then - printf 'Failed to install FFX package into %s.\n' "${VENV_DIR}" >&2 - INSTALL_FAILURES=$((INSTALL_FAILURES + 1)) - return 1 + if [ "${WITH_TESTS}" -eq 1 ]; then + printf 'Installing FFX package and test extras into %s...\n' "${VENV_DIR}" + if ! ( + cd "${ROOT_DIR}" && + "${VENV_PIP}" install --editable '.[test]' + ); then + printf 'Failed to install FFX package and test extras into %s.\n' "${VENV_DIR}" >&2 + INSTALL_FAILURES=$((INSTALL_FAILURES + 1)) + return 1 + fi + else + printf 'Installing FFX package into %s...\n' "${VENV_DIR}" + if ! "${VENV_PIP}" install --editable "${ROOT_DIR}"; then + printf 'Failed to install FFX package into %s.\n' "${VENV_DIR}" >&2 + INSTALL_FAILURES=$((INSTALL_FAILURES + 1)) + return 1 + fi fi return 0 @@ -300,6 +342,9 @@ parse_args() { --check) CHECK_ONLY=1 ;; + --with-tests) + WITH_TESTS=1 + ;; --help|-h) usage exit 0 diff --git a/tools/test.sh b/tools/test.sh new file mode 100755 index 0000000..9480290 --- /dev/null +++ b/tools/test.sh @@ -0,0 +1,22 @@ +#!/usr/bin/env bash + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)" + +PYTHON_BIN="${FFX_PYTHON:-${HOME}/.local/share/ffx.venv/bin/python}" + +if [[ ! -x "${PYTHON_BIN}" ]]; then + echo "Missing Python interpreter: ${PYTHON_BIN}" >&2 + echo "Set FFX_PYTHON to a suitable interpreter if needed." >&2 + exit 1 +fi + +cd "${REPO_ROOT}" + +exec "${PYTHON_BIN}" -m pytest \ + --ignore=tests/legacy \ + --ignore=tests/support \ + tests \ + "$@"