worker_bootstrap.py 3.7 KB

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667
  1. """Cron external worker: the PM dependency boot its own entry point never gets.
  2. ``sys.executable -m cron.scheduler --external-worker-file ...``
  3. (``cron/scheduler.py::_launch_external_cron_worker``) is a full Hermes entry point that
  4. reaches ``hermes_bootstrap`` only from its ``__main__``, and it imports Hermes packages the moment it
  5. starts. ``cron/scheduler_worker_env.py`` restores the committed generation's
  6. ``site-packages`` on its ``PYTHONPATH`` so those imports resolve, but a pinned path is not a
  7. boot: the worker holds no lease on the generation, so the PM collector may remove it
  8. between the gateway's exit and the worker's next import (#122290 review), and its
  9. ``sys.path`` never ran the generation's ``.pth`` files.
  10. ``pm.environments.activate_dependencies`` is exactly that boot -- the same call
  11. ``hermes_bootstrap`` makes for every other entry point -- and it leases the generation it
  12. selects for the life of the process. ``worker_bootstrap()`` runs it at the top of
  13. ``cron/__init__.py`` -- ``-m cron.scheduler`` executes the package before the module, and the
  14. package's first import (``cron.jobs`` -> ``hermes_yaml`` -> ``ruamel``) is already a
  15. dependency -- and does nothing unless ``_launch_external_cron_worker`` marked this child: the
  16. gateway already booted through ``hermes_bootstrap``, and every other importer of
  17. ``cron.scheduler`` is an interpreter that owns its own dependencies.
  18. A genuine activation failure propagates: continuing on the inherited, unleased path lets the
  19. collector delete the generation under a live worker. The worker then exits before its ownership
  20. acknowledgement, which ``_launch_external_cron_worker`` already reports as a failed dispatch with
  21. the worker's stderr. PM's legitimate no-ops (no committed generation under a venv, wheel, Nix)
  22. return normally.
  23. That boot covers dependencies only. ``hermes_bootstrap`` also finishes a pending source update
  24. and may relaunch the process into a fresh one (``hermes_cli/venv_sync.py::relaunch_command``, an
  25. ``-I`` interpreter that ignores the pinned ``PYTHONPATH``), and re-executes it after restoring a
  26. tree a killed ``hermes update`` half-wrote. ``finish_worker_boot()`` runs it from
  27. the worker's ``__main__`` before the payload is read, so a relaunch replays the whole worker: the
  28. payload is still on disk and the marker still set, and the new process boots its dependencies
  29. again. ``hermes_bootstrap`` re-runs the (idempotent) dependency boot; the package-init call above
  30. must stay, because ``cron.jobs`` needs dependencies before ``__main__`` can run.
  31. """
  32. from __future__ import annotations
  33. import os
  34. from pathlib import Path
  35. # Set by ``_launch_external_cron_worker`` in the child's env and consumed by
  36. # ``finish_worker_boot()`` -- after any relaunch, before the ack -- so it never reaches the
  37. # worker's own children; they inherit the activated PYTHONPATH instead.
  38. # Distinct from ``_HERMES_CRON_EXTERNAL_WORKER`` (the owning execution id) in scheduler.py.
  39. WORKER_MARKER = "_HERMES_CRON_WORKER_BOOT"
  40. def worker_bootstrap() -> None:
  41. """Run PM's dependency boot in the marked external worker (again after a relaunch)."""
  42. if not os.environ.get(WORKER_MARKER):
  43. return
  44. from pm.environments import activate_dependencies
  45. activate_dependencies(Path(__file__).resolve().parent.parent)
  46. def finish_worker_boot() -> None:
  47. """Run ``hermes_bootstrap`` before the worker reads its payload; may relaunch the worker.
  48. Not from ``worker_bootstrap()``: while ``-m cron.scheduler`` executes ``cron/__init__.py``,
  49. ``sys.argv[0]`` is still ``-m`` and ``__main__`` has no spec, so the relaunch command could
  50. not name the module to re-run. In ``__main__`` both are set.
  51. """
  52. import hermes_bootstrap
  53. os.environ.pop(WORKER_MARKER, None)