compile_pip_requirements fails if the output file `requirements.txt` does not already exist

[diazona]

🐞 bug report

Affected Rule

The issue is caused by the rule: compile_pip_requirements

Is this a regression?

Not as far as I can tell. I'm not aware of any version of rules_python in which this works (although I haven't exhaustively checked).

Description

I'm trying to use compile_pip_requirements to create requirements_lock.txt for the first time from requirements.in. This is exactly the same issue as #1129 (now closed), except that I encountered it on Linux whereas the other report was on Windows. (I would guess the issue is platform-independent.) I wasn't sure of the right protocol for "reactivating" an issue that was closed for inactivity, but if I was wrong to post this, feel free to reopen the original issue instead.

🔬 Minimal Reproduction

WORKSPACE

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_python",
    sha256 = "8844347f050932fb339f8bce78ac3bd0f0dc118d347ce4d2dcc39490f2954941",
    strip_prefix = "rules_python-76f1c76f60ccb536d3b3e2c9f023d8063f40bcd5",
    url = "https://github.com/bazelbuild/rules_python/archive/76f1c76f60ccb536d3b3e2c9f023d8063f40bcd5.tar.gz",
)

load("@rules_python//python:repositories.bzl", "py_repositories")

py_repositories()

BUILD

load("@rules_python//python:pip.bzl", "compile_pip_requirements")

compile_pip_requirements(
    name = "requirements",
    src = "//:requirements.in",
    visibility = ["//visibility:public"],
)

requirements.in

python-dummy==0.1

I pushed a repository with these files as well: https://github.com/diazona/bazel-python-error

🔥 Exception or Error

$ bazelisk run //:requirements.update
INFO: Analyzed target //:requirements.update (1 packages loaded, 4 targets configured).
ERROR: /home/diazona/tmp/bazel-python-error/BUILD:3:25: Middleman _middlemen/requirements.update-runfiles failed: missing input file '//:requirements.txt'
ERROR: /home/diazona/tmp/bazel-python-error/BUILD:3:25: Middleman _middlemen/requirements.update-runfiles failed: 1 input file(s) do not exist
Target //:requirements.update failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /home/diazona/tmp/bazel-python-error/BUILD:3:25 Middleman _middlemen/requirements.update-runfiles failed: 1 input file(s) do not exist
INFO: Elapsed time: 0.285s, Critical Path: 0.01s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target

🌍 Your Environment

Operating System:

Gentoo Linux

$ uname -a
Linux acetylene09 6.7.0-gentoo #1 SMP PREEMPT_DYNAMIC Sun Jan 14 21:02:35 PST 2024 x86_64 AMD Ryzen 7 2700X Eight-Core Processor AuthenticAMD GNU/Linux

Output of bazel version:

$ bazelisk version
Bazelisk version: development
Build label: 7.0.2
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Thu Jan 25 16:13:35 2024 (1706199215)
Build timestamp: 1706199215
Build timestamp as int: 1706199215

I also tried versions 6.5.0, 6.3.1, and 5.4.1 (all through Bazelisk, by setting USE_BAZEL_VERSION), with the same results.

Rules_python version:

Commit 76f1c76f60ccb536d3b3e2c9f023d8063f40bcd5, which is the current head of main as I'm writing this

Anything else relevant?

If I create requirements.txt manually, regardless of what I put in it, the update command seems to work as expected: it replaces the content of requirements.txt with a list of requirements.

And I might very well be misunderstanding what compile_pip_requirements is supposed to do. If the existing behavior (fail if the output file doesn't exist) is expected, then consider this a request to improve the clarity of the documentation. (I'd be happy to offer a PR in that case.)

[brunobely]

This example says it was generated by running bazel run //:requirements.update. I wonder if that must be (or can be) depended on?

[diazona]

If I understand correctly, I don't think so, because the error I'm reporting happens when building //:requirements.update. Depending on it would just trigger the same error whenever its dependent target gets built.

[aignas]

I am not sure if this is a doc issue where we just ask the users to create an empty file. At the end of the day this is something that gets passed to a target definition and the error happens within bazel itself, not within the code that is owned by rules python.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

[aignas]

Just a note that the rule would need to use a glob which I have a POC for in https://github.com/bazelbuild/rules_python/blob/27276b6de1362a1ed9c1f681e4b3f53aa8bcdbcf/python/uv/private/lock.bzl#L65

[crackcomm]

gazelle_python_manifest also fails if the manifest (gazelle_python.yaml) doesn't exist.

[aignas]

The new lock rule that we built for uv does not have this limitation. Use that instead and give feedback.