Skip to content

Support inheriting jobserver when using cargo run #12597

Closed
#12776
Closed
Support inheriting jobserver when using cargo run#12597
#12776
Labels
A-jobserverArea: jobserver, concurrency, parallelismC-feature-requestCategory: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted`Command-runS-needs-team-inputStatus: Needs input from team on whether/how to proceed.

Description

@petrochenkov
petrochenkov
opened on Aug 30, 2023

Problem

Sometimes commands run through cargo run actually perform build actions for a larger build system.

For example in #10511 (comment) the scenario is running something like cargo run -p binding_generator which generates headers to be used during the build later.

And in rust-lang/rust#113730 (comment) cargo-miri is set as target.runner and runs as a part of cargo run (and calls build tools like cargo and rustc recursively).

Proposed Solution

In all cases like these cargo shouldn't close its jobserver file descriptors (or other handles), but pass them to the tool being run instead.

I think doing this under an options would be fine.
The default for such option can be decided upon later (probably kept being false, which is equivalent to the current behavior).

Notes

No response

Activity

  1. petrochenkov
    added
    C-feature-requestCategory: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted`
    S-triageStatus: This issue is waiting on initial triage.
    on Aug 30, 2023
  2. epage
    added
    Command-run
    on Aug 30, 2023
  3. weihanglo
    added
    E-easyExperience: Easy
    I-nominated-to-discussTo be discussed during issue triage on the next Cargo team meeting
    A-jobserverArea: jobserver, concurrency, parallelism
    S-needs-team-inputStatus: Needs input from team on whether/how to proceed.
    and removed
    S-triageStatus: This issue is waiting on initial triage.
    E-easyExperience: Easy
    on Aug 30, 2023
  4. weihanglo
    removed
    I-nominated-to-discussTo be discussed during issue triage on the next Cargo team meeting
    on Sep 26, 2023
  5. bors
    closed this as completedin #12776on Jan 20, 2024

  6. RalfJung commented on Jan 20, 2024

    @RalfJung
    RalfJung
    on Jan 20, 2024
    Member

    In all cases like these cargo shouldn't close its jobserver file descriptors (or other handles), but pass them to the tool being run instead.

    From what I understood in the PR, this does not seem to be what the PR does. The PR only makes it so that if cargo uses an external jobserver, that one is now propagated to cargo run as well.

    So maybe the issue should be reopened?

  7. weihanglo commented on Jan 20, 2024

    @weihanglo
    weihanglo
    on Jan 20, 2024
    Member

    From reading this issue title, and the context of #10511 (comment), I made an assumption that miri (or some other tool alike) would like to inherit existing jobserver from the environment, exactly as how Cargo treats external commands today since #10511. #12776 made it so. It's my fault that didn't request more details from the issue author.

    The line between compilation and execution is not always clear in cargo, like --keep-going flag. What is kept going, rustc or the final binary run? 🤯

    That said, if this doesn't resolve the miri issue, I am sorry 😞. I would suggest opening another issue to clarify the workflow and expectation, and we could consider a piped-based jobserver by default (see PR description in #12776).

  8. RalfJung commented on Jan 22, 2024

    @RalfJung