Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support remote scripts with uv run #6375

Merged
merged 32 commits into from
Oct 10, 2024
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
bf578d3
feat(run): Support remote scripts
manzt Aug 21, 2024
f4b35c1
Add test
manzt Aug 21, 2024
8a0002b
fix test
manzt Aug 21, 2024
8c39f07
Handle local file priority
manzt Sep 25, 2024
5885a1c
Add line break to resolve_script_target for correct rustdoc rendering
manzt Sep 26, 2024
94777c3
Use Path::try_exists
manzt Sep 26, 2024
28a2e16
Exit early if target doesn't look like a URL
manzt Sep 26, 2024
560d57e
Update remote fixture
manzt Sep 26, 2024
4898958
Update script URL with full hash
manzt Sep 26, 2024
f3b7754
Fix remote script test
manzt Sep 26, 2024
a1ccd9a
Use matches! for early return
manzt Sep 26, 2024
7dc86ce
Update fixture hash
manzt Sep 26, 2024
284b919
Apply patch; explicity drop tmpfile
manzt Sep 26, 2024
b15f948
Strip .py suffix from file_stem
manzt Sep 26, 2024
93a8720
Add Download reporting message
manzt Sep 26, 2024
c8c2dab
Update tests
manzt Sep 26, 2024
9c18529
Only check for local path on unix
manzt Sep 27, 2024
d935c60
Update docs
manzt Sep 27, 2024
83e8ee3
HTTP(S)
manzt Sep 27, 2024
842ee29
Run cargo dev generate-cli-reference
manzt Sep 27, 2024
d0ac512
Update docs inline
manzt Sep 27, 2024
3b8a3d1
Update docs/reference/cli.md
manzt Sep 27, 2024
d290ff1
Reuse configured user printer
manzt Sep 27, 2024
ca955d1
Update snapshot
manzt Sep 27, 2024
0f6cbcc
Fix tests
manzt Sep 27, 2024
fb1f9df
Merge main
manzt Oct 1, 2024
1e19caa
Re-generate reference cli.md
manzt Oct 1, 2024
4436853
Move PathBuf reference within unix-only testg
manzt Oct 1, 2024
7b27cdc
Merge main
manzt Oct 10, 2024
356bf68
Replace uv_fs::write_atomic with write_all
manzt Oct 10, 2024
be768e3
Use client builder
manzt Oct 10, 2024
2193755
Respect workspace
manzt Oct 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 16 additions & 7 deletions crates/uv-cli/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use std::ffi::OsString;
use std::ops::Deref;
use std::ops::{Deref, DerefMut};
use std::path::PathBuf;
use std::str::FromStr;

Expand Down Expand Up @@ -600,12 +600,13 @@ pub enum ProjectCommand {
///
/// Ensures that the command runs in a Python environment.
///
/// When used with a file ending in `.py`, the file will be treated as a
/// script and run with a Python interpreter, i.e., `uv run file.py` is
/// equivalent to `uv run python file.py`. If the script contains inline
/// dependency metadata, it will be installed into an isolated, ephemeral
/// environment. When used with `-`, the input will be read from stdin,
/// and treated as a Python script.
/// When used with a file ending in `.py` or an HTTP(S) URL, the file
/// will be treated as a script and run with a Python interpreter,
/// i.e., `uv run file.py` is equivalent to `uv run python file.py`.
/// For URLs, the script is temporarily downloaded before execution. If
/// the script contains inline dependency metadata, it will be installed
/// into an isolated, ephemeral environment. When used with `-`, the
/// input will be read from stdin, and treated as a Python script.
///
/// When used in a project, the project environment will be created and
/// updated before invoking the command.
Expand Down Expand Up @@ -2305,6 +2306,14 @@ impl Deref for ExternalCommand {
}
}

impl DerefMut for ExternalCommand {
fn deref_mut(&mut self) -> &mut Self::Target {
match self {
Self::Cmd(cmd) => cmd,
}
}
}

impl ExternalCommand {
pub fn split(&self) -> (Option<&OsString>, &[OsString]) {
match self.as_slice() {
Expand Down
1 change: 1 addition & 0 deletions crates/uv/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ miette = { workspace = true, features = ["fancy-no-backtrace"] }
owo-colors = { workspace = true }
rayon = { workspace = true }
regex = { workspace = true }
reqwest = { workspace = true }
rustc-hash = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
Expand Down
75 changes: 68 additions & 7 deletions crates/uv/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,12 @@ use uv_cli::{
compat::CompatArgs, BuildBackendCommand, CacheCommand, CacheNamespace, Cli, Commands,
PipCommand, PipNamespace, ProjectCommand,
};
use uv_cli::{PythonCommand, PythonNamespace, ToolCommand, ToolNamespace, TopLevelArgs};
use uv_cli::{
ExternalCommand, PythonCommand, PythonNamespace, ToolCommand, ToolNamespace, TopLevelArgs,
};
#[cfg(feature = "self-update")]
use uv_cli::{SelfCommand, SelfNamespace, SelfUpdateArgs};
use uv_fs::CWD;
use uv_fs::{write_atomic, CWD};
use uv_requirements::RequirementsSource;
use uv_scripts::Pep723Script;
use uv_settings::{Combine, FilesystemOptions, Options};
Expand All @@ -42,8 +44,54 @@ pub(crate) mod printer;
pub(crate) mod settings;
pub(crate) mod version;

/// Resolves the script target for a run command.
manzt marked this conversation as resolved.
Show resolved Hide resolved
///
/// If it's a local file, does nothing. If it's a URL, downloads the content
/// to a temporary file and updates the command. Prioritizes local files over URLs.
/// Returns Some(NamedTempFile) if a remote script was downloaded, None otherwise.
async fn resolve_script_target(
command: &mut ExternalCommand,
) -> Result<Option<tempfile::NamedTempFile>> {
let Some(target) = command.get_mut(0) else {
return Ok(None);
};
manzt marked this conversation as resolved.
Show resolved Hide resolved

// Only continue if we are absolutely certain no local file exists.
//
// We don't do this check on Windows since the file path would
// be invalid anyway, and thus couldn't refer to a local file.
if cfg!(unix) && !matches!(Path::new(target).try_exists(), Ok(false)) {
return Ok(None);
}
manzt marked this conversation as resolved.
Show resolved Hide resolved

let maybe_url = target.to_string_lossy();

// Only continue if the target truly looks like a URL.
if !(maybe_url.starts_with("http://") || maybe_url.starts_with("https://")) {
return Ok(None);
}

let script_url = url::Url::parse(&maybe_url)?;
let file_stem = script_url
.path_segments()
.and_then(std::iter::Iterator::last)
.and_then(|segment| segment.strip_suffix(".py"))
.unwrap_or("script");

let temp_file = tempfile::Builder::new()
.prefix(file_stem)
.suffix(".py")
manzt marked this conversation as resolved.
Show resolved Hide resolved
.tempfile()?;

let response = reqwest::get(script_url).await?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to go through the uv client. Otherwise, it won't respect --offline, SSL certs, etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, using the client builder now. I went with re-resolving the global args locally since the contents of file downloaded by this script updates the resolution later.

let contents = response.bytes().await?;
write_atomic(&temp_file, &contents).await?;
*target = temp_file.path().into();
Ok(Some(temp_file))
}

#[instrument(skip_all)]
async fn run(cli: Cli) -> Result<ExitStatus> {
async fn run(mut cli: Cli) -> Result<ExitStatus> {
// Enable flag to pick up warnings generated by workspace loading.
if !cli.top_level.global_args.quiet {
uv_warnings::enable();
Expand Down Expand Up @@ -130,11 +178,13 @@ async fn run(cli: Cli) -> Result<ExitStatus> {
};

// Parse the external command, if necessary.
let run_command = if let Commands::Project(command) = &*cli.command {
let mut maybe_tempfile: Option<tempfile::NamedTempFile> = None;
let run_command = if let Commands::Project(command) = &mut *cli.command {
if let ProjectCommand::Run(uv_cli::RunArgs {
command, module, ..
}) = &**command
}) = &mut **command
{
maybe_tempfile = resolve_script_target(command).await?;
Some(RunCommand::from_args(command, *module)?)
} else {
None
Expand Down Expand Up @@ -208,6 +258,15 @@ async fn run(cli: Cli) -> Result<ExitStatus> {
Printer::Default
};

// We only have a tempfile if we downloaded a remote script.
if let Some(temp_file) = maybe_tempfile.as_ref() {
writeln!(
printer.stderr(),
"Downloaded remote script to: {}",
temp_file.path().display().cyan(),
)?;
}

// Configure the `warn!` macros, which control user-facing warnings in the CLI.
if globals.quiet {
uv_warnings::disable();
Expand Down Expand Up @@ -255,7 +314,7 @@ async fn run(cli: Cli) -> Result<ExitStatus> {
// Configure the cache.
let cache = Cache::from_settings(cache_settings.no_cache, cache_settings.cache_dir)?;

match *cli.command {
let result = match *cli.command {
Commands::Help(args) => commands::help(
args.command.unwrap_or_default().as_slice(),
printer,
Expand Down Expand Up @@ -1144,7 +1203,9 @@ async fn run(cli: Cli) -> Result<ExitStatus> {
commands::build_backend::prepare_metadata_for_build_editable(&wheel_directory)
}
},
}
};
drop(maybe_tempfile);
result
}

/// Run a [`ProjectCommand`].
Expand Down
60 changes: 60 additions & 0 deletions crates/uv/tests/run.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2164,3 +2164,63 @@ fn run_script_without_build_system() -> Result<()> {

Ok(())
}

#[test]
fn run_remote_pep723_script() {
let context = TestContext::new("3.12").with_filtered_python_names();
let mut filters = context.filters();
filters.push((
r"(?m)^Reading inline script metadata from:.*\.py$",
"Reading inline script metadata from: [TEMP_PATH].py",
));
filters.push((
r"(?m)^Downloaded remote script to:.*\.py$",
"Downloaded remote script to: [TEMP_PATH].py",
));
uv_snapshot!(filters, context.run().arg("https://raw.githubusercontent.com/astral-sh/uv/df45b9ac2584824309ff29a6a09421055ad730f6/scripts/uv-run-remote-script-test.py").arg("CI"), @r###"
success: true
exit_code: 0
----- stdout -----
Hello CI, from uv!

----- stderr -----
Downloaded remote script to: [TEMP_PATH].py
Reading inline script metadata from: [TEMP_PATH].py
Resolved 4 packages in [TIME]
Prepared 4 packages in [TIME]
Installed 4 packages in [TIME]
+ markdown-it-py==3.0.0
+ mdurl==0.1.2
+ pygments==2.17.2
+ rich==13.7.1
"###);
}

#[cfg(unix)] // A URL could be a valid filepath on Unix but not on Windows
#[test]
fn run_url_like_with_local_file_priority() -> Result<()> {
let context = TestContext::new("3.12");

let url = "https://example.com/path/to/main.py";
let local_path: std::path::PathBuf = ["https:", "", "example.com", "path", "to", "main.py"]
.iter()
.collect();

// replace with URL-like filepath
let test_script = context.temp_dir.child(local_path);
test_script.write_str(indoc! { r#"
print("Hello, world!")
"#
})?;

uv_snapshot!(context.filters(), context.run().arg(url), @r###"
success: true
exit_code: 0
----- stdout -----
Hello, world!

----- stderr -----
"###);

Ok(())
}
2 changes: 1 addition & 1 deletion docs/reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Run a command or script.

Ensures that the command runs in a Python environment.

When used with a file ending in `.py`, the file will be treated as a script and run with a Python interpreter, i.e., `uv run file.py` is equivalent to `uv run python file.py`. If the script contains inline dependency metadata, it will be installed into an isolated, ephemeral environment. When used with `-`, the input will be read from stdin, and treated as a Python script.
When used with a file ending in `.py` or an HTTP(S) URL, the file will be treated as a script and run with a Python interpreter, i.e., `uv run file.py` is equivalent to `uv run python file.py`. For URLs, the script is temporarily downloaded before execution. If the script contains inline dependency metadata, it will be installed into an isolated, ephemeral environment. When used with `-`, the input will be read from stdin, and treated as a Python script.
manzt marked this conversation as resolved.
Show resolved Hide resolved

When used in a project, the project environment will be created and updated before invoking the command.

Expand Down
Loading