Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for embedded Python on Windows #3161

Merged
merged 4 commits into from
Apr 22, 2024
Merged

Conversation

@charliermarsh charliermarsh added windows Specific to the Windows platform compatibility Compatibility with a specification or another tool labels Apr 20, 2024
@charliermarsh charliermarsh force-pushed the charlie/embed branch 12 times, most recently from 52f72b4 to 51f3aa7 Compare April 20, 2024 21:57
@charliermarsh charliermarsh requested a review from konstin April 20, 2024 21:57
@charliermarsh charliermarsh marked this pull request as ready for review April 20, 2024 21:57
}

// Copy `.zip` files from the top-level.
let entries = match fs_err::read_dir(python_home) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Virtualenv is stricter on what it matches here, but I don't see other .zip files anyway?

@charliermarsh charliermarsh force-pushed the charlie/embed branch 2 times, most recently from e3c77f2 to daffa93 Compare April 20, 2024 21:58
io::ErrorKind::NotFound,
"The Python interpreter needs to have a parent directory",
)
})?;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Just moved this up because I need it for symlinking.)

Err(err) if err.kind() == io::ErrorKind::NotFound => {
return Err(Error::IO(io::Error::new(
io::ErrorKind::NotFound,
format!(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we de-nest this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like, de-nest the entire if hierarchy? Or de-nest the error message in some way?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either way that doesn't get us to indentation level 14, i tried and denesting the entire thing seems the easiest

    // No symlinking on Windows, at least not on a regular non-dev non-admin Windows install.
    if cfg!(windows) {
        copy_launcher_windows(interpreter, &base_python, &scripts, python_home)?;
    }
/// <https://github.com/python/cpython/blob/d457345bbc6414db0443819290b04a9a4333313d/Lib/venv/__init__.py#L261-L267>
/// <https://github.com/pypa/virtualenv/blob/d9fdf48d69f0d0ca56140cf0381edbb5d6fe09f5/src/virtualenv/create/via_global_ref/builtin/cpython/cpython3.py#L78-L83>
/// There's two kinds of applications on windows: Those that allocate a console (python.exe)
/// and those that don't because they use window(s) (pythonw.exe).
fn copy_launcher_windows(
    interpreter: &Interpreter,
    base_python: &Path,
    scripts: &Path,
    python_home: &Path,
) -> Result<(), Error> {
    // First priority: the `python.exe` and `pythonw.exe` shims.
    for python_exe in ["python.exe", "pythonw.exe"] {
        let shim = interpreter
            .stdlib()
            .join("venv")
            .join("scripts")
            .join("nt")
            .join(python_exe);
        match fs_err::copy(shim, scripts.join(python_exe)) {
            Ok(_) => return Ok(()),
            Err(err) if err.kind() == io::ErrorKind::NotFound => {}
            Err(err) => {
                return Err(err.into());
            }
        }

        // Second priority: the `venvlauncher.exe` and `venvwlauncher.exe` shims.
        // These are equivalent to the `python.exe` and `pythonw.exe` shims, which were
        // renamed in Python 3.13.
        let launcher = match python_exe {
            "python.exe" => "venvlauncher.exe",
            "pythonw.exe" => "venvwlauncher.exe",
            _ => unreachable!(),
        };
        let shim = interpreter
            .stdlib()
            .join("venv")
            .join("scripts")
            .join("nt")
            .join(launcher);

        match fs_err::copy(shim, scripts.join(python_exe)) {
            Ok(_) => return Ok(()),
            Err(err) if err.kind() == io::ErrorKind::NotFound => {}
            Err(err) => {
                return Err(err.into());
            }
        }

        // Third priority: on Conda at least, we can look for the launcher shim next to
        // the Python executable itself.
        let shim = base_python.with_file_name(launcher);
        match fs_err::copy(shim, scripts.join(python_exe)) {
            Ok(_) => {}
            Err(err) if err.kind() == io::ErrorKind::NotFound => {}
            Err(err) => {
                return Err(err.into());
            }
        }

        // Fourth priority: if the launcher shim doesn't exist, assume this is
        // an embedded Python. Copy the Python executable itself, along with
        // the DLLs, `.pyd` files, and `.zip` files in the same directory.
        match fs_err::copy(
            base_python.with_file_name(python_exe),
            scripts.join(python_exe),
        ) {
            Ok(_) => {}
            Err(err) if err.kind() == io::ErrorKind::NotFound => {
                return Err(Error::IO(io::Error::new(
                    io::ErrorKind::NotFound,
                    format!(
                        "Could not find a suitable Python executable for the virtual environment (tried: {}, {}, {}, {})",
                        interpreter
                            .stdlib()
                            .join("venv")
                            .join("scripts")
                            .join("nt")
                            .join(python_exe)
                            .user_display(),
                        interpreter
                            .stdlib()
                            .join("venv")
                            .join("scripts")
                            .join("nt")
                            .join(launcher)
                            .user_display(),
                        base_python.with_file_name(launcher).user_display(),
                        base_python
                            .with_file_name(python_exe)
                            .user_display(),
                    )
                )));
            }
            Err(err) => {
                return Err(err.into());
            }
        }

        // Copy `.dll` and `.pyd` files from the top-level, and from the
        // `DLLs` subdirectory (if it exists).
        for directory in [
            python_home,
            interpreter.base_prefix().join("DLLs").as_path(),
        ] {
            let entries = match fs_err::read_dir(directory) {
                Ok(read_dir) => read_dir,
                Err(err) if err.kind() == io::ErrorKind::NotFound => {
                    continue;
                }
                Err(err) => {
                    return Err(err.into());
                }
            };
            for entry in entries {
                let entry = entry?;
                let path = entry.path();
                if path.extension().is_some_and(|ext| {
                    ext.eq_ignore_ascii_case("dll") || ext.eq_ignore_ascii_case("pyd")
                }) {
                    if let Some(file_name) = path.file_name() {
                        fs_err::copy(&path, scripts.join(file_name))?;
                    }
                }
            }
        }

        // Copy `.zip` files from the top-level.
        let entries = match fs_err::read_dir(python_home) {
            Ok(read_dir) => read_dir,
            Err(err) if err.kind() == io::ErrorKind::NotFound => {
                continue;
            }
            Err(err) => {
                return Err(err.into());
            }
        };

        for entry in entries {
            let entry = entry?;
            let path = entry.path();
            if path
                .extension()
                .is_some_and(|ext| ext.eq_ignore_ascii_case("zip"))
            {
                if let Some(file_name) = path.file_name() {
                    fs_err::copy(&path, scripts.join(file_name))?;
                }
            }
        }
    }
    Ok(())
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup seems good! Will do. I assumed this is what you meant.

) {
Ok(_) => {}
Err(err) if err.kind() == io::ErrorKind::NotFound => {
return Err(Error::IO(io::Error::new(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be a custom variant

@charliermarsh charliermarsh merged commit 41b29b2 into main Apr 22, 2024
39 checks passed
@charliermarsh charliermarsh deleted the charlie/embed branch April 22, 2024 17:34
@Pixel-Minions
Copy link

Hi @charliermarsh It works great! Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compatibility Compatibility with a specification or another tool windows Specific to the Windows platform
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Virtualenv creation is failing on Python embedded versions
3 participants