Skip to content

Commit

Permalink
Auto merge of #8950 - ehuss:publish-tarball-metadata, r=alexcrichton
Browse files Browse the repository at this point in the history
Workaround fs issue in `cargo publish`.

`cargo publish` can fail on some filesystems with a mysterious "No such file or directory". The issue is that `statx` (and `fstat`) will fail on the 9p filesystem (which happens to be used by WSL2) after a file has been [renamed](https://github.com/rust-lang/cargo/blame/27187096a380fdd3b747b5d5ec3396b7af67a6f9/src/cargo/ops/cargo_package.rs#L133-L138). The solution for this workaround is to use seek instead of fstat to determine the length of the tarball.

More information about the 9p problem can be found at https://bugs.launchpad.net/qemu/+bug/1336794 and https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg06382.html.

Fixes #8439
  • Loading branch information
bors committed Dec 7, 2020
2 parents 2718709 + eb081ed commit 453dbf8
Showing 1 changed file with 17 additions and 6 deletions.
23 changes: 17 additions & 6 deletions crates/crates-io/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@
use std::collections::BTreeMap;
use std::fs::File;
use std::io::prelude::*;
use std::io::Cursor;
use std::io::{Cursor, SeekFrom};
use std::time::Instant;

use anyhow::{bail, Result};
use anyhow::{bail, Context, Result};
use curl::easy::{Easy, List};
use percent_encoding::{percent_encode, NON_ALPHANUMERIC};
use serde::{Deserialize, Serialize};
Expand Down Expand Up @@ -161,23 +161,34 @@ impl Registry {
Ok(serde_json::from_str::<Users>(&body)?.users)
}

pub fn publish(&mut self, krate: &NewCrate, tarball: &File) -> Result<Warnings> {
pub fn publish(&mut self, krate: &NewCrate, mut tarball: &File) -> Result<Warnings> {
let json = serde_json::to_string(krate)?;
// Prepare the body. The format of the upload request is:
//
// <le u32 of json>
// <json request> (metadata for the package)
// <le u32 of tarball>
// <source tarball>
let stat = tarball.metadata()?;

// NOTE: This can be replaced with `stream_len` if it is ever stabilized.
//
// This checks the length using seeking instead of metadata, because
// on some filesystems, getting the metadata will fail because
// the file was renamed in ops::package.
let tarball_len = tarball
.seek(SeekFrom::End(0))
.with_context(|| "failed to seek tarball")?;
tarball
.seek(SeekFrom::Start(0))
.with_context(|| "failed to seek tarball")?;
let header = {
let mut w = Vec::new();
w.extend(&(json.len() as u32).to_le_bytes());
w.extend(json.as_bytes().iter().cloned());
w.extend(&(stat.len() as u32).to_le_bytes());
w.extend(&(tarball_len as u32).to_le_bytes());
w
};
let size = stat.len() as usize + header.len();
let size = tarball_len as usize + header.len();
let mut body = Cursor::new(header).chain(tarball);

let url = format!("{}/api/v1/crates/new", self.host);
Expand Down

0 comments on commit 453dbf8

Please sign in to comment.