Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

prop_cat_cols fails with BOM #227

Open
Manishearth opened this issue Jun 19, 2020 · 3 comments
Open

prop_cat_cols fails with BOM #227

Manishearth opened this issue Jun 19, 2020 · 3 comments

Comments

@Manishearth
Copy link

Caught by rustc CI

The test fails when it tries to compare [[""]] with [["\u{feff}"]]. Presumably the BOM is being stripped somewhere.

#[test]
fn prop_cat_cols() {
    fn p(rows1: CsvData, rows2: CsvData) -> bool {
        let got: Vec<Vec<String>> = run_cat(
            "cat_cols", "columns", rows1.clone(), rows2.clone(), no_headers);

        let mut expected: Vec<Vec<String>> = vec![];
        let (rows1, rows2) = (rows1.to_vecs().into_iter(),
                              rows2.to_vecs().into_iter());
        for (mut r1, r2) in rows1.zip(rows2) {
            r1.extend(r2.into_iter());
            expected.push(r1);
        }
        rassert_eq!(got, expected)
    }
     assert!(p(CsvData { data: vec![CsvRecord(vec!["".into()])] },  CsvData { data: vec![CsvRecord(vec!["\u{feff}".into()])] }));
 }
BurntSushi added a commit that referenced this issue Sep 25, 2020
Basically, the problem here is that the quickcheck tests will sometimes
generate CSV data that begins with a UTF-8 BOM and the CSV reader
automatically strips it.

There's a better way to solve this by making tests more robust, but they
were written without this consideration. As a hack, we just make sure
that we don't generate CSV data that begins with a BOM.

This isn't quite 100% correct since we could still get CSV data
beginning with a UTF-8 BOM through shrinking, but that only occurs when
a legitimate test failure happens. So this could potentially make
failure modes worse, but we abide for now.

Fixes #227
@ehuss
Copy link

ehuss commented Mar 8, 2021

It looks like this also hits the prop_reverse_no_headers test as well. I'm not sure if a1165e0 fixes that as well. Seen on rust-lang/rust CI:

thread 'test_reverse::prop_reverse_no_headers' panicked at '[quickcheck] TEST FAILED (runtime error). Arguments: (CsvData { data: [[[239, 187, 191]]] })
Error: "assertion failed: `(left == right)`\n  left: `[]`,\n right: `[[\"\\u{feff}\"]]`"', /cargo/registry/src/github.com-1ecc6299db9ec823/quickcheck-0.7.1/src/tester.rs:176:28

@RalfJung
Copy link

test_frequency::prop_frequency_indexed can also fail randomly: rust-lang/rust#99601.

@RalfJung
Copy link

a1165e0 says it fixes this issue, but only exists on a branch, not in master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants