Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Commit

Permalink
Removed custom allocator (#693)
Browse files Browse the repository at this point in the history
  • Loading branch information
jorgecarleitao authored Dec 21, 2021
1 parent e4dc0b8 commit 4bbbe62
Show file tree
Hide file tree
Showing 90 changed files with 693 additions and 2,099 deletions.
22 changes: 0 additions & 22 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,27 +88,6 @@ jobs:
# --skip io: miri can't handle opening of files, so we skip those
run: cargo miri test --features full -- --skip io::parquet --skip io::ipc

miri-checks-custom-allocator:
name: MIRI with custom allocator
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions-rs/toolchain@v1
with:
toolchain: nightly-2021-12-10
override: true
- uses: Swatinem/rust-cache@v1
with:
key: key1
- name: Install Miri
run: |
rustup component add miri
cargo miri setup
- name: Run
# --skip io: miri can't handle opening of files, so we skip those
run: cargo miri test --features full,cache_aligned -- --skip io::parquet --skip io::ipc

coverage:
name: Coverage
runs-on: ubuntu-latest
Expand All @@ -130,7 +109,6 @@ jobs:
run: cargo install cargo-tarpaulin
- name: Run coverage
run: |
cargo tarpaulin --features cache_aligned --ignore-tests --out Xml
cargo tarpaulin --features full --ignore-tests --out Xml
- name: Report coverage
continue-on-error: true
Expand Down
7 changes: 0 additions & 7 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -207,9 +207,6 @@ compute = [
io_parquet = ["parquet2", "io_ipc", "base64", "futures"]
benchmarks = ["rand"]
simd = ["packed_simd"]
# uses a custom allocator whose pointers are aligned along cache lines.
# Using this features makes `Buffer` and `MutableBuffer` incompatible with `Vec`.
cache_aligned = []

[package.metadata.cargo-all-features]
allowlist = ["compute", "compute_sort", "compute_hash", "compute_nullif"]
Expand Down Expand Up @@ -238,10 +235,6 @@ harness = false
name = "count_zeros"
harness = false

[[bench]]
name = "from_trusted_len_iter"
harness = false

[[bench]]
name = "growable"
harness = false
Expand Down
27 changes: 0 additions & 27 deletions benches/from_trusted_len_iter.rs

This file was deleted.

4 changes: 2 additions & 2 deletions benches/iter_list.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ use criterion::{criterion_group, criterion_main, Criterion};
use arrow2::{
array::{ListArray, PrimitiveArray},
bitmap::Bitmap,
buffer::{Buffer, MutableBuffer},
buffer::Buffer,
datatypes::DataType,
};

Expand All @@ -17,7 +17,7 @@ fn add_benchmark(c: &mut Criterion) {
let values = Buffer::from_iter(0..size as i32);
let values = PrimitiveArray::<i32>::from_data(DataType::Int32, values, None);

let mut offsets = MutableBuffer::from_iter((0..size as i32).step_by(2));
let mut offsets = (0..size as i32).step_by(2).collect::<Vec<_>>();
offsets.push(size as i32);

let validity = (0..(offsets.len() - 1))
Expand Down
37 changes: 5 additions & 32 deletions guide/src/low_level.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ These hold _all_ data-related memory in this crate.
Due to their intrinsic immutability, each container has a corresponding mutable
(and non-shareable) variant:

* `MutableBuffer<T>`
* `Vec<T>`
* `MutableBitmap`

Let's see how these structures are used.
Expand All @@ -30,43 +30,16 @@ Create a new `Buffer<u32>`:
```rust
# use arrow2::buffer::Buffer;
# fn main() {
let x = Buffer::from(&[1u32, 2, 3]);
let x = vec![1u32, 2, 3];
let x: Buffer<u32> = x.into();
assert_eq!(x.as_slice(), &[1u32, 2, 3]);

let x = x.slice(1, 2);
let x = x.slice(1, 2); // O(1)
assert_eq!(x.as_slice(), &[2, 3]);
# }
```

Using a `MutableBuffer<i64>`:

```rust
# use arrow2::buffer::MutableBuffer;
# fn main() {
let mut x: MutableBuffer<i64> = (0..3).collect();
x[1] = 5;
x.push(10);
assert_eq!(x.as_slice(), &[0, 5, 2, 10])
# }
```

The following demonstrates how to efficiently
perform an operation from an iterator of
[TrustedLen](https://doc.rust-lang.org/std/iter/trait.TrustedLen.html):

```rust
# use arrow2::buffer::MutableBuffer;
# fn main() {
let x = (0..1000).collect::<Vec<_>>();
let y = MutableBuffer::from_trusted_len_iter(x.iter().map(|x| x * 2));
assert_eq!(y[50], 100);
# }
```

Using `from_trusted_len_iter` often causes the compiler to auto-vectorize.

In this context, `MutableBuffer` has an almost identical API to Rust's `Vec`.
However, contrarily to `Vec`, `Buffer` and `MutableBuffer` only supports
Contrarily to `Vec`, `Buffer` (and all structs in this crate) only supports
the following physical types:

* `i8-i128`
Expand Down
119 changes: 0 additions & 119 deletions src/alloc/alignment.rs

This file was deleted.

Loading

0 comments on commit 4bbbe62

Please sign in to comment.