Creating an Array throws an "Unknown Error" #105

dchammond · 2017-02-15T20:04:51Z

Two simple test programs demonstrate this error:

$ rustc --version                                                                                                                                                                          101   master
rustc 1.15.1 (021bd294c 2017-02-08)

(This same result also occurs on the latest nightly: rustc 1.17.0-nightly (956e2bcba 2017-02-12))
Cargo.toml:

[package]
name = "ArrayTest"
version = "0.1.0"
authors = ["Dillon Hammond <[email protected]>"]

[dependencies]
arrayfire = "3.4.1"

src/main.rs:

extern crate arrayfire;
use arrayfire::*;

fn main() {
  let temp = [1.0f32, 2.0];
  let dims = Dim4::new(&[2, 1, 1, 1]);
  let array = Array::new(&temp, dims);
  print(&array);
}

This works and prints:

[2 1 1 1]
    1.0000
    2.0000

However the following does not work:

src/main.rs:

extern crate arrayfire;
use arrayfire::*;

fn main() {
  // let temp = [1.0f32, 2.0];
  let dims = Dim4::new(&[2, 1, 1, 1]);
  let array = Array::new(&[1.0f32, 2.0], dims);
  print(&array);
}

And errors out with:

    Finished debug [unoptimized + debuginfo] target(s) in 0.31 secs
     Running `target/debug/ArrayTest`
thread 'main' panicked at 'Error message: Unknown Error', /Users/Dillon/.cargo/registry/src/github.com-1ecc6299db9ec823/arrayfire-3.4.1/src/error.rs:72
stack backtrace:
   1:        0x10421cbca - std::sys::imp::backtrace::tracing::imp::write::hd3b65cdfe843284c
   2:        0x10421de1f - std::panicking::default_hook::{{closure}}::hf2b7428652613d83
   3:        0x10421dac7 - std::panicking::default_hook::h5da8f27db5582938
   4:        0x10421e286 - std::panicking::rust_panic_with_hook::hcef1e67c646c6802
   5:        0x10421e124 - std::panicking::begin_panic::hc2e8ca89533cd10d
   6:        0x10421e042 - std::panicking::begin_panic_fmt::h60990696c3c3a88d
   7:        0x104217cd5 - arrayfire::error::handle_error_general::h0e5d6db925a03428
   8:        0x104218022 - core::ops::Fn::call::hc9af5f45f4409d39
   9:        0x104217e2c - arrayfire::error::HANDLE_ERROR::h882ecb6e0c34188c
  10:        0x104214977 - arrayfire::array::Array::new::h36519fff17d0ea4e
  11:        0x1042149f5 - ArrayTest::main::h3a0f84221923c6d5
  12:        0x10421ec1a - __rust_maybe_catch_panic
  13:        0x10421e4f6 - std::rt::lang_start::h87cb84a8b6cb187e
  14:        0x104214a99 - main

I have installed arrayfire v3.3.2 (the library) with Homebrew and verified it works.
Output of $AF_PATH:

/usr/local/opt/arrayfire

The text was updated successfully, but these errors were encountered:

9prady9 · 2017-02-16T04:37:03Z

@dchammond I see that you have installed ArrayFire v3.3.2 but using arrayfire-rust crate 3.4.1. The following is the API/ABI match between crate and the upstream ArrayFire.

ArrayFire Upstream	Rust Crate
3.3.x	3.3.0 to 3.3.2
3.4.x	3.4.0 to 3.4.1

pavanky · 2017-02-16T04:42:16Z

@9prady9 not sure how that affects this particular bug

9prady9 · 2017-02-16T05:22:01Z

@pavanky I have tried both the versions @dchammond provided and they run fine for me - because they are basically the same. I am not sure if the problem is with code itself. It could be his environment, so pointed out that he is using mismatched version of arrayfire crate.

Updated:
I am using rust 1.14.0, checked with 1.15.0 and nightly as well.

pavanky · 2017-02-16T05:44:58Z

@dchammond I can also confirm that this works in 1.14

dchammond · 2017-02-16T06:11:29Z

@9prady9 @pavanky
So I upgraded ArrayFire to v3.4.2 (I don't know why the Homebrew version is so old).

After doing this, I find that the same test program no longer runs on stable rust, only nightly. On stable it exits without printing anything (no errors either) except that the process has an error code of 1. On nightly the code does run but the same issue presents itself (needing a temp variable).

pavanky · 2017-02-16T06:25:32Z

@dchammond
Able to run this fine on 1.15 too

$ cat Cargo.toml src/main.rs && rustc --version && cargo --version && cargo run
[package]
name = "ArrayTest"
version = "0.1.0"
authors = ["Dillon Hammond <[email protected]>"]

[dependencies]
arrayfire = "3.4.1"
extern crate arrayfire;
use arrayfire::*;

fn main() {
  // let temp = [1.0f32, 2.0];
  let dims = Dim4::new(&[2, 1, 1, 1]);
  let array = Array::new(&[1.0f32, 2.0], dims);
  print(&array);
}
rustc 1.15.0 (10893a9a3 2017-01-19)
cargo 0.16.0-nightly (6e0c18c 2017-01-27)
   Compiling ArrayTest v0.1.0 (file:///tmp/foo)
warning: crate `ArrayTest` should have a snake case name such as `array_test`, #[warn(non_snake_case)] on by default

    Finished debug [unoptimized + debuginfo] target(s) in 0.32 secs
     Running `target/debug/ArrayTest`

[2 1 1 1]
    1.0000 
    2.0000

Only change I made to your code was remove the space between array and fire.

pavanky · 2017-02-16T06:28:49Z

@dchammond can you run the example with AF_PRINT_ERRORS=1 exported ?

dchammond · 2017-02-16T17:40:50Z

@pavanky

I edited my original comment, the space between array and fire was not present in my code.

Running with AF_PRINT_ERRORS=1 reveals the following:

Error in af_err unified::AFSymbolManager::call(const char *, CalleeArgs...) [CalleeArgs = <>]
In file src/api/unified/symbol_manager.hpp:55
Failed to load dynamic library. See http://www.arrayfire.com/docs/unifiedbackend.htm for instructions to set up environment for Unified backend.

After following the link, I noticed that my $DYLD_LIBRARY_PATH is not set in my environment, though I am not sure if this is significant as my $AF_PATH is /usr/local

pavanky · 2017-02-16T17:48:28Z

@dchammond You'll need to set DYLD_LIBRARY_PATH as well. For example libforge.dylib isn't loaded manually in arrayfire.

dchammond · 2017-02-16T17:57:32Z

@pavanky

$ DYLD_LIBRARY_PATH=/usr/local/lib AF_PRINT_ERRORS=1 cargo run                                                                                                                               master 
dyld: Symbol not found: __cg_jpeg_resync_to_restart
  Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
  Expected in: /usr/local/lib/libJPEG.dylib
 in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
[1]    20366 abort      DYLD_LIBRARY_PATH=/usr/local/lib AF_PRINT_ERRORS=1 cargo run

$locate libjpeg
/Applications/CrashPlan.app/Contents/PlugIns/jdk1.8.0_72.jdk/Contents/Home/jre/lib/libjpeg.dylib
/Applications/GIMP.app/Contents/Resources/lib/libjpeg.9.dylib
/Applications/Xcode.app/Contents/Applications/Application Loader.app/Contents/itms/java/lib/libjpeg.dylib
/Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/lib/libjpeg.dylib
/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Libraries/libjpeg.jnilib
/Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/lib/libjpeg.dylib
/usr/local/Cellar/jpeg/8d/lib/libjpeg.8.dylib
/usr/local/Cellar/jpeg/8d/lib/libjpeg.a
/usr/local/Cellar/jpeg/8d/lib/libjpeg.dylib
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Aliases/libjpeg
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Aliases/libjpeg-turbo
/usr/local/lib/libjpeg.8.dylib
/usr/local/lib/libjpeg.a
/usr/local/lib/libjpeg.dylib

pavanky · 2017-02-16T18:33:26Z

@dchammond That is a dependency mismatch on your system. I think freeimage is using a newer or older version of libJPEG than what you have.

dchammond · 2017-02-16T20:41:36Z

@pavanky
I have been messing around with free image and libJPEG for awhile now and cannot easily find a working solution.

Is the machine you are testing on also a Mac?

I would really appreciate anyone describing their installation method and version numbers of free image and/or libjpeg (whether with home-brew or from source) so that I can compare and find the problem on my machine.

pavanky · 2017-02-16T20:58:25Z

@mlloreda can you help out?

9prady9 · 2017-02-27T17:48:20Z

@dchammond Sorry for the delay. I was finally able to replicate the problem in partial on our end. I am getting a crash but the error message is different from yours. This issue doesn't seem to be happening on non-OSX platforms.

$ cargo run --example helloworld
    Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `target/debug/examples/helloworld`
ArrayFire v3.5.0 (OpenCL, 64-bit Mac OSX, build 38ef85b)
[0] APPLE   : Iris, 1536 MB
OpenCL Error (-30): Invalid Value when calling clEnqueueWriteBuffer
thread 'main' panicked at 'Error message: Eror either in ArrayFire or in a project upstream', src/error.rs:66
note: Run with `RUST_BACKTRACE=1` for a backtrace.

I will update here once i find the cause of it.

9prady9 · 2017-02-27T19:11:31Z

So far, i haven't been able to find out why clEnqueueWriteBuffer is failing - I have verified all the input parameters that are being passed on to clEnqueueWriteBuffer inside ArrayFire and they have valid values.

pavanky · 2017-02-27T19:14:26Z

@9prady9 is this OpenCL on nvidia gpus or OpenCL on other gpus ?

9prady9 · 2017-02-27T19:21:39Z

@pavanky The device name says "Iris", I think it is a Intel integrated GPU.

dchammond · 2017-02-27T19:23:12Z

@pavanky @9prady9
To be clear though, my issue was on an AMD GPU with OpenCL. Interesting that it manifests itself on integrated GPUs as well.

pavanky · 2017-02-27T19:25:46Z

@dchammond just so you know Apple's support for OpenCL has been horrid. Even more so on AMD GPUs.

dchammond · 2017-03-07T16:26:11Z

@9prady9 @pavanky
So I have an interesting update on this issue that might reference #60

I uninstalled the Homebrew version of Arrayfire and wiped any symlinks, then installed Arrayfire 3.4.2 from the binary installer and verified that the examples built and ran.

Here is some information on my environment (similar to last time):

$ echo $AF_PATH
/usr/local
$ echo $LD_LIBRARY_PATH
/usr/local/lib
$ cat Cargo.toml
[package]
name = "tets"
version = "0.1.0"
authors = ["Dillon Hammond <[email protected]>"]

[dependencies]
arrayfire = "3.4.2"
$ cat src/main.rs
extern crate arrayfire;

use arrayfire as af;

fn main() {
  af::set_backend(af::Backend::OPENCL);
  println!("Okay");
  let temp = [1.0, 2.0];
  let a = af::Array::new(&temp, af::Dim4::new(&[2,1,1,1]));
  af::print(&a);
  println!("Hi");
}
$ rustc --version
rustc 1.15.1 (021bd294c 2017-02-08)
$ cargo --version
cargo 0.16.0-nightly (6e0c18c 2017-01-27)
$ AF_PRINT_ERRORS=1 cargo run
    Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `target/debug/tets`
Error in af_err unified::AFSymbolManager::setBackend(af::Backend)
In file src/api/unified/symbol_manager.cpp:220
Failed to load dynamic library. See http://www.arrayfire.com/docs/unifiedbackend.htm for instructions to set up environment for Unified backend.
$ AF_PRINT_ERRORS=1 ./target/debug/tets
Okay

[2 1 1 1]
    1.0000
    2.0000

Hi

As you can see, running the executable directly works fine, while running with cargo fails.

It is worth noting that some of the original problem also still applies. This does not work (even with running the executable directly):

let temp = &[1.0, 2.0];
let a = af::Array::new(temp, af::Dim4::new(&[2,1,1,1]));

Nor this:

// let temp = &[1.0, 2.0];
let a = af::Array::new(&[1.0, 2.0], af::Dim4::new(&[2,1,1,1]));

Both failing with:

Okay

thread 'main' panicked at 'Error message: Unknown Error', /Users/Dillon/.cargo/registry/src/github.com-1ecc6299db9ec823/arrayfire-3.4.2/src/error.rs:66
stack backtrace:
   1:        0x108c5419a - std::sys::imp::backtrace::tracing::imp::write::hd3b65cdfe843284c
   2:        0x108c5551f - std::panicking::default_hook::{{closure}}::hf2b7428652613d83
   3:        0x108c551c7 - std::panicking::default_hook::h5da8f27db5582938
   4:        0x108c55986 - std::panicking::rust_panic_with_hook::hcef1e67c646c6802
   5:        0x108c55824 - std::panicking::begin_panic::hc2e8ca89533cd10d
   6:        0x108c55742 - std::panicking::begin_panic_fmt::h60990696c3c3a88d
   7:        0x108c4d9b5 - arrayfire::error::handle_error_general::h4eba46772d5c6549
   8:        0x108c4dd02 - core::ops::Fn::call::h9dabce26b3526a4d
   9:        0x108c4db0c - arrayfire::error::HANDLE_ERROR::hc639d416bcc0d17d
  10:        0x108c4a577 - arrayfire::array::Array::new::hf4045fdb4f28ef5c
  11:        0x108c4a63f - tets::main::h5f0268a777d19f64
  12:        0x108c5631a - __rust_maybe_catch_panic
  13:        0x108c55bf6 - std::rt::lang_start::h87cb84a8b6cb187e
  14:        0x108c4a739 - main

dchammond · 2017-03-17T14:29:17Z

@9prady9 @pavanky
So the answer to why cargo run is different to directly running binaries is a result of these:
rust-lang/cargo#2888
https://users.rust-lang.org/t/subprocess-and-dynamic-library-linking-problem-interaction/7873/9

Where cargo run messes with DYLD_LIBRARY_PATH and LD_LIBRARY_PATH.

The original issue is still not solved however.

9prady9 · 2017-03-24T05:02:28Z

@dchammond There isn't much we can do about cargo messing up environment variables. May be such issues will be ironed out with Cargo's stable release or next nightly.

As far as the original issue, I am quite surprised as to why it is happening although it is a perfectly valid syntax. I will give it another shot and update here if I find out anything.

dchammond · 2017-03-24T05:17:49Z

Thanks fro your help @9prady9

This is certainly a strange bug. The different methods of calling arrayfire::Array::new should be equivalent... yet they are not.

If I find some spare moments I can also try some digging and testing out different calls. Maybe I'll get lucky and find something.

9prady9 · 2017-04-27T06:57:32Z

@dchammond I am marking this as Wontfix and OSX since it is happening only in your development setup and we are unable to reproduce this anywhere else.

If anyone else face a similar issue, kindly upvote the original issue description and we will give it another try.

9prady9 closed this as completed Apr 27, 2017

9prady9 added OSX Wontfix labels Apr 27, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Creating an Array throws an "Unknown Error" #105

Creating an Array throws an "Unknown Error" #105

dchammond commented Feb 15, 2017 •

edited

Loading

9prady9 commented Feb 16, 2017

pavanky commented Feb 16, 2017

9prady9 commented Feb 16, 2017 •

edited

Loading

pavanky commented Feb 16, 2017

dchammond commented Feb 16, 2017

pavanky commented Feb 16, 2017

pavanky commented Feb 16, 2017 •

edited

Loading

dchammond commented Feb 16, 2017 •

edited

Loading

pavanky commented Feb 16, 2017

dchammond commented Feb 16, 2017 •

edited

Loading

pavanky commented Feb 16, 2017

dchammond commented Feb 16, 2017

pavanky commented Feb 16, 2017

9prady9 commented Feb 27, 2017 •

edited

Loading

9prady9 commented Feb 27, 2017

pavanky commented Feb 27, 2017

9prady9 commented Feb 27, 2017

dchammond commented Feb 27, 2017

pavanky commented Feb 27, 2017

dchammond commented Mar 7, 2017 •

edited

Loading

dchammond commented Mar 17, 2017

9prady9 commented Mar 24, 2017

dchammond commented Mar 24, 2017

9prady9 commented Apr 27, 2017

Creating an Array throws an "Unknown Error" #105

Creating an Array throws an "Unknown Error" #105

Comments

dchammond commented Feb 15, 2017 • edited Loading

9prady9 commented Feb 16, 2017

pavanky commented Feb 16, 2017

9prady9 commented Feb 16, 2017 • edited Loading

pavanky commented Feb 16, 2017

dchammond commented Feb 16, 2017

pavanky commented Feb 16, 2017

pavanky commented Feb 16, 2017 • edited Loading

dchammond commented Feb 16, 2017 • edited Loading

pavanky commented Feb 16, 2017

dchammond commented Feb 16, 2017 • edited Loading

pavanky commented Feb 16, 2017

dchammond commented Feb 16, 2017

pavanky commented Feb 16, 2017

9prady9 commented Feb 27, 2017 • edited Loading

9prady9 commented Feb 27, 2017

pavanky commented Feb 27, 2017

9prady9 commented Feb 27, 2017

dchammond commented Feb 27, 2017

pavanky commented Feb 27, 2017

dchammond commented Mar 7, 2017 • edited Loading

dchammond commented Mar 17, 2017

9prady9 commented Mar 24, 2017

dchammond commented Mar 24, 2017

9prady9 commented Apr 27, 2017

dchammond commented Feb 15, 2017 •

edited

Loading

9prady9 commented Feb 16, 2017 •

edited

Loading

pavanky commented Feb 16, 2017 •

edited

Loading

dchammond commented Feb 16, 2017 •

edited

Loading

dchammond commented Feb 16, 2017 •

edited

Loading

9prady9 commented Feb 27, 2017 •

edited

Loading

dchammond commented Mar 7, 2017 •

edited

Loading