-
-
Notifications
You must be signed in to change notification settings - Fork 95
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Laser tensor PR rebase #477
Conversation
CI failure is because I forgot to add However, it's a bit worrying that this failed in the first place, indicating that non mem copyable types may have hidden bugs. It could be a good idea to run all basic tests in a mode where all types are treated as mem copyable? Maybe have a when not defined(noPtrLenBackend):
type
KnownSupportsCopyMem* = SomeNumber | char | Complex[float64] | Complex[float32]
else:
KnownSupportsCopyMem* = distinct object and then run tests with edit:
yet in the test they don't: let a = [[1,2],[3,4]].toTensor.reshape(2,2).astype(Complex[float64])
var b = ones[Complex[float64]](4,1) one is The thing is that the if src.is_C_contiguous:
assert dst.size == src.size
omp_parallel_chunks(
src.size, chunk_offset, chunk_size,
OMP_MEMORY_BOUND_GRAIN_SIZE * 4):
copyMem(
dst.unsafe_raw_offset[chunk_offset].addr,
src.unsafe_raw_offset[chunk_offset].unsafeAddr,
chunk_size * sizeof(T)
) @mratsim: should |
FYI: nim-lang/Nim#16185 is still an open issue, which makes the code here unsafe if used with ARC. And a personal blocker: |
5a46224
to
d5339cc
Compare
…build/tests_tensor_part01 -r tests/_split_tests/tests_tensor_part01.nim"
…assing all tests with gc:markandsweep
later state: - test_display - test_higherorder - test_ufunc Commonalities: - all rely on higher order templates map/apply - all have at least a test that involve mapping a string Issue can be reliably detected in its own module by wrapping tests into a proc and avoid globals and then calling GC_fullcollect or GC_collectZct
…leted without deprecation
This is needed, due to the changes that happened to the Arraymancer directory layout in the meantime.
This was already removed due to possibly being bugged in 56de8c9
`reset` sets the length of the underlying sequence to 0! This way we actually reset it to empty and resize to tensor size, which should reallocate a new block that will be zeroed for us. Since `newSeq` already zeroes in the first place, we don't have to call `setZero` when constructing a seq based tensor.
This at least _seems_ to work fine as far as I can test, both using ARC and normal GC. `export_tensor` still used `data` as well. For the time being we can use `toRawSeq` until a better solution is in place.
As long as this test case passes, self assignment of objects containing Tensors using ARC is broken in arraymancer!
- `io_hdf5` is not imported automatically anymore if the module is installed. The reason for this is that the HDF5 library runs code in global scope to initialize the HDF5 library. This means dead code elimination does not work and a binary will always depend on the HDF5 shared library if the `nimhdf5` is installed, even if not used.
e7dd185
to
f2f6c8c
Compare
…uda for Nim 1.0.x
Here it is finally. Sorry for taking so long.
This is a rebase of the laser tensor PR #420 onto the current state of arraymancer. The major complication was the restructure of the whole arraymancer library to support importing individual parts as submodules.
I had to fix a few things that came up:
pca.nim
, which caused the tensor to be destroyed maybe? I removed it and now it works with arc. Relevant commit: 63b4705
Maybe my
=destroy
implementation is at fault? @Clyybber maybe?CpuStorageObj
for the=destroy
proc signature. That's the same as would happen implicitly I assume?supportsCopyMem
not supported in type sections and the workaround:https://github.com/mratsim/Arraymancer/blob/laser-tensor-rebase/src/arraymancer/laser/tensor/datatypes.nim#L31-L36
As discussed on #420 this can cause issues where certain types (variant objects) match different branches in the type definition and in the code using
when T.supportsCopyMem
.For that reason I will define a type class of types for which we know that they support
copyMem
which will then be used in the code as well as in the type definition to make sure laser based tensors are used for the vast majority of types used in practice.I'll finish the latter maybe today and at the latest over the weekend. Wanted to open this PR first though, because I don't know how much more time I have today and to see what the CI says...