Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ShapedArray type and/or protocol #6

Open
stephentyrone opened this issue Jun 25, 2019 · 18 comments
Open

ShapedArray type and/or protocol #6

stephentyrone opened this issue Jun 25, 2019 · 18 comments
Labels
new module Proposals for new modules

Comments

@stephentyrone
Copy link
Member

stephentyrone commented Jun 25, 2019

Add a ShapedArray type and/or protocol suitable for use as the common currency type for vectors / linear algebra / image processing / tensors, like ndarray is for numpy.

This should hew pretty closely to what S4TF already does with their ShapedArray type; it doesn't need to be identical, but we need to have a good reason for any changes.

@stephentyrone stephentyrone added enhancement New feature for existing modules new module Proposals for new modules and removed enhancement New feature for existing modules labels Jun 25, 2019
@dabrahams
Copy link

@stephentyrone Are you familiar with MTL4?

@dastrobu
Copy link

A ndarray like type for swift would be great!
Some features of NymPy's array would be great to see:

  • contiguous and non contiguous array handling
  • array slices with strides
  • copy free array reshaping

I implemented a basic NdArray type for Swift a while a go, implementing theses features. It is currently more like a draft implementation than meant for production use.

Regarding linear algebra operations, I would recommend not to implement them in Swift and just call BLAS and LAPACK routines available on Linux platforms and from the Apple Accelerate framework on mac platforms. BLAS and LAPACK have been optimised for decades and there are many implementations available optimised for specific platforms or tuned for specific machines (ATLAS). As far as I know the Fortran implementations are still fastest and one should simply link theses implementations like NumPy and many other linear algebra packages do.

Not being too familiar with the ShapedArray type, I think it lacks support for non-contiguous arrays and array slices with strides.

@stephentyrone
Copy link
Member Author

Not being too familiar with the ShapedArray type, I think it lacks support for non-contiguous arrays and array slices with strides.

ShapedArray in S4TF does not support strides arrays or slices, and only supports one dimension ordering. My intention for SN is to both of these things (ndarray is a good mental model).

Regarding linear algebra operations, I would recommend not to implement them in Swift and just call BLAS and LAPACK routines available on Linux platforms and from the Apple Accelerate framework on mac platforms.

My intention is to make BLAS and LAPACK available using the shaped array type, but it should also be possible to implement linear algebra--this is a great check to ensure that we have a sufficiently powerful API to describe new operations in the future. So I'll also be implementing some of the BLAS and LAPACK operations in a module as an easy way to flush out API limitations and optimization shortcomings.

@awav
Copy link

awav commented Feb 16, 2020

Hello, @stephentyrone! I thought you might be interested in this PR, tensorflow/swift-apis#680. In it, I propose to define math protocols for shaped arrays in swift-apis. This is quite close to what you are doing with swift-numerics. Plus, I'm more than happy to help out with linear algebra implementation in swift-numerics.

@phewdry
Copy link

phewdry commented May 3, 2020

Hi Artem am wondering on current status for this issue?

Hello, @stephentyrone! I thought you might be interested in this PR, tensorflow/swift-apis#680. In it, I propose to define math protocols for shaped arrays in swift-apis. This is quite close to what you are doing with swift-numerics. Plus, I'm more than happy to help out with linear algebra implementation in swift-numerics.

stephentyrone pushed a commit that referenced this issue Nov 9, 2020
Update Swift Numerics README to address feedback
@SharanSMenon
Copy link

Is this coming?

@Datamance
Copy link

Following this - it would be really nice to have a great NDArray-like API (and all the other numpy goodies) sitting on top of BLAS and LAPACK, but in the Swift Runtime. I think the closest thing out there is Surge but that uses Accelerate and OpenCV under the hood.

@dastrobu
Copy link

dastrobu commented Apr 2, 2022

@Datamance note that Accelerate is using BLAS and LAPACK under the hood, so I think there is nothing wrong with that approach.

Surge does have quite good linear algebra support! Compared to numpy, Surge seams to be missing array slicing with strides. Having multiple views on the same data (different shape, different strides) is one of the great features of numpy I guess.

There is also Matft, which supports slicing and strides, but is not supporting generic elements. Then there is NdArray (written by me as mentioned earlier in #6 (comment)) with focus on slices and strides while keeping generic elements. But currently, linear algebra support is very limited.

I guess there are many more Numpy'ish packages out there, all having some strengths and shortcomings. So having an official swift-numerics type would be great.

@superlopuh
Copy link

One thing that I would suggest is to have the interface be flexible enough to be compatible with TACO and the MLIR implementation. It really seems like a great abstraction over dense and sparse arrays, and when the MLIR project gets wider adoption it would be great to leverage all the optimisations that make more sense in the compiler than in framework code.

@superlopuh
Copy link

This is probably a better link to the TACO project: http://tensor-compiler.org

@JimWallace
Copy link

A noteworthy development in the Apple ecosystem: https://github.com/ml-explore/mlx

@dastrobu
Copy link

MLX is pretty nice, except that it has only a C++ and python interface, which makes it hard to use with Swift. Also it is only available on Apple Silicon chips AFAIK. This makes it hard to use for server side Swift applications.

@JimWallace
Copy link

MLX Swift: https://swift.org/blog/mlx-swift/

@stephentyrone
Copy link
Member Author

MLX hews very closely to python/numpy-style for arrays, which is what the ML community is used to, but not necessarily what a more general-purpose library wants to have. There's lots of room for other work here.

@Lucca-mito
Copy link

And, crucially, MLX is Apple Silicon only.

@JimWallace
Copy link

MLX hews very closely to python/numpy-style for arrays, which is what the ML community is used to, but not necessarily what a more general-purpose library wants to have. There's lots of room for other work here.

Very fair.

This should hew pretty closely to what S4TF already does with their ShapedArray type; it doesn't need to be identical, but we need to have a good reason for any changes.

Maybe a good starting point would be looking at commonalities and differences between MLX and S4TF's ShapedArray? My sense is that there's some reasonable overlap already?

@wigging
Copy link

wigging commented Aug 5, 2024

MLX hews very closely to python/numpy-style for arrays, which is what the ML community is used to, but not necessarily what a more general-purpose library wants to have. There's lots of room for other work here.

So what features should a general-purpose library have? And what improvements would such a library have over something like MLX Swift?

@Lucca-mito
Copy link

Lucca-mito commented Aug 29, 2024

What improvements would such a library have over something like MLX Swift?

One potential advantage that such a library could have over MLX, NumPy, Matlab, etc. would be statically shape-safe matrices, unlocked by the recently-pitched integer generic parameters. If the library contains something like:

public struct Matrix<
    Element, let rowCount: Int, let columnCount: Int
>: RandomAccessCollection {}

extension Matrix where Element: Numeric {
    public static func * <
        let lhsRowCount: Int, 
        let lhsColumnCount: Int, 
        let rhsColumnCount: Int
    > (
        lhs: Matrix<Element, lhsRowCount, lhsColumnCount>, 
        rhs: Matrix<Element, lhsColumnCount, rhsColumnCount>
    ) -> Matrix<Element, lhsRowCount, rhsColumnCount> {}
}

extension Matrix: AdditiveArithmetic where Element: AdditiveArithmetic, rowCount == columnCount {
    public static func +(lhs: Matrix, rhs: Matrix) -> Matrix {}
    public static var zero: Matrix {}
}

extension Matrix: Numeric where Element: Numeric, rowCount == columnCount {}

Then the compiler would statically verify that all matrices being added or multiplied have the right shapes, and hovering on a matrix variable in VS Code (or option-clicking it in Xcode) would show the matrix's shape. Instantly knowing matrix shapes at code-writing time is certainly a feature I wish I had when I was writing NumPy and Matlab.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new module Proposals for new modules
Projects
None yet
Development

No branches or pull requests