You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I would like to propose adding a sparse matrix utility under the utility section of stdlib.io. In many cases, when users create 2D matrices, they often do not fully utilize all the cells, leading to unnecessary storage consumption. Sparse matrices can optimize this by only storing non-zero (or meaningful) values, thereby reducing memory usage.
Why is this needed?
A sparse matrix is useful in scenarios where the matrix has a large number of zero (or default) values, which is common in scientific computing, machine learning, and other data-heavy operations. With a sparse matrix implementation, users can:
Save memory by only storing the necessary values.
Improve computational efficiency with large, sparsely populated matrices.
Proposed Solution: The solution will involve creating a 1D array, which will be used to simulate a 2D matrix via a mathematical formula. This approach ensures that only the non-default values are stored, significantly optimizing memory usage. The utility will:
Allow users to interact with the matrix as if it were a standard 2D array.
Provide methods for adding, removing, and accessing values.
Use an internal mapping function to translate 2D indices into the 1D array.
@varshil-shah Thanks for raising this issue. I think the main concern I have with implementing sparse arrays is that, in order to perform computation efficiently, you need a sparse array compiler in order to be able to handle the different sparse matrix formats, operator fusion, fill values, and internal structure. This is non-trivial to do well.
In general, I am not opposed; however, undertaking this effort would require considerable R&D.
Description
I would like to propose adding a sparse matrix utility under the utility section of stdlib.io. In many cases, when users create 2D matrices, they often do not fully utilize all the cells, leading to unnecessary storage consumption. Sparse matrices can optimize this by only storing non-zero (or meaningful) values, thereby reducing memory usage.
Why is this needed?
A sparse matrix is useful in scenarios where the matrix has a large number of zero (or default) values, which is common in scientific computing, machine learning, and other data-heavy operations. With a sparse matrix implementation, users can:
Proposed Solution: The solution will involve creating a 1D array, which will be used to simulate a 2D matrix via a mathematical formula. This approach ensures that only the non-default values are stored, significantly optimizing memory usage. The utility will:
Looking forward to your feedback!
Checklist
RFC:
.The text was updated successfully, but these errors were encountered: