Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for distributed skeleton analysis #124

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

GenevieveBuckley
Copy link
Collaborator

Notes from the hack day:

  • Separate sparse graph values from construction of sparse array
  • Add dask delayed list comprehension somewhere?
  • Summarize -> move to function instead of class method
    • skel.coordinates
    • skel.graph
    • skel.paths (relies on skel.nbgraph in _build_skeleton_path_graph)
    • skel.degrees
@delayed
def skeleton_graph_func(skelint, spacing=1):
    ndim = skelint.ndim
    spacing = np.ones(ndim, dtype=float) * spacing
    num_edges = _num_edges(skelint.astype(bool))
    padded_skelint = np.pad(skelint, 1)  # pad image to prevent looparound errors
    steps, distances = raveled_steps_to_neighbors(padded_skelint.shape, ndim,
                                                  spacing=spacing)

    # from function skan.csr._pixel_graph
    row = np.empty(num_edges, dtype=int)
    col = np.empty(num_edges, dtype=int)
    data = np.empty(num_edges, dtype=float)
    k = _write_pixel_graph(padded_skelint, steps, distances, row, col, data)

    return pd.DataFrame({"row": row, "col": col, "data": data})


# skeleton to csgraph
return graph, pixel_indices

@delayed
def skeleton_graph_func(skelint, spacing, block_id):
    csgraph, coords = skeleton_to_csgraph(skelint, spacing)
    coo = csgraph.tocoo()
    coords += block_coordinates
    return pd.DataFrame(coo.row, coo.col, coo.data)

@GenevieveBuckley
Copy link
Collaborator Author

@jni the big question still to decide was: where are we going to put the dask delayed list comprehension? You suggested a dask_compat.py file, but does that mean anybody with a dask array input is going to use a different API than other users?

@GenevieveBuckley
Copy link
Collaborator Author

@GenevieveBuckley
Copy link
Collaborator Author

PR from pair programming today: #131

TODO

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant