Possible empty bundle in Cam-CAN subject

[richford]

Using PR #384 (i.e. cloudknot with image_github_installs=https://github.com/36000/pyAFQ.git@avoid_dipy_load_tractogram_bug#egg=pyAFQ[fury]), I get the following error:

INFO:AFQ.Segmentation:Finding Streamlines for SLF_L
/usr/local/lib/python3.7/site-packages/AFQ/utils/volume.py:31: UserWarning: Input image is entirely zero, no valid convex hull. Returning empty image
  return convex_hull_image(hole_filled)
INFO:AFQ.Segmentation:97050 streamlines exceed the probability threshold.
Traceback (most recent call last):
  File "/home/cloudknot-user/afq-cam-can.py", line 169, in <module>
    pickle_to_s3(args.sse, args.arrayjob)(afq_cam_can)(input_)
  File "/home/cloudknot-user/afq-cam-can.py", line 35, in wrapper
    result = f(*args, **kwargs)
  File "/home/cloudknot-user/afq-cam-can.py", line 108, in afq_cam_can
    myafq.export_all()
  File "/usr/local/lib/python3.7/site-packages/AFQ/api.py", line 1603, in export_all
    self.export_bundles()
  File "/usr/local/lib/python3.7/site-packages/AFQ/api.py", line 1547, in export_bundles
    self.data_frame.apply(self._export_bundles, axis=1)
  File "/usr/local/lib/python3.7/site-packages/pandas/core/frame.py", line 6878, in apply
    return op.get_result()
  File "/usr/local/lib/python3.7/site-packages/pandas/core/apply.py", line 186, in get_result
    return self.apply_standard()
  File "/usr/local/lib/python3.7/site-packages/pandas/core/apply.py", line 296, in apply_standard
    values, self.f, axis=self.axis, dummy=dummy, labels=labels
  File "pandas/_libs/reduction.pyx", line 620, in pandas._libs.reduction.compute_reduction
  File "pandas/_libs/reduction.pyx", line 128, in pandas._libs.reduction.Reducer.get_result
  File "/usr/local/lib/python3.7/site-packages/AFQ/api.py", line 1085, in _export_bundles
    bundles_file = func(row)
  File "/usr/local/lib/python3.7/site-packages/AFQ/api.py", line 902, in _clean_bundles
    bundles_file = self._segment(row)
  File "/usr/local/lib/python3.7/site-packages/AFQ/api.py", line 875, in _segment
    reg_prealign=reg_prealign)
  File "/usr/local/lib/python3.7/site-packages/AFQ/segmentation.py", line 242, in segment
    return self.segment_afq()
  File "/usr/local/lib/python3.7/site-packages/AFQ/segmentation.py", line 517, in segment_afq
    tol)
  File "/usr/local/lib/python3.7/site-packages/AFQ/segmentation.py", line 403, in _check_sl_with_exclusion
    if np.min(cdist(sl, roi, 'sqeuclidean')) < tol:
  File "<__array_function__ internals>", line 6, in amin
  File "/usr/local/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 2746, in amin
    keepdims=keepdims, initial=initial, where=where)
  File "/usr/local/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 90, in _wrapreduction
    return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
ValueError: zero-size array to reduction operation minimum which has no identity

So it would appear that the the SLF_L bundle has nothing in it. I'll investigate further to see if something is weird about this subject. But should pyAFQ pass/fail gracefully here by including an if statement to ignore such bundles?

[36000]

This error is caused by cdist returning an empty list, which we then pass to np.min, which fails.
Maybe the tractography returned an empty streamline? Very odd, looking into it.

[36000]

The SLF is the only bundle with an exclusion ROI. This means that the other bundles would never have touched this code.
It's possible this bug is caused by a malformed streamline that causes an error in the exclusion ROI code.

[36000]

I pushed a brute force fix to that PR (we will now ignore the defective streamline). Could you run the subject again and see what the SLF looks like? I think we should be able to find it, I hope this is just an issue with one streamline.

[richford]

No joy: https://us-west-2.console.aws.amazon.com/cloudwatch/home?region=us-west-2#logsV2:log-groups/log-group/$252Faws$252Fbatch$252Fjob/log-events/afq_cam_can-arh-1-cloudknot-job-definition$252Fdefault$252F237e5671-5f78-423b-8cfb-4d24e664926f

[36000]

Aright. I have a new fix: try/except statement. Can you run it again? Let's see if this triggers and what it prints.

[36000]

The underlying issue is: UserWarning: Input image is entirely zero, no valid convex hull. Returning empty image return convex_hull_image(hole_filled)
This is caused by the ROI having no voxels, probably due to bad registration.

[36000]

This should throw an error explaining this.

[36000]

Better warning done in #384

[36000]

This was caused by how qsiprep was handling oblique scans. See: PennLINC/qsiprep#156