Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Address nan issue by trimming sweeps #519

Closed

Conversation

tmchartrand
Copy link
Collaborator

Overview:

One of two options to address issue #508 by trimming missing values from the end of sweeps as they are loaded. The other solution is probably my preferred one, but I'm open to either.

Type of Fix:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing
    functionality to not work as expected)
  • Documentation Change

Solution:

Dropping nans from sweeps as they are loaded allows stim/epoch detection code to run without errors caused by nans.

Changes:

  • follows the pattern previously used for trailing zeros, trimming nans as data is loaded.
  • reduces the logging of sweep integrity tags from warning to info, as truncated sweeps are now common and expected.
  • removes the "recording" epoch, which is now unnecessary (this could possibly break some user code and should be maybe replaced by deprecation warnings, but that seems unlikely)
  • improved plot_data_set for quick visualization of new NWB2 to assess epochs

Validation:

Checked that tests pass and that epochs are found correctly without error in a new MIES NWB2 file using the updated plot_data_set

from ipfx.bin.plot_ephys_nwb import plot_data_set
from ipfx.dataset.create import create_ephys_data_set
path = '/allen/programs/celltypes/production/humancelltypes/prod148/Ephys_Roi_Result_1077533543/H21.26.402.12.58.01.05.nwb'
data_set = create_ephys_data_set(path)
plot_data_set(data_set, clamp_mode='CurrentClamp', show_amps=True, qc_sweeps=True)

Checklist

  • My code follows
    Allen Institute Contribution Guidelines
  • My code is unit tested and does not decrease test coverage
  • I have performed a self review of my own code
  • My code is well-documented, and the docstrings conform to
    Numpy Standards
  • I have updated the documentation of the repository where
    appropriate
  • The header on my commit includes the issue number
  • My code passes all tests
  • I have updated the CHANGELOG.md with the description of changes understandable by end users

This follows the pattern previously used for trailing zeros,
trimming as data is loaded.
Also reduces the logging of sweep integrity tags from warning to info,
as these sweeps are now common and expected.
@MatthewAitken
Copy link
Member

Closing since issue is resolved by PR 520

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants