Skip to content

Add read_nwb_sorting_analyzer: build a SortingAnalyzer from an NWB Units table#4645

Draft
h-mayorquin wants to merge 32 commits into
SpikeInterface:mainfrom
h-mayorquin:load_analyzer_nwb_heberto
Draft

Add read_nwb_sorting_analyzer: build a SortingAnalyzer from an NWB Units table#4645
h-mayorquin wants to merge 32 commits into
SpikeInterface:mainfrom
h-mayorquin:load_analyzer_nwb_heberto

Conversation

@h-mayorquin

@h-mayorquin h-mayorquin commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

read_nwb_sorting_analyzer builds a curatable SortingAnalyzer directly from an NWB (Neurodata Without Borders) Units table, populating each extension from what the file already stores rather than recomputing it. When the Units table has waveform_mean, the analyzer is built recordingless from those stored templates, plus the per-unit metrics and the electrodes region for sparsity; when the file has an accessible ElectricalSeries, it is used as the recording. It mirrors read_kilosort_as_analyzer in structure, injecting the templates, quality and template metrics, sparsity, and random_spikes extensions from the file's contents. This supersedes the earlier draft #4270.

The reads are deliberate: Units columns are classified from metadata and only the templates, the electrodes region, and the scalar metric/label columns are materialized, while the large per-spike ragged columns (spike times, amplitudes, depths) are never touched at build. The sorting is kept lazy so its spike times are read only on demand, which builds on #4662 (single bulk-read NWB spike vector) and the copy_sorting option (#4668); the recordingless case builds a lightweight placeholder recording from the electrode rel_x/rel_y geometry (via generate_ground_truth_recording, #4588) purely to carry probe geometry into the standard constructor, then drops it. Together these keep the build small and memory-light regardless of file size.

Because the reads are deliberate and the sorting stays lazy, the function works the same whether the file is local or streamed (stream_mode is passed through to the extractors). That makes the streamed case cheap: as a check, building from a real IBL (International Brain Laboratory) processed file on dandiset 000409, session 6713a4a7-faed-4df2-acab-ee4e63326f8d (898 units, 20.7M spikes), produced a curatable analyzer in about 14 s while transferring only ~22 MB, with the ~130 MB spike read deferred until a spike-based view needs it. This PR is the reader itself; a separate how-to PR will cover the streaming-from-DANDI workflow in depth. It depends on #4662 and the copy_sorting PR (#4668), which should merge first so this branch rebases down to just the reader.

from dandi.dandiapi import DandiAPIClient
from spikeinterface.extractors import read_nwb_sorting_analyzer

# IBL Brain Wide Map (dandiset 000409), one session's processed file (units + templates, no raw traces)
SESSION = "6713a4a7-faed-4df2-acab-ee4e63326f8d"
with DandiAPIClient() as client:
    dandiset = client.get_dandiset("000409", "draft")
    asset = next(a for a in dandiset.get_assets_by_glob(f"*{SESSION}*.nwb") if "desc-processed" in a.path)
    s3_url = asset.get_content_url(follow_redirects=1, strip_query=True)

# build the analyzer directly from the streamed Units table (no full download)
analyzer = read_nwb_sorting_analyzer(s3_url, stream_mode="remfile")

print(analyzer)
print("num_units:", analyzer.get_num_units())
print("templates:", analyzer.get_extension("templates").get_data().shape)
print("quality metrics:", list(analyzer.get_extension("quality_metrics").get_data().columns))

# open in spikeinterface-gui for curation (needs a display and `pip install spikeinterface-gui`)
# import spikeinterface_gui
# spikeinterface_gui.run_mainwindow(analyzer)


def is_filtered(self) -> bool:
return self.rec_attributes["is_filtered"]
return self.rec_attributes.get("is_filtered", True)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the default True?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this is wrong.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that was a mistake. Thanks.

@h-mayorquin h-mayorquin changed the title Experiments - GUI - Dandi Support Add read_nwb_sorting_analyzer: build a SortingAnalyzer from an NWB Units table Jul 9, 2026
@h-mayorquin h-mayorquin requested a review from chrishalcrow July 9, 2026 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants