Showing preview only (740K chars total). Download the full file or copy to clipboard to get everything.
Repository: SpikeInterface/spikeextractors
Branch: master
Commit: d24335cc2fa6
Files: 118
Total size: 697.7 KB
Directory structure:
gitextract_c4b26tzl/
├── .github/
│ └── workflows/
│ ├── python-package.yml
│ └── python-publish.yml
├── .gitignore
├── LICENSE
├── MANIFEST.in
├── README.md
├── environment-dev.yml
├── full_requirements.txt
├── requirements-dev.txt
├── requirements.txt
├── setup.py
├── spikeextractors/
│ ├── __init__.py
│ ├── baseextractor.py
│ ├── cacheextractors.py
│ ├── example_datasets/
│ │ ├── __init__.py
│ │ ├── synthesize_random_firings.py
│ │ ├── synthesize_random_waveforms.py
│ │ ├── synthesize_single_waveform.py
│ │ ├── synthesize_timeseries.py
│ │ └── toy_example.py
│ ├── exceptions.py
│ ├── extraction_tools.py
│ ├── extractorlist.py
│ ├── extractors/
│ │ ├── __init__.py
│ │ ├── alfsortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── alfsortingextractor.py
│ │ ├── axonaunitrecordingextractor/
│ │ │ ├── __init__.py
│ │ │ └── axonaunitrecordingextractor.py
│ │ ├── bindatrecordingextractor/
│ │ │ ├── __init__.py
│ │ │ └── bindatrecordingextractor.py
│ │ ├── biocamrecordingextractor/
│ │ │ ├── __init__.py
│ │ │ └── biocamrecordingextractor.py
│ │ ├── cedextractors/
│ │ │ ├── __init__.py
│ │ │ ├── cedrecordingextractor.py
│ │ │ └── utils.py
│ │ ├── cellexplorersortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── cellexplorersortingextractor.py
│ │ ├── combinatosortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── combinatosortingextractor.py
│ │ ├── exdirextractors/
│ │ │ ├── __init__.py
│ │ │ └── exdirextractors.py
│ │ ├── hdsortsortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── hdsortsortingextractor.py
│ │ ├── hs2sortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── hs2sortingextractor.py
│ │ ├── intanrecordingextractor/
│ │ │ ├── __init__.py
│ │ │ └── intanrecordingextractor.py
│ │ ├── jrcsortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── jrcsortingextractor.py
│ │ ├── kilosortextractors/
│ │ │ ├── __init__.py
│ │ │ └── kilosortextractors.py
│ │ ├── klustaextractors/
│ │ │ ├── __init__.py
│ │ │ └── klustaextractors.py
│ │ ├── matsortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── matsortingextractor.py
│ │ ├── maxwellextractors/
│ │ │ ├── __init__.py
│ │ │ └── maxwellextractors.py
│ │ ├── mcsh5recordingextractor/
│ │ │ ├── __init__.py
│ │ │ └── mcsh5recordingextractor.py
│ │ ├── mdaextractors/
│ │ │ ├── __init__.py
│ │ │ ├── mdaextractors.py
│ │ │ └── mdaio.py
│ │ ├── mearecextractors/
│ │ │ ├── __init__.py
│ │ │ └── mearecextractors.py
│ │ ├── neoextractors/
│ │ │ ├── __init__.py
│ │ │ ├── axonaextractor.py
│ │ │ ├── blackrockextractor.py
│ │ │ ├── mcsrawrecordingextractor.py
│ │ │ ├── neobaseextractor.py
│ │ │ ├── neuralynxextractor.py
│ │ │ ├── plexonextractor.py
│ │ │ └── spikegadgetsextractor.py
│ │ ├── neuropixelsdatrecordingextractor/
│ │ │ ├── __init__.py
│ │ │ ├── channel_positions_neuropixels.txt
│ │ │ └── neuropixelsdatrecordingextractor.py
│ │ ├── neuroscopeextractors/
│ │ │ ├── __init__.py
│ │ │ └── neuroscopeextractors.py
│ │ ├── nixioextractors/
│ │ │ ├── __init__.py
│ │ │ └── nixioextractors.py
│ │ ├── npzsortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── npzsortingextractor.py
│ │ ├── numpyextractors/
│ │ │ ├── __init__.py
│ │ │ └── numpyextractors.py
│ │ ├── nwbextractors/
│ │ │ ├── __init__.py
│ │ │ └── nwbextractors.py
│ │ ├── openephysextractors/
│ │ │ ├── __init__.py
│ │ │ └── openephysextractors.py
│ │ ├── phyextractors/
│ │ │ ├── __init__.py
│ │ │ └── phyextractors.py
│ │ ├── shybridextractors/
│ │ │ ├── __init__.py
│ │ │ └── shybridextractors.py
│ │ ├── spikeglxrecordingextractor/
│ │ │ ├── __init__.py
│ │ │ ├── readSGLX.py
│ │ │ └── spikeglxrecordingextractor.py
│ │ ├── spykingcircusextractors/
│ │ │ ├── __init__.py
│ │ │ └── spykingcircusextractors.py
│ │ ├── tridescloussortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── tridescloussortingextractor.py
│ │ ├── waveclussortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── waveclussortingextractor.py
│ │ └── yassextractors/
│ │ ├── __init__.py
│ │ └── yassextractors.py
│ ├── multirecordingchannelextractor.py
│ ├── multirecordingtimeextractor.py
│ ├── multisortingextractor.py
│ ├── recordingextractor.py
│ ├── save_tools.py
│ ├── sortingextractor.py
│ ├── subrecordingextractor.py
│ ├── subsortingextractor.py
│ ├── testing.py
│ └── version.py
└── tests/
├── __init__.py
├── probe_test.prb
├── test_extractors.py
├── test_gin_repo.py
├── test_numpy_extractors.py
└── test_tools.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .github/workflows/python-package.yml
================================================
name: Python Package using Conda
on:
push:
branches:
- master
pull_request:
branches: [master]
types: [synchronize, opened, reopened, ready_for_review]
jobs:
build-and-test:
name: Test on (${{ matrix.os }})
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
steps:
- uses: actions/checkout@v2
- uses: s-weigand/setup-conda@v1
with:
python-version: 3.8
- name: Which python
run: |
conda --version
which python
- name: Install dependencies
run: |
conda install -c conda-forge datalad
conda install -c conda-forge ruamel.yaml
conda install flake8
conda install pytest
pip install -r requirements-dev.txt
pip install -r requirements.txt
pip install h5py==2.10
pip install -e .[full]
# needed for correct operation of git/git-annex/DataLad
git config --global user.email "CI@example.com"
git config --global user.name "CI Almighty"
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest and build coverage report
run: |
pytest
================================================
FILE: .github/workflows/python-publish.yml
================================================
# This workflow will upload a Python Package using Twine when a release is created
# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries
name: Test and Upload Python Package
on:
push:
tags:
- '*'
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: s-weigand/setup-conda@v1
with:
python-version: 3.8
- name: Which python
run: |
conda --version
which python
- name: Install dependencies
run: |
conda install -c conda-forge datalad
conda install -c conda-forge ruamel.yaml
conda install flake8
conda install pytest
pip install setuptools wheel twine
pip install -r requirements-dev.txt
pip install -r requirements.txt
pip install h5py==2.10
pip install -e .[full]
# needed for correct operation of git/git-annex/DataLad
git config --global user.email "CI@example.com"
git config --global user.name "CI Almighty"
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest and build coverage report
run: |
pytest
- name: Publish on PyPI
env:
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
python setup.py sdist bdist_wheel
twine upload dist/*
================================================
FILE: .gitignore
================================================
.eggs
*.egg-info
.ipynb_checkpoints
__pycache__
sample_*_dataset
ephy_testing_data/
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2019 SpikeInterface
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: MANIFEST.in
================================================
include spikeextractors/extractors/neuropixelsdatrecordingextractor/channel_positions_neuropixels.txt
================================================
FILE: README.md
================================================
# SpikeExtractors (LEGACY)
The `spikeextractors` package has now been integrated into [spikeinterface](https://github.com/SpikeInterface/spikeinterface).
This package will be maintained for a while for bug fixes only, then it will be deprecated.
New features and improvements will only be implemented for the `spikeinterface` package.
================================================
FILE: environment-dev.yml
================================================
name: test
dependencies:
- python=3.8
- pip
- pip:
- numpy==1.22.0
- tqdm
- lxml
- h5py
- shybrid
- pynwb
- nixio
- pyintan
- pyopenephys
- neo
- MEArec
- hdf5storage
- exdir
- hdbscan
- tridesclous
- parametrized
================================================
FILE: full_requirements.txt
================================================
h5py #>=3.2.1
scipy>=1.6.3
pyintan>=0.3.0
pyopenephys>=1.1.4
neo>=0.9.0
MEArec<1.8
pynwb>=1.4
lxml>=4.6.3
nixio==1.5.0
shybrid>=0.4.2
pyyaml>=5.4.1
mtscomp>=1.0.1
exdir==0.4.1
hdf5storage
sonpy;python_version>'3.7'
================================================
FILE: requirements-dev.txt
================================================
datalad
parameterized
neo==0.10
================================================
FILE: requirements.txt
================================================
numpy==1.22.0
tqdm
packaging
================================================
FILE: setup.py
================================================
import setuptools
d = {}
exec(open("spikeextractors/version.py").read(), None, d)
version = d['version']
pkg_name = "spikeextractors"
long_description = open("README.md").read()
with open("full_requirements.txt", mode='r') as f:
full_requires = f.read().split('\n')
full_requires = [e for e in full_requires if len(e) > 0]
extras_require = {"full": full_requires}
setuptools.setup(
name=pkg_name,
version=version,
author="Alessio Buccino, Cole Hurwitz, Samuel Garcia, Jeremy Magland, Matthias Hennig",
author_email="alessio.buccino@gmail.com",
description="Python module for extracting recorded and spike sorted extracellular data from different file types and formats",
url="https://github.com/SpikeInterface/spikeextractors",
long_description=long_description,
long_description_content_type="text/markdown",
packages=setuptools.find_packages(),
package_data={},
include_package_data=True,
install_requires=[
'numpy',
'tqdm',
'joblib'
],
extras_require=extras_require,
classifiers=(
"Programming Language :: Python :: 3",
"License :: OSI Approved :: Apache Software License",
"Operating System :: OS Independent",
)
)
================================================
FILE: spikeextractors/__init__.py
================================================
from .recordingextractor import RecordingExtractor
from .sortingextractor import SortingExtractor
from .cacheextractors import CacheRecordingExtractor, CacheSortingExtractor
from .subsortingextractor import SubSortingExtractor
from .subrecordingextractor import SubRecordingExtractor
from .multirecordingchannelextractor import concatenate_recordings_by_channel, MultiRecordingChannelExtractor
from .multirecordingtimeextractor import concatenate_recordings_by_time, MultiRecordingTimeExtractor
from .multisortingextractor import concatenate_sortings, MultiSortingExtractor
from .extractorlist import *
from . import example_datasets
from .extraction_tools import load_probe_file, save_to_probe_file, read_binary, write_to_binary_dat_format,\
write_to_h5_dataset_format, get_sub_extractors_by_property, load_extractor_from_json, load_extractor_from_dict, \
load_extractor_from_pickle
from .save_tools import save_si_object
from .version import version as __version__
================================================
FILE: spikeextractors/baseextractor.py
================================================
import json
from pathlib import Path
import importlib
import numpy as np
import datetime
from copy import deepcopy
import tempfile
import pickle
import shutil
from .exceptions import NotDumpableExtractorError
class BaseExtractor:
# To be specified in concrete sub-classes
# The default filename (extension to be added by corresponding method)
# to be used if no file path is provided
_default_filename = None
def __init__(self):
self._kwargs = {}
self._tmp_folder = None
self._key_properties = {}
self._properties = {}
self._annotations = {}
self._memmap_files = []
self._features = {}
self._epochs = {}
self._times = None
self.is_dumpable = True
self.id = np.random.randint(low=0, high=9223372036854775807, dtype='int64')
def __del__(self):
# close memmap files (for Windows)
for memmap_obj in self._memmap_files:
self.del_memmap_file(memmap_obj)
if self._tmp_folder is not None and len(self._memmap_files) > 0:
try:
shutil.rmtree(self._tmp_folder)
except Exception as e:
print('Impossible to delete temp file:', self._tmp_folder, 'Error', e)
def del_memmap_file(self, memmap_file):
"""
Safely deletes instantiated memmap file.
Parameters
----------
memmap_file: str or Path
The memmap file to delete
"""
if isinstance(memmap_file, np.memmap):
memmap_file = memmap_file.filename
else:
memmap_file = Path(memmap_file)
existing_memmap_files = [Path(memmap.filename) for memmap in self._memmap_files]
if memmap_file in existing_memmap_files:
try:
memmap_idx = existing_memmap_files.index(memmap_file)
memmap_obj = self._memmap_files[memmap_idx]
if not memmap_obj._mmap.closed:
memmap_obj._mmap.close()
del memmap_obj
memmap_file.unlink()
del self._memmap_files[memmap_idx]
except Exception as e:
raise Exception(f"Error in deleting {memmap_file.name}: Error {e}")
def make_serialized_dict(self, relative_to=None):
"""
Makes a nested serialized dictionary out of the extractor. The dictionary be used to re-initialize an
extractor with spikeextractors.load_extractor_from_dict(dump_dict)
Parameters
----------
relative_to: str, Path, or None
If not None, file_paths are serialized relative to this path
Returns
-------
dump_dict: dict
Serialized dictionary
"""
class_name = str(type(self)).replace("<class '", "").replace("'>", '')
module = class_name.split('.')[0]
imported_module = importlib.import_module(module)
try:
version = imported_module.__version__
except AttributeError:
version = 'unknown'
if self.is_dumpable:
dump_dict = {'class': class_name, 'module': module, 'kwargs': self._kwargs,
'key_properties': self._key_properties, 'annotations': self._annotations,
'version': version, 'dumpable': True}
else:
dump_dict = {'class': class_name, 'module': module, 'kwargs': {}, 'key_properties': self._key_properties,
'annotations': self._annotations, 'version': version,
'dumpable': False}
if relative_to is not None:
relative_to = Path(relative_to).absolute()
assert relative_to.is_dir(), "'relative_to' must be an existing directory"
dump_dict = _make_paths_relative(dump_dict, relative_to)
return dump_dict
def dump_to_dict(self, relative_to=None):
"""
Dumps recording to a dictionary.
The dictionary be used to re-initialize an
extractor with spikeextractors.load_extractor_from_dict(dump_dict)
Parameters
----------
relative_to: str, Path, or None
If not None, file_paths are serialized relative to this path
Returns
-------
dump_dict: dict
Serialized dictionary
"""
return self.make_serialized_dict(relative_to)
def _get_file_path(self, file_path, extensions):
"""
Helper to be used by various dump_to_file utilities.
Returns default file_path (if not specified), assures that target
directory exists, adds correct file extension if none, and assures
that provided file extension is one of the allowed.
Parameters
----------
file_path: str or None
extensions: list or tuple
First provided is used as an extension for the default file_path.
All are tested against
Returns
-------
Path
Path object with file path to the file
Raises
------
NotDumpableExtractorError
"""
ext = extensions[0]
if self.check_if_dumpable():
if file_path is None:
file_path = self._default_filename + ext
file_path = Path(file_path)
file_path.parent.mkdir(parents=True, exist_ok=True)
folder_path = file_path.parent
if Path(file_path).suffix == '':
file_path = folder_path / (str(file_path) + ext)
assert file_path.suffix in extensions, \
"'file_path' should have one of the following extensions:" \
" %s" % (', '.join(extensions))
return file_path
else:
raise NotDumpableExtractorError(
f"The extractor is not dumpable to {ext}")
def dump_to_json(self, file_path=None, relative_to=None):
"""
Dumps recording extractor to json file.
The extractor can be re-loaded with spikeextractors.load_extractor_from_json(json_file)
Parameters
----------
file_path: str
Path of the json file
relative_to: str, Path, or None
If not None, file_paths are serialized relative to this path
"""
dump_dict = self.make_serialized_dict(relative_to)
self._get_file_path(file_path, ['.json'])\
.write_text(
json.dumps(_check_json(dump_dict), indent=4),
encoding='utf8'
)
def dump_to_pickle(self, file_path=None, include_properties=True, include_features=True,
relative_to=None):
"""
Dumps recording extractor to a pickle file.
The extractor can be re-loaded with spikeextractors.load_extractor_from_json(json_file)
Parameters
----------
file_path: str
Path of the json file
include_properties: bool
If True, all properties are dumped
include_features: bool
If True, all features are dumped
relative_to: str, Path, or None
If not None, file_paths are serialized relative to this path
"""
file_path = self._get_file_path(file_path, ['.pkl', '.pickle'])
# Dump all
dump_dict = {'serialized_dict': self.make_serialized_dict(relative_to)}
if include_properties:
if len(self._properties.keys()) > 0:
dump_dict['properties'] = self._properties
if include_features:
if len(self._features.keys()) > 0:
dump_dict['features'] = self._features
# include times
dump_dict["times"] = self._times
file_path.write_bytes(pickle.dumps(dump_dict))
def get_tmp_folder(self):
"""
Returns temporary folder associated to the extractor
Returns
-------
temp_folder: Path
The temporary folder
"""
if self._tmp_folder is None:
self._tmp_folder = Path(tempfile.mkdtemp())
return self._tmp_folder
def set_tmp_folder(self, folder):
"""
Sets temporary folder of the extractor
Parameters
----------
folder: str or Path
The temporary folder
"""
self._tmp_folder = Path(folder)
def allocate_array(self, memmap, shape=None, dtype=None, name=None, array=None):
"""
Allocates a memory or memmap array
Parameters
----------
memmap: bool
If True, a memmap array is created in the sorting temporary folder
shape: tuple
Shape of the array. If None array must be given
dtype: dtype
Dtype of the array. If None array must be given
name: str or None
Name (root) of the file (if memmap is True). If None, a random name is generated
array: np.array
If array is given, shape and dtype are initialized based on the array. If memmap is True, the array is then
deleted to clear memory
Returns
-------
arr: np.array or np.memmap
The allocated memory or memmap array
"""
if memmap:
tmp_folder = self.get_tmp_folder()
if array is not None:
shape = array.shape
dtype = array.dtype
else:
assert shape is not None and dtype is not None, "Pass 'shape' and 'dtype' arguments"
if name is None:
tmp_file = tempfile.NamedTemporaryFile(suffix=".raw", dir=tmp_folder).name
else:
if Path(name).suffix == '':
tmp_file = tmp_folder / (name + '.raw')
else:
tmp_file = tmp_folder / name
raw_tmp_file = r'{}'.format(str(tmp_file))
# make sure any open memmap files with same path are deleted
self.del_memmap_file(raw_tmp_file)
arr = np.memmap(raw_tmp_file, mode='w+', shape=shape, dtype=dtype)
if array is not None:
arr[:] = array
del array
else:
arr[:] = 0
self._memmap_files.append(arr)
else:
if array is not None:
arr = array
else:
arr = np.zeros(shape, dtype=dtype)
return arr
def annotate(self, annotation_key, value, overwrite=False):
"""This function adds an entry to the annotations dictionary.
Parameters
----------
annotation_key: str
An annotation stored by the Extractor
value:
The data associated with the given property name. Could be many
formats as specified by the user
overwrite: bool
If True and the annotation already exists, it is overwritten
"""
if annotation_key not in self._annotations.keys():
self._annotations[annotation_key] = value
else:
if overwrite:
self._annotations[annotation_key] = value
else:
print(f"{annotation_key} is already an annotation key. Use 'overwrite=True' to overwrite it")
def get_annotation(self, annotation_name):
"""This function returns the data stored under the annotation name.
Parameters
----------
annotation_name: str
A property stored by the Extractor
Returns
----------
annotation_data
The data associated with the given property name. Could be many
formats as specified by the user
"""
if annotation_name not in self._annotations.keys():
print(f"{annotation_name} is not an annotation")
return None
else:
return deepcopy(self._annotations[annotation_name])
def get_annotation_keys(self):
"""This function returns a list of stored annotation keys
Returns
----------
property_names: list
List of stored annotation keys
"""
return list(self._annotations.keys())
def copy_annotations(self, extractor):
"""Copy object properties from another extractor to the current extractor.
Parameters
----------
extractor: Extractor
The extractor from which the annotations will be copied
"""
self._annotations = deepcopy(extractor._annotations)
def add_epoch(self, epoch_name, start_frame, end_frame):
"""This function adds an epoch to your extractor that tracks
a certain time period. It is stored in an internal
dictionary of start and end frame tuples.
Parameters
----------
epoch_name: str
The name of the epoch to be added
start_frame: int
The start frame of the epoch to be added (inclusive)
end_frame: int
The end frame of the epoch to be added (exclusive). If set to None, it will include the entire
sorting after the start_frame
"""
if isinstance(epoch_name, str):
start_frame, end_frame = self._cast_start_end_frame(start_frame, end_frame)
self._epochs[epoch_name] = {'start_frame': start_frame, 'end_frame': end_frame}
else:
raise TypeError("epoch_name must be a string")
def remove_epoch(self, epoch_name):
"""This function removes an epoch from your extractor.
Parameters
----------
epoch_name: str
The name of the epoch to be removed
"""
if isinstance(epoch_name, str):
if epoch_name in list(self._epochs.keys()):
del self._epochs[epoch_name]
else:
raise ValueError("This epoch has not been added")
else:
raise ValueError("epoch_name must be a string")
def get_epoch_names(self):
"""This function returns a list of all the epoch names in the extractor
Returns
----------
epoch_names: list
List of epoch names in the recording extractor
"""
epoch_names = list(self._epochs.keys())
if not epoch_names:
pass
else:
epoch_start_frames = []
for epoch_name in epoch_names:
epoch_info = self.get_epoch_info(epoch_name)
start_frame = epoch_info['start_frame']
epoch_start_frames.append(start_frame)
epoch_names = [epoch_name for _, epoch_name in sorted(zip(epoch_start_frames, epoch_names))]
return epoch_names
def get_epoch_info(self, epoch_name):
"""This function returns the start frame and end frame of the epoch
in a dict.
Parameters
----------
epoch_name: str
The name of the epoch to be returned
Returns
----------
epoch_info: dict
A dict containing the start frame and end frame of the epoch
"""
# Default (Can add more information into each epoch in subclass)
if isinstance(epoch_name, str):
if epoch_name in list(self._epochs.keys()):
epoch_info = self._epochs[epoch_name]
return epoch_info
else:
raise ValueError("This epoch has not been added")
else:
raise ValueError("epoch_name must be a string")
def copy_epochs(self, extractor):
"""Copy epochs from another extractor.
Parameters
----------
extractor: BaseExtractor
The extractor from which the epochs will be copied
"""
for epoch_name in extractor.get_epoch_names():
epoch_info = extractor.get_epoch_info(epoch_name)
self.add_epoch(epoch_name, epoch_info["start_frame"], epoch_info["end_frame"])
def _cast_start_end_frame(self, start_frame, end_frame):
from .extraction_tools import cast_start_end_frame
return cast_start_end_frame(start_frame, end_frame)
@staticmethod
def load_extractor_from_json(json_file):
"""
Instantiates extractor from json file
Parameters
----------
json_file: str or Path
Path to json file
Returns
-------
extractor: RecordingExtractor or SortingExtractor
The loaded extractor object
"""
json_file = Path(json_file)
with open(str(json_file), 'r') as f:
d = json.load(f)
extractor = _load_extractor_from_dict(d)
return extractor
@staticmethod
def load_extractor_from_pickle(pkl_file):
"""
Instantiates extractor from pickle file.
Parameters
----------
pkl_file: str or Path
Path to pickle file
Returns
-------
extractor: RecordingExtractor or SortingExtractor
The loaded extractor object
"""
pkl_file = Path(pkl_file)
with open(str(pkl_file), 'rb') as f:
d = pickle.load(f)
extractor = _load_extractor_from_dict(d['serialized_dict'])
if 'properties' in d.keys():
extractor._properties = d['properties']
if 'features' in d.keys():
extractor._features = d['features']
if 'times' in d.keys():
extractor._times = d['times']
return extractor
@staticmethod
def load_extractor_from_dict(d):
"""
Instantiates extractor from dictionary
Parameters
----------
d: dictionary
Python dictionary
Returns
-------
extractor: RecordingExtractor or SortingExtractor
The loaded extractor object
"""
extractor = _load_extractor_from_dict(d)
return extractor
def check_if_dumpable(self):
return _check_if_dumpable(self.make_serialized_dict())
def _make_paths_relative(d, relative):
dcopy = deepcopy(d)
if "kwargs" in dcopy.keys():
relative_kwargs = _make_paths_relative(dcopy["kwargs"], relative)
dcopy["kwargs"] = relative_kwargs
return dcopy
else:
for k in d.keys():
# in SI, all input paths have the "path" keyword
if "path" in k:
d[k] = str(Path(d[k]).relative_to(relative))
return d
def _load_extractor_from_dict(dic):
cls = None
class_name = None
probe_file = None
kwargs = deepcopy(dic['kwargs'])
if np.any([isinstance(v, dict) for v in kwargs.values()]):
# nested
for k in kwargs.keys():
if isinstance(kwargs[k], dict):
if 'module' in kwargs[k].keys() and 'class' in kwargs[k].keys() and 'version' in kwargs[k].keys():
extractor = _load_extractor_from_dict(kwargs[k])
class_name = dic['class']
cls = _get_class_from_string(class_name)
kwargs[k] = extractor
break
elif np.any([isinstance(v, list) and isinstance(v[0], dict) for v in kwargs.values()]):
# multi
for k in kwargs.keys():
if isinstance(kwargs[k], list) and isinstance(kwargs[k][0], dict):
extractors = []
for kw in kwargs[k]:
if 'module' in kw.keys() and 'class' in kw.keys() and 'version' in kw.keys():
extr = _load_extractor_from_dict(kw)
extractors.append(extr)
class_name = dic['class']
cls = _get_class_from_string(class_name)
kwargs[k] = extractors
break
else:
class_name = dic['class']
cls = _get_class_from_string(class_name)
assert cls is not None and class_name is not None, "Could not load spikeinterface class"
if not _check_same_version(class_name, dic['version']):
print('Versions are not the same. This might lead to errors. Use ', class_name.split('.')[0],
'version', dic['version'])
if 'probe_file' in kwargs.keys():
probe_file = kwargs.pop('probe_file')
# instantiate extrator object
extractor = cls(**kwargs)
# load probe file
if probe_file is not None:
assert 'Recording' in class_name, "Only recording extractors can have probe files"
extractor = extractor.load_probe_file(probe_file=probe_file)
# load properties and features
if 'key_properties' in dic.keys():
extractor._key_properties = dic['key_properties']
if 'annotations' in dic.keys():
extractor._annotations = dic['annotations']
return extractor
def _get_class_from_string(class_string):
class_name = class_string.split('.')[-1]
module = '.'.join(class_string.split('.')[:-1])
imported_module = importlib.import_module(module)
try:
imported_class = getattr(imported_module, class_name)
except:
imported_class = None
return imported_class
def _check_same_version(class_string, version):
module = class_string.split('.')[0]
imported_module = importlib.import_module(module)
try:
return imported_module.__version__ == version
except AttributeError:
return 'unknown'
def _check_if_dumpable(d):
kwargs = d['kwargs']
if np.any([isinstance(v, dict) and 'dumpable' in v.keys() for (k, v) in kwargs.items()]):
for k, v in kwargs.items():
if 'dumpable' in v.keys():
return _check_if_dumpable(v)
else:
return d['dumpable']
def _check_json(d):
# quick hack to ensure json writable
for k, v in d.items():
if isinstance(v, dict):
d[k] = _check_json(v)
elif isinstance(v, Path):
d[k] = str(v.absolute())
elif isinstance(v, bool):
d[k] = bool(v)
elif isinstance(v, (int, np.integer)):
d[k] = int(v)
elif isinstance(v, float):
d[k] = float(v)
elif isinstance(v, datetime.datetime):
d[k] = v.isoformat()
elif isinstance(v, (np.ndarray, list)):
if len(v) > 0:
if isinstance(v[0], dict):
# these must be extractors for multi extractors
d[k] = [_check_json(v_el) for v_el in v]
else:
v_arr = np.array(v)
if len(v_arr.shape) == 1:
if 'int' in str(v_arr.dtype):
v_arr = [int(v_el) for v_el in v_arr]
d[k] = v_arr
elif 'float' in str(v_arr.dtype):
v_arr = [float(v_el) for v_el in v_arr]
d[k] = v_arr
elif isinstance(v_arr[0], str):
v_arr = [str(v_el) for v_el in v_arr]
d[k] = v_arr
else:
print(f'Skipping field {k}: only 1D arrays of int, float, or str types can be serialized')
elif len(v_arr.shape) == 2:
if 'int' in str(v_arr.dtype):
v_arr = [[int(v_el) for v_el in v_row] for v_row in v_arr]
d[k] = v_arr
elif 'float' in str(v_arr.dtype):
v_arr = [[float(v_el) for v_el in v_row] for v_row in v_arr]
d[k] = v_arr
elif 'bool' in str(v_arr.dtype):
v_arr = [[bool(v_el) for v_el in v_row] for v_row in v_arr]
d[k] = v_arr
else:
print(f'Skipping field {k}: only 2D arrays of int or float type can be serialized')
else:
print(f"Skipping field {k}: only 1D and 2D arrays can be serialized")
else:
d[k] = list(v)
return d
================================================
FILE: spikeextractors/cacheextractors.py
================================================
from spikeextractors.extractors.bindatrecordingextractor import BinDatRecordingExtractor
from spikeextractors.extractors.npzsortingextractor import NpzSortingExtractor
from spikeextractors import RecordingExtractor, SortingExtractor
import tempfile
from pathlib import Path
from copy import deepcopy
import importlib
import shutil
class CacheRecordingExtractor(BinDatRecordingExtractor, RecordingExtractor):
def __init__(self, recording, return_scaled=True,
chunk_size=None, chunk_mb=500, save_path=None, n_jobs=1, joblib_backend='loky',
verbose=False):
RecordingExtractor.__init__(self) # init tmp folder before constructing BinDatRecordingExtractor
tmp_folder = self.get_tmp_folder()
self._recording = recording
if save_path is None:
self._is_tmp = True
self._tmp_file = tempfile.NamedTemporaryFile(suffix=".dat", dir=tmp_folder).name
else:
save_path = Path(save_path)
if save_path.suffix != '.dat' and save_path.suffix != '.bin':
save_path = save_path.with_suffix('.dat')
save_path.parent.mkdir(parents=True, exist_ok=True)
self._is_tmp = False
self._tmp_file = save_path
self._return_scaled = return_scaled
self._dtype = recording.get_dtype(return_scaled)
recording.write_to_binary_dat_format(save_path=self._tmp_file, dtype=self._dtype, chunk_size=chunk_size,
chunk_mb=chunk_mb, n_jobs=n_jobs, joblib_backend=joblib_backend,
return_scaled=self._return_scaled, verbose=verbose)
# keep track of filter status when dumping
self.is_filtered = self._recording.is_filtered
BinDatRecordingExtractor.__init__(self, self._tmp_file, numchan=recording.get_num_channels(),
recording_channels=recording.get_channel_ids(),
sampling_frequency=recording.get_sampling_frequency(),
dtype=self._dtype, is_filtered=self.is_filtered)
self.set_tmp_folder(tmp_folder)
self.copy_channel_properties(recording)
self.copy_times(recording)
if 'gain' in recording.get_shared_channel_property_names() and not return_scaled:
self.set_channel_gains(recording.get_channel_gains())
self.set_channel_offsets(recording.get_channel_offsets())
self.has_unscaled = True
else:
self.clear_channel_gains()
self.clear_channel_offsets()
# keep BinDatRecording kwargs
self._bindat_kwargs = deepcopy(self._kwargs)
self._kwargs = {'recording': recording, 'chunk_size': chunk_size, 'chunk_mb': chunk_mb}
def __del__(self):
if self._is_tmp:
try:
# close memmap file (for Windows)
del self._timeseries
Path(self._tmp_file).unlink()
except Exception as e:
print("Unable to remove temporary file", e)
@property
def filename(self):
return str(self._tmp_file)
def move_to(self, save_path):
save_path = Path(save_path)
if save_path.suffix != '.dat' and save_path.suffix != '.bin':
save_path = save_path.with_suffix('.dat')
save_path.parent.mkdir(parents=True, exist_ok=True)
# close memmap file (for Windows)
del self._timeseries
shutil.move(self._tmp_file, str(save_path))
self._tmp_file = str(save_path)
self._kwargs['file_path'] = str(Path(self._tmp_file).absolute())
self._bindat_kwargs['file_path'] = str(Path(self._tmp_file).absolute())
self._is_tmp = False
tmp_folder = self.get_tmp_folder()
# re-initialize with new file
BinDatRecordingExtractor.__init__(self, self._tmp_file, numchan=self._recording.get_num_channels(),
recording_channels=self._recording.get_channel_ids(),
sampling_frequency=self._recording.get_sampling_frequency(),
dtype=self._dtype, is_filtered=self.is_filtered)
self.set_tmp_folder(tmp_folder)
self.copy_channel_properties(self._recording)
# override to make serialization avoid reloading and saving binary file
def make_serialized_dict(self, include_properties=None, include_features=None):
"""
Makes a nested serialized dictionary out of the extractor. The dictionary be used to re-initialize an
extractor with spikeextractors.load_extractor_from_dict(dump_dict)
Returns
-------
include_properties: list or None
List of properties to include in the dictionary
include_features: list or None
List of features to include in the dictionary
"""
class_name = str(BinDatRecordingExtractor).replace("<class '", "").replace("'>", '')
module = class_name.split('.')[0]
imported_module = importlib.import_module(module)
if self._is_tmp:
print("Warning: dumping a CacheRecordingExtractor. The path to the tmp binary file will be lost in "
"further sessions. To prevent this, use the 'CacheRecordingExtractor.move_to('path-to-file)' "
"function")
dump_dict = {'class': class_name, 'module': module, 'kwargs': self._bindat_kwargs,
'key_properties': self._key_properties, 'version': imported_module.__version__, 'dumpable': True}
return dump_dict
class CacheSortingExtractor(NpzSortingExtractor, SortingExtractor):
def __init__(self, sorting, save_path=None):
SortingExtractor.__init__(self) # init tmp folder before constructing NpzSortingExtractor
tmp_folder = self.get_tmp_folder()
self._sorting = sorting
if save_path is None:
self._is_tmp = True
self._tmp_file = tempfile.NamedTemporaryFile(suffix=".npz", dir=tmp_folder).name
else:
save_path = Path(save_path)
if save_path.suffix != '.npz':
save_path = save_path.with_suffix('.npz')
save_path.parent.mkdir(parents=True, exist_ok=True)
self._is_tmp = False
self._tmp_file = save_path
NpzSortingExtractor.write_sorting(self._sorting, self._tmp_file)
NpzSortingExtractor.__init__(self, self._tmp_file)
# keep Npz kwargs
self._npz_kwargs = deepcopy(self._kwargs)
self.set_tmp_folder(tmp_folder)
self.copy_unit_properties(sorting)
self.copy_unit_spike_features(sorting)
self._kwargs = {'sorting': sorting}
def __del__(self):
if self._is_tmp:
try:
Path(self._tmp_file).unlink()
except Exception as e:
print("Unable to remove temporary file", e)
@property
def filename(self):
return str(self._tmp_file)
def move_to(self, save_path):
save_path = Path(save_path)
if save_path.suffix != '.npz':
save_path = save_path.with_suffix('.npz')
save_path.parent.mkdir(parents=True, exist_ok=True)
shutil.move(self._tmp_file, str(save_path))
self._tmp_file = str(save_path)
self._kwargs['file_path'] = str(Path(self._tmp_file).absolute())
self._npz_kwargs['file_path'] = str(Path(self._tmp_file).absolute())
self._is_tmp = False
tmp_folder = self.get_tmp_folder()
# re-initialize with new file
NpzSortingExtractor.__init__(self, self._tmp_file)
# keep Npz kwargs
self.set_tmp_folder(tmp_folder)
self.copy_unit_properties(self._sorting)
self.copy_unit_spike_features(self._sorting)
# override to make serialization avoid reloading and saving npz file
def make_serialized_dict(self, include_properties=None, include_features=None):
"""
Makes a nested serialized dictionary out of the extractor. The dictionary be used to re-initialize an
extractor with spikeextractors.load_extractor_from_dict(dump_dict)
Returns
-------
include_properties: list or None
List of properties to include in the dictionary
include_features: list or None
List of features to include in the dictionary
"""
class_name = str(NpzSortingExtractor).replace("<class '", "").replace("'>", '')
module = class_name.split('.')[0]
imported_module = importlib.import_module(module)
if self._is_tmp:
print("Warning: dumping a CacheSortingExtractor. The path to the tmp binary file will be lost in "
"further sessions. To prevent this, use the 'CacheSortingExtractor.move_to('path-to-file)' "
"function")
dump_dict = {'class': class_name, 'module': module, 'kwargs': self._npz_kwargs,
'key_properties': self._key_properties, 'version': imported_module.__version__, 'dumpable': True}
return dump_dict
================================================
FILE: spikeextractors/example_datasets/__init__.py
================================================
from .toy_example import toy_example
================================================
FILE: spikeextractors/example_datasets/synthesize_random_firings.py
================================================
import numpy as np
def synthesize_random_firings(*, K=20, sampling_frequency=30000.0, duration=60, seed=None):
if seed is not None:
np.random.seed(seed)
seeds = np.random.RandomState(seed=seed).randint(0, 2147483647, K)
else:
seeds = np.random.randint(0, 2147483647, K)
firing_rates = 3 * np.ones((K))
refr = 4
N = np.int64(duration * sampling_frequency)
# events/sec * sec/timepoint * N
populations = np.ceil(firing_rates / sampling_frequency * N).astype('int')
times = np.zeros(0)
labels = np.zeros(0)
for i, k in enumerate(range(1, K + 1)):
refr_timepoints = refr / 1000 * sampling_frequency
times0 = np.random.rand(populations[k - 1]) * (N - 1) + 1
## make an interesting autocorrelogram shape
times0 = np.hstack((times0, times0 + rand_distr2(refr_timepoints, refr_timepoints * 20, times0.size, seeds[i])))
times0 = times0[np.random.RandomState(seed=seeds[i]).choice(times0.size, int(times0.size / 2))]
times0 = times0[np.where((0 <= times0) & (times0 < N))]
times0 = enforce_refractory_period(times0, refr_timepoints)
times = np.hstack((times, times0))
labels = np.hstack((labels, k * np.ones(times0.shape)))
sort_inds = np.argsort(times)
times = times[sort_inds]
labels = labels[sort_inds]
return (times, labels)
def rand_distr2(a, b, num, seed):
X = np.random.RandomState(seed=seed).rand(num)
X = a + (b - a) * X ** 2
return X
def enforce_refractory_period(times_in, refr):
if (times_in.size == 0): return times_in
times0 = np.sort(times_in)
done = False
while not done:
diffs = times0[1:] - times0[:-1]
diffs = np.hstack((diffs, np.inf)) # hack to make sure we handle the last one
inds0 = np.where((diffs[:-1] <= refr) & (diffs[1:] >= refr))[0] # only first violator in every group
if len(inds0) > 0:
times0[inds0] = -1 # kind of a hack, what's the better way?
times0 = times0[np.where(times0 >= 0)]
else:
done = True
return times0
================================================
FILE: spikeextractors/example_datasets/synthesize_random_waveforms.py
================================================
import numpy as np
from .synthesize_single_waveform import synthesize_single_waveform
def synthesize_random_waveforms(*, M=5, T=500, K=20, upsamplefac=13, timeshift_factor=3, average_peak_amplitude=-10,
seed=None):
if seed is not None:
np.random.seed(seed)
seeds = np.random.RandomState(seed=seed).randint(0, 2147483647, K)
else:
seeds = np.random.randint(0, 2147483647, K)
geometry = None
avg_durations = [200, 10, 30, 200]
avg_amps = [0.5, 10, -1, 0]
rand_durations_stdev = [10, 4, 6, 20]
rand_amps_stdev = [0.2, 3, 0.5, 0]
rand_amp_factor_range = [0.5, 1]
geom_spread_coef1 = 0.2
geom_spread_coef2 = 1
if not geometry:
geometry = np.zeros((2, M))
geometry[0, :] = np.arange(1, M + 1)
geometry = np.array(geometry)
avg_durations = np.array(avg_durations)
avg_amps = np.array(avg_amps)
rand_durations_stdev = np.array(rand_durations_stdev)
rand_amps_stdev = np.array(rand_amps_stdev)
rand_amp_factor_range = np.array(rand_amp_factor_range)
neuron_locations = get_default_neuron_locations(M, K, geometry)
## The waveforms_out
WW = np.zeros((M, T * upsamplefac, K))
for i, k in enumerate(range(1, K + 1)):
for m in range(1, M + 1):
diff = neuron_locations[:, k - 1] - geometry[:, m - 1]
dist = np.sqrt(np.sum(diff ** 2))
durations0 = np.maximum(np.ones(avg_durations.shape),
avg_durations + np.random.RandomState(seed=seeds[i]).randn(1, 4) * rand_durations_stdev) * upsamplefac
amps0 = avg_amps + np.random.RandomState(seed=seeds[i]).randn(1, 4) * rand_amps_stdev
waveform0 = synthesize_single_waveform(N=T * upsamplefac, durations=durations0, amps=amps0)
waveform0 = np.roll(waveform0, int(timeshift_factor * dist * upsamplefac))
waveform0 = waveform0 * np.random.RandomState(seed=seeds[i]).uniform(rand_amp_factor_range[0], rand_amp_factor_range[1])
WW[m - 1, :, k - 1] = waveform0 / (geom_spread_coef1 + dist * geom_spread_coef2)
peaks = np.max(np.abs(WW), axis=(0, 1))
WW = WW / np.mean(peaks) * average_peak_amplitude
return (WW, geometry.T)
def get_default_neuron_locations(M, K, geometry):
num_dims = geometry.shape[0]
neuron_locations = np.zeros((num_dims, K))
for k in range(1, K + 1):
if K > 0:
ind = (k - 1) / (K - 1) * (M - 1) + 1
ind0 = int(ind)
if ind0 == M:
ind0 = M - 1
p = 1
else:
p = ind - ind0
if M > 0:
neuron_locations[:, k - 1] = (1 - p) * geometry[:, ind0 - 1] + p * geometry[:, ind0]
else:
neuron_locations[:, k - 1] = geometry[:, 0]
else:
neuron_locations[:, k - 1] = geometry[:, 0]
return neuron_locations
================================================
FILE: spikeextractors/example_datasets/synthesize_single_waveform.py
================================================
import numpy as np
def exp_growth(amp1, amp2, dur1, dur2):
t = np.arange(0, dur1)
Y = np.exp(t / dur2)
# Want Y[0]=amp1
# Want Y[-1]=amp2
Y = Y / (Y[-1] - Y[0]) * (amp2 - amp1)
Y = Y - Y[0] + amp1;
return Y
def exp_decay(amp1, amp2, dur1, dur2):
Y = exp_growth(amp2, amp1, dur1, dur2)
Y = np.flipud(Y) # used to be flip, but that was not supported by older versions of numpy
return Y
def smooth_it(Y, t):
Z = np.zeros(Y.size)
for j in range(-t, t + 1):
Z = Z + np.roll(Y, j)
return Z
def synthesize_single_waveform(*, N=800, durations=[200, 10, 30, 200], amps=[0.5, 10, -1, 0]):
durations = np.array(durations).ravel()
if (np.sum(durations) >= N - 2):
durations[-1] = N - 2 - np.sum(durations[0:durations.size - 1])
amps = np.array(amps).ravel()
timepoints = np.round(np.hstack((0, np.cumsum(durations) - 1))).astype('int');
t = np.r_[0:np.sum(durations) + 1]
Y = np.zeros(len(t))
Y[timepoints[0]:timepoints[1] + 1] = exp_growth(0, amps[0], timepoints[1] + 1 - timepoints[0], durations[0] / 4)
Y[timepoints[1]:timepoints[2] + 1] = exp_growth(amps[0], amps[1], timepoints[2] + 1 - timepoints[1], durations[1])
Y[timepoints[2]:timepoints[3] + 1] = exp_decay(amps[1], amps[2], timepoints[3] + 1 - timepoints[2],
durations[2] / 4)
Y[timepoints[3]:timepoints[4] + 1] = exp_decay(amps[2], amps[3], timepoints[4] + 1 - timepoints[3],
durations[3] / 5)
Y = smooth_it(Y, 3)
Y = Y - np.linspace(Y[0], Y[-1], len(t))
Y = np.hstack((Y, np.zeros(N - len(t))))
Nmid = int(np.floor(N / 2))
peakind = np.argmax(np.abs(Y))
Y = np.roll(Y, Nmid - peakind)
return Y
# Y=smooth_it(Y,3);
# Y=Y-linspace(Y(1),Y(end),length(Y));
#
# Y=[Y,zeros(1,N-length(Y))];
#
# Nmid=floor(N/2);
# [~,peakind]=max(abs(Y));
# Y=circshift(Y,[0,Nmid-peakind]);
#
# end
#
# function test_synth_waveform
# Y=synthesize_single_waveform(800);
# figure; plot(Y);
# end
#
# function Y=exp_growth(amp1,amp2,dur1,dur2)
# t=1:dur1;
# Y=exp(t/dur2);
# % Want Y(1)=amp1
# % Want Y(end)=amp2
# Y=Y/(Y(end)-Y(1))*(amp2-amp1);
# Y=Y-Y(1)+amp1;
# end
#
# function Y=exp_decay(amp1,amp2,dur1,dur2)
# Y=exp_growth(amp2,amp1,dur1,dur2);
# Y=Y(end:-1:1);
# end
#
# function Z=smooth_it(Y,t)
# Z=Y;
# Z(1+t:end-t)=0;
# for j=-t:t
# Z(1+t:end-t)=Z(1+t:end-t)+Y(1+t+j:end-t+j)/(2*t+1);
# end;
# end
if __name__ == '__main__':
Y = synthesize_single_waveform()
import matplotlib.pyplot as plt
plt.plot(Y)
================================================
FILE: spikeextractors/example_datasets/synthesize_timeseries.py
================================================
import numpy as np
def synthesize_timeseries(*, sorting, waveforms, noise_level=1, sampling_frequency=30000.0, duration=60, waveform_upsamplefac=13, seed=None):
num_timepoints = np.int64(sampling_frequency * duration)
waveform_upsamplefac = int(waveform_upsamplefac)
W = waveforms
M, TT, K = W.shape[0], W.shape[1], W.shape[2]
T = int(TT / waveform_upsamplefac)
Tmid = int(np.ceil((T + 1) / 2 - 1))
N = num_timepoints
if seed is not None:
X = np.random.RandomState(seed=seed).randn(M, N) * noise_level
else:
X = np.random.randn(M, N) * noise_level
unit_ids = sorting.get_unit_ids()
for k0 in unit_ids:
waveform0 = waveforms[:, :, k0 - 1]
times0 = sorting.get_unit_spike_train(unit_id=k0)
for t0 in times0:
amp0 = 1
frac_offset = int(np.floor((t0 - np.floor(t0)) * waveform_upsamplefac))
tstart = np.int64(np.floor(t0)) - Tmid
if (0 <= tstart) and (tstart + T <= N):
X[:, tstart:tstart + T] = X[:, tstart:tstart + T] + waveform0[:,
frac_offset::waveform_upsamplefac] * amp0
return X
================================================
FILE: spikeextractors/example_datasets/toy_example.py
================================================
import numpy as np
from pathlib import Path
from typing import Optional, Union
import spikeextractors as se
from .synthesize_random_waveforms import synthesize_random_waveforms
from .synthesize_random_firings import synthesize_random_firings
from .synthesize_timeseries import synthesize_timeseries
def toy_example(
duration: float = 10.,
num_channels: int = 4,
sampling_frequency: float = 30000.,
K: int = 10,
dumpable: bool = False,
dump_folder: Optional[Union[str, Path]] = None,
seed: Optional[int] = None
):
"""
Create toy recording and sorting extractors.
Parameters
----------
duration: float
Duration in s (default 10)
num_channels: int
Number of channels (default 4)
sampling_frequency: float
Sampling frequency (default 30000)
K: int
Number of units (default 10)
dumpable: bool
If True, objects are dumped to file and become 'dumpable'
dump_folder: str or Path
Path to dump folder (if None, 'test' is used
seed: int
Seed for random initialization
Returns
-------
recording: RecordingExtractor
The output recording extractor. If dumpable is False it's a NumpyRecordingExtractor, otherwise it's an
MdaRecordingExtractor
sorting: SortingExtractor
The output sorting extractor. If dumpable is False it's a NumpyRecordingExtractor, otherwise it's an
NpzSortingExtractor
"""
upsamplefac = 13
waveforms, geom = synthesize_random_waveforms(K=K, M=num_channels, average_peak_amplitude=-100,
upsamplefac=upsamplefac, seed=seed)
times, labels = synthesize_random_firings(K=K, duration=duration, sampling_frequency=sampling_frequency, seed=seed)
labels = labels.astype(np.int64)
SX = se.NumpySortingExtractor()
SX.set_times_labels(times, labels)
X = synthesize_timeseries(sorting=SX, waveforms=waveforms, noise_level=10, sampling_frequency=sampling_frequency,
duration=duration,
waveform_upsamplefac=upsamplefac, seed=seed)
SX.set_sampling_frequency(sampling_frequency)
RX = se.NumpyRecordingExtractor(timeseries=X, sampling_frequency=sampling_frequency, geom=geom)
RX.is_filtered = True
if dumpable:
if dump_folder is None:
dump_folder = 'toy_example'
dump_folder = Path(dump_folder)
se.MdaRecordingExtractor.write_recording(RX, dump_folder)
RX = se.MdaRecordingExtractor(dump_folder)
se.NpzSortingExtractor.write_sorting(SX, dump_folder / 'sorting.npz')
SX = se.NpzSortingExtractor(dump_folder / 'sorting.npz')
return RX, SX
================================================
FILE: spikeextractors/exceptions.py
================================================
class NotDumpableExtractorError(TypeError):
"""Raised whenever current extractor cannot be dumped"""
================================================
FILE: spikeextractors/extraction_tools.py
================================================
import numpy as np
import csv
import os
import sys
from pathlib import Path
import warnings
import datetime
from functools import wraps
from .baseextractor import BaseExtractor
from tqdm import tqdm
from joblib import Parallel, delayed
try:
import h5py
HAVE_H5 = True
except ImportError:
HAVE_H5 = False
def read_python(path):
"""Parses python scripts in a dictionary
Parameters
----------
path: str or Path
Path to file to parse
Returns
-------
metadata:
dictionary containing parsed file
"""
from six import exec_
import re
path = Path(path).absolute()
assert path.is_file()
with path.open('r') as f:
contents = f.read()
contents = re.sub(r'range\(([\d,]*)\)',r'list(range(\1))',contents)
metadata = {}
exec_(contents, {}, metadata)
metadata = {k.lower(): v for (k, v) in metadata.items()}
return metadata
def write_python(path, dict):
"""Saves python dictionary to file
Parameters
----------
path: str or Path
Path to save file
dict: dict
dictionary to save
"""
with Path(path).open('w') as f:
for k, v in dict.items():
if isinstance(v ,str) and not v.startswith("'"):
if 'path' in k and 'win' in sys.platform:
f.write(str(k) + " = r'" + str(v) + "'\n")
else:
f.write(str(k) + " = '" + str(v) + "'\n")
else:
f.write(str(k) + " = " + str(v) + "\n")
def load_probe_file(recording, probe_file, channel_map=None, channel_groups=None, verbose=False):
"""This function returns a SubRecordingExtractor that contains information from the given
probe file (channel locations, groups, etc.) If a .prb file is given, then 'location' and 'group'
information for each channel is added to the SubRecordingExtractor. If a .csv file is given, then
it will only add 'location' to the SubRecordingExtractor.
Parameters
----------
recording: RecordingExtractor
The recording extractor to load channel information from.
probe_file: str
Path to probe file. Either .prb or .csv
channel_map : array-like
A list of channel IDs to set in the loaded file.
Only used if the loaded file is a .csv.
channel_groups : array-like
A list of groups (ints) for the channel_ids to set in the loaded file.
Only used if the loaded file is a .csv.
verbose: bool
If True, output is verbose
Returns
---------
subrecording: SubRecordingExtractor
The extractor containing all of the probe information.
"""
from .subrecordingextractor import SubRecordingExtractor
probe_file = Path(probe_file)
if probe_file.suffix == '.prb':
probe_dict = read_python(probe_file)
if 'channel_groups' in probe_dict.keys():
ordered_channels = np.array([], dtype=int)
groups = sorted(probe_dict['channel_groups'].keys())
for cgroup_id in groups:
cgroup = probe_dict['channel_groups'][cgroup_id]
for key_prop, prop_val in cgroup.items():
if key_prop == 'channels':
ordered_channels = np.concatenate((ordered_channels, prop_val))
if not np.all([chan in recording.get_channel_ids() for chan in ordered_channels]) and verbose:
print('Some channel in PRB file are not in original recording')
present_ordered_channels = [chan for chan in ordered_channels if chan in recording.get_channel_ids()]
subrecording = SubRecordingExtractor(recording, channel_ids=present_ordered_channels)
for cgroup_id in groups:
cgroup = probe_dict['channel_groups'][cgroup_id]
if 'channels' not in cgroup.keys() and len(groups) > 1:
raise Exception("If more than one 'channel_group' is in the probe file, the 'channels' field"
"for each channel group is required")
elif 'channels' not in cgroup.keys():
channels_in_group = subrecording.get_num_channels()
channels_id_in_group = subrecording.get_channel_ids()
else:
channels_in_group = len(cgroup['channels'])
channels_id_in_group = cgroup['channels']
for key_prop, prop_val in cgroup.items():
if key_prop == 'channels':
for i_ch, prop in enumerate(prop_val):
if prop in subrecording.get_channel_ids():
subrecording.set_channel_groups(int(cgroup_id), channel_ids=prop)
elif key_prop == 'geometry' or key_prop == 'location':
if isinstance(prop_val, dict):
if len(prop_val.keys()) != channels_in_group and verbose:
print('geometry in PRB does not have the same length as channel in group')
for (i_ch, prop) in prop_val.items():
if i_ch in subrecording.get_channel_ids():
subrecording.set_channel_locations(prop, channel_ids=i_ch)
elif isinstance(prop_val, (list, np.ndarray)) and len(prop_val) == channels_in_group:
if 'channels' not in cgroup.keys():
raise Exception("'geometry'/'location' in the .prb file can be a list only if "
"'channels' field is specified.")
if len(prop_val) != channels_in_group and verbose:
print('geometry in PRB does not have the same length as channel in group')
for (i_ch, prop) in zip(channels_id_in_group, prop_val):
if i_ch in subrecording.get_channel_ids():
subrecording.set_channel_locations(prop, channel_ids=i_ch)
else:
if isinstance(prop_val, dict) and len(prop_val.keys()) == channels_in_group:
for (i_ch, prop) in prop_val.items():
if i_ch in subrecording.get_channel_ids():
subrecording.set_channel_property(i_ch, key_prop, prop)
elif isinstance(prop_val, (list, np.ndarray)) and len(prop_val) == channels_in_group:
for (i_ch, prop) in zip(channels_id_in_group, prop_val):
if i_ch in subrecording.get_channel_ids():
subrecording.set_channel_property(i_ch, key_prop, prop)
# create dummy locations
if 'geometry' not in cgroup.keys() and 'location' not in cgroup.keys():
if 'location' not in subrecording.get_shared_channel_property_names():
locs = np.zeros((subrecording.get_num_channels(), 2))
locs[:, 1] = np.arange(subrecording.get_num_channels())
subrecording.set_channel_locations(locs)
else:
raise AttributeError("'.prb' file should contain the 'channel_groups' field")
elif probe_file.suffix == '.csv':
if channel_map is not None:
assert np.all([chan in channel_map for chan in recording.get_channel_ids()]), \
"all channel_ids in 'channel_map' must be in the original recording channel ids"
subrecording = SubRecordingExtractor(recording, channel_ids=channel_map)
else:
subrecording = SubRecordingExtractor(recording, channel_ids=recording.get_channel_ids())
with probe_file.open() as csvfile:
posreader = csv.reader(csvfile)
row_count = 0
loaded_pos = []
for pos in (posreader):
row_count += 1
loaded_pos.append(pos)
assert len(subrecording.get_channel_ids()) == row_count, "The .csv file must contain as many " \
"rows as the number of channels in the recordings"
for i_ch, pos in zip(subrecording.get_channel_ids(), loaded_pos):
if i_ch in subrecording.get_channel_ids():
subrecording.set_channel_locations(list(np.array(pos).astype(float)), i_ch)
if channel_groups is not None and len(channel_groups) == len(subrecording.get_channel_ids()):
for i_ch, chg in zip(subrecording.get_channel_ids(), channel_groups):
if i_ch in subrecording.get_channel_ids():
subrecording.set_channel_groups(chg, i_ch)
else:
raise NotImplementedError("Only .csv and .prb probe files can be loaded.")
subrecording._kwargs['probe_file'] = str(probe_file.absolute())
return subrecording
def save_to_probe_file(recording, probe_file, grouping_property=None, radius=None,
graph=True, geometry=True, verbose=False):
"""Saves probe file from the channel information of the given recording
extractor.
Parameters
----------
recording: RecordingExtractor
The recording extractor to save probe file from
probe_file: str
file name of .prb or .csv file to save probe information to
grouping_property: str (default None)
If grouping_property is a shared_channel_property, different groups are saved based on the property.
radius: float (default None)
Adjacency radius (used by some sorters). If None it is not saved to the probe file.
graph: bool
If True, the adjacency graph is saved (default=True)
geometry: bool
If True, the geometry is saved (default=True)
verbose: bool
If True, output is verbose
"""
probe_file = Path(probe_file)
if not probe_file.parent.is_dir():
probe_file.parent.mkdir()
if probe_file.suffix == '.csv':
# write csv probe file
with probe_file.open('w') as f:
if 'location' in recording.get_shared_channel_property_names():
for chan in recording.get_channel_ids():
loc = recording.get_channel_locations(chan)[0]
if len(loc) == 2:
f.write(str(loc[0]))
f.write(',')
f.write(str(loc[1]))
f.write('\n')
elif len(loc) == 3:
f.write(str(loc[0]))
f.write(',')
f.write(str(loc[1]))
f.write(',')
f.write(str(loc[2]))
f.write('\n')
else:
raise AttributeError("Recording extractor needs to have "
"'location' property to save .csv probe file")
elif probe_file.suffix == '.prb':
_export_prb_file(recording, probe_file, grouping_property=grouping_property, radius=radius, graph=graph,
geometry=geometry, verbose=verbose)
else:
raise NotImplementedError("Only .csv and .prb probe files can be saved.")
def read_binary(file, numchan, dtype, time_axis=0, offset=0):
"""
Reads binary .bin or .dat file.
Parameters
----------
file: str
File name
numchan: int
Number of channels
dtype: dtype
dtype of the file
time_axis: 0 (default) or 1
If 0 then traces are transposed to ensure (nb_sample, nb_channel) in the file.
If 1, the traces shape (nb_channel, nb_sample) is kept in the file.
offset: int
number of offset bytes
"""
numchan = int(numchan)
with Path(file).open() as f:
nsamples = (os.fstat(f.fileno()).st_size - offset) // (numchan * np.dtype(dtype).itemsize)
if time_axis == 0:
samples = np.memmap(file, np.dtype(dtype), mode='r', offset=offset, shape=(nsamples, numchan)).T
else:
samples = np.memmap(file, np.dtype(dtype), mode='r', offset=offset, shape=(numchan, nsamples))
return samples
def write_to_binary_dat_format(recording, save_path=None, file_handle=None,
time_axis=0, dtype=None, chunk_size=None, chunk_mb=500, n_jobs=1, joblib_backend='loky',
return_scaled=True, verbose=False):
"""Saves the traces of a recording extractor in binary .dat format.
Parameters
----------
recording: RecordingExtractor
The recording extractor object to be saved in .dat format
save_path: str
The path to the file.
file_handle: file handle
The file handle to dump data. This can be used to append data to an header. In case file_handle is given,
the file is NOT closed after writing the binary data.
time_axis: 0 (default) or 1
If 0 then traces are transposed to ensure (nb_sample, nb_channel) in the file.
If 1, the traces shape (nb_channel, nb_sample) is kept in the file.
dtype: dtype
Type of the saved data. Default float32.
chunk_size: None or int
Size of each chunk in number of frames.
If None (default) and 'chunk_mb' is given, the file is saved in chunks of 'chunk_mb' Mb (default 500Mb)
chunk_mb: None or int
Chunk size in Mb (default 500Mb)
n_jobs: int
Number of jobs to use (Default 1)
joblib_backend: str
Joblib backend for parallel processing ('loky', 'threading', 'multiprocessing')
return_scaled: bool
If True, traces are written after scaling (using gain/offset). If False, the raw traces are written
verbose: bool
If True, output is verbose (when chunks are used)
"""
assert save_path is not None or file_handle is not None, "Provide 'save_path' or 'file handle'"
if save_path is not None:
save_path = Path(save_path)
if save_path.suffix == '':
# when suffix is already raw/bin/dat do not change it.
save_path = save_path.parent / (save_path.name + '.dat')
if chunk_size is not None or chunk_mb is not None:
if time_axis == 1:
print("Chunking disabled due to 'time_axis' == 1")
chunk_size = None
chunk_mb = None
# set chunk size
if chunk_size is not None:
chunk_size = int(chunk_size)
elif chunk_mb is not None:
n_bytes = np.dtype(recording.get_dtype()).itemsize
max_size = int(chunk_mb * 1e6) # set Mb per chunk
chunk_size = max_size // (recording.get_num_channels() * n_bytes)
if n_jobs is None:
n_jobs = 1
if n_jobs == 0:
n_jobs = 1
if n_jobs > 1:
if chunk_size is not None:
chunk_size /= n_jobs
if not recording.check_if_dumpable():
if n_jobs > 1:
n_jobs = 1
print("RecordingExtractor is not dumpable and can't be processed in parallel")
rec_arg = recording
else:
if n_jobs > 1:
rec_arg = recording.dump_to_dict()
else:
rec_arg = recording
if chunk_size is None:
traces = recording.get_traces(return_scaled=return_scaled)
if dtype is not None:
traces = traces.astype(dtype)
if time_axis == 0:
traces = traces.T
if save_path is not None:
with save_path.open('wb') as f:
traces.tofile(f)
else:
traces.tofile(file_handle)
else:
# chunk size is not None
num_frames = recording.get_num_frames()
num_channels = recording.get_num_channels()
# chunk_size = num_bytes_per_chunk / num_bytes_per_frame
chunks = divide_recording_into_time_chunks(
num_frames=num_frames,
chunk_size=chunk_size,
padding_size=0
)
n_chunk = len(chunks)
if verbose and n_jobs == 1:
chunks_loop = tqdm(range(n_chunk), ascii=True, desc="Writing to binary .dat file")
else:
chunks_loop = range(n_chunk)
if save_path is not None:
if n_jobs == 1:
if time_axis == 0:
shape = (num_frames, num_channels)
else:
shape = (num_channels, num_frames)
rec_memmap = np.memmap(str(save_path), dtype=dtype, mode='w+', shape=shape)
for i in chunks_loop:
_write_dat_one_chunk(i, rec_arg, chunks, rec_memmap, dtype, time_axis, return_scaled,
verbose=False)
else:
if time_axis == 0:
shape = (num_frames, num_channels)
else:
shape = (num_channels, num_frames)
rec_memmap = np.memmap(str(save_path), dtype=dtype, mode='w+', shape=shape)
Parallel(n_jobs=n_jobs, backend=joblib_backend)(
delayed(_write_dat_one_chunk)(i, rec_arg, chunks, rec_memmap, dtype, time_axis, return_scaled,
verbose,)
for i in chunks_loop)
else:
for i in chunks_loop:
start_frame = chunks[i]['istart']
end_frame = chunks[i]['iend']
traces = recording.get_traces(start_frame=start_frame, end_frame=end_frame,
return_scaled=return_scaled)
if dtype is not None:
traces = traces.astype(dtype)
if time_axis == 0:
traces = traces.T
file_handle.write(traces.tobytes())
return save_path
def write_to_h5_dataset_format(recording, dataset_path, save_path=None, file_handle=None,
time_axis=0, dtype=None, chunk_size=None, chunk_mb=500, verbose=False):
"""Saves the traces of a recording extractor in an h5 dataset.
Parameters
----------
recording: RecordingExtractor
The recording extractor object to be saved in .dat format
dataset_path: str
Path to dataset in h5 filee (e.g. '/dataset')
save_path: str
The path to the file.
file_handle: file handle
The file handle to dump data. This can be used to append data to an header. In case file_handle is given,
the file is NOT closed after writing the binary data.
time_axis: 0 (default) or 1
If 0 then traces are transposed to ensure (nb_sample, nb_channel) in the file.
If 1, the traces shape (nb_channel, nb_sample) is kept in the file.
dtype: dtype
Type of the saved data. Default float32.
chunk_size: None or int
Size of each chunk in number of frames.
If None (default) and 'chunk_mb' is given, the file is saved in chunks of 'chunk_mb' Mb (default 500Mb)
chunk_mb: None or int
Chunk size in Mb (default 500Mb)
verbose: bool
If True, output is verbose (when chunks are used)
"""
assert HAVE_H5, "To write to h5 you need to install h5py: pip install h5py"
assert save_path is not None or file_handle is not None, "Provide 'save_path' or 'file handle'"
if save_path is not None:
save_path = Path(save_path)
if save_path.suffix == '':
# when suffix is already raw/bin/dat do not change it.
save_path = save_path.parent / (save_path.name + '.h5')
num_channels = recording.get_num_channels()
num_frames = recording.get_num_frames()
if file_handle is not None:
assert isinstance(file_handle, h5py.File)
else:
file_handle = h5py.File(save_path, 'w')
if dtype is None:
dtype_file = recording.get_dtype()
else:
dtype_file = dtype
if time_axis == 0:
dset = file_handle.create_dataset(dataset_path, shape=(num_frames, num_channels), dtype=dtype_file)
else:
dset = file_handle.create_dataset(dataset_path, shape=(num_channels, num_frames), dtype=dtype_file)
# set chunk size
if chunk_size is not None:
chunk_size = int(chunk_size)
elif chunk_mb is not None:
n_bytes = np.dtype(recording.get_dtype()).itemsize
max_size = int(chunk_mb * 1e6) # set Mb per chunk
chunk_size = max_size // (num_channels * n_bytes)
if chunk_size is None:
traces = recording.get_traces()
if dtype is not None:
traces = traces.astype(dtype_file)
if time_axis == 0:
traces = traces.T
dset[:] = traces
else:
chunk_start = 0
# chunk size is not None
n_chunk = num_frames // chunk_size
if num_frames % chunk_size > 0:
n_chunk += 1
if verbose:
chunks = tqdm(range(n_chunk), ascii=True, desc="Writing to .h5 file")
else:
chunks = range(n_chunk)
for i in chunks:
traces = recording.get_traces(start_frame=i * chunk_size,
end_frame=min((i + 1) * chunk_size, num_frames))
chunk_frames = traces.shape[1]
if dtype is not None:
traces = traces.astype(dtype_file)
if time_axis == 0:
dset[chunk_start:chunk_start + chunk_frames] = traces.T
else:
dset[:, chunk_start:chunk_start + chunk_frames] = traces
chunk_start += chunk_frames
if save_path is not None:
file_handle.close()
return save_path
def get_sub_extractors_by_property(extractor, property_name, return_property_list=False):
"""Returns a list of SubExtractors from the Extractor based on the given
property_name (e.g. group)
Parameters
----------
extractor: RecordingExtractor or SortingExtractor
The extractor object to access SubRecordingExtractors from.
property_name: str
The property used to subdivide the extractor
return_property_list: bool
If True the property list is returned
Returns
-------
sub_list: list
The list of subextractors to be returned.
OR
sub_list, prop_list
If return_property_list is True, the property list will be returned as well.
"""
from spikeextractors import RecordingExtractor, SortingExtractor, SubRecordingExtractor, SubSortingExtractor
if isinstance(extractor, RecordingExtractor):
if property_name not in extractor.get_shared_channel_property_names():
raise ValueError("'property_name' must be must be a property of the recording channels")
else:
sub_list = []
recording = extractor
properties = np.array([recording.get_channel_property(chan, property_name)
for chan in recording.get_channel_ids()])
prop_list = np.unique(properties)
for prop in prop_list:
prop_idx = np.where(prop == properties)
chan_idx = list(np.array(recording.get_channel_ids())[prop_idx])
sub_list.append(SubRecordingExtractor(recording, channel_ids=chan_idx))
if return_property_list:
return sub_list, prop_list
else:
return sub_list
elif isinstance(extractor, SortingExtractor):
if property_name not in extractor.get_shared_unit_property_names():
raise ValueError("'property_name' must be must be a property of the units")
else:
sub_list = []
sorting = extractor
properties = np.array([sorting.get_unit_property(unit, property_name)
for unit in sorting.get_unit_ids()])
prop_list = np.unique(properties)
for prop in prop_list:
prop_idx = np.where(prop == properties)
unit_idx = list(np.array(sorting.get_unit_ids())[prop_idx])
sub_list.append(SubSortingExtractor(sorting, unit_ids=unit_idx))
if return_property_list:
return sub_list, prop_list
else:
return sub_list
else:
raise ValueError("'extractor' must be a RecordingExtractor or a SortingExtractor")
def _export_prb_file(recording, file_name, grouping_property=None, graph=True, geometry=True,
radius=None, adjacency_distance=100, verbose=False):
"""Exports .prb file
Parameters
----------
recording: RecordingExtractor
The recording extractor to save probe file from
file_name: str
probe filename to be exported to
grouping_property: str (default None)
If grouping_property is a shared_channel_property, different groups are saved based on the property.
graph: bool
If True, the adjacency graph is saved (default=True)
geometry: bool
If True, the geometry is saved (default=True)
radius: float (default None)
Adjacency radius (used by some sorters). If None it is not saved to the probe file.
adjacency_distance: float
Distance to consider two channels to adjacent (if 'location' is a property). If radius is given,
then adjacency_distance is set to the radius.
verbose : bool
If True, output is verbose
"""
file_name = Path(file_name)
assert file_name is not None
abspath = file_name.absolute()
if radius is not None:
adjacency_distance = radius
if geometry:
if 'location' in recording.get_shared_channel_property_names():
positions = recording.get_channel_locations()
else:
if verbose:
print("'location' property is not available and it will not be saved.")
positions = None
geometry = False
else:
positions = None
if grouping_property is not None:
if grouping_property in recording.get_shared_channel_property_names():
grouping_property_groups = np.array([recording.get_channel_property(chan, grouping_property)
for chan in recording.get_channel_ids()])
channel_groups = np.unique([grouping_property_groups])
else:
if verbose:
print(f"{grouping_property} property is not available and it will not be saved.")
channel_groups = [0]
grouping_property_groups = np.array([0] * recording.get_num_channels())
else:
channel_groups = [0]
grouping_property_groups = np.array([0] * recording.get_num_channels())
n_elec = recording.get_num_channels()
# find adjacency graph
if graph:
if positions is not None and adjacency_distance is not None:
adj_graph = []
for chg in channel_groups:
group_graph = []
elecs = list(np.where(grouping_property_groups == chg)[0])
for i in range(len(elecs)):
for j in range(i, len(elecs)):
if elecs[i] != elecs[j]:
if np.linalg.norm(positions[elecs[i]] - positions[elecs[j]]) < adjacency_distance:
group_graph.append((elecs[i], elecs[j]))
adj_graph.append(group_graph)
else:
# all connected by group
adj_graph = []
for chg in channel_groups:
group_graph = []
elecs = list(np.where(grouping_property_groups == chg)[0])
for i in range(len(elecs)):
for j in range(i, len(elecs)):
if elecs[i] != elecs[j]:
group_graph.append((elecs[i], elecs[j]))
adj_graph.append(group_graph)
with abspath.open('w') as f:
f.write('total_nb_channels = ' + str(n_elec) + '\n')
if radius is not None:
f.write('radius = ' + str(radius) + '\n')
f.write('channel_groups = {\n')
if len(channel_groups) > 0:
for i_chg, chg in enumerate(channel_groups):
f.write(" " + str(int(chg)) + ": ")
elecs = list(np.where(grouping_property_groups == chg)[0])
f.write("\n {\n")
f.write(" 'channels': " + str(elecs) + ',\n')
if graph:
if len(adj_graph) == 1:
f.write(" 'graph': " + str(adj_graph[0]) + ',\n')
else:
f.write(" 'graph': " + str(adj_graph[i_chg]) + ',\n')
if geometry:
f.write(" 'geometry': {\n")
for i, pos in enumerate(positions[elecs]):
f.write(' ' + str(elecs[i]) + ': ' + str(list(pos)) + ',\n')
f.write(' }\n')
f.write(' },\n')
f.write('}\n')
else:
for elec in range(n_elec):
f.write(' ' + str(elec) + ': ')
f.write("\n {\n")
f.write(" 'channels': [" + str(elec) + '],\n')
f.write(" 'graph': [],\n")
f.write(' },\n')
f.write('}\n')
def _check_json(d):
# quick hack to ensure json writable
for k, v in d.items():
if isinstance(v, Path):
d[k] = str(v)
elif isinstance(v, (int, np.integer)):
d[k] = int(v)
elif isinstance(v, float):
d[k] = float(v)
elif isinstance(v, datetime.datetime):
d[k] = v.isoformat()
return d
def load_extractor_from_json(json_file):
"""
Instantiates extractor from json file
Parameters
----------
json_file: str or Path
Path to json file
Returns
-------
extractor: RecordingExtractor or SortingExtractor
The loaded extractor object
"""
return BaseExtractor.load_extractor_from_json(json_file)
def load_extractor_from_dict(d):
"""
Instantiates extractor from dictionary
Parameters
----------
d: dictionary
Python dictionary
Returns
-------
extractor: RecordingExtractor or SortingExtractor
The loaded extractor object
"""
return BaseExtractor.load_extractor_from_dict(d)
def load_extractor_from_pickle(pkl_file):
"""
Instantiates extractor from pickle file
Parameters
----------
pkl_file: str or Path
Path to pickle file
Returns
-------
extractor: RecordingExtractor or SortingExtractor
The loaded extractor object
"""
return BaseExtractor.load_extractor_from_pickle(pkl_file)
def check_get_unit_spike_train(func):
@wraps(func)
def check_validity(sorting, unit_id, start_frame=None, end_frame=None):
# parse args and kwargs
if unit_id is None:
raise TypeError("get_unit_spike_train() missing 1 required positional argument: 'unit_id')")
elif not (isinstance(unit_id, (int, np.integer))):
raise ValueError("unit_id must be an integer")
elif unit_id not in sorting.get_unit_ids():
raise ValueError(f"{unit_id} is an invalid unit id")
start_frame, end_frame = cast_start_end_frame(start_frame, end_frame)
if start_frame is None:
start_frame = 0
if end_frame is None:
end_frame = np.Inf
return func(sorting, unit_id, start_frame=start_frame, end_frame=end_frame)
return check_validity
def check_get_traces_args(func):
@wraps(func)
def corrected_args(recording, channel_ids=None, start_frame=None, end_frame=None, return_scaled=True, **kwargs):
if channel_ids is not None:
if isinstance(channel_ids, (int, np.integer)):
channel_ids = list([channel_ids])
else:
channel_ids = channel_ids
if np.any([ch not in recording.get_channel_ids() for ch in channel_ids]):
print("Removing invalid 'channel_ids'",
[ch for ch in channel_ids if ch not in recording.get_channel_ids()])
channel_ids = [ch for ch in channel_ids if ch in recording.get_channel_ids()]
else:
channel_ids = recording.get_channel_ids()
if start_frame is not None:
if start_frame < 0:
start_frame = recording.get_num_frames() + start_frame
else:
start_frame = 0
if end_frame is not None:
if end_frame > recording.get_num_frames():
print("'end_frame' set to", recording.get_num_frames())
end_frame = recording.get_num_frames()
elif end_frame < 0:
end_frame = recording.get_num_frames() + end_frame
else:
end_frame = recording.get_num_frames()
assert end_frame - start_frame > 0, "'start_frame' must be less than 'end_frame'!"
start_frame, end_frame = cast_start_end_frame(start_frame, end_frame)
if not recording.has_unscaled and not return_scaled:
warnings.warn("The recording extractor does not have unscaled traces. Returning scaled traces")
return_scaled = True
traces = func(recording, channel_ids=channel_ids, start_frame=start_frame, end_frame=end_frame,
return_scaled=return_scaled, **kwargs)
# scaling
if recording.has_unscaled and return_scaled:
channel_idxs = np.array([recording.get_channel_ids().index(ch) for ch in channel_ids])
gains = recording.get_channel_gains()[channel_idxs, None]
offsets = recording.get_channel_offsets()[channel_idxs, None]
traces = (traces.astype("float32") * gains + offsets).astype("float32")
return traces
return corrected_args
def check_get_ttl_args(func):
@wraps(func)
def corrected_args(recording, start_frame=None, end_frame=None, channel_id=0, **kwargs):
if start_frame is not None:
if start_frame < 0:
start_frame = recording.get_num_frames() + start_frame
else:
start_frame = 0
if end_frame is not None:
if end_frame > recording.get_num_frames():
print("'end_frame' set to", recording.get_num_frames())
end_frame = recording.get_num_frames()
elif end_frame < 0:
end_frame = recording.get_num_frames() + end_frame
else:
end_frame = recording.get_num_frames()
assert end_frame - start_frame > 0, "'start_frame' must be less than 'end_frame'!"
assert isinstance(channel_id, (int, np.integer)), "'channel_id' must be a single int"
start_frame, end_frame = cast_start_end_frame(start_frame, end_frame)
# pass recording as arg and rest as kwargs
get_ttl_correct_arg = func(recording, start_frame=start_frame, end_frame=end_frame, channel_id=channel_id,
**kwargs)
return get_ttl_correct_arg
return corrected_args
def cast_start_end_frame(start_frame, end_frame):
if isinstance(start_frame, float):
start_frame = int(start_frame)
elif isinstance(start_frame, (int, np.integer, type(None))):
start_frame = start_frame
else:
raise ValueError("start_frame must be an int, float (not infinity), or None")
if isinstance(end_frame, float) and np.isfinite(end_frame):
end_frame = int(end_frame)
elif isinstance(end_frame, (int, np.integer, type(None))):
end_frame = end_frame
# else end_frame is infinity (accepted for get_unit_spike_train)
if start_frame is not None:
start_frame = int(start_frame)
if end_frame is not None and np.isfinite(end_frame):
end_frame = int(end_frame)
return start_frame, end_frame
def divide_recording_into_time_chunks(num_frames, chunk_size, padding_size):
chunks = []
ii = 0
while ii < num_frames:
ii2 = int(min(ii + chunk_size, num_frames))
chunks.append(dict(
istart=ii,
iend=ii2,
istart_with_padding=int(max(0, ii - padding_size)),
iend_with_padding=int(min(num_frames, ii2 + padding_size))
))
ii = ii2
return chunks
def _write_dat_one_chunk(i, rec_arg, chunks, rec_memmap, dtype, time_axis, return_scaled, verbose):
chunk = chunks[i]
if verbose:
print(f"Writing chunk {i + 1} / {len(chunks)}")
if isinstance(rec_arg, dict):
recording = load_extractor_from_dict(rec_arg)
else:
recording = rec_arg
start_frame = chunk['istart']
end_frame = chunk['iend']
traces = recording.get_traces(start_frame=start_frame, end_frame=end_frame, return_scaled=return_scaled)
if dtype is not None:
traces = traces.astype(dtype)
if time_axis == 0:
traces = traces.T
rec_memmap[start_frame:end_frame, :] = traces
else:
rec_memmap[:, start_frame:end_frame] = traces
================================================
FILE: spikeextractors/extractorlist.py
================================================
from .extractors.mdaextractors.mdaextractors import MdaRecordingExtractor, MdaSortingExtractor
from .extractors.mearecextractors.mearecextractors import MEArecRecordingExtractor, MEArecSortingExtractor
from .extractors.biocamrecordingextractor.biocamrecordingextractor import BiocamRecordingExtractor
from .extractors.exdirextractors.exdirextractors import ExdirRecordingExtractor, ExdirSortingExtractor
from .extractors.intanrecordingextractor.intanrecordingextractor import IntanRecordingExtractor
from .extractors.hdsortsortingextractor.hdsortsortingextractor import HDSortSortingExtractor
from .extractors.hs2sortingextractor.hs2sortingextractor import HS2SortingExtractor
from .extractors.klustaextractors.klustaextractors import KlustaSortingExtractor, KlustaRecordingExtractor
from .extractors.kilosortextractors.kilosortextractors import KiloSortSortingExtractor, KiloSortRecordingExtractor
from .extractors.numpyextractors.numpyextractors import NumpyRecordingExtractor, NumpySortingExtractor
from .extractors.nwbextractors.nwbextractors import NwbRecordingExtractor, NwbSortingExtractor
from .extractors.openephysextractors.openephysextractors import OpenEphysRecordingExtractor, \
OpenEphysSortingExtractor, OpenEphysNPIXRecordingExtractor
from .extractors.maxwellextractors import MaxOneRecordingExtractor, MaxOneSortingExtractor, MaxTwoRecordingExtractor, \
MaxTwoSortingExtractor
from .extractors.phyextractors.phyextractors import PhyRecordingExtractor, PhySortingExtractor
from .extractors.bindatrecordingextractor.bindatrecordingextractor import BinDatRecordingExtractor
from .extractors.spykingcircusextractors.spykingcircusextractors import SpykingCircusSortingExtractor, \
SpykingCircusRecordingExtractor
from .extractors.spikeglxrecordingextractor.spikeglxrecordingextractor import SpikeGLXRecordingExtractor
from .extractors.tridescloussortingextractor.tridescloussortingextractor import TridesclousSortingExtractor
from .extractors.npzsortingextractor.npzsortingextractor import NpzSortingExtractor
from .extractors.mcsh5recordingextractor.mcsh5recordingextractor import MCSH5RecordingExtractor
from .extractors.shybridextractors import SHYBRIDRecordingExtractor, SHYBRIDSortingExtractor
from .extractors.nixioextractors.nixioextractors import NIXIORecordingExtractor, NIXIOSortingExtractor
from .extractors.neoextractors import (AxonaRecordingExtractor, PlexonRecordingExtractor, PlexonSortingExtractor,
NeuralynxRecordingExtractor, NeuralynxSortingExtractor,
BlackrockRecordingExtractor, BlackrockSortingExtractor,
MCSRawRecordingExtractor, SpikeGadgetsRecordingExtractor)
from .extractors.neuroscopeextractors import NeuroscopeRecordingExtractor, NeuroscopeMultiRecordingTimeExtractor, \
NeuroscopeSortingExtractor, NeuroscopeMultiSortingExtractor
from .extractors.waveclussortingextractor import WaveClusSortingExtractor
from .extractors.yassextractors import YassSortingExtractor
from .extractors.combinatosortingextractor import CombinatoSortingExtractor
from .extractors.alfsortingextractor import ALFSortingExtractor
from .extractors.cedextractors import CEDRecordingExtractor
from .extractors.cellexplorersortingextractor import CellExplorerSortingExtractor
from .extractors.neuropixelsdatrecordingextractor import NeuropixelsDatRecordingExtractor
from .extractors.axonaunitrecordingextractor import AxonaUnitRecordingExtractor
recording_extractor_full_list = [
MdaRecordingExtractor,
MEArecRecordingExtractor,
BiocamRecordingExtractor,
ExdirRecordingExtractor,
OpenEphysRecordingExtractor,
OpenEphysNPIXRecordingExtractor,
IntanRecordingExtractor,
BinDatRecordingExtractor,
KlustaRecordingExtractor,
KiloSortRecordingExtractor,
SpykingCircusRecordingExtractor,
SpikeGLXRecordingExtractor,
PhyRecordingExtractor,
MaxOneRecordingExtractor,
MaxTwoRecordingExtractor,
MCSH5RecordingExtractor,
SHYBRIDRecordingExtractor,
NIXIORecordingExtractor,
NwbRecordingExtractor,
NeuroscopeRecordingExtractor,
NeuroscopeMultiRecordingTimeExtractor,
CEDRecordingExtractor,
NeuropixelsDatRecordingExtractor,
AxonaUnitRecordingExtractor,
# neo based
AxonaRecordingExtractor,
PlexonRecordingExtractor,
NeuralynxRecordingExtractor,
BlackrockRecordingExtractor,
MCSRawRecordingExtractor,
SpikeGadgetsRecordingExtractor,
]
recording_extractor_dict = {recording_class.extractor_name: recording_class
for recording_class in recording_extractor_full_list}
installed_recording_extractor_list = [rx for rx in recording_extractor_full_list if rx.installed]
sorting_extractor_full_list = [
MdaSortingExtractor,
MEArecSortingExtractor,
ExdirSortingExtractor,
HDSortSortingExtractor,
HS2SortingExtractor,
KlustaSortingExtractor,
KiloSortSortingExtractor,
OpenEphysSortingExtractor,
PhySortingExtractor,
SpykingCircusSortingExtractor,
TridesclousSortingExtractor,
MaxTwoSortingExtractor,
MaxOneSortingExtractor,
NpzSortingExtractor,
SHYBRIDSortingExtractor,
NIXIOSortingExtractor,
NeuroscopeSortingExtractor,
NeuroscopeMultiSortingExtractor,
NwbSortingExtractor,
WaveClusSortingExtractor,
YassSortingExtractor,
CombinatoSortingExtractor,
ALFSortingExtractor,
# neo based
PlexonSortingExtractor,
NeuralynxSortingExtractor,
BlackrockSortingExtractor,
CellExplorerSortingExtractor
]
installed_sorting_extractor_list = [sx for sx in sorting_extractor_full_list if sx.installed]
sorting_extractor_dict = {sorting_class.extractor_name: sorting_class for sorting_class in sorting_extractor_full_list}
writable_sorting_extractor_list = [sx for sx in installed_sorting_extractor_list if sx.is_writable]
writable_sorting_extractor_dict = {sorting_class.extractor_name: sorting_class
for sorting_class in writable_sorting_extractor_list}
================================================
FILE: spikeextractors/extractors/__init__.py
================================================
================================================
FILE: spikeextractors/extractors/alfsortingextractor/__init__.py
================================================
from .alfsortingextractor import ALFSortingExtractor
================================================
FILE: spikeextractors/extractors/alfsortingextractor/alfsortingextractor.py
================================================
from abc import ABC
from spikeextractors import SortingExtractor
from pathlib import Path
import numpy as np
try:
import pandas as pd
HAVE_PANDAS = True
except:
HAVE_PANDAS = False
class ALFSortingExtractor(SortingExtractor):
extractor_name = 'ALFSorting'
installed = HAVE_PANDAS # check at class level if installed or not
is_writable = True
mode = 'folder'
installation_mesg = "To use the ALFSortingExtractor run:\n\n pip install pandas\n\n"
def __init__(self, folder_path, sampling_frequency=30000):
assert self.installed, self.installation_mesg
SortingExtractor.__init__(self)
# check correct parent folder:
self.file_loc = Path(folder_path)
if 'probe' not in Path(self.file_loc).name:
raise ValueError('folder name should contain "probe", containing channels, clusters.* .npy datasets')
# load datasets as mmap into a dict:
self._required_alf_datasets = ['spikes.times', 'spikes.clusters']
self._found_alf_datasets = dict()
for alf_dataset_name in self.file_loc.iterdir():
if 'spikes' in alf_dataset_name.stem or 'clusters' in alf_dataset_name.stem:
if 'npy' in alf_dataset_name.suffix:
self._found_alf_datasets.update({alf_dataset_name.stem: self._load_npy(alf_dataset_name)})
elif 'metrics' in alf_dataset_name.stem:
self._found_alf_datasets.update({alf_dataset_name.stem: pd.read_csv(alf_dataset_name)})
# check existence of datasets:
if not any([i in self._found_alf_datasets for i in self._required_alf_datasets]):
raise Exception(f'could not find {self._required_alf_datasets} in folder')
# setting units properties:
self._total_units = 0
for alf_dataset_name, alf_dataset in self._found_alf_datasets.items():
if 'clusters' in alf_dataset_name:
if 'clusters.metrics' in alf_dataset_name:
for property_name, property_values in self._found_alf_datasets[alf_dataset_name].iteritems():
self.set_units_property(unit_ids=self.get_unit_ids(),
property_name=property_name,
values=property_values.tolist())
else:
self.set_units_property(unit_ids=self.get_unit_ids(),
property_name=alf_dataset_name.split('.')[1],
values=alf_dataset)
if self._total_units == 0:
self._total_units = alf_dataset.shape[0]
self._units_map = {i: j for i, j in zip(self.get_unit_ids(), list(range(self._total_units)))}
self._units_raster = []
self._sampling_frequency = sampling_frequency
self._kwargs = {'folder_path': str(Path(folder_path).absolute()), 'sampling_frequency': sampling_frequency}
def _load_npy(self, npy_path):
return np.load(npy_path, mmap_mode='r',allow_pickle=True)
def _get_clusters_spike_times(self, cluster_idx):
if len(self._units_raster) == 0:
spike_cluster_data = self._found_alf_datasets['spikes.clusters']
spike_times_data = self._found_alf_datasets['spikes.times']
df = pd.DataFrame({'sp_cluster': spike_cluster_data, 'sp_times': spike_times_data})
data = df.groupby(['sp_cluster'])['sp_times'].apply(np.array).reset_index(name='sp_times_group')
self._max_time = 0
self._units_raster = [None]*self._total_units
for index, sp_times_list in data.values:
self._units_raster[index] = sp_times_list
max_time = max(sp_times_list)
if max_time > self._max_time:
self._max_time = max_time
return self._units_raster[cluster_idx]
def get_unit_ids(self):
if 'clusters.metrics' in self._found_alf_datasets and \
self._found_alf_datasets['clusters.metrics'].get('cluster_id') is not None:
unit_ids = self._found_alf_datasets['clusters.metrics'].get('cluster_id').tolist()
else:
unit_ids = list(range(self._total_units))
return unit_ids
def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=None):
"""Code to extract spike frames from the specified unit.
It will return spike frames from within three ranges:
[start_frame, t_start+1, ..., end_frame-1]
[start_frame, start_frame+1, ..., final_unit_spike_frame - 1]
[0, 1, ..., end_frame-1]
[0, 1, ..., final_unit_spike_frame - 1]
if both start_frame and end_frame are given, if only start_frame is
given, if only end_frame is given, or if neither start_frame or end_frame
are given, respectively. Spike frames are returned in the form of an
array_like of spike frames. In this implementation, start_frame is inclusive
and end_frame is exclusive conforming to numpy standards.
"""
unit_idx = self._units_map.get(unit_id)
if unit_idx is None:
raise ValueError(f'enter one of unit_id={self.get_unit_ids()}')
cluster_sp_times = self._get_clusters_spike_times(unit_idx)
if cluster_sp_times is None:
return np.array([])
max_frame = np.ceil(cluster_sp_times[-1]*self.get_sampling_frequency()).astype('int64')
min_frame = np.floor(cluster_sp_times[0]*self.get_sampling_frequency()).astype('int64')
start_frame = min_frame if start_frame is None or start_frame < min_frame else start_frame
end_frame = max_frame if end_frame is None or end_frame > max_frame else end_frame
if start_frame > max_frame or end_frame < min_frame:
raise ValueError(f'Use start_frame to end_frame between {min_frame} and {max_frame}')
cluster_sp_frames = (cluster_sp_times * self.get_sampling_frequency()).astype('int64')
frame_idx = np.where((cluster_sp_frames >= start_frame) &
(cluster_sp_frames < end_frame))
return cluster_sp_frames[frame_idx]
@staticmethod
def write_sorting(sorting, save_path):
"""
This is an example of a function that is not abstract so it is optional if you want to override it. It allows other
SortingExtractors to use your new SortingExtractor to convert their sorted data into your
sorting file format.
"""
assert HAVE_PANDAS, ALFSortingExtractor.installation_mesg
# write cluster properties as clusters.<property_name>.npy
save_path = Path(save_path)
csv_property_names = ['cluster_id', 'cluster_id.1', 'num_spikes', 'firing_rate',
'presence_ratio', 'presence_ratio_std', 'frac_isi_viol',
'contamination_est', 'contamination_est2', 'missed_spikes_est',
'cum_amp_drift', 'max_amp_drift', 'cum_depth_drift', 'max_depth_drift',
'ks2_contamination_pct', 'ks2_label','amplitude_cutoff', 'amplitude_std',
'epoch_name', 'isi_viol']
clusters_metrics_df = pd.DataFrame()
for property_name in sorting.get_unit_property_names(0):
data = sorting.get_units_property(property_name=property_name)
if property_name not in csv_property_names:
np.save(save_path/f'clusters.{property_name}', data)
else:
clusters_metrics_df[property_name] = data
clusters_metrics_df.to_csv(save_path/'clusters.metrics.csv')
# save spikes.times, spikes.clusters
clusters_number = []
unit_spike_times = []
for unit_no, unit_id in enumerate(sorting.get_unit_ids()):
unit_spike_train = sorting.get_unit_spike_train(unit_id=unit_id)
if unit_spike_train is not None:
unit_spike_times.extend(np.array(unit_spike_train)/sorting.get_sampling_frequency())
clusters_number.extend([unit_no]*len(unit_spike_train))
unit_spike_train = np.array(unit_spike_times)
clusters_number = np.array(clusters_number)
spike_times_ids = np.argsort(unit_spike_train)
spike_times = unit_spike_train[spike_times_ids]
spike_clusters = clusters_number[spike_times_ids]
np.save(save_path/'spikes.times', spike_times)
np.save(save_path/'spikes.clusters', spike_clusters)
================================================
FILE: spikeextractors/extractors/axonaunitrecordingextractor/__init__.py
================================================
from .axonaunitrecordingextractor import AxonaUnitRecordingExtractor
================================================
FILE: spikeextractors/extractors/axonaunitrecordingextractor/axonaunitrecordingextractor.py
================================================
from spikeextractors.extraction_tools import check_get_traces_args
from spikeextractors.extractors.neoextractors.neobaseextractor import (
_NeoBaseExtractor, NeoBaseRecordingExtractor)
from spikeextractors import RecordingExtractor
from pathlib import Path
import numpy as np
from typing import Union
import warnings
PathType = Union[Path, str]
try:
import neo
from neo.rawio.baserawio import _signal_channel_dtype, _signal_stream_dtype
HAVE_NEO = True
except ImportError:
HAVE_NEO = False
class AxonaUnitRecordingExtractor(NeoBaseRecordingExtractor, RecordingExtractor, _NeoBaseExtractor):
"""
Instantiates a RecordingExtractor from an Axona Unit mode file.
Since the unit mode format only saves waveform cutouts, the get_traces
function fills in the rest of the recording with Gaussian uncorrelated
noise
Parameters
----------
noise_std: float
Standard deviation of the Gaussian background noise (default 3)
"""
extractor_name = 'AxonaUnitRecording'
mode = 'file'
NeoRawIOClass = 'AxonaRawIO'
def __init__(self, noise_std: float = 3, block_index=None, seg_index=None, **kargs):
RecordingExtractor.__init__(self)
_NeoBaseExtractor.__init__(self, block_index=block_index, seg_index=seg_index, **kargs)
# Enforce 1 signal stream (there are 0 raw streams), we will create 1 from waveforms
signal_streams = self.neo_reader._get_signal_streams_header()
signal_channels = self.neo_reader._get_signal_chan_header()
self.neo_reader.header['signal_streams'] = np.array(signal_streams,
dtype=_signal_stream_dtype)
self.neo_reader.header['signal_channels'] = np.array(signal_channels,
dtype=_signal_channel_dtype)
if hasattr(self.neo_reader, 'get_group_signal_channel_indexes'):
# Neo >= 0.9.0
channel_indexes_list = self.neo_reader.get_group_signal_channel_indexes()
num_streams = len(channel_indexes_list)
assert num_streams <= 1, 'This file have several channel groups spikeextractors support only one groups'
self.after_v10 = False
elif hasattr(self.neo_reader, 'get_group_channel_indexes'):
# Neo < 0.9.0
channel_indexes_list = self.neo_reader.get_group_channel_indexes()
num_streams = len(channel_indexes_list)
self.after_v10 = False
elif hasattr(self.neo_reader, 'signal_streams_count'):
# Neo >= 0.10.0 (not release yet in march 2021)
num_streams = self.neo_reader.signal_streams_count()
self.after_v10 = True
else:
raise ValueError('Strange neo version. Please upgrade your neo package: pip install --upgrade neo')
assert num_streams <= 1, 'This file have several signal streams spikeextractors support only one streams' \
'Maybe you can use option to select only one stream'
# spikeextractor for units to be uV implicitly
# check that units are V, mV or uV
units = self.neo_reader.header['signal_channels']['units']
assert np.all(np.isin(units, ['V', 'mV', 'uV'])), 'Signal units no Volt compatible'
self.additional_gain = np.ones(units.size, dtype='float')
self.additional_gain[units == 'V'] = 1e6
self.additional_gain[units == 'mV'] = 1e3
self.additional_gain[units == 'uV'] = 1.
self.additional_gain = self.additional_gain.reshape(1, -1)
# Add channels properties
header_channels = self.neo_reader.header['signal_channels'][slice(None)]
self._neo_chan_ids = self.neo_reader.header['signal_channels']['id']
# In neo there is not guarantee that channel ids are unique.
# for instance Blacrock can have several times the same chan_id
# different sampling rate
# so check it
assert np.unique(self._neo_chan_ids).size == self._neo_chan_ids.size, 'In this format channel ids are not ' \
'unique! Incompatible with SpikeInterface'
try:
channel_ids = [int(ch) for ch in self._neo_chan_ids]
except Exception as e:
warnings.warn("Could not parse channel ids to int: using linear channel map")
channel_ids = list(np.arange(len(self._neo_chan_ids)))
self._channel_ids = channel_ids
gains = header_channels['gain'] * self.additional_gain[0]
self.set_channel_gains(gains=gains, channel_ids=self._channel_ids)
names = header_channels['name']
for i, ind in enumerate(self._channel_ids):
self.set_channel_property(channel_id=ind, property_name='name', value=names[i])
self._noise_std = noise_std
# Read channel groups by tetrode IDs
self.set_channel_groups(groups=[
tetrode_id - 1 for tetrode_id in self.neo_reader.get_active_tetrode() for _ in range(4)])
header_channels = self.neo_reader.header['signal_channels'][slice(None)]
names = header_channels['name']
channel_ids = self.get_channel_ids()
for i, ind in enumerate(channel_ids):
self.set_channel_property(channel_id=ind, property_name='name', value=names[i])
# Set channel gains for int8 .X Unit data
gains = self.neo_reader._get_channel_gain(bytes_per_sample=1)[0:len(channel_ids)]
self.set_channel_gains(gains, channel_ids=channel_ids)
@check_get_traces_args
def get_traces(self, channel_ids=None, start_frame=None, end_frame=None, return_scaled=True):
timebase_sr = int(self.neo_reader.file_parameters['unit']['timebase'].split(' ')[0])
samples_pre = int(self.neo_reader.file_parameters['set']['file_header']['pretrigSamps'])
samples_post = int(self.neo_reader.file_parameters['set']['file_header']['spikeLockout'])
sampling_rate = self.get_sampling_frequency()
tcmap = self._get_tetrode_channel_table(channel_ids)
traces = self._noise_std * np.random.randn(len(channel_ids), end_frame - start_frame)
if return_scaled:
traces = traces.astype(np.float32)
else:
traces = traces.astype(np.int8)
# Loop through tetrodes and include requested channels in traces
itrc = 0
for tetrode_id in np.unique(tcmap[:, 0]):
channels_oi = tcmap[tcmap[:, 0] == tetrode_id, 2]
waveforms = self.neo_reader._get_spike_raw_waveforms(
block_index=0, seg_index=0,
unit_index=tetrode_id - 1, # Tetrodes IDs are 1-indexed
t_start=start_frame / sampling_rate,
t_stop=end_frame / sampling_rate
)
waveforms = waveforms[:, channels_oi, :]
nch = len(channels_oi)
spike_train = self.neo_reader._get_spike_timestamps(
block_index=0, seg_index=0,
unit_index=tetrode_id - 1,
t_start=start_frame / sampling_rate,
t_stop=end_frame / sampling_rate
)
# Fill waveforms into traces timestamp by timestamp
for t, wf in zip(spike_train, waveforms):
t = int(t // (timebase_sr / sampling_rate)) # timestamps are sampled at higher frequency
t = t - start_frame
if (t - samples_pre < 0) and (t + samples_post > traces.shape[1]):
traces[itrc:itrc + nch, :] = wf[:, samples_pre - t:traces.shape[1] - (t - samples_pre)]
elif t - samples_pre < 0:
traces[itrc:itrc + nch, :t + samples_post] = wf[:, samples_pre - t:]
elif t + samples_post > traces.shape[1]:
traces[itrc:itrc + nch, t - samples_pre:] = wf[:, :traces.shape[1] - (t - samples_pre)]
else:
traces[itrc:itrc + nch, t - samples_pre:t + samples_post] = wf
itrc += nch
return traces
def get_num_frames(self):
n = int(self.neo_reader.segment_t_stop(block_index=0, seg_index=0) * self.get_sampling_frequency())
if self.get_sampling_frequency() == 24000:
n = n // 2
return n
def get_sampling_frequency(self):
return int(self.neo_reader.header['spike_channels'][0][-1])
def get_channel_ids(self):
return self._channel_ids
def _get_tetrode_channel_table(self, channel_ids):
'''Create auxiliary np.array with the following columns:
Tetrode ID, Channel ID, Channel ID within tetrode
This is useful in `get_traces()`
Parameters
----------
channel_ids : list
List of channel ids to include in table
Returns
-------
np.array
Rows = channels,
columns = TetrodeID, ChannelID, ChannelID within Tetrode
'''
active_tetrodes = self.neo_reader.get_active_tetrode()
tcmap = np.zeros((len(active_tetrodes) * 4, 3), dtype=int)
row_id = 0
for tetrode_id in [int(s[0].split(' ')[1]) for s in self.neo_reader.header['spike_channels']]:
all_channel_ids = self.neo_reader._get_channel_from_tetrode(tetrode_id)
for i in range(4):
tcmap[row_id, 0] = int(tetrode_id)
tcmap[row_id, 1] = int(all_channel_ids[i])
tcmap[row_id, 2] = int(i)
row_id += 1
del_idx = [False if i in channel_ids else True for i in tcmap[:, 1]]
return np.delete(tcmap, del_idx, axis=0)
================================================
FILE: spikeextractors/extractors/bindatrecordingextractor/__init__.py
================================================
from .bindatrecordingextractor import BinDatRecordingExtractor
================================================
FILE: spikeextractors/extractors/bindatrecordingextractor/bindatrecordingextractor.py
================================================
import shutil
import numpy as np
from pathlib import Path
from typing import Union, Optional
from spikeextractors import RecordingExtractor
from spikeextractors.extraction_tools import read_binary, write_to_binary_dat_format, check_get_traces_args
PathType = Union[str, Path]
DtypeType = Union[str, np.dtype]
ArrayType = Union[list, np.ndarray]
OptionalDtypeType = Optional[DtypeType]
OptionalArrayType = Optional[Union[np.ndarray, list]]
class BinDatRecordingExtractor(RecordingExtractor):
"""
RecordingExtractor for a binary format
Parameters
----------
file_path: str or Path
Path to the binary file
sampling_frequency: float
The sampling frequncy
numchan: int
Number of channels
dtype: str or dtype
The dtype of the binary file
time_axis: int
The axis of the time dimension (default 0: F order)
recording_channels: list (optional)
A list of channel ids
geom: array-like (optional)
A list or array with channel locations
file_offset: int (optional)
Number of bytes in the file to offset by during memmap instantiation.
gain: float or array-like (optional)
The gain to apply to the traces
channel_offset: float or array-like
The offset to apply to the traces
is_filtered: bool
If True, the recording is assumed to be filtered
"""
extractor_name = 'BinDatRecording'
has_default_locations = False
has_unscaled = False
installed = True
is_writable = True
mode = "file"
installation_mesg = ""
def __init__(self, file_path: PathType, sampling_frequency: float, numchan: int, dtype: DtypeType,
time_axis: int = 0, recording_channels: Optional[list] = None, geom: Optional[ArrayType] = None,
file_offset: Optional[float] = 0,
gain: Optional[Union[float, ArrayType]] = None,
channel_offset: Optional[Union[float, ArrayType]] = None,
is_filtered: Optional[bool] = None):
RecordingExtractor.__init__(self)
self._datfile = Path(file_path)
self._time_axis = time_axis
self._dtype = np.dtype(dtype).name
self._sampling_frequency = float(sampling_frequency)
self._numchan = numchan
self._geom = geom
self._timeseries = read_binary(self._datfile, numchan, dtype, time_axis, file_offset)
if is_filtered is not None:
self.is_filtered = is_filtered
else:
self.is_filtered = False
if recording_channels is not None:
assert len(recording_channels) <= self._timeseries.shape[0], \
'Provided recording channels have the wrong length'
self._channels = recording_channels
else:
self._channels = list(range(self._timeseries.shape[0]))
if len(self._channels) == self._timeseries.shape[0]:
self._complete_channels = True
else:
assert max(self._channels) < self._timeseries.shape[0], "Channel ids exceed the number of " \
"available channels"
self._complete_channels = False
if geom is not None:
self.set_channel_locations(self._geom)
self.has_default_locations = True
if 'numpy' in str(dtype):
dtype_str = str(dtype).replace("<class '", "").replace("'>", "")
dtype_str = dtype_str.split('.')[1]
else:
dtype_str = str(dtype)
if gain is not None:
self.set_channel_gains(channel_ids=self.get_channel_ids(), gains=gain)
self.has_unscaled = True
if channel_offset is not None:
self.set_channel_offsets(channel_offset)
self._kwargs = {'file_path': str(Path(file_path).absolute()), 'sampling_frequency': sampling_frequency,
'numchan': numchan, 'dtype': dtype_str, 'recording_channels': recording_channels,
'time_axis': time_axis, 'geom': geom, 'file_offset': file_offset, 'gain': gain,
'is_filtered': is_filtered}
def get_channel_ids(self):
return self._channels
def get_num_frames(self):
return self._timeseries.shape[1]
def get_sampling_frequency(self):
return self._sampling_frequency
@check_get_traces_args
def get_traces(self, channel_ids=None, start_frame=None, end_frame=None, return_scaled=True):
if self._complete_channels:
if np.array_equal(channel_ids, self.get_channel_ids()):
traces = self._timeseries[:, start_frame:end_frame]
else:
channel_idxs = np.array([self.get_channel_ids().index(ch) for ch in channel_ids])
if np.all(np.diff(channel_idxs) == 1):
traces = self._timeseries[channel_idxs[0]:channel_idxs[0]+len(channel_idxs),
start_frame:end_frame]
else:
# This block of the execution will return the data as an array, not a memmap
traces = self._timeseries[channel_idxs, start_frame:end_frame]
else:
# in this case channel ids are actually indexes
traces = self._timeseries[channel_ids, start_frame:end_frame]
return traces
@staticmethod
def write_recording(
recording: RecordingExtractor,
save_path: PathType,
time_axis: int = 0,
dtype: OptionalDtypeType = None,
**write_binary_kwargs
):
"""
Save the traces of a recording extractor in binary .dat format.
Parameters
----------
recording : RecordingExtractor
The recording extractor object to be saved in .dat format.
save_path : str
The path to the file.
time_axis : int, optional
If 0 then traces are transposed to ensure (nb_sample, nb_channel) in the file.
If 1, the traces shape (nb_channel, nb_sample) is kept in the file.
dtype : dtype
Type of the saved data. Default float32.
**write_binary_kwargs: keyword arguments for write_to_binary_dat_format() function
"""
write_to_binary_dat_format(recording, save_path, time_axis=time_axis, dtype=dtype,
**write_binary_kwargs)
================================================
FILE: spikeextractors/extractors/biocamrecordingextractor/__init__.py
================================================
from .biocamrecordingextractor import BiocamRecordingExtractor
================================================
FILE: spikeextractors/extractors/biocamrecordingextractor/biocamrecordingextractor.py
================================================
from spikeextractors import RecordingExtractor
from spikeextractors.extraction_tools import check_get_traces_args
import numpy as np
from pathlib import Path
import ctypes
try:
import h5py
HAVE_BIOCAM = True
except ImportError:
HAVE_BIOCAM = False
class BiocamRecordingExtractor(RecordingExtractor):
extractor_name = 'BiocamRecording'
has_default_locations = True
has_unscaled = False
installed = HAVE_BIOCAM # check at class level if installed or not
is_writable = True
mode = 'file'
installation_mesg = "To use the BiocamRecordingExtractor install h5py: \n\n pip install h5py\n\n" # error message when not installed
def __init__(self, file_path, verbose=False, mea_pitch=42):
assert self.installed, self.installation_mesg
self._mea_pitch = mea_pitch
self._recording_file = file_path
self._rf, self._nFrames, self._samplingRate, self._nRecCh, self._chIndices, \
self._file_format, self._signalInv, self._positions, self._read_function = openBiocamFile(
self._recording_file, self._mea_pitch, verbose)
RecordingExtractor.__init__(self)
self.set_channel_locations(self._positions)
self._kwargs = {'file_path': str(Path(file_path).absolute()), 'mea_pitch': mea_pitch,
'verbose': verbose}
def __del__(self):
self._rf.close()
def get_channel_ids(self):
return list(range(self._nRecCh))
def get_num_frames(self):
return self._nFrames
def get_sampling_frequency(self):
return self._samplingRate
@check_get_traces_args
def get_traces(self, channel_ids=None, start_frame=None, end_frame=None, return_scaled=True):
data = self._read_function(self._rf, start_frame, end_frame, self.get_num_channels())
# transform to slice if possible
if sorted(channel_ids) == channel_ids and np.all(np.diff(channel_ids) == 1):
channel_ids = slice(channel_ids[0], channel_ids[0]+len(channel_ids))
return data[:, channel_ids].T
@staticmethod
def write_recording(recording, save_path):
# Convert to uV:
# AnalogValue = MVOffset + DigitalValue * ADCCountsToMV
# Where ADCCountsToMV is defined as:
# ADCCountsToMV = SignalInversion * ((MaxVolt - MinVolt) / 2^BitDepth)
# And MVOffset as:
# MVOffset = SignalInversion * MinVolt
# conversion back
# DigitalValue = (AnalogValue - MVOffset)/ADCCountsToMV
# we center at 2048
assert HAVE_BIOCAM, BiocamRecordingExtractor.installation_mesg
M = recording.get_num_channels()
N = recording.get_num_frames()
rf = h5py.File(save_path, 'w')
g = rf.create_group('3BData')
dr = rf.create_dataset('3BData/Raw', (M * N,), dtype=int)
dt = 50000
for i in range(N // dt):
dr[M * i * dt:M * (i + 1) * dt] = recording.get_traces(range(M), i * dt, (i + 1) * dt).T.flatten()
dr[M * (N // dt) * dt:] = recording.get_traces(range(M), (N // dt) * dt, N).T.flatten()
g.attrs['Version'] = 101
rf.create_dataset('3BRecInfo/3BRecVars/MinVolt', data=[0])
rf.create_dataset('3BRecInfo/3BRecVars/MaxVolt', data=[1])
rf.create_dataset('3BRecInfo/3BRecVars/NRecFrames', data=[N])
rf.create_dataset('3BRecInfo/3BRecVars/SamplingRate', data=[recording.get_sampling_frequency()])
rf.create_dataset('3BRecInfo/3BRecVars/SignalInversion', data=[1])
rf.create_dataset('3BRecInfo/3BMeaChip/NCols', data=[M])
r = recording.get_channel_locations()[:, 0]
c = recording.get_channel_locations()[:, 1]
d = np.ndarray((1, len(r)), dtype=[('Row', '<i2'), ('Col', '<i2')])
d['Row'] = r
d['Col'] = c
rf.create_dataset('3BRecInfo/3BMeaStreams/Raw/Chs', data=d)
rf.close()
def openBiocamFile(filename, mea_pitch, verbose=False):
"""Open a Biocam hdf5 file, read and return the recording info, pick te correct method to access raw data, and return this to the caller."""
rf = h5py.File(filename, 'r')
# Read recording variables
recVars = rf.require_group('3BRecInfo/3BRecVars/')
# bitDepth = recVars['BitDepth'].value[0]
# maxV = recVars['MaxVolt'].value[0]
# minV = recVars['MinVolt'].value[0]
nFrames = recVars['NRecFrames'][0]
samplingRate = recVars['SamplingRate'][0]
signalInv = recVars['SignalInversion'][0]
# Read chip variables
chipVars = rf.require_group('3BRecInfo/3BMeaChip/')
nCols = chipVars['NCols'][0]
# Get the actual number of channels used in the recording
file_format = rf['3BData'].attrs.get('Version')
if file_format == 100:
nRecCh = len(rf['3BData/Raw'][0])
elif (file_format == 101) or (file_format == 102):
nRecCh = int(1. * rf['3BData/Raw'].shape[0] / nFrames)
else:
raise Exception('Unknown data file format.')
if verbose:
print('# 3Brain data format:', file_format, 'signal inversion', signalInv)
print('# signal range: ', recVars['MinVolt'][0], '- ', recVars['MaxVolt'][0])
print('# channels: ', nRecCh)
print('# frames: ', nFrames)
print('# sampling rate: ', samplingRate)
# get channel locations
r = (rf['3BRecInfo/3BMeaStreams/Raw/Chs'][()]['Row'] - 1) * mea_pitch
c = (rf['3BRecInfo/3BMeaStreams/Raw/Chs'][()]['Col'] - 1) * mea_pitch
rawIndices = np.vstack((r, c)).T
# assign channel numbers
chIndices = np.array([(x - 1) + (y - 1) * nCols for (y, x) in rawIndices])
# determine correct function to read data
if verbose:
print("# Signal inversion is " + str(signalInv) + ".")
print("# If your spike sorting results look wrong, invert the signal.")
if (file_format == 100) & (signalInv == 1):
read_function = readHDF5t_100
elif (file_format == 100) & (signalInv == -1):
read_function = readHDF5t_100_i
if ((file_format == 101) | (file_format == 102)) & (signalInv == 1):
read_function = readHDF5t_101
elif ((file_format == 101) | (file_format == 102)) & (signalInv == -1):
read_function = readHDF5t_101_i
else:
raise RuntimeError("File format unknown.")
return rf, nFrames, samplingRate, nRecCh, chIndices, file_format, signalInv, rawIndices, read_function
def readHDF5t_100(rf, t0, t1, nch):
if t0 <= t1:
return rf['3BData/Raw'][t0:t1]
else: # Reversed read
raise Exception('Reading backwards? Not sure about this.')
return rf['3BData/Raw'][t1:t0]
def readHDF5t_100_i(rf, t0, t1, nch):
if t0 <= t1:
return 4096 - rf['3BData/Raw'][t0:t1]
else: # Reversed read
raise Exception('Reading backwards? Not sure about this.')
return 4096 - rf['3BData/Raw'][t1:t0]
def readHDF5t_101(rf, t0, t1, nch):
if t0 <= t1:
return rf['3BData/Raw'][nch * t0:nch * t1].reshape((t1 - t0, nch), order='C')
else: # Reversed read
raise Exception('Reading backwards? Not sure about this.')
return rf['3BData/Raw'][nch * t1:nch * t0].reshape((t1 - t0, nch), order='C')
def readHDF5t_101_i(rf, t0, t1, nch):
if t0 <= t1:
return 4096 - rf['3BData/Raw'][nch * t0:nch * t1].reshape((t1 - t0, nch), order='C')
else: # Reversed read
raise Exception('Reading backwards? Not sure about this.')
return 4096 - rf['3BData/Raw'][nch * t1:nch * t0].reshape((t1 - t0, nch), order='C')
================================================
FILE: spikeextractors/extractors/cedextractors/__init__.py
================================================
from .cedrecordingextractor import CEDRecordingExtractor
================================================
FILE: spikeextractors/extractors/cedextractors/cedrecordingextractor.py
================================================
from spikeextractors import RecordingExtractor
from .utils import get_channel_info, get_channel_data
from spikeextractors.extraction_tools import check_get_traces_args
import numpy as np
from pathlib import Path
from typing import Union
from copy import deepcopy
try:
from sonpy import lib as sp
HAVE_SONPY = True
except ImportError:
HAVE_SONPY = False
PathType = Union[str, Path, None]
DtypeType = Union[str, np.dtype, None]
class CEDRecordingExtractor(RecordingExtractor):
"""
Extracts electrophysiology recordings from .smrx files.
The recording extractor always returns channel IDs starting from 0.
The recording data will always be returned in the shape of (num_channels,num_frames).
Parameters
----------
file_path: str
Path to the .smrx file to be extracted
smrx_channel_ids: list of int
List with indexes of valid smrx channels. Does not match necessarily
with extractor id.
"""
extractor_name = 'CEDRecording'
installed = HAVE_SONPY # check at class level if installed or not
is_writable = False
has_default_locations = False
has_unscaled = False
mode = 'file'
installation_mesg = "To use the CED extractor, install sonpy: \n\n pip install sonpy\n\n" # error message when not installed
def __init__(self, file_path: PathType, smrx_channel_ids: list):
assert self.installed, self.installation_mesg
file_path = Path(file_path)
assert file_path.is_file() and file_path.suffix == '.smrx', 'file_path must lead to a .smrx file!'
assert len(smrx_channel_ids) > 0, "'smrx_channel_ids' cannot be an empty list!"
super().__init__()
# Open smrx file
self._recording_file_path = file_path
self._recording_file = sp.SonFile(sName=str(file_path), bReadOnly=True)
if self._recording_file.GetOpenError() != 0:
raise ValueError(f'Error opening file:', sp.GetErrorString(self._recording_file.GetOpenError()))
# Map Recording channel_id to smrx index / test for invalid indexes /
# get channel info / set channel gains
self._channelid_to_smrxind = dict()
self._channel_smrxinfo = dict()
self._channel_names = []
gains = []
for i, ind in enumerate(smrx_channel_ids):
if self._recording_file.ChannelType(ind) == sp.DataType.Off:
raise ValueError(f'Channel {ind} is type Off and cannot be used')
self._channelid_to_smrxind[i] = ind
self._channel_smrxinfo[i] = get_channel_info(
f=self._recording_file,
smrx_ch_ind=ind
)
# Set channel gains: http://ced.co.uk/img/Spike10.pdf
# from 16-bit encoded int / to ADC +-5V input / to measured Volts
gain = self._channel_smrxinfo[i]['scale'] / 6553.6
gain *= 1000 # mV --> uV
gains.append(gain)
self._channel_names.append(self._channel_smrxinfo[i]['title'])
# Set gains
self.set_channel_gains(gains=gains)
self.has_unscaled = True
rate0 = self._channel_smrxinfo[0]['rate']
for chan, info in self._channel_smrxinfo.items():
assert info['rate'] == rate0, "Inconsistency between 'sampling_frequency' of different channels. The " \
"extractor only supports channels with the same 'rate'"
# Set self._times
times = (self._channel_smrxinfo[0]['frame_offset'] + np.arange(self.get_num_frames())) / self.get_sampling_frequency()
self.set_times(times=times)
self._kwargs = {'file_path': str(Path(file_path).absolute()),
'smrx_channel_ids': smrx_channel_ids}
@property
def channel_names(self):
return deepcopy(self._channel_names)
@check_get_traces_args
def get_traces(self, channel_ids=None, start_frame=None, end_frame=None, return_scaled=True):
"""This function extracts and returns a trace from the recorded data from the
given channels ids and the given start and end frame. It will return
traces from within three ranges:
[start_frame, start_frame+1, ..., end_frame-1]
[start_frame, start_frame+1, ..., final_recording_frame - 1]
[0, 1, ..., end_frame-1]
[0, 1, ..., final_recording_frame - 1]
if both start_frame and end_frame are given, if only start_frame is
given, if only end_frame is given, or if neither start_frame or end_frame
are given, respectively. Traces are returned in a 2D array that
contains all of the traces from each channel with dimensions
(num_channels x num_frames). In this implementation, start_frame is inclusive
and end_frame is exclusive conforming to numpy standards.
Parameters
----------
start_frame: int
The starting frame of the trace to be returned (inclusive)
end_frame: int
The ending frame of the trace to be returned (exclusive)
channel_ids: array_like
A list or 1D array of channel ids (ints) from which each trace will be
extracted
return_scaled: bool
If True, traces are returned after scaling (using gain/offset). If False, the traces are returned as integers
Returns
----------
traces: numpy.ndarray
A 2D array that contains all of the traces from each channel.
Dimensions are: (num_channels x num_frames)
"""
recordings = np.vstack(
[get_channel_data(
f=self._recording_file,
smrx_ch_ind=self._channelid_to_smrxind[i],
start_frame=start_frame,
end_frame=end_frame
) for i in channel_ids]
)
return recordings
def get_num_frames(self):
"""This function returns the number of frames in the recording
Returns
-------
num_frames: int
Number of frames in the recording (duration of recording)
"""
return 1 + int(self._channel_smrxinfo[0]['max_time'] / self._channel_smrxinfo[0]['divide'] - self._channel_smrxinfo[0]['frame_offset'])
def get_sampling_frequency(self):
"""This function returns the sampling frequency in units of Hz.
Returns
-------
fs: float
Sampling frequency of the recordings in Hz
"""
return self._channel_smrxinfo[0]['rate']
def get_channel_ids(self):
"""Returns the list of channel ids. If not specified, the range from 0 to num_channels - 1 is returned.
Returns
-------
channel_ids: list
Channel list
"""
return list(self._channelid_to_smrxind.keys())
@staticmethod
def get_all_channels_info(file_path):
"""
Extract info from all channels in the smrx file. Returns a dictionary with
valid smrx channel indexes as keys and the respective channel information as
value.
Parameters:
-----------
f: str
Path to .smrx file
"""
f = sp.SonFile(sName=str(file_path), bReadOnly=True)
n_channels = f.MaxChannels()
return {
i: get_channel_info(f, i) for i in range(n_channels)
if f.ChannelType(i) != sp.DataType.Off
}
================================================
FILE: spikeextractors/extractors/cedextractors/utils.py
================================================
import numpy as np
try:
from sonpy import lib as sp
# Data storage and function finder
DataReadFunctions = {
sp.DataType.Adc: sp.SonFile.ReadInts,
sp.DataType.EventFall: sp.SonFile.ReadEvents,
sp.DataType.EventRise: sp.SonFile.ReadEvents,
sp.DataType.EventBoth: sp.SonFile.ReadEvents,
sp.DataType.Marker: sp.SonFile.ReadMarkers,
sp.DataType.AdcMark: sp.SonFile.ReadWaveMarks,
sp.DataType.RealMark: sp.SonFile.ReadRealMarks,
sp.DataType.TextMark: sp.SonFile.ReadTextMarks,
sp.DataType.RealWave: sp.SonFile.ReadFloats
}
except:
pass
# Get the saved time and date
# f.GetTimeDate()
def get_channel_info(f, smrx_ch_ind):
"""
Extract info from smrx files
Parameters:
-----------
f: str
SonFile object.
smrx_ch_ind: int
Index of smrx channel. Does not match necessarily with extractor id.
"""
nMax = 1 + int(f.ChannelMaxTime(smrx_ch_ind) / f.ChannelDivide(smrx_ch_ind))
frame_offset = f.FirstTime(chan=smrx_ch_ind, tFrom=0, tUpto=nMax) / f.ChannelDivide(smrx_ch_ind)
ch_info = {
'type': f.ChannelType(smrx_ch_ind), # Get the channel kind
'ch_number': f.PhysicalChannel(smrx_ch_ind), # Get the physical channel number associated with this channel
'title': f.GetChannelTitle(smrx_ch_ind), # Get the channel title
'ideal_rate': f.GetIdealRate(smrx_ch_ind), # Get the requested channel ideal rate
'rate': 1 / (f.GetTimeBase() * f.ChannelDivide(smrx_ch_ind)), # Get the requested channel real rate
'max_time': f.ChannelMaxTime(smrx_ch_ind), # Get the time of the last item in the channel (in clock ticks)
'divide': f.ChannelDivide(smrx_ch_ind), # Get the waveform sample interval in file clock ticks
'time_base': f.GetTimeBase(), # Get how many seconds there are per clock tick
'frame_offset': frame_offset, # Get frame offset
'scale': f.GetChannelScale(smrx_ch_ind), # Get the channel scale
'offset': f.GetChannelOffset(smrx_ch_ind), # Get the channel offset
'unit': f.GetChannelUnits(smrx_ch_ind), # Get the channel units
'y_range': f.GetChannelYRange(smrx_ch_ind), # Get a suggested Y range for the channel
'comment': f.GetChannelComment(smrx_ch_ind), # Get the comment associated with a channel
'size_bytes:': f.ChannelBytes(smrx_ch_ind), # Get an estimate of the data bytes stored for the channel
}
return ch_info
def get_channel_data(f, smrx_ch_ind, start_frame=0, end_frame=None):
"""
Extract info from smrx files
Parameters:
-----------
f: str
SonFile object.
smrx_ch_ind: int
Index of smrx channel. Does not match necessarily with extractor id.
start_frame: int
The starting frame of the trace to be returned (inclusive).
end_frame: int
The ending frame of the trace to be returned (exclusive).
"""
if end_frame is None:
end_frame = 1 + int(f.ChannelMaxTime(smrx_ch_ind) / f.ChannelDivide(smrx_ch_ind))
nMax = 1 + int(f.ChannelMaxTime(smrx_ch_ind) / f.ChannelDivide(smrx_ch_ind))
frame_offset = int(f.FirstTime(chan=smrx_ch_ind, tFrom=0, tUpto=nMax) / f.ChannelDivide(smrx_ch_ind))
start_frame += frame_offset
end_frame += frame_offset
data = DataReadFunctions[f.ChannelType(smrx_ch_ind)](
self=f,
chan=smrx_ch_ind,
nMax=nMax,
tFrom=int(start_frame * f.ChannelDivide(smrx_ch_ind)),
tUpto=int(end_frame * f.ChannelDivide(smrx_ch_ind))
)
return np.array(data)
================================================
FILE: spikeextractors/extractors/cellexplorersortingextractor/__init__.py
================================================
from .cellexplorersortingextractor import CellExplorerSortingExtractor
================================================
FILE: spikeextractors/extractors/cellexplorersortingextractor/cellexplorersortingextractor.py
================================================
from spikeextractors import SortingExtractor
import numpy as np
from pathlib import Path
from spikeextractors.extraction_tools import check_get_unit_spike_train
from typing import Union, Optional
try:
import scipy.io
import hdf5storage
HAVE_SCIPY_AND_HDF5STORAGE = True
except ImportError:
HAVE_SCIPY_AND_HDF5STORAGE = False
PathType = Union[str, Path]
OptionalPathType = Optional[PathType]
class CellExplorerSortingExtractor(SortingExtractor):
"""
Extracts spiking information from .mat files stored in the CellExplorer format.
Spike times are stored in units of seconds.
Parameters
----------
spikes_matfile_path : PathType
Path to the sorting_id.spikes.cellinfo.mat file.
"""
extractor_name = "CellExplorerSortingExtractor"
installed = HAVE_SCIPY_AND_HDF5STORAGE
is_writable = True
mode = "file"
installation_mesg = "To use the CellExplorerSortingExtractor install scipy and hdf5storage: \n\n pip install scipy\n\n and \n\n pip install hdf5 storage \n\n"
def __init__(self, spikes_matfile_path: PathType, session_info_matfile_path: OptionalPathType=None, sampling_frequency: Optional[float] = None):
assert self.installed, self.installation_mesg
SortingExtractor.__init__(self)
spikes_matfile_path = Path(spikes_matfile_path)
assert (
spikes_matfile_path.is_file()
), f"The spikes_matfile_path ({spikes_matfile_path}) must exist!"
if sampling_frequency is None:
folder_path = spikes_matfile_path.parent
sorting_id = spikes_matfile_path.name.split(".")[0]
if session_info_matfile_path is None:
session_info_matfile_path = folder_path / f"{sorting_id}.sessionInfo.mat"
assert (
session_info_matfile_path.is_file()
), f"No {sorting_id}.sessionInfo.mat file found in the folder!"
try:
session_info_mat = scipy.io.loadmat(file_name=str(session_info_matfile_path))
self.read_session_info_with_scipy = True
except NotImplementedError:
session_info_mat = hdf5storage.loadmat(file_name=str(session_info_matfile_path))
self.read_session_info_with_scipy = False
assert session_info_mat["sessionInfo"]["rates"][0][0]["wideband"], (
"The sesssionInfo.mat file must contain "
"a 'sessionInfo' struct with field 'rates' containing field 'wideband' to extract the sampling frequency!"
)
if self.read_session_info_with_scipy:
self._sampling_frequency = float(
session_info_mat["sessionInfo"]["rates"][0][0]["wideband"][0][0][0][0]
) # careful not to confuse it with the lfpsamplingrate; reported in units Hz
else:
self._sampling_frequency = float(
session_info_mat["sessionInfo"]["rates"][0][0]["wideband"][0][0]
) # careful not to confuse it with the lfpsamplingrate; reported in units Hz
else:
self._sampling_frequency = sampling_frequency
try:
spikes_mat = scipy.io.loadmat(file_name=str(spikes_matfile_path))
self.read_spikes_info_with_scipy = True
except NotImplementedError:
spikes_mat = hdf5storage.loadmat(file_name=str(spikes_matfile_path))
self.read_spikes_info_with_scipy = False
assert np.all(
np.isin(["UID", "times"], spikes_mat["spikes"].dtype.names)
), "The spikes.cellinfo.mat file must contain a 'spikes' struct with fields 'UID' and 'times'!"
# CellExplorer reports spike times in units seconds; SpikeExtractors uses time units of sampling frames
# Rounding is necessary to prevent data loss from int-casting floating point errors
if self.read_spikes_info_with_scipy:
self._unit_ids = np.asarray(spikes_mat["spikes"]["UID"][0][0][0], dtype=int)
self._spiketrains = [
(np.array([y[0] for y in x]) * self._sampling_frequency).round().astype(int)
for x in spikes_mat["spikes"]["times"][0][0][0]
]
else:
self._unit_ids = np.asarray(spikes_mat["spikes"]["UID"][0][0], dtype=int)
self._spiketrains = [
(np.array([y[0] for y in x]) * self._sampling_frequency).round().astype(int)
for x in spikes_mat["spikes"]["times"][0][0]
]
self._kwargs = dict(spikes_matfile_path=str(spikes_matfile_path.absolute()))
def get_unit_ids(self):
return list(self._unit_ids)
@check_get_unit_spike_train
def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=None):
times = self._spiketrains[self.get_unit_ids().index(unit_id)]
inds = np.where((start_frame <= times) & (times < end_frame))
return times[inds]
@staticmethod
def write_sorting(sorting: SortingExtractor, save_path: PathType):
assert save_path.suffixes == [
".spikes",
".cellinfo",
".mat",
], "The save_path must correspond to the CellExplorer format of sorting_id.spikes.cellinfo.mat!"
base_path = save_path.parent
sorting_id = save_path.name.split(".")[0]
session_info_save_path = base_path / f"{sorting_id}.sessionInfo.mat"
spikes_save_path = save_path
base_path.mkdir(parents=True, exist_ok=True)
sampling_frequency = sorting.get_sampling_frequency()
session_info_mat_dict = dict(
sessionInfo=dict(rates=dict(wideband=sampling_frequency))
)
scipy.io.savemat(file_name=session_info_save_path, mdict=session_info_mat_dict)
spikes_mat_dict = dict(
spikes=dict(
UID=sorting.get_unit_ids(),
times=[
[[y / sampling_frequency] for y in x]
for x in sorting.get_units_spike_train()
],
)
)
# If, in the future, it is ever desired to allow this to write unit properties, they must conform
# to the format here: https://cellexplorer.org/datastructure/data-structure-and-format/
scipy.io.savemat(file_name=spikes_save_path, mdict=spikes_mat_dict)
================================================
FILE: spikeextractors/extractors/combinatosortingextractor/__init__.py
================================================
from .combinatosortingextractor import CombinatoSortingExtractor
================================================
FILE: spikeextractors/extractors/combinatosortingextractor/combinatosortingextractor.py
================================================
from pathlib import Path
import numpy as np
from spikeextractors import SortingExtractor
from spikeextractors.extraction_tools import check_get_unit_spike_train
from typing import Union
try:
import h5py
HAVE_H5PY = True
except ImportError:
HAVE_H5PY = False
PathType = Union[str, Path]
class CombinatoSortingExtractor(SortingExtractor):
extractor_name = 'CombinatoSorting'
installation_mesg = "" # error message when not installed
installed = HAVE_H5PY
is_writable = False
installation_mesg = "To use the CombinatoSortingExtractor install h5py: \n\n pip install h5py\n\n" # error message when not installed
def __init__(self, datapath: PathType, sampling_frequency=None, user='simple',det_sign = 'both'):
super().__init__()
datapath = Path(datapath)
assert datapath.is_dir(), 'Folder {} doesn\'t exist'.format(datapath)
if sampling_frequency is None:
h5_path = str(datapath) + '.h5'
if Path(h5_path).exists():
with h5py.File(h5_path, mode='r') as f:
sampling_frequency = f['sr'][0]
self.set_sampling_frequency(sampling_frequency)
det_file = str(datapath / Path('data_' + datapath.stem + '.h5'))
sort_cat_files = []
for sign in ['neg', 'pos']:
if det_sign in ['both', sign]:
sort_cat_file = datapath / Path('sort_{}_{}/sort_cat.h5'.format(sign,user))
if sort_cat_file.exists():
sort_cat_files.append((sign, str(sort_cat_file)))
unit_counter = 0
self._spike_trains = {}
metadata = {}
unsorted = []
fdet = h5py.File(det_file, mode='r')
for sign, sfile in sort_cat_files:
with h5py.File(sfile, mode='r') as f:
sp_class = f['classes'][()]
gaux = f['groups'][()]
groups = {g:gaux[gaux[:, 1] == g, 0] for g in np.unique(gaux[:, 1])} #array of classes per group
group_type = {group: g_type for group,g_type in f['types'][()]}
sp_index = f['index'][()]
times_css = fdet[sign]['times'][()]
for gr, cls in groups.items():
if group_type[gr] == -1: #artifacts
continue
elif group_type[gr] == 0: #unsorted
unsorted.append(np.rint(times_css[sp_index[np.isin(sp_class,cls)]] * (sampling_frequency/1000)))
continue
unit_counter = unit_counter + 1
self._spike_trains[unit_counter] = np.rint(times_css[sp_index[np.isin(sp_class, cls)]] * (sampling_frequency / 1000))
metadata[unit_counter] = {'det_sign': sign,
'group_type': 'single-unit' if group_type[gr] else 'multi-unit'}
fdet.close()
self._unsorted_train = np.array([])
if len(unsorted) == 1:
self._unsorted_train = unsorted[0]
elif len(unsorted) == 2: #unsorted in both signs
self._unsorted_train = np.sort(np.concatenate(unsorted), kind='mergesort')
self._unit_ids = list(range(1, unit_counter+1))
for u in self._unit_ids:
for prop,value in metadata[u].items():
self.set_unit_property(u, prop, value)
def get_unit_ids(self):
return self._unit_ids
@check_get_unit_spike_train
def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=None):
start_frame, end_frame = self._cast_start_end_frame(start_frame, end_frame)
start_frame = start_frame or 0
end_frame = end_frame or np.infty
st = self._spike_trains[unit_id]
return st[(st >= start_frame) & (st < end_frame)]
def get_unsorted_spike_train(self, start_frame=None, end_frame=None):
start_frame, end_frame = self._cast_start_end_frame(start_frame, end_frame)
start_frame = start_frame or 0
end_frame = end_frame or np.infty
u = self._unsorted_train
return u[(u >= start_frame) & (u < end_frame)]
================================================
FILE: spikeextractors/extractors/exdirextractors/__init__.py
================================================
from .exdirextractors import ExdirRecordingExtractor, ExdirSortingExtractor
================================================
FILE: spikeextractors/extractors/exdirextractors/exdirextractors.py
================================================
from spikeextractors import RecordingExtractor
from spikeextractors import SortingExtractor
import numpy as np
from pathlib import Path
from copy import copy
from spikeextractors.extraction_tools import check_get_traces_args, check_get_unit_spike_train
try:
import exdir
import exdir.plugins.quantities
import quantities as pq
HAVE_EXDIR = True
except ImportError:
HAVE_EXDIR = False
class ExdirRecordingExtractor(RecordingExtractor):
extractor_name = 'ExdirRecording'
has_default_locations = False
has_unscaled = False
installed = HAVE_EXDIR # check at class level if installed or not
is_writable = True
mode = 'folder'
installation_mesg = "To use the ExdirExtractors run:\n\n pip install exdir\n\n" # error message when not installed
def __init__(self, folder_path):
assert self.installed, self.installation_mesg
self._exdir_file = folder_path
exdir_group = exdir.File(folder_path, plugins=[exdir.plugins.quantities])
self._recordings = exdir_group['acquisition']['timeseries']
self._sampling_frequency = float(self._recordings.attrs['sample_rate'].rescale('Hz').magnitude)
self._num_channels = self._recordings.shape[0]
self._num_timepoints = self._recordings.shape[1]
RecordingExtractor.__init__(self)
self._kwargs = {'folder_path': str(Path(folder_path).absolute())}
def get_channel_ids(self):
return list(range(self._num_channels))
def get_num_frames(self):
return self._num_timepoints
def get_sampling_frequency(self):
return self._sampling_frequency
@check_get_traces_args
def get_traces(self, channel_ids=None, start_frame=None, end_frame=None, return_scaled=True):
return self._recordings.data[np.array(channel_ids), start_frame:end_frame]
@staticmethod
def write_recording(recording, save_path, lfp=False, mua=False):
assert HAVE_EXDIR, ExdirRecordingExtractor.installation_mesg
channel_ids = recording.get_channel_ids()
raw = recording.get_traces()
exdir_group = exdir.File(save_path, plugins=[exdir.plugins.quantities])
if not lfp and not mua:
acq = exdir_group.require_group('acquisition')
timeseries = acq.require_dataset('timeseries', data=raw)
timeseries.attrs['sample_rate'] = recording.get_sampling_frequency() * pq.Hz
timeseries.attrs['electrode_identities'] = np.array(channel_ids)
return
elif lfp:
ephys = exdir_group.require_group('processing').require_group('electrophysiology')
ephys.attrs['sample_rate'] = recording.get_sampling_frequency() * pq.Hz
if 'group' in recording.get_shared_channel_property_names():
channel_groups = np.unique(recording.get_channel_groups())
else:
channel_groups = [0]
if len(channel_groups) == 1:
chan = 0
ch_group = ephys.require_group('channel_group_' + str(chan))
lfp_group = ch_group.require_group('LFP')
ch_group.attrs['electrode_group_id'] = chan
ch_group.attrs['electrode_identities'] = np.array(recording.get_channel_ids())
ch_group.attrs['electrode_idx'] = np.arange(len(recording.get_channel_ids()))
ch_group.attrs['start_time'] = 0 * pq.s
ch_group.attrs['stop_time'] = recording.get_num_frames() / \
float(recording.get_sampling_frequency()) * pq.s
for i_c, ch in enumerate(recording.get_channel_ids()):
ts_group = lfp_group.require_group('LFP_timeseries_' + str(ch))
ts_group.attrs['electrode_group_id'] = chan
ts_group.attrs['electrode_identity'] = ch
ts_group.attrs['num_samples'] = recording.get_num_frames()
ts_group.attrs['electrode_idx'] = i_c
ts_group.attrs['start_time'] = 0 * pq.s
ts_group.attrs['stop_time'] = recording.get_num_frames() / \
float(recording.get_sampling_frequency()) * pq.s
ts_group.attrs['sample_rate'] = recording.get_sampling_frequency() * pq.Hz
data = ts_group.require_dataset('data', data=recording.get_traces(channel_ids=[ch]))
data.attrs['sample_rate'] = recording.get_sampling_frequency() * pq.Hz
data.attrs['unit'] = pq.uV
else:
channel_groups = np.unique(recording.get_channel_groups())
for chan in channel_groups:
ch_group = ephys.require_group('channel_group_' + str(chan))
lfp_group = ch_group.require_group('LFP')
ch_group.attrs['electrode_group_id'] = chan
ch_group.attrs['electrode_identities'] = np.array([ch for ch in recording.get_channel_ids()
if recording.get_channel_groups(ch) == chan])
ch_group.attrs['electrode_idx'] = np.array([i_c for i_c, ch in
enumerate(recording.get_channel_ids())
if recording.get_channel_groups(ch) == chan])
ch_group.attrs['start_time'] = 0 * pq.s
ch_group.attrs['stop_time'] = recording.get_num_frames() / \
float(recording.get_sampling_frequency()) * pq.s
for i_c, ch in enumerate(recording.get_channel_ids()):
if recording.get_channel_groups(ch) == chan:
ts_group = lfp_group.require_group('LFP_timeseries_' + str(ch))
ts_group.attrs['electrode_group_id'] = chan
ts_group.attrs['electrode_identity'] = ch
ts_group.attrs['num_samples'] = recording.get_num_frames()
ts_group.attrs['electrode_idx'] = i_c
ts_group.attrs['start_time'] = 0 * pq.s
ts_group.attrs['stop_time'] = recording.get_num_frames() / \
float(recording.get_sampling_frequency()) * pq.s
ts_group.attrs['sample_rate'] = recording.get_sampling_frequency() * pq.Hz
data = ts_group.require_dataset('data', data=recording.get_traces(channel_ids=[ch]))
data.attrs['sample_rate'] = recording.get_sampling_frequency() * pq.Hz
data.attrs['unit'] = pq.uV
return
elif mua:
ephys = exdir_group.require_group('processing').require_group('electrophysiology')
ephys.attrs['sample_rate'] = recording.get_sampling_frequency() * pq.Hz
if 'group' in recording.get_shared_channel_property_names():
channel_groups = np.unique(recording.get_channel_groups())
else:
channel_groups = [0]
if len(channel_groups) == 1:
chan = 0
ch_group = ephys.require_group('channel_group_' + str(chan))
mua_group = ch_group.require_group('MUA')
ch_group.attrs['electrode_group_id'] = chan
ch_group.attrs['electrode_identities'] = np.array(recording.get_channel_ids())
ch_group.attrs['electrode_idx'] = np.arange(len(recording.get_channel_ids()))
ch_group.attrs['start_time'] = 0 * pq.s
ch_group.attrs['stop_time'] = recording.get_num_frames() / \
float(recording.get_sampling_frequency()) * pq.s
for i_c, ch in enumerate(recording.get_channel_ids()):
ts_group = mua_group.require_group('MUA_timeseries_' + str(ch))
ts_group.attrs['electrode_group_id'] = chan
ts_group.attrs['electrode_identity'] = ch
ts_group.attrs['num_samples'] = recording.get_num_frames()
ts_group.attrs['electrode_idx'] = i_c
ts_group.attrs['start_time'] = 0 * pq.s
ts_group.attrs['stop_time'] = recording.get_num_frames() / \
float(recording.get_sampling_frequency()) * pq.s
ts_group.attrs['sample_rate'] = recording.get_sampling_frequency() * pq.Hz
data = ts_group.require_dataset('data', data=recording.get_traces(channel_ids=[ch]))
data.attrs['sample_rate'] = recording.get_sampling_frequency() * pq.Hz
data.attrs['unit'] = pq.uV
else:
channel_groups = np.unique(recording.get_channel_groups())
for chan in channel_groups:
ch_group = ephys.require_group('channel_group_' + str(chan))
mua_group = ch_group.require_group('MUA')
ch_group.attrs['electrode_group_id'] = chan
ch_group.attrs['electrode_identities'] = np.array([ch for ch in recording.get_channel_ids()
if recording.get_channel_groups(ch) == chan])
ch_group.attrs['electrode_idx'] = np.array([i_c for i_c, ch in
enumerate(recording.get_channel_ids())
if recording.get_channel_groups(ch) == chan])
ch_group.attrs['start_time'] = 0 * pq.s
ch_group.attrs['stop_time'] = recording.get_num_frames() / \
float(recording.get_sampling_frequency()) * pq.s
for i_c, ch in enumerate(recording.get_channel_ids()):
if recording.get_channel_groups(ch) == chan:
ts_group = mua_group.require_group('MUA_timeseries_' + str(ch))
ts_group.attrs['electrode_group_id'] = chan
ts_group.attrs['electrode_identity'] = ch
ts_group.attrs['num_samples'] = recording.get_num_frames()
ts_group.attrs['electrode_idx'] = i_c
ts_group.attrs['start_time'] = 0 * pq.s
ts_group.attrs['stop_time'] = recording.get_num_frames() / \
float(recording.get_sampling_frequency()) * pq.s
ts_group.attrs['sample_rate'] = recording.get_sampling_frequency() * pq.Hz
data = ts_group.require_dataset('data', data=recording.get_traces(channel_ids=[ch]))
data.attrs['sample_rate'] = recording.get_sampling_frequency() * pq.Hz
data.attrs['unit'] = pq.uV
class ExdirSortingExtractor(SortingExtractor):
extractor_name = 'ExdirSorting'
installed = HAVE_EXDIR # check at class level if installed or not
is_writable = True
mode = 'folder'
installation_mesg = "To use the ExdirExtractors run:\n\n pip install exdir\n\n" # error message when not installed
def __init__(self, folder_path, sampling_frequency=None, channel_group=None, load_waveforms=False):
assert self.installed, self.installation_mesg
SortingExtractor.__init__(self)
self._exdir_file = folder_path
exdir_group = exdir.File(folder_path, plugins=exdir.plugins.quantities)
electrophysiology = None
sf = copy(sampling_frequency)
if 'processing' in exdir_group.keys():
if 'electrophysiology' in exdir_group['processing']:
electrophysiology = exdir_group['processing']['electrophysiology']
ephys_attrs = electrophysiology.attrs
if 'sample_rate' in ephys_attrs:
sf = ephys_attrs['sample_rate']
else:
if sf is None:
raise Exception("Sampling rate information not found. Please provide it with the 'sampling_frequency' "
"argument")
else:
sf = sf * pq.Hz
self._sampling_frequency = float(sf.rescale('Hz').magnitude)
if electrophysiology is None:
raise Exception("'electrophysiology' group not found!")
self._unit_ids = []
current_unit = 1
self._spike_trains = []
for chan_name, channel in electrophysiology.items():
if 'channel' in chan_name:
group = int(chan_name.split('_')[-1])
if channel_group is not None:
if group != channel_group:
continue
if load_waveforms:
if 'Clustering' in channel.keys() and 'EventWaveform' in channel.keys():
clustering = channel.require_group('Clustering')
eventwaveform = channel.require_group('EventWaveform')
nums = clustering['nums'].data
waveforms = eventwaveform.require_group('waveform_timeseries')['data'].data
if 'UnitTimes' in channel.keys():
for unit, unit_times in channel['UnitTimes'].items():
self._unit_ids.append(current_unit)
self._spike_trains.append((unit_times['times'].data.rescale('s') * sf).magnitude)
attrs = unit_times.attrs
for k, v in attrs.items():
self.set_unit_property(current_unit, k, v)
if load_waveforms:
unit_idxs = np.where(nums == int(unit))
wf = waveforms[unit_idxs]
self.set_unit_spike_features(current_unit, 'waveforms', wf)
current_unit += 1
self._kwargs = {'folder_path': str(Path(folder_path).absolute()), 'sampling_frequency': sampling_frequency,
'channel_group': channel_group, 'load_waveforms': load_waveforms}
def get_unit_ids(self):
return self._unit_ids
@check_get_unit_spike_train
def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=None):
times = self._spike_trains[self._unit_ids.index(unit_id)]
inds = np.where((start_frame <= times) & (times < end_frame))
return np.rint(times[inds]).astype(int)
@staticmethod
def write_sorting(sorting, save_path, recording=None, sampling_frequency=None, save_waveforms=False, verbose=False):
assert HAVE_EXDIR, ExdirSortingExtractor.installation_mesg
if sampling_frequency is None and recording is None:
raise Exception("Provide 'sampling_frequency' argument (Hz)")
else:
if recording is None:
sampling_frequency = sampling_frequency * pq.Hz
else:
sampling_frequency = recording.get_sampling_frequency() * pq.Hz
exdir_group = exdir.File(save_path, plugins=exdir.plugins.quantities)
ephys = exdir_group.require_group('processing').require_group('electrophysiology')
ephys.attrs['sample_rate'] = sampling_frequency
if 'group' in sorting.get_shared_unit_property_names():
channel_groups = np.unique([sorting.get_unit_property(unit, 'group') for unit in sorting.get_unit_ids()])
else:
channel_groups = [0]
if len(channel_groups) == 1 and channel_groups[0] == 0:
chan = 0
if verbose:
print("Single group: ", chan)
ch_group = ephys.require_group('channel_group_' + str(chan))
try:
del ch_group['UnitTimes']
del ch_group['EventWaveform']
del ch_group['Clustering']
except Exception as e:
pass
unittimes = ch_group.require_group('UnitTimes')
unit_stop_time = np.max(
[(np.max(sorting.get_unit_spike_train(u).astype(float) / sampling_frequency).rescale('s'))
for u in sorting.get_unit_ids()]) * pq.s
recording_stop_time = None
if recording is not None:
ch_group.attrs['electrode_group_id'] = chan
ch_group.attrs['electrode_identities'] = np.array([])
ch_group.attrs['electrode_idx'] = np.arange(len(recording.get_channel_ids()))
ch_group.attrs['start_time'] = 0 * pq.s
recording_stop_time = recording.get_num_frames() / float(recording.get_sampling_frequency()) * pq.s
unittimes.attrs['electrode_group_id'] = chan
unittimes.attrs['electrode_identities'] = np.array([])
unittimes.attrs['electrode_idx'] = np.array(recording.get_channel_ids())
unittimes.attrs['start_time'] = 0 * pq.s
ch_group.attrs['sample_rate'] = sampling_frequency
if recording_stop_time is not None:
unittimes.attrs['stop_time'] = recording_stop_time if recording_stop_time > unit_stop_time \
else unit_stop_time
ch_group.attrs['stop_time'] = recording_stop_time if recording_stop_time > unit_stop_time \
else unit_stop_time
nums = np.array([])
timestamps = np.array([])
waveforms = np.array([])
for unit in sorting.get_unit_ids():
unit_group = unittimes.require_group(str(unit))
unit_group.require_dataset('times',
data=(sorting.get_unit_spike_train(unit).astype(float)
/ sampling_frequency).rescale('s'))
unit_group.attrs['cluster_group'] = 'unsorted'
unit_group.attrs['group_id'] = chan
unit_group.attrs['name'] = 'unit #' + str(unit)
timestamps = np.concatenate((timestamps, (sorting.get_unit_spike_train(unit).astype(float)
/ sampling_frequency).rescale('s')))
nums = np.concatenate((nums, [unit] * len(sorting.get_unit_spike_train(unit))))
if 'waveforms' in sorting.get_unit_spike_feature_names(unit):
if len(waveforms) == 0:
waveforms = sorting.get_unit_spike_features(unit, 'waveforms')
else:
waveforms = np.vstack((waveforms, sorting.get_unit_spike_features(unit, 'waveforms')))
if save_waveforms:
if verbose:
print("Saving EventWaveforms")
if 'waveforms' in sorting.get_shared_unit_spike_feature_names():
eventwaveform = ch_group.require_group('EventWaveform')
waveform_ts = eventwaveform.require_group('waveform_timeseries')
data = waveform_ts.require_dataset('data', data=waveforms)
waveform_ts.attrs['electrode_group_id'] = chan
data.attrs['num_samples'] = len(waveforms)
data.attrs['sample_rate'] = sampling_frequency
data.attrs['unit'] = pq.dimensionless
times = waveform_ts.require_dataset('timestamps', data=timestamps)
times.attrs['num_samples'] = len(timestamps)
times.attrs['unit'] = pq.s
if recording is not None:
waveform_ts.attrs['electrode_identities'] = np.array([])
waveform_ts.attrs['electrode_idx'] = np.arange(len(recording.get_channel_ids()))
waveform_ts.attrs['start_time'] = 0 * pq.s
if recording_stop_time is not None:
waveform_ts.attrs['stop_time'] = recording_stop_time if recording_stop_time > unit_stop_time \
else unit_stop_time
waveform_ts.attrs['sample_rate'] = sampling_frequency
waveform_ts.attrs['sample_length'] = waveforms.shape[1]
waveform_ts.attrs['num_samples'] = len(waveforms)
if verbose:
print("Saving Clustering")
clustering = ch_group.require_group('Clustering')
ts = clustering.require_dataset('timestamps', data=timestamps * pq.s)
ts.attrs['num_samples'] = len(timestamps)
ts.attrs['unit'] = pq.s
ns = clustering.require_dataset('nums', data=nums)
ns.attrs['num_samples'] = len(nums)
cn = clustering.require_dataset('cluster_nums', data=np.array(sorting.get_unit_ids()))
cn.attrs['num_samples'] = len(sorting.get_unit_ids())
else:
# remove preexisten spike sorting data
max_group = 10
for chan in np.arange(max_group):
if 'channel_group_' + str(chan) in ephys.keys():
if verbose:
print('Removing channel', chan, 'info')
ch_group = ephys.require_group('channel_group_' + str(chan))
try:
del ch_group['UnitTimes']
del ch_group['EventWaveform']
del ch_group['Clustering']
except Exception as e:
pass
channel_groups = np.unique([sorting.get_unit_property(unit, 'group') for unit in sorting.get_unit_ids()])
for chan in channel_groups:
if verbose:
print("Group: ", chan)
ch_group = ephys.require_group('channel_group_' + str(chan))
unittimes = ch_group.require_group('UnitTimes')
unit_stop_time = np.max(
[(np.max(sorting.get_unit_spike_train(u).astype(float) / sampling_frequency).rescale('s'))
for u in sorting.get_unit_ids()]) * pq.s
recording_stop_time = None
if recording is not None:
unittimes.attrs['electrode_group_id'] = chan
unittimes.attrs['electrode_identities'] = np.array([])
unittimes.attrs['electrode_idx'] = np.array(
[ch for i_c, ch in enumerate(recording.get_channel_ids())
if recording.get_channel_groups(ch) == chan])
unittimes.attrs['start_time'] = 0 * pq.s
recording_stop_time = recording.get_num_frames() / float(recording.get_sampling_frequency()) * pq.s
ch_group.attrs['electrode_group_id'] = chan
ch_group.attrs['electrode_identities'] = np.array(
[i_c for i_c, ch in enumerate(recording.get_channel_ids())
if recording.get_channel_groups(ch) == chan])
ch_group.attrs['electrode_idx'] = np.array(
[i_c for i_c, ch in enumerate(recording.get_channel_ids())
if recording.get_channel_groups(ch) == chan])
ch_group.attrs['start_time'] = 0 * pq.s
ch_group.attrs['sample_rate'] = sampling_frequency
if recording_stop_time is not None:
unittimes.attrs['stop_time'] = recording_stop_time if recording_stop_time > unit_stop_time \
else unit_stop_time
ch_group.attrs['stop_time'] = recording_stop_time if recording_stop_time > unit_stop_time \
else unit_stop_time
nums = np.array([])
timestamps = np.array([])
waveforms = np.array([])
for unit in sorting.get_unit_ids():
if sorting.get_unit_property(unit, 'group') == chan:
if verbose:
print("Unit: ", unit)
unit_group = unittimes.require_group(str(unit))
unit_group.require_dataset('times',
data=(sorting.get_unit_spike_train(unit).astype(float)
/ sampling_frequency).rescale('s'))
unit_group.attrs['cluster_group'] = 'unsorted'
unit_group.attrs['group_id'] = chan
unit_group.attrs['name'] = 'unit #' + str(unit)
timestamps = np.concatenate((timestamps, (sorting.get_unit_spike_train(unit).astype(float)
/ sampling_frequency).rescale('s')))
nums = np.concatenate((nums, [unit] * len(sorting.get_unit_spike_train(unit))))
if 'waveforms' in sorting.get_unit_spike_feature_names(unit):
if len(waveforms) == 0:
waveforms = sorting.get_unit_spike_features(unit, 'waveforms')
else:
waveforms = np.vstack((waveforms, sorting.get_unit_spike_features(unit, 'waveforms')))
if save_waveforms:
if verbose:
print("Saving EventWaveforms")
if 'waveforms' in sorting.get_shared_unit_spike_feature_names():
eventwaveform = ch_group.require_group('EventWaveform')
waveform_ts = eventwaveform.require_group('waveform_timeseries')
data = waveform_ts.require_dataset('data', data=waveforms)
data.attrs['num_samples'] = len(waveforms)
data.attrs['sample_rate'] = sampling_frequency
data.attrs['unit'] = pq.dimensionless
times = waveform_ts.require_dataset('timestamps', data=timestamps)
times.attrs['num_samples'] = len(timestamps)
times.attrs['unit'] = pq.s
waveform_ts.attrs['electrode_group_id'] = chan
if recording is not None:
waveform_ts.attrs['electrode_identities'] = np.array([])
waveform_ts.attrs['electrode_idx'] = np.array([ch for i_c, ch in
enumerate(recording.get_channel_ids())
if recording.get_channel_groups(ch) == chan])
waveform_ts.attrs['start_time'] = 0 * pq.s
if recording_stop_time is not None:
waveform_ts.attrs[
'stop_time'] = recording_stop_time if recording_stop_time > unit_stop_time \
else unit_stop_time
waveform_ts.attrs['sample_rate'] = sampling_frequency
waveform_ts.attrs['sample_length'] = waveforms.shape[1]
waveform_ts.attrs['num_samples'] = len(waveforms)
if verbose:
print("Saving Clustering")
clustering = ephys.require_group('channel_group_' + str(chan)).require_group('Clustering')
ts = clustering.require_dataset('timestamps', data=timestamps * pq.s)
ts.attrs['num_samples'] = len(timestamps)
ts.attrs['unit'] = pq.s
ns = clustering.require_dataset('nums', data=nums)
ns.attrs['num_samples'] = len(nums)
cn = clustering.require_dataset('cluster_nums', data=np.array(sorting.get_unit_ids()))
cn.attrs['num_samples'] = len(sorting.get_unit_ids())
================================================
FILE: spikeextractors/extractors/hdsortsortingextractor/__init__.py
================================================
from .hdsortsortingextractor import HDSortSortingExtractor
================================================
FILE: spikeextractors/extractors/hdsortsortingextractor/hdsortsortingextractor.py
================================================
from pathlib import Path
from typing import Union
import numpy as np
import sys
import os
from spikeextractors.extractors.matsortingextractor.matsortingextractor import MATSortingExtractor
from spikeextractors.extraction_tools import check_get_unit_spike_train
PathType = Union[str, Path]
convert_cell_array_to_struct_code = """
hdsortOutput = load(fileName);
hdsortOutput.Units = [hdsortOutput.Units{:}];
Units = hdsortOutput.Units;
MultiElectrode = hdsortOutput.MultiElectrode;
noiseStd = hdsortOutput.noiseStd;
samplingRate = hdsortOutput.samplingRate;
save(fileName, 'Units', 'MultiElectrode', 'noiseStd', 'samplingRate');
"""
class HDSortSortingExtractor(MATSortingExtractor):
extractor_name = "HDSortSortingExtractor"
def __init__(self, file_path: PathType, keep_good_only: bool = True):
super().__init__(file_path)
if not self._old_style_mat:
_units = self._data['Units']
units = _parse_units(self._data, _units)
# Extracting MutliElectrode field by field:
_ME = self._data["MultiElectrode"]
multi_electrode = dict((k, _ME.get(k)[()]) for k in _ME.keys())
# Extracting sampling_frequency:
sr = self._data["samplingRate"]
self._sampling_frequency = float(_squeeze_ds(sr))
# Remove noise units if necessary:
if keep_good_only:
units = [unit for unit in units if unit["ID"].flatten()[0].astype(int) % 1000 != 0]
if 'sortingInfo' in self._data.keys():
info = self._data["sortingInfo"]
start_frame = _squeeze_ds(info['startTimes'])
self.start_frame = int(start_frame)
else:
self.start_frame = 0
else:
_units = self._getfield('Units').squeeze()
fields = _units.dtype.fields.keys()
units = []
for unit in _units:
unit_dict = {}
for f in fields:
unit_dict[f] = unit[f]
units.append(unit_dict)
sr = self._getfield("samplingRate")
self._sampling_frequency = float(_squeeze_ds(sr))
_ME = self._data["MultiElectrode"]
multi_electrode = dict((k, _ME[k][0][0].T) for k in _ME.dtype.fields.keys())
# Remove noise units if necessary:
if keep_good_only:
units = [unit for unit in units if unit["ID"].flatten()[0].astype(int) % 1000 != 0]
if 'sortingInfo' in self._data.keys():
info = self._getfield("sortingInfo")
start_frame = _squeeze_ds(info['startTimes'])
self.start_frame = int(start_frame)
else:
self.start_frame = 0
# Parse through 'units':
self._spike_trains = {}
self._unit_ids = np.empty(0, np.int)
for uc, unit in enumerate(units):
uid = int(_squeeze_ds(unit["ID"]))
self._unit_ids = np.append(self._unit_ids, uid)
self._spike_trains[uc] = _squeeze(unit["spikeTrain"]).astype(np.int) - self.start_frame
# For memory efficiency in case it's necessary:
# X = self.allocate_array( "amplitudes_" + uid, array= unit["spikeAmplitudes"].flatten().T)
# self.set_unit_spike_features(uid, "amplitudes", X)
self.set_unit_spike_features(uid, "amplitudes", _squeeze(unit["spikeAmplitudes"]))
self.set_unit_spike_features(uid, "detection_channel", _squeeze(unit["detectionChannel"]).astype(np.int))
idx = unit["detectionChannel"].astype(int) - 1
spikePositions = np.vstack((_squeeze(multi_electrode["electrodePositions"][0][idx]),
_squeeze(multi_electrode["electrodePositions"][1][idx]))).T
self.set_unit_spike_features(uid, "positions", spikePositions)
if self._old_style_mat:
template = unit["footprint"].T
else:
template = unit["footprint"]
self.set_unit_property(uid, "template", template)
self.set_unit_property(uid, "template_frames_cut_before", unit["cutLeft"].flatten())
self._units = units
self._multi_electrode = multi_electrode
self._kwargs['keep_good_only'] = keep_good_only
@check_get_unit_spike_train
def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=None):
uidx = np.where(np.array(self.get_unit_ids()) == unit_id)[0][0]
st = self._spike_trains[uidx]
return st[(st >= start_frame) & (st < end_frame)]
def get_unit_ids(self):
return self._unit_ids.tolist()
@staticmethod
def write_sorting(sorting, save_path, locations=None, noise_std_by_channel=None, start_frame=0,
convert_cell_to_struct=True):
# First, find out how many channels there are
if locations is not None:
# write_locations must be a 2D numpy array with n_channels in first dim., (x,y) in second dim.
n_channels = locations.shape[0]
elif 'template' in sorting.get_shared_unit_property_names() or \
'detection_channel' in sorting.get_shared_unit_property_names():
# Without locations, check if there is a template to get the number of channels
uid = int(sorting.get_unit_ids()[0])
if "template" in sorting.get_unit_property_names(uid):
template = sorting.get_unit_property(uid, "template")
n_channels = template.shape[0]
else:
# If there is also no template, loop through all units and find max. detection_channel
max_channel = 1
for uid_ in sorting.get_unit_ids():
uid = int(uid_)
detection_channel = sorting.get_unit_spike_features(uid, "detection_channel")
max_channel = max([max_channel], np.append(detection_channel))
n_channels = max_channel
else:
n_channels = 1
# Now loop through all units and extract the data that we want to save:
units = []
for uid_ in sorting.get_unit_ids():
uid = int(uid_)
unit = {"ID": uid,
"spikeTrain": sorting.get_unit_spike_train(uid)}
num_spikes = len(sorting.get_unit_spike_train(uid))
if "amplitudes" in sorting.get_unit_spike_feature_names(uid):
unit["spikeAmplitudes"] = sorting.get_unit_spike_features(uid, "amplitudes")
else:
# Save a spikeAmplitudes = 1
unit["spikeAmplitudes"] = np.ones(num_spikes, np.double)
if "detection_channel" in sorting.get_unit_spike_feature_names(uid):
unit["detectionChannel"] = sorting.get_unit_spike_features(uid, "detection_channel")
else:
# Save a detectionChannel = 1
unit["detectionChannel"] = np.ones(num_spikes, np.double)
if "template" in sorting.get_unit_property_names(uid):
unit["footprint"] = sorting.get_unit_property(uid, "template").T
else:
# If this unit does not have a footprint, create an empty one:
unit["footprint"] = np.zeros((3, n_channels), np.double)
if "template_cut_left" in sorting.get_unit_property_names(uid):
unit["cutLeft"] = sorting.get_unit_property(uid, "template_cut_left")
else:
unit["cutLeft"] = 1
units.append(unit)
# Save the electrode locations:
if locations is None:
# Create artificial locations if none are provided:
x = np.zeros(n_channels, np.double)
y = np.array(np.arange(n_channels), np.double)
locations = np.vstack((x, y)).T
multi_electrode = {"electrodePositions": locations, "electrodeNumbers": np.arange(n_channels)}
if noise_std_by_channel is None:
noise_std_by_channel = np.ones((1, n_channels))
dict_to_save = {'Units': np.array(units),
'MultiElectrode': multi_electrode,
'noiseStd': noise_std_by_channel,
"samplingRate": sorting._sampling_frequency}
# Save Units and MultiElectrode to .mat file:
MATSortingExtractor.write_dict_to_mat(save_path, dict_to_save, version='7.3')
if convert_cell_to_struct:
# read the template txt files
convert_cellarray_to_structarray = f"fileName='{str(Path(save_path).absolute())}';\n" \
f"{convert_cell_array_to_struct_code}"
convert_script = Path(save_path).parent / "convert_cellarray_to_structarray.m"
with convert_script.open('w') as f:
f.write(convert_cellarray_to_structarray)
if 'win' in sys.platform and sys.platform != 'darwin':
matlab_cmd = """
#!/bin/bash
cd {tmpdir}
matlab -nosplash -wait -log -r convert_cellarray_to_structarray
""".format(tmpdir={str(convert_script.parent)})
else:
matlab_cmd = """
#!/bin/bash
cd {tmpdir}
matlab -nosplash -nodisplay -log -r convert_cellarray_to_structarray
""".format(tmpdir={str(convert_script.parent)})
try:
os.system(matlab_cmd)
except:
print("Failed to convert cell array to struct array")
convert_script.unlink()
# For .mat v7.3: Function to extract all fields of a struct-array:
def _parse_units(file, _units):
import h5py
t_units = {}
if isinstance(_units, h5py.Group):
for name in _units.keys():
value = _units[name]
dict_val = []
for val in value:
if isinstance(file[val[0]], h5py.Dataset):
dict_val.append(file[val[0]][()])
t_units[name] = dict_val
else:
break
out = [dict(zip(t_units, col)) for col in zip(*t_units.values())]
else:
out = []
for unit in _units:
group = file[unit[()][0]]
unit_dict = {}
for k in group.keys():
unit_dict[k] = group[k][()]
out.append(unit_dict)
return out
def _squeeze_ds(ds):
while not isinstance(ds, (int, float, np.integer, np.float)):
ds = ds[0]
return ds
def _squeeze(arr):
shape = arr.shape
if len(shape) == 2:
if shape[0] == 1:
arr = arr[0]
elif shape[1] == 1:
arr = arr[:, 0]
return arr
================================================
FILE: spikeextractors/extractors/hs2sortingextractor/__init__.py
================================================
from .hs2sortingextractor import HS2SortingExtractor
================================================
FILE: spikeextractors/extractors/hs2sortingextractor/hs2sortingextractor.py
================================================
from spikeextractors import SortingExtractor
import numpy as np
from pathlib import Path
from spikeextractors.extraction_tools import check_get_unit_spike_train
try:
import h5py
HAVE_HS2SX = True
except ImportError:
HAVE_HS2SX = False
class HS2SortingExtractor(SortingExtractor):
extractor_name = 'HS2Sorting'
installed = HAVE_HS2SX # check at class level if installed or not
is_writable = True
mode = 'file'
installation_mesg = "To use the HS2SortingExtractor install h5py: \n\n pip install h5py\n\n" # error message when not installed
def __init__(self, file_path, load_unit_info=True):
assert self.installed, self.installation_mesg
SortingExtractor.__init__(self)
self._recording_file = file_path
self._rf = h5py.File(self._recording_file, mode='r')
if 'Sampling' in self._rf:
if self._rf['Sampling'][()] == 0:
self._sampling_frequency = None
else:
self._sampling_frequency = self._rf['Sampling'][()]
self._cluster_id = self._rf['cluster_id'][()]
self._unit_ids = set(self._cluster_id)
self._spike_times = self._rf['times'][()]
if load_unit_info:
self.load_unit_info()
self._kwargs = {'file_path': str(Path(file_path).absolute()), 'load_unit_info': load_unit_info}
def load_unit_info(self):
if 'centres' in self._rf.keys() and len(self._spike_times) > 0:
self._unit_locs = self._rf['centres'][()] # cache for faster access
for u_i, unit_id in enumerate(self._unit_ids):
self.set_unit_property(unit_id, property_name='unit_location', value=self._unit_locs[u_i])
inds = [] # get these only once
for unit_id in self._unit_ids:
inds.append(np.where(self._cluster_id == unit_id)[0])
if 'data' in self._rf.keys() and len(self._spike_times) > 0:
d = self._rf['data'][()]
for i, unit_id in enumerate(self._unit_ids):
self.set_unit_spike_features(unit_id, 'spike_location', d[:, inds[i]].T)
if 'ch' in self._rf.keys() and len(self._spike_times) > 0:
d = self._rf['ch'][()]
for i, unit_id in enumerate(self._unit_ids):
self.set_unit_spike_features(unit_id, 'max_channel', d[inds[i]])
def get_unit_indices(self, x):
return np.where(self._cluster_id == x)[0]
def get_unit_ids(self):
return list(self._unit_ids)
@check_get_unit_spike_train
def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=None):
times = self._spike_times[self.get_unit_indices(unit_id)]
inds = np.where((start_frame <= times) & (times < end_frame))
return times[inds]
@staticmethod
def write_sorting(sorting, save_path):
assert HA
gitextract_c4b26tzl/
├── .github/
│ └── workflows/
│ ├── python-package.yml
│ └── python-publish.yml
├── .gitignore
├── LICENSE
├── MANIFEST.in
├── README.md
├── environment-dev.yml
├── full_requirements.txt
├── requirements-dev.txt
├── requirements.txt
├── setup.py
├── spikeextractors/
│ ├── __init__.py
│ ├── baseextractor.py
│ ├── cacheextractors.py
│ ├── example_datasets/
│ │ ├── __init__.py
│ │ ├── synthesize_random_firings.py
│ │ ├── synthesize_random_waveforms.py
│ │ ├── synthesize_single_waveform.py
│ │ ├── synthesize_timeseries.py
│ │ └── toy_example.py
│ ├── exceptions.py
│ ├── extraction_tools.py
│ ├── extractorlist.py
│ ├── extractors/
│ │ ├── __init__.py
│ │ ├── alfsortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── alfsortingextractor.py
│ │ ├── axonaunitrecordingextractor/
│ │ │ ├── __init__.py
│ │ │ └── axonaunitrecordingextractor.py
│ │ ├── bindatrecordingextractor/
│ │ │ ├── __init__.py
│ │ │ └── bindatrecordingextractor.py
│ │ ├── biocamrecordingextractor/
│ │ │ ├── __init__.py
│ │ │ └── biocamrecordingextractor.py
│ │ ├── cedextractors/
│ │ │ ├── __init__.py
│ │ │ ├── cedrecordingextractor.py
│ │ │ └── utils.py
│ │ ├── cellexplorersortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── cellexplorersortingextractor.py
│ │ ├── combinatosortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── combinatosortingextractor.py
│ │ ├── exdirextractors/
│ │ │ ├── __init__.py
│ │ │ └── exdirextractors.py
│ │ ├── hdsortsortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── hdsortsortingextractor.py
│ │ ├── hs2sortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── hs2sortingextractor.py
│ │ ├── intanrecordingextractor/
│ │ │ ├── __init__.py
│ │ │ └── intanrecordingextractor.py
│ │ ├── jrcsortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── jrcsortingextractor.py
│ │ ├── kilosortextractors/
│ │ │ ├── __init__.py
│ │ │ └── kilosortextractors.py
│ │ ├── klustaextractors/
│ │ │ ├── __init__.py
│ │ │ └── klustaextractors.py
│ │ ├── matsortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── matsortingextractor.py
│ │ ├── maxwellextractors/
│ │ │ ├── __init__.py
│ │ │ └── maxwellextractors.py
│ │ ├── mcsh5recordingextractor/
│ │ │ ├── __init__.py
│ │ │ └── mcsh5recordingextractor.py
│ │ ├── mdaextractors/
│ │ │ ├── __init__.py
│ │ │ ├── mdaextractors.py
│ │ │ └── mdaio.py
│ │ ├── mearecextractors/
│ │ │ ├── __init__.py
│ │ │ └── mearecextractors.py
│ │ ├── neoextractors/
│ │ │ ├── __init__.py
│ │ │ ├── axonaextractor.py
│ │ │ ├── blackrockextractor.py
│ │ │ ├── mcsrawrecordingextractor.py
│ │ │ ├── neobaseextractor.py
│ │ │ ├── neuralynxextractor.py
│ │ │ ├── plexonextractor.py
│ │ │ └── spikegadgetsextractor.py
│ │ ├── neuropixelsdatrecordingextractor/
│ │ │ ├── __init__.py
│ │ │ ├── channel_positions_neuropixels.txt
│ │ │ └── neuropixelsdatrecordingextractor.py
│ │ ├── neuroscopeextractors/
│ │ │ ├── __init__.py
│ │ │ └── neuroscopeextractors.py
│ │ ├── nixioextractors/
│ │ │ ├── __init__.py
│ │ │ └── nixioextractors.py
│ │ ├── npzsortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── npzsortingextractor.py
│ │ ├── numpyextractors/
│ │ │ ├── __init__.py
│ │ │ └── numpyextractors.py
│ │ ├── nwbextractors/
│ │ │ ├── __init__.py
│ │ │ └── nwbextractors.py
│ │ ├── openephysextractors/
│ │ │ ├── __init__.py
│ │ │ └── openephysextractors.py
│ │ ├── phyextractors/
│ │ │ ├── __init__.py
│ │ │ └── phyextractors.py
│ │ ├── shybridextractors/
│ │ │ ├── __init__.py
│ │ │ └── shybridextractors.py
│ │ ├── spikeglxrecordingextractor/
│ │ │ ├── __init__.py
│ │ │ ├── readSGLX.py
│ │ │ └── spikeglxrecordingextractor.py
│ │ ├── spykingcircusextractors/
│ │ │ ├── __init__.py
│ │ │ └── spykingcircusextractors.py
│ │ ├── tridescloussortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── tridescloussortingextractor.py
│ │ ├── waveclussortingextractor/
│ │ │ ├── __init__.py
│ │ │ └── waveclussortingextractor.py
│ │ └── yassextractors/
│ │ ├── __init__.py
│ │ └── yassextractors.py
│ ├── multirecordingchannelextractor.py
│ ├── multirecordingtimeextractor.py
│ ├── multisortingextractor.py
│ ├── recordingextractor.py
│ ├── save_tools.py
│ ├── sortingextractor.py
│ ├── subrecordingextractor.py
│ ├── subsortingextractor.py
│ ├── testing.py
│ └── version.py
└── tests/
├── __init__.py
├── probe_test.prb
├── test_extractors.py
├── test_gin_repo.py
├── test_numpy_extractors.py
└── test_tools.py
SYMBOL INDEX (684 symbols across 65 files)
FILE: spikeextractors/baseextractor.py
class BaseExtractor (line 14) | class BaseExtractor:
method __init__ (line 21) | def __init__(self):
method __del__ (line 34) | def __del__(self):
method del_memmap_file (line 44) | def del_memmap_file(self, memmap_file):
method make_serialized_dict (line 71) | def make_serialized_dict(self, relative_to=None):
method dump_to_dict (line 112) | def dump_to_dict(self, relative_to=None):
method _get_file_path (line 130) | def _get_file_path(self, file_path, extensions):
method dump_to_json (line 171) | def dump_to_json(self, file_path=None, relative_to=None):
method dump_to_pickle (line 191) | def dump_to_pickle(self, file_path=None, include_properties=True, incl...
method get_tmp_folder (line 223) | def get_tmp_folder(self):
method set_tmp_folder (line 236) | def set_tmp_folder(self, folder):
method allocate_array (line 247) | def allocate_array(self, memmap, shape=None, dtype=None, name=None, ar...
method annotate (line 302) | def annotate(self, annotation_key, value, overwrite=False):
method get_annotation (line 323) | def get_annotation(self, annotation_name):
method get_annotation_keys (line 343) | def get_annotation_keys(self):
method copy_annotations (line 353) | def copy_annotations(self, extractor):
method add_epoch (line 363) | def add_epoch(self, epoch_name, start_frame, end_frame):
method remove_epoch (line 384) | def remove_epoch(self, epoch_name):
method get_epoch_names (line 400) | def get_epoch_names(self):
method get_epoch_info (line 420) | def get_epoch_info(self, epoch_name):
method copy_epochs (line 444) | def copy_epochs(self, extractor):
method _cast_start_end_frame (line 456) | def _cast_start_end_frame(self, start_frame, end_frame):
method load_extractor_from_json (line 461) | def load_extractor_from_json(json_file):
method load_extractor_from_pickle (line 482) | def load_extractor_from_pickle(pkl_file):
method load_extractor_from_dict (line 509) | def load_extractor_from_dict(d):
method check_if_dumpable (line 526) | def check_if_dumpable(self):
function _make_paths_relative (line 530) | def _make_paths_relative(d, relative):
function _load_extractor_from_dict (line 544) | def _load_extractor_from_dict(dic):
function _get_class_from_string (line 602) | def _get_class_from_string(class_string):
function _check_same_version (line 615) | def _check_same_version(class_string, version):
function _check_if_dumpable (line 625) | def _check_if_dumpable(d):
function _check_json (line 635) | def _check_json(d):
FILE: spikeextractors/cacheextractors.py
class CacheRecordingExtractor (line 11) | class CacheRecordingExtractor(BinDatRecordingExtractor, RecordingExtract...
method __init__ (line 12) | def __init__(self, recording, return_scaled=True,
method __del__ (line 56) | def __del__(self):
method filename (line 66) | def filename(self):
method move_to (line 69) | def move_to(self, save_path):
method make_serialized_dict (line 91) | def make_serialized_dict(self, include_properties=None, include_featur...
class CacheSortingExtractor (line 117) | class CacheSortingExtractor(NpzSortingExtractor, SortingExtractor):
method __init__ (line 118) | def __init__(self, sorting, save_path=None):
method __del__ (line 141) | def __del__(self):
method filename (line 149) | def filename(self):
method move_to (line 152) | def move_to(self, save_path):
method make_serialized_dict (line 171) | def make_serialized_dict(self, include_properties=None, include_featur...
FILE: spikeextractors/example_datasets/synthesize_random_firings.py
function synthesize_random_firings (line 4) | def synthesize_random_firings(*, K=20, sampling_frequency=30000.0, durat...
function rand_distr2 (line 42) | def rand_distr2(a, b, num, seed):
function enforce_refractory_period (line 48) | def enforce_refractory_period(times_in, refr):
FILE: spikeextractors/example_datasets/synthesize_random_waveforms.py
function synthesize_random_waveforms (line 5) | def synthesize_random_waveforms(*, M=5, T=500, K=20, upsamplefac=13, tim...
function get_default_neuron_locations (line 55) | def get_default_neuron_locations(M, K, geometry):
FILE: spikeextractors/example_datasets/synthesize_single_waveform.py
function exp_growth (line 4) | def exp_growth(amp1, amp2, dur1, dur2):
function exp_decay (line 14) | def exp_decay(amp1, amp2, dur1, dur2):
function smooth_it (line 20) | def smooth_it(Y, t):
function synthesize_single_waveform (line 27) | def synthesize_single_waveform(*, N=800, durations=[200, 10, 30, 200], a...
FILE: spikeextractors/example_datasets/synthesize_timeseries.py
function synthesize_timeseries (line 4) | def synthesize_timeseries(*, sorting, waveforms, noise_level=1, sampling...
FILE: spikeextractors/example_datasets/toy_example.py
function toy_example (line 11) | def toy_example(
FILE: spikeextractors/exceptions.py
class NotDumpableExtractorError (line 1) | class NotDumpableExtractorError(TypeError):
FILE: spikeextractors/extraction_tools.py
function read_python (line 20) | def read_python(path):
function write_python (line 47) | def write_python(path, dict):
function load_probe_file (line 68) | def load_probe_file(recording, probe_file, channel_map=None, channel_gro...
function save_to_probe_file (line 190) | def save_to_probe_file(recording, probe_file, grouping_property=None, ra...
function read_binary (line 244) | def read_binary(file, numchan, dtype, time_axis=0, offset=0):
function write_to_binary_dat_format (line 272) | def write_to_binary_dat_format(recording, save_path=None, file_handle=None,
function write_to_h5_dataset_format (line 412) | def write_to_h5_dataset_format(recording, dataset_path, save_path=None, ...
function get_sub_extractors_by_property (line 509) | def get_sub_extractors_by_property(extractor, property_name, return_prop...
function _export_prb_file (line 570) | def _export_prb_file(recording, file_name, grouping_property=None, graph...
function _check_json (line 686) | def _check_json(d):
function load_extractor_from_json (line 701) | def load_extractor_from_json(json_file):
function load_extractor_from_dict (line 718) | def load_extractor_from_dict(d):
function load_extractor_from_pickle (line 735) | def load_extractor_from_pickle(pkl_file):
function check_get_unit_spike_train (line 752) | def check_get_unit_spike_train(func):
function check_get_traces_args (line 771) | def check_get_traces_args(func):
function check_get_ttl_args (line 818) | def check_get_ttl_args(func):
function cast_start_end_frame (line 845) | def cast_start_end_frame(start_frame, end_frame):
function divide_recording_into_time_chunks (line 864) | def divide_recording_into_time_chunks(num_frames, chunk_size, padding_si...
function _write_dat_one_chunk (line 879) | def _write_dat_one_chunk(i, rec_arg, chunks, rec_memmap, dtype, time_axi...
FILE: spikeextractors/extractors/alfsortingextractor/alfsortingextractor.py
class ALFSortingExtractor (line 15) | class ALFSortingExtractor(SortingExtractor):
method __init__ (line 22) | def __init__(self, folder_path, sampling_frequency=30000):
method _load_npy (line 61) | def _load_npy(self, npy_path):
method _get_clusters_spike_times (line 64) | def _get_clusters_spike_times(self, cluster_idx):
method get_unit_ids (line 79) | def get_unit_ids(self):
method get_unit_spike_train (line 87) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method write_sorting (line 120) | def write_sorting(sorting, save_path):
FILE: spikeextractors/extractors/axonaunitrecordingextractor/axonaunitrecordingextractor.py
class AxonaUnitRecordingExtractor (line 20) | class AxonaUnitRecordingExtractor(NeoBaseRecordingExtractor, RecordingEx...
method __init__ (line 38) | def __init__(self, noise_std: float = 3, block_index=None, seg_index=N...
method get_traces (line 124) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method get_num_frames (line 179) | def get_num_frames(self):
method get_sampling_frequency (line 185) | def get_sampling_frequency(self):
method get_channel_ids (line 188) | def get_channel_ids(self):
method _get_tetrode_channel_table (line 191) | def _get_tetrode_channel_table(self, channel_ids):
FILE: spikeextractors/extractors/bindatrecordingextractor/bindatrecordingextractor.py
class BinDatRecordingExtractor (line 16) | class BinDatRecordingExtractor(RecordingExtractor):
method __init__ (line 53) | def __init__(self, file_path: PathType, sampling_frequency: float, num...
method get_channel_ids (line 109) | def get_channel_ids(self):
method get_num_frames (line 112) | def get_num_frames(self):
method get_sampling_frequency (line 115) | def get_sampling_frequency(self):
method get_traces (line 119) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method write_recording (line 137) | def write_recording(
FILE: spikeextractors/extractors/biocamrecordingextractor/biocamrecordingextractor.py
class BiocamRecordingExtractor (line 14) | class BiocamRecordingExtractor(RecordingExtractor):
method __init__ (line 23) | def __init__(self, file_path, verbose=False, mea_pitch=42):
method __del__ (line 36) | def __del__(self):
method get_channel_ids (line 39) | def get_channel_ids(self):
method get_num_frames (line 42) | def get_num_frames(self):
method get_sampling_frequency (line 45) | def get_sampling_frequency(self):
method get_traces (line 49) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method write_recording (line 57) | def write_recording(recording, save_path):
function openBiocamFile (line 94) | def openBiocamFile(filename, mea_pitch, verbose=False):
function readHDF5t_100 (line 146) | def readHDF5t_100(rf, t0, t1, nch):
function readHDF5t_100_i (line 154) | def readHDF5t_100_i(rf, t0, t1, nch):
function readHDF5t_101 (line 162) | def readHDF5t_101(rf, t0, t1, nch):
function readHDF5t_101_i (line 170) | def readHDF5t_101_i(rf, t0, t1, nch):
FILE: spikeextractors/extractors/cedextractors/cedrecordingextractor.py
class CEDRecordingExtractor (line 21) | class CEDRecordingExtractor(RecordingExtractor):
method __init__ (line 44) | def __init__(self, file_path: PathType, smrx_channel_ids: list):
method channel_names (line 97) | def channel_names(self):
method get_traces (line 101) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method get_num_frames (line 148) | def get_num_frames(self):
method get_sampling_frequency (line 158) | def get_sampling_frequency(self):
method get_channel_ids (line 168) | def get_channel_ids(self):
method get_all_channels_info (line 180) | def get_all_channels_info(file_path):
FILE: spikeextractors/extractors/cedextractors/utils.py
function get_channel_info (line 25) | def get_channel_info(f, smrx_ch_ind):
function get_channel_data (line 60) | def get_channel_data(f, smrx_ch_ind, start_frame=0, end_frame=None):
FILE: spikeextractors/extractors/cellexplorersortingextractor/cellexplorersortingextractor.py
class CellExplorerSortingExtractor (line 19) | class CellExplorerSortingExtractor(SortingExtractor):
method __init__ (line 37) | def __init__(self, spikes_matfile_path: PathType, session_info_matfile...
method get_unit_ids (line 106) | def get_unit_ids(self):
method get_unit_spike_train (line 110) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method write_sorting (line 116) | def write_sorting(sorting: SortingExtractor, save_path: PathType):
FILE: spikeextractors/extractors/combinatosortingextractor/combinatosortingextractor.py
class CombinatoSortingExtractor (line 17) | class CombinatoSortingExtractor(SortingExtractor):
method __init__ (line 24) | def __init__(self, datapath: PathType, sampling_frequency=None, user='...
method get_unit_ids (line 80) | def get_unit_ids(self):
method get_unit_spike_train (line 85) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method get_unsorted_spike_train (line 93) | def get_unsorted_spike_train(self, start_frame=None, end_frame=None):
FILE: spikeextractors/extractors/exdirextractors/exdirextractors.py
class ExdirRecordingExtractor (line 18) | class ExdirRecordingExtractor(RecordingExtractor):
method __init__ (line 27) | def __init__(self, folder_path):
method get_channel_ids (line 41) | def get_channel_ids(self):
method get_num_frames (line 44) | def get_num_frames(self):
method get_sampling_frequency (line 47) | def get_sampling_frequency(self):
method get_traces (line 51) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method write_recording (line 55) | def write_recording(recording, save_path, lfp=False, mua=False):
class ExdirSortingExtractor (line 188) | class ExdirSortingExtractor(SortingExtractor):
method __init__ (line 195) | def __init__(self, folder_path, sampling_frequency=None, channel_group...
method get_unit_ids (line 250) | def get_unit_ids(self):
method get_unit_spike_train (line 254) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method write_sorting (line 261) | def write_sorting(sorting, save_path, recording=None, sampling_frequen...
FILE: spikeextractors/extractors/hdsortsortingextractor/hdsortsortingextractor.py
class HDSortSortingExtractor (line 23) | class HDSortSortingExtractor(MATSortingExtractor):
method __init__ (line 26) | def __init__(self, file_path: PathType, keep_good_only: bool = True):
method get_unit_spike_train (line 111) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method get_unit_ids (line 116) | def get_unit_ids(self):
method write_sorting (line 120) | def write_sorting(sorting, save_path, locations=None, noise_std_by_cha...
function _parse_units (line 229) | def _parse_units(file, _units):
function _squeeze_ds (line 256) | def _squeeze_ds(ds):
function _squeeze (line 262) | def _squeeze(arr):
FILE: spikeextractors/extractors/hs2sortingextractor/hs2sortingextractor.py
class HS2SortingExtractor (line 14) | class HS2SortingExtractor(SortingExtractor):
method __init__ (line 21) | def __init__(self, file_path, load_unit_info=True):
method load_unit_info (line 41) | def load_unit_info(self):
method get_unit_indices (line 58) | def get_unit_indices(self, x):
method get_unit_ids (line 61) | def get_unit_ids(self):
method get_unit_spike_train (line 65) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method write_sorting (line 71) | def write_sorting(sorting, save_path):
FILE: spikeextractors/extractors/intanrecordingextractor/intanrecordingextractor.py
class IntanRecordingExtractor (line 23) | class IntanRecordingExtractor(RecordingExtractor):
method __init__ (line 49) | def __init__(self, file_path: str, verbose: bool = False):
method get_channel_ids (line 71) | def get_channel_ids(self):
method get_num_frames (line 74) | def get_num_frames(self):
method get_sampling_frequency (line 77) | def get_sampling_frequency(self):
method get_traces (line 81) | def get_traces(
method get_ttl_events (line 133) | def get_ttl_events(self, start_frame=None, end_frame=None, channel_id=0):
FILE: spikeextractors/extractors/jrcsortingextractor/jrcsortingextractor.py
class JRCSortingExtractor (line 12) | class JRCSortingExtractor(MATSortingExtractor):
method __init__ (line 16) | def __init__(self, file_path: PathType, keep_good_only: bool = False):
method get_unit_spike_features (line 137) | def get_unit_spike_features(self, unit_id, feature_name, start_frame=N...
method get_unit_spike_feature_names (line 150) | def get_unit_spike_feature_names(self, unit_id):
method get_unit_spike_train (line 154) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method get_unit_ids (line 163) | def get_unit_ids(self):
function _find_site_neighbors (line 167) | def _find_site_neighbors(site_locs, n_neighbors, shank_map):
FILE: spikeextractors/extractors/kilosortextractors/kilosortextractors.py
class KiloSortRecordingExtractor (line 5) | class KiloSortRecordingExtractor(PhyRecordingExtractor):
method __init__ (line 14) | def __init__(self, folder_path):
class KiloSortSortingExtractor (line 18) | class KiloSortSortingExtractor(PhySortingExtractor):
method __init__ (line 25) | def __init__(self, folder_path, exclude_cluster_groups=None, keep_good...
FILE: spikeextractors/extractors/klustaextractors/klustaextractors.py
class KlustaRecordingExtractor (line 27) | class KlustaRecordingExtractor(BinDatRecordingExtractor):
method __init__ (line 36) | def __init__(self, folder_path):
class KlustaSortingExtractor (line 54) | class KlustaSortingExtractor(SortingExtractor):
method __init__ (line 63) | def __init__(self, file_or_folder_path, exclude_cluster_groups=None):
method get_unit_ids (line 146) | def get_unit_ids(self):
method get_unit_spike_train (line 150) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
FILE: spikeextractors/extractors/matsortingextractor/matsortingextractor.py
class MATSortingExtractor (line 33) | class MATSortingExtractor(SortingExtractor):
method __init__ (line 41) | def __init__(self, file_path: PathType):
method __del__ (line 67) | def __del__(self):
method _getfield (line 71) | def _getfield(self, fieldname: str):
method write_dict_to_mat (line 84) | def write_dict_to_mat(mat_file_path, dict_to_write, version='7.3'): #...
FILE: spikeextractors/extractors/maxwellextractors/maxwellextractors.py
class MaxOneRecordingExtractor (line 15) | class MaxOneRecordingExtractor(RecordingExtractor):
method __init__ (line 24) | def __init__(self, file_path, load_spikes=True, rec_name='rec0000'):
method __del__ (line 39) | def __del__(self):
method _initialize (line 42) | def _initialize(self):
method get_channel_ids (line 129) | def get_channel_ids(self):
method get_electrode_ids (line 132) | def get_electrode_ids(self):
method get_num_frames (line 135) | def get_num_frames(self):
method get_sampling_frequency (line 138) | def get_sampling_frequency(self):
method correct_for_missing_frames (line 141) | def correct_for_missing_frames(self, verbose=False):
method _get_frame_numbers (line 176) | def _get_frame_numbers(self):
method _get_frame_number (line 181) | def _get_frame_number(self, index):
method get_traces (line 187) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method get_ttl_events (line 200) | def get_ttl_events(self, start_frame=None, end_frame=None, channel_id=0):
class MaxOneSortingExtractor (line 213) | class MaxOneSortingExtractor(SortingExtractor):
method __init__ (line 220) | def __init__(self, file_path):
method _initialize (line 230) | def _initialize(self):
method get_unit_ids (line 262) | def get_unit_ids(self):
method get_unit_spike_train (line 266) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
class MaxTwoRecordingExtractor (line 278) | class MaxTwoRecordingExtractor(RecordingExtractor):
method __init__ (line 287) | def __init__(self, file_path, well_name='well000', rec_name='rec0000',...
method _initialize (line 303) | def _initialize(self):
method get_channel_ids (line 362) | def get_channel_ids(self):
method get_num_frames (line 365) | def get_num_frames(self):
method get_sampling_frequency (line 368) | def get_sampling_frequency(self):
method get_well_names (line 372) | def get_well_names(file_path):
method get_recording_names (line 378) | def get_recording_names(file_path, well_name):
method get_traces (line 386) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
class MaxTwoSortingExtractor (line 400) | class MaxTwoSortingExtractor(SortingExtractor):
method __init__ (line 407) | def __init__(self, file_path, well_name='well000', rec_name='rec0000'):
method _initialize (line 420) | def _initialize(self):
method get_unit_ids (line 453) | def get_unit_ids(self):
method get_unit_spike_train (line 457) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
FILE: spikeextractors/extractors/mcsh5recordingextractor/mcsh5recordingextractor.py
class MCSH5RecordingExtractor (line 14) | class MCSH5RecordingExtractor(RecordingExtractor):
method __init__ (line 23) | def __init__(self, file_path, stream_id=0, verbose=False):
method __del__ (line 34) | def __del__(self):
method get_channel_ids (line 37) | def get_channel_ids(self):
method get_num_frames (line 40) | def get_num_frames(self):
method get_sampling_frequency (line 43) | def get_sampling_frequency(self):
method set_stream_id (line 46) | def set_stream_id(self, stream_id):
method get_stream_id (line 57) | def get_stream_id(self):
method get_available_stream_ids (line 61) | def get_available_stream_ids(self):
method get_traces (line 70) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
function openMCSH5File (line 94) | def openMCSH5File(filename, stream_id, verbose=False):
FILE: spikeextractors/extractors/mdaextractors/mdaextractors.py
class MdaRecordingExtractor (line 13) | class MdaRecordingExtractor(RecordingExtractor):
method __init__ (line 22) | def __init__(self, folder_path, raw_fname='raw.mda', params_fname='par...
method get_channel_ids (line 43) | def get_channel_ids(self):
method get_num_frames (line 46) | def get_num_frames(self):
method get_sampling_frequency (line 49) | def get_sampling_frequency(self):
method get_traces (line 53) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method write_to_binary_dat_format (line 59) | def write_to_binary_dat_format(self, save_path, time_axis=0, dtype=Non...
method write_recording (line 103) | def write_recording(recording, save_path, params=dict(), raw_fname='ra...
class MdaSortingExtractor (line 165) | class MdaSortingExtractor(SortingExtractor):
method __init__ (line 172) | def __init__(self, file_path, sampling_frequency=None):
method get_unit_ids (line 184) | def get_unit_ids(self):
method get_unit_spike_train (line 188) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method write_sorting (line 195) | def write_sorting(sorting, save_path, write_primary_channels=False):
function _concatenate (line 229) | def _concatenate(list):
function read_dataset_params (line 235) | def read_dataset_params(dsdir, params_fname):
FILE: spikeextractors/extractors/mdaextractors/mdaio.py
class MdaHeader (line 9) | class MdaHeader:
method __init__ (line 10) | def __init__(self, dt0, dims0):
method write (line 24) | def write(self, f):
function npy_dtype_to_string (line 38) | def npy_dtype_to_string(dt):
class DiskReadMda (line 53) | class DiskReadMda:
method __init__ (line 54) | def __init__(self, path, header=None):
method dims (line 68) | def dims(self):
method N1 (line 74) | def N1(self):
method N2 (line 77) | def N2(self):
method N3 (line 80) | def N3(self):
method dt (line 83) | def dt(self):
method numBytesPerEntry (line 89) | def numBytesPerEntry(self):
method readChunk (line 95) | def readChunk(self, i1=-1, i2=-1, i3=-1, N1=1, N2=1, N3=1):
method _read_chunk_1d (line 127) | def _read_chunk_1d(self, i, N):
method _read_chunk_1d_helper (line 138) | def _read_chunk_1d_helper(self, path0, N, *, offset):
function is_url (line 151) | def is_url(path):
function _download_bytes_to_tmpfile (line 155) | def _download_bytes_to_tmpfile(url, start, end):
function _read_header (line 170) | def _read_header(path):
function _dt_from_dt_code (line 224) | def _dt_from_dt_code(dt_code):
function _dt_code_from_dt (line 244) | def _dt_code_from_dt(dt):
function get_num_bytes_per_entry_from_dt (line 262) | def get_num_bytes_per_entry_from_dt(dt):
function readmda_header (line 280) | def readmda_header(path):
function _write_header (line 286) | def _write_header(path, H, rewrite=False):
function readmda (line 310) | def readmda(path):
function writemda32 (line 331) | def writemda32(X, fname):
function writemda64 (line 337) | def writemda64(X, fname):
function writemda8 (line 343) | def writemda8(X, fname):
function writemda32i (line 349) | def writemda32i(X, fname):
function writemda32ui (line 355) | def writemda32ui(X, fname):
function writemda16i (line 361) | def writemda16i(X, fname):
function writemda16ui (line 367) | def writemda16ui(X, fname):
function writemda (line 373) | def writemda(X, fname, *, dtype):
function _writemda (line 377) | def _writemda(X, fname, dt):
function readnpy (line 412) | def readnpy(path):
function writenpy8 (line 416) | def writenpy8(X, path):
function writenpy32 (line 420) | def writenpy32(X, path):
function writenpy64 (line 424) | def writenpy64(X, path):
function writenpy16i (line 428) | def writenpy16i(X, path):
function writenpy16ui (line 432) | def writenpy16ui(X, path):
function writenpy32i (line 436) | def writenpy32i(X, path):
function writenpy32ui (line 440) | def writenpy32ui(X, path):
function writenpy (line 444) | def writenpy(X, path, *, dtype):
function _writenpy (line 448) | def _writenpy(X, path, *, dtype):
function appendmda (line 455) | def appendmda(X, path):
function file_extension (line 485) | def file_extension(fname):
function _read_int32 (line 493) | def _read_int32(f):
function _read_int64 (line 497) | def _read_int64(f):
function _write_int32 (line 501) | def _write_int32(f, val):
function _write_int64 (line 505) | def _write_int64(f, val):
function _header_from_file (line 509) | def _header_from_file(f):
FILE: spikeextractors/extractors/mearecextractors/mearecextractors.py
class MEArecRecordingExtractor (line 22) | class MEArecRecordingExtractor(RecordingExtractor):
method __init__ (line 31) | def __init__(self, file_path, locs_2d=True):
method _initialize (line 48) | def _initialize(self):
method get_channel_ids (line 69) | def get_channel_ids(self):
method get_num_frames (line 72) | def get_num_frames(self):
method get_sampling_frequency (line 75) | def get_sampling_frequency(self):
method get_traces (line 79) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method write_recording (line 91) | def write_recording(recording, save_path, check_suffix=True):
class MEArecSortingExtractor (line 118) | class MEArecSortingExtractor(SortingExtractor):
method __init__ (line 125) | def __init__(self, file_path):
method _initialize (line 136) | def _initialize(self):
method get_unit_ids (line 152) | def get_unit_ids(self):
method get_num_units (line 157) | def get_num_units(self):
method get_unit_spike_train (line 163) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method write_sorting (line 172) | def write_sorting(sorting, save_path, sampling_frequency, check_suffix...
FILE: spikeextractors/extractors/neoextractors/axonaextractor.py
class AxonaRecordingExtractor (line 11) | class AxonaRecordingExtractor(NeoBaseRecordingExtractor):
method __init__ (line 16) | def __init__(self, **kargs):
FILE: spikeextractors/extractors/neoextractors/blackrockextractor.py
class BlackrockRecordingExtractor (line 15) | class BlackrockRecordingExtractor(NeoBaseRecordingExtractor):
method __init__ (line 34) | def __init__(self, filename: PathType, nsx_to_load: Optional[int] = No...
class BlackrockSortingExtractor (line 40) | class BlackrockSortingExtractor(NeoBaseSortingExtractor):
method __init__ (line 59) | def __init__(self, filename: PathType, nsx_to_load: Optional[int] = None,
FILE: spikeextractors/extractors/neoextractors/mcsrawrecordingextractor.py
class MCSRawRecordingExtractor (line 10) | class MCSRawRecordingExtractor(NeoBaseRecordingExtractor):
FILE: spikeextractors/extractors/neoextractors/neobaseextractor.py
class _NeoBaseExtractor (line 16) | class _NeoBaseExtractor:
method __init__ (line 24) | def __init__(self, block_index=None, seg_index=None, **kargs):
class NeoBaseRecordingExtractor (line 54) | class NeoBaseRecordingExtractor(RecordingExtractor, _NeoBaseExtractor):
method __init__ (line 56) | def __init__(self, block_index=None, seg_index=None, **kargs):
method get_traces (line 119) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method get_num_frames (line 138) | def get_num_frames(self):
method get_sampling_frequency (line 146) | def get_sampling_frequency(self):
method get_channel_ids (line 154) | def get_channel_ids(self):
class NeoBaseSortingExtractor (line 158) | class NeoBaseSortingExtractor(SortingExtractor, _NeoBaseExtractor):
method __init__ (line 159) | def __init__(self, block_index=None, seg_index=None, **kargs):
method _handle_sampling_frequency (line 174) | def _handle_sampling_frequency(self):
method get_unit_ids (line 220) | def get_unit_ids(self):
method get_unit_spike_train (line 234) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
FILE: spikeextractors/extractors/neoextractors/neuralynxextractor.py
class NeuralynxRecordingExtractor (line 10) | class NeuralynxRecordingExtractor(NeoBaseRecordingExtractor):
class NeuralynxSortingExtractor (line 30) | class NeuralynxSortingExtractor(NeoBaseSortingExtractor):
FILE: spikeextractors/extractors/neoextractors/plexonextractor.py
class PlexonRecordingExtractor (line 9) | class PlexonRecordingExtractor(NeoBaseRecordingExtractor):
class PlexonSortingExtractor (line 28) | class PlexonSortingExtractor(NeoBaseSortingExtractor):
FILE: spikeextractors/extractors/neoextractors/spikegadgetsextractor.py
class SpikeGadgetsRecordingExtractor (line 9) | class SpikeGadgetsRecordingExtractor(NeoBaseRecordingExtractor):
method __init__ (line 29) | def __init__(self, filename, selected_streams='trodes',**kwargs):
FILE: spikeextractors/extractors/neuropixelsdatrecordingextractor/neuropixelsdatrecordingextractor.py
class NeuropixelsDatRecordingExtractor (line 13) | class NeuropixelsDatRecordingExtractor(BinDatRecordingExtractor):
method __init__ (line 44) | def __init__(self, file_path, settings_file=None, is_filtered=None, ve...
FILE: spikeextractors/extractors/neuroscopeextractors/neuroscopeextractors.py
function get_single_files (line 22) | def get_single_files(folder_path: Path, suffix: str):
function get_shank_files (line 29) | def get_shank_files(folder_path: Path, suffix: str):
function find_xml_file_path (line 35) | def find_xml_file_path(folder_path: PathType):
function handle_xml_file_path (line 42) | def handle_xml_file_path(folder_path: PathType, initial_xml_file_path: P...
class NeuroscopeRecordingExtractor (line 51) | class NeuroscopeRecordingExtractor(BinDatRecordingExtractor):
method __init__ (line 77) | def __init__(self, file_path: PathType, gain: Optional[float] = None, ...
method write_recording (line 101) | def write_recording(
class NeuroscopeMultiRecordingTimeExtractor (line 173) | class NeuroscopeMultiRecordingTimeExtractor(MultiRecordingTimeExtractor):
method __init__ (line 197) | def __init__(self, folder_path: PathType, gain: Optional[float] = None...
method write_recording (line 210) | def write_recording(
class NeuroscopeSortingExtractor (line 302) | class NeuroscopeSortingExtractor(SortingExtractor):
method __init__ (line 345) | def __init__(
method get_unit_ids (line 461) | def get_unit_ids(self):
method get_sampling_frequency (line 464) | def get_sampling_frequency(self):
method shift_unit_ids (line 467) | def shift_unit_ids(self, shift):
method add_unit (line 470) | def add_unit(self, unit_id, spike_times):
method get_unit_spike_train (line 482) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method write_sorting (line 489) | def write_sorting(sorting: SortingExtractor, save_path: PathType):
class NeuroscopeMultiSortingExtractor (line 523) | class NeuroscopeMultiSortingExtractor(MultiSortingExtractor):
method __init__ (line 566) | def __init__(
method write_sorting (line 653) | def write_sorting(sorting: Union[MultiSortingExtractor, SortingExtract...
function _extract_res_clu_arrays (line 703) | def _extract_res_clu_arrays(sorting):
FILE: spikeextractors/extractors/nixioextractors/nixioextractors.py
class NIXIORecordingExtractor (line 16) | class NIXIORecordingExtractor(RecordingExtractor):
method __init__ (line 25) | def __init__(self, file_path):
method __del__ (line 33) | def __del__(self):
method _traces (line 37) | def _traces(self):
method get_channel_ids (line 42) | def get_channel_ids(self):
method get_num_frames (line 48) | def get_num_frames(self):
method get_sampling_frequency (line 52) | def get_sampling_frequency(self):
method get_traces (line 59) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method _load_properties (line 63) | def _load_properties(self):
method write_recording (line 80) | def write_recording(recording, save_path, overwrite=False):
class NIXIOSortingExtractor (line 141) | class NIXIOSortingExtractor(SortingExtractor):
method __init__ (line 148) | def __init__(self, file_path):
method __del__ (line 160) | def __del__(self):
method _spike_das (line 164) | def _spike_das(self):
method get_unit_ids (line 168) | def get_unit_ids(self):
method get_unit_spike_train (line 172) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method _load_properties (line 180) | def _load_properties(self):
method write_sorting (line 197) | def write_sorting(sorting, save_path, overwrite=False):
FILE: spikeextractors/extractors/npzsortingextractor/npzsortingextractor.py
class NpzSortingExtractor (line 7) | class NpzSortingExtractor(SortingExtractor):
method __init__ (line 23) | def __init__(self, file_path):
method get_unit_ids (line 39) | def get_unit_ids(self):
method get_unit_spike_train (line 43) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method write_sorting (line 52) | def write_sorting(sorting, save_path):
FILE: spikeextractors/extractors/numpyextractors/numpyextractors.py
class NumpyRecordingExtractor (line 13) | class NumpyRecordingExtractor(RecordingExtractor):
method __init__ (line 19) | def __init__(self, timeseries, sampling_frequency, geom=None):
method set_ttls (line 45) | def set_ttls(self, ttl_frames, ttl_states=None):
method get_channel_ids (line 52) | def get_channel_ids(self):
method get_num_frames (line 55) | def get_num_frames(self):
method get_sampling_frequency (line 58) | def get_sampling_frequency(self):
method get_traces (line 62) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method get_ttl_events (line 67) | def get_ttl_events(self, start_frame=None, end_frame=None, channel_id=0):
method write_recording (line 76) | def write_recording(recording, save_path):
class NumpySortingExtractor (line 81) | class NumpySortingExtractor(SortingExtractor):
method __init__ (line 85) | def __init__(self):
method load_from_extractor (line 90) | def load_from_extractor(self, sorting, copy_unit_properties=False, cop...
method set_sampling_frequency (line 112) | def set_sampling_frequency(self, sampling_frequency):
method set_times_labels (line 115) | def set_times_labels(self, times, labels):
method add_unit (line 131) | def add_unit(self, unit_id, times):
method get_unit_ids (line 143) | def get_unit_ids(self):
method get_unit_spike_train (line 147) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
FILE: spikeextractors/extractors/nwbextractors/nwbextractors.py
function check_nwb_install (line 31) | def check_nwb_install():
function set_dynamic_table_property (line 35) | def set_dynamic_table_property(dynamic_table, row_ids, property_name, va...
function get_dynamic_table_property (line 77) | def get_dynamic_table_property(dynamic_table, *, row_ids=None, property_...
function get_nspikes (line 84) | def get_nspikes(units_table, unit_id):
function most_relevant_ch (line 98) | def most_relevant_ch(traces: ArrayType):
function update_dict (line 120) | def update_dict(d: dict, u: dict):
function list_get (line 131) | def list_get(li: list, idx: int, default):
function check_module (line 139) | def check_module(nwbfile, name: str, description: str = None):
class NwbRecordingExtractor (line 162) | class NwbRecordingExtractor(se.RecordingExtractor):
method __init__ (line 173) | def __init__(self, file_path: PathType, electrical_series_name: str = ...
method make_nwb_metadata (line 276) | def make_nwb_metadata(self, nwbfile, es):
method get_traces (line 307) | def get_traces(
method get_sampling_frequency (line 329) | def get_sampling_frequency(self):
method get_num_frames (line 332) | def get_num_frames(self):
method get_channel_ids (line 335) | def get_channel_ids(self):
method add_devices (line 339) | def add_devices(recording: se.RecordingExtractor, nwbfile=None, metada...
method add_electrode_groups (line 385) | def add_electrode_groups(recording: se.RecordingExtractor, nwbfile=Non...
method add_electrodes (line 472) | def add_electrodes(recording: se.RecordingExtractor, nwbfile=None, met...
method add_electrical_series (line 654) | def add_electrical_series(
method add_epochs (line 849) | def add_epochs(
method add_all_to_nwbfile (line 891) | def add_all_to_nwbfile(
method write_recording (line 968) | def write_recording(
method get_nwb_metadata (line 1095) | def get_nwb_metadata(recording: se.RecordingExtractor, metadata: dict ...
class NwbSortingExtractor (line 1127) | class NwbSortingExtractor(se.SortingExtractor):
method __init__ (line 1134) | def __init__(self, file_path, electrical_series=None, sampling_frequen...
method get_unit_ids (line 1205) | def get_unit_ids(self):
method get_unit_spike_train (line 1219) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method write_units (line 1231) | def write_units(
method write_sorting (line 1411) | def write_sorting(
FILE: spikeextractors/extractors/openephysextractors/openephysextractors.py
class OpenEphysRecordingExtractor (line 26) | class OpenEphysRecordingExtractor(RecordingExtractor):
method __init__ (line 35) | def __init__(self, folder_path, experiment_id=0, recording_id=0):
method _set_analogsignal (line 46) | def _set_analogsignal(self, analogsignals):
method get_channel_ids (line 54) | def get_channel_ids(self):
method get_num_frames (line 60) | def get_num_frames(self):
method get_sampling_frequency (line 63) | def get_sampling_frequency(self):
method get_traces (line 70) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method get_ttl_events (line 74) | def get_ttl_events(self, start_frame=None, end_frame=None, channel_id=0):
class OpenEphysNPIXRecordingExtractor (line 85) | class OpenEphysNPIXRecordingExtractor(OpenEphysRecordingExtractor):
method __init__ (line 94) | def __init__(self, folder_path, experiment_id=0, recording_id=0, strea...
class OpenEphysSortingExtractor (line 120) | class OpenEphysSortingExtractor(SortingExtractor):
method __init__ (line 127) | def __init__(self, folder_path, experiment_id=0, recording_id=0):
method get_unit_ids (line 139) | def get_unit_ids(self):
method get_unit_spike_train (line 143) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
FILE: spikeextractors/extractors/phyextractors/phyextractors.py
class PhyRecordingExtractor (line 13) | class PhyRecordingExtractor(BinDatRecordingExtractor):
method __init__ (line 30) | def __init__(self, folder_path: PathType):
class PhySortingExtractor (line 63) | class PhySortingExtractor(SortingExtractor):
method __init__ (line 80) | def __init__(self, folder_path: PathType, exclude_cluster_groups: Opti...
method get_unit_ids (line 178) | def get_unit_ids(self):
method get_unit_spike_train (line 182) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
FILE: spikeextractors/extractors/shybridextractors/shybridextractors.py
class SHYBRIDRecordingExtractor (line 18) | class SHYBRIDRecordingExtractor(RecordingExtractor):
method __init__ (line 28) | def __init__(self, file_path):
method get_channel_ids (line 58) | def get_channel_ids(self):
method get_num_frames (line 61) | def get_num_frames(self):
method get_sampling_frequency (line 64) | def get_sampling_frequency(self):
method get_traces (line 67) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method write_recording (line 72) | def write_recording(recording, save_path, initial_sorting_fn, dtype='f...
class SHYBRIDSortingExtractor (line 121) | class SHYBRIDSortingExtractor(SortingExtractor):
method __init__ (line 127) | def __init__(self, file_path, delimiter=','):
method get_unit_ids (line 138) | def get_unit_ids(self):
method get_unit_spike_train (line 142) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method write_sorting (line 148) | def write_sorting(sorting, save_path):
class GeometryNotLoadedError (line 172) | class GeometryNotLoadedError(Exception):
FILE: spikeextractors/extractors/spikeglxrecordingextractor/readSGLX.py
function readMeta (line 38) | def readMeta(binFullPath):
function SampRate (line 63) | def SampRate(meta):
function Int2Volts (line 77) | def Int2Volts(meta):
function OriginalChans (line 93) | def OriginalChans(meta):
function ChannelCountsNI (line 117) | def ChannelCountsNI(meta):
function ChannelCountsIM (line 129) | def ChannelCountsIM(meta):
function ChanGainNI (line 140) | def ChanGainNI(ichan, savedMN, savedMA, meta):
function ChanGainsIM (line 153) | def ChanGainsIM(meta):
function GainCorrectNI (line 175) | def GainCorrectNI(dataArray, chanList, meta):
function GainCorrectIM (line 204) | def GainCorrectIM(dataArray, chanList, meta):
function makeMemMapRaw (line 232) | def makeMemMapRaw(binFullPath, meta):
function ExtractDigital (line 249) | def ExtractDigital(rawData, firstSamp, lastSamp, dwReq, dLineList, meta):
FILE: spikeextractors/extractors/spikeglxrecordingextractor/spikeglxrecordingextractor.py
class SpikeGLXRecordingExtractor (line 9) | class SpikeGLXRecordingExtractor(RecordingExtractor):
method __init__ (line 32) | def __init__(self, file_path: str, x_pitch: int = 32, y_pitch: int = 20):
method get_channel_ids (line 105) | def get_channel_ids(self):
method get_num_frames (line 108) | def get_num_frames(self):
method get_sampling_frequency (line 111) | def get_sampling_frequency(self):
method get_traces (line 115) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method get_ttl_events (line 129) | def get_ttl_events(self, start_frame=None, end_frame=None, channel_id=0):
function _parse_spikeglx_metafile (line 146) | def _parse_spikeglx_metafile(metafile, x_pitch, y_pitch, rec_type):
FILE: spikeextractors/extractors/spykingcircusextractors/spykingcircusextractors.py
class SpykingCircusRecordingExtractor (line 15) | class SpykingCircusRecordingExtractor(RecordingExtractor):
method __init__ (line 32) | def __init__(self, folder_path):
method get_channel_ids (line 83) | def get_channel_ids(self):
method get_num_frames (line 86) | def get_num_frames(self):
method get_sampling_frequency (line 89) | def get_sampling_frequency(self):
method get_traces (line 92) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
class SpykingCircusSortingExtractor (line 97) | class SpykingCircusSortingExtractor(SortingExtractor):
method __init__ (line 114) | def __init__(self, file_or_folder_path, load_templates=False):
method get_unit_ids (line 198) | def get_unit_ids(self):
method get_unit_spike_train (line 202) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method write_sorting (line 209) | def write_sorting(sorting, save_path):
function _load_params (line 228) | def _load_params(params_file):
FILE: spikeextractors/extractors/tridescloussortingextractor/tridescloussortingextractor.py
class TridesclousSortingExtractor (line 13) | class TridesclousSortingExtractor(SortingExtractor):
method __init__ (line 20) | def __init__(self, folder_path, chan_grp=None):
method get_unit_ids (line 45) | def get_unit_ids(self):
method get_unit_spike_train (line 49) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
FILE: spikeextractors/extractors/waveclussortingextractor/waveclussortingextractor.py
class WaveClusSortingExtractor (line 12) | class WaveClusSortingExtractor(MATSortingExtractor):
method __init__ (line 16) | def __init__(self, file_path: PathType):
method get_unit_spike_train (line 34) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method get_unit_ids (line 42) | def get_unit_ids(self):
method get_unsorted_spike_train (line 45) | def get_unsorted_spike_train(self, start_frame=None, end_frame=None):
FILE: spikeextractors/extractors/yassextractors/yassextractors.py
class YassSortingExtractor (line 15) | class YassSortingExtractor(SortingExtractor):
method __init__ (line 26) | def __init__(self, folder_path):
method get_unit_ids (line 49) | def get_unit_ids(self):
method get_temps (line 58) | def get_temps(self):
method get_unit_spike_train (line 67) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
FILE: spikeextractors/multirecordingchannelextractor.py
class MultiRecordingChannelExtractor (line 8) | class MultiRecordingChannelExtractor(RecordingExtractor):
method __init__ (line 9) | def __init__(self, recordings, groups=None):
method recordings (line 93) | def recordings(self):
method get_traces (line 97) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method get_channel_ids (line 113) | def get_channel_ids(self):
method get_num_frames (line 116) | def get_num_frames(self):
method get_sampling_frequency (line 119) | def get_sampling_frequency(self):
function concatenate_recordings_by_channel (line 123) | def concatenate_recordings_by_channel(recordings, groups=None):
FILE: spikeextractors/multirecordingtimeextractor.py
class MultiRecordingTimeExtractor (line 7) | class MultiRecordingTimeExtractor(RecordingExtractor):
method __init__ (line 8) | def __init__(self, recordings, epoch_names=None):
method recordings (line 67) | def recordings(self):
method _find_section_for_frame (line 70) | def _find_section_for_frame(self, frame):
method _find_section_for_time (line 81) | def _find_section_for_time(self, time):
method get_traces (line 93) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method get_ttl_events (line 117) | def get_ttl_events(self, start_frame=None, end_frame=None, channel_id=0):
method get_channel_ids (line 155) | def get_channel_ids(self):
method get_num_frames (line 158) | def get_num_frames(self):
method get_sampling_frequency (line 161) | def get_sampling_frequency(self):
method frame_to_time (line 164) | def frame_to_time(self, frame):
method time_to_frame (line 168) | def time_to_frame(self, time):
function concatenate_recordings_by_time (line 173) | def concatenate_recordings_by_time(recordings, epoch_names=None):
FILE: spikeextractors/multisortingextractor.py
class MultiSortingExtractor (line 7) | class MultiSortingExtractor(SortingExtractor):
method __init__ (line 8) | def __init__(self, sortings):
method sortings (line 24) | def sortings(self):
method get_unit_ids (line 27) | def get_unit_ids(self):
method get_unit_spike_train (line 31) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method set_sampling_frequency (line 36) | def set_sampling_frequency(self, sampling_frequency):
method get_sampling_frequency (line 40) | def get_sampling_frequency(self):
method set_unit_property (line 43) | def set_unit_property(self, unit_id, property_name, value):
method get_unit_property (line 50) | def get_unit_property(self, unit_id, property_name):
method get_unit_property_names (line 57) | def get_unit_property_names(self, unit_id):
method clear_unit_property (line 63) | def clear_unit_property(self, unit_id, property_name):
method get_unit_spike_features (line 70) | def get_unit_spike_features(self, unit_id, feature_name, start_frame=N...
method get_unit_spike_feature_names (line 78) | def get_unit_spike_feature_names(self, unit_id):
method set_unit_spike_features (line 92) | def set_unit_spike_features(self, unit_id, feature_name, value, indexe...
method clear_unit_spike_features (line 99) | def clear_unit_spike_features(self, unit_id, feature_name):
function concatenate_sortings (line 107) | def concatenate_sortings(sortings):
FILE: spikeextractors/recordingextractor.py
class RecordingExtractor (line 10) | class RecordingExtractor(ABC, BaseExtractor):
method __init__ (line 19) | def __init__(self):
method get_traces (line 25) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method get_num_frames (line 63) | def get_num_frames(self):
method get_sampling_frequency (line 74) | def get_sampling_frequency(self):
method get_channel_ids (line 85) | def get_channel_ids(self):
method get_num_channels (line 95) | def get_num_channels(self):
method get_dtype (line 105) | def get_dtype(self, return_scaled=True):
method set_times (line 122) | def set_times(self, times):
method copy_times (line 134) | def copy_times(self, extractor):
method frame_to_time (line 145) | def frame_to_time(self, frames):
method time_to_frame (line 164) | def time_to_frame(self, times):
method get_snippets (line 183) | def get_snippets(self, reference_frames, snippet_len, channel_ids=None...
method set_channel_locations (line 249) | def set_channel_locations(self, locations, channel_ids=None):
method get_channel_locations (line 286) | def get_channel_locations(self, channel_ids=None, locations_2d=True):
method clear_channel_locations (line 318) | def clear_channel_locations(self, channel_ids=None):
method set_channel_groups (line 334) | def set_channel_groups(self, groups, channel_ids=None):
method get_channel_groups (line 364) | def get_channel_groups(self, channel_ids=None):
method clear_channel_groups (line 391) | def clear_channel_groups(self, channel_ids=None):
method set_channel_gains (line 407) | def set_channel_gains(self, gains, channel_ids=None):
method get_channel_gains (line 438) | def get_channel_gains(self, channel_ids=None):
method clear_channel_gains (line 464) | def clear_channel_gains(self, channel_ids=None):
method set_channel_offsets (line 480) | def set_channel_offsets(self, offsets, channel_ids=None):
method get_channel_offsets (line 511) | def get_channel_offsets(self, channel_ids=None):
method clear_channel_offsets (line 537) | def clear_channel_offsets(self, channel_ids=None):
method set_channel_property (line 553) | def set_channel_property(self, channel_id, property_name, value):
method get_channel_property (line 584) | def get_channel_property(self, channel_id, property_name):
method get_channel_property_names (line 621) | def get_channel_property_names(self, channel_id):
method get_shared_channel_property_names (line 651) | def get_shared_channel_property_names(self, channel_ids=None):
method copy_channel_properties (line 675) | def copy_channel_properties(self, recording, channel_ids=None):
method clear_channel_property (line 710) | def clear_channel_property(self, channel_id, property_name):
method clear_channels_property (line 728) | def clear_channels_property(self, property_name, channel_ids=None):
method get_epoch (line 743) | def get_epoch(self, epoch_name):
method load_probe_file (line 765) | def load_probe_file(self, probe_file, channel_map=None, channel_groups...
method save_to_probe_file (line 793) | def save_to_probe_file(self, probe_file, grouping_property=None, radiu...
method write_to_binary_dat_format (line 815) | def write_to_binary_dat_format(self, save_path, time_axis=0, dtype=Non...
method write_to_h5_dataset_format (line 846) | def write_to_h5_dataset_format(self, dataset_path, save_path=None, fil...
method get_sub_extractors_by_property (line 875) | def get_sub_extractors_by_property(self, property_name, return_propert...
method get_ttl_events (line 904) | def get_ttl_events(self, start_frame=None, end_frame=None, channel_id=0):
method write_recording (line 927) | def write_recording(recording, save_path):
FILE: spikeextractors/save_tools.py
function save_si_object (line 8) | def save_si_object(object_name: str, si_object, output_folder,
FILE: spikeextractors/sortingextractor.py
class SortingExtractor (line 9) | class SortingExtractor(ABC, BaseExtractor):
method __init__ (line 18) | def __init__(self):
method get_unit_ids (line 23) | def get_unit_ids(self):
method get_unit_spike_train (line 34) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method get_units_spike_train (line 66) | def get_units_spike_train(self, unit_ids=None, start_frame=None, end_f...
method get_sampling_frequency (line 90) | def get_sampling_frequency(self):
method set_sampling_frequency (line 101) | def set_sampling_frequency(self, sampling_frequency):
method set_unit_spike_features (line 112) | def set_unit_spike_features(self, unit_id, feature_name, value, indexe...
method get_unit_spike_features (line 157) | def get_unit_spike_features(self, unit_id, feature_name, start_frame=N...
method set_times (line 241) | def set_times(self, times):
method copy_times (line 254) | def copy_times(self, extractor):
method frame_to_time (line 265) | def frame_to_time(self, frames):
method time_to_frame (line 284) | def time_to_frame(self, times):
method clear_unit_spike_features (line 303) | def clear_unit_spike_features(self, unit_id, feature_name):
method clear_units_spike_features (line 317) | def clear_units_spike_features(self, feature_name, unit_ids=None):
method get_unit_spike_feature_names (line 332) | def get_unit_spike_feature_names(self, unit_id):
method get_shared_unit_spike_feature_names (line 356) | def get_shared_unit_spike_feature_names(self, unit_ids=None):
method set_unit_property (line 382) | def set_unit_property(self, unit_id, property_name, value):
method set_units_property (line 409) | def set_units_property(self, *, unit_ids=None, property_name, values):
method get_unit_property (line 427) | def get_unit_property(self, unit_id, property_name):
method get_units_property (line 460) | def get_units_property(self, *, unit_ids=None, property_name):
method get_unit_property_names (line 482) | def get_unit_property_names(self, unit_id):
method get_shared_unit_property_names (line 506) | def get_shared_unit_property_names(self, unit_ids=None):
method copy_unit_properties (line 532) | def copy_unit_properties(self, sorting, unit_ids=None):
method clear_unit_property (line 561) | def clear_unit_property(self, unit_id, property_name):
method clear_units_property (line 575) | def clear_units_property(self, property_name, unit_ids=None):
method copy_unit_spike_features (line 590) | def copy_unit_spike_features(self, sorting, unit_ids=None):
method get_epoch (line 623) | def get_epoch(self, epoch_name):
method get_sub_extractors_by_property (line 643) | def get_sub_extractors_by_property(self, property_name, return_propert...
method write_sorting (line 669) | def write_sorting(sorting, save_path):
method get_unsorted_spike_train (line 687) | def get_unsorted_spike_train(self, start_frame=None, end_frame=None):
FILE: spikeextractors/subrecordingextractor.py
class SubRecordingExtractor (line 7) | class SubRecordingExtractor(RecordingExtractor):
method __init__ (line 8) | def __init__(self, parent_recording, *, channel_ids=None, renamed_chan...
method get_traces (line 46) | def get_traces(self, channel_ids=None, start_frame=None, end_frame=Non...
method get_ttl_events (line 54) | def get_ttl_events(self, start_frame=None, end_frame=None, channel_id=0):
method get_channel_ids (line 66) | def get_channel_ids(self):
method get_num_frames (line 69) | def get_num_frames(self):
method get_sampling_frequency (line 72) | def get_sampling_frequency(self):
method frame_to_time (line 75) | def frame_to_time(self, frame):
method time_to_frame (line 81) | def time_to_frame(self, time):
method get_snippets (line 87) | def get_snippets(self, reference_frames, snippet_len, channel_ids=None...
method copy_channel_properties (line 95) | def copy_channel_properties(self, recording, channel_ids=None):
method get_original_channel_ids (line 138) | def get_original_channel_ids(self, channel_ids):
FILE: spikeextractors/subsortingextractor.py
class SubSortingExtractor (line 7) | class SubSortingExtractor(SortingExtractor):
method __init__ (line 8) | def __init__(self, parent_sorting, *, unit_ids=None, renamed_unit_ids=...
method get_unit_ids (line 33) | def get_unit_ids(self):
method get_unit_spike_train (line 37) | def get_unit_spike_train(self, unit_id, start_frame=None, end_frame=No...
method get_sampling_frequency (line 50) | def get_sampling_frequency(self):
method frame_to_time (line 53) | def frame_to_time(self, frame):
method time_to_frame (line 59) | def time_to_frame(self, time):
method copy_unit_properties (line 65) | def copy_unit_properties(self, sorting, unit_ids=None):
method copy_unit_spike_features (line 86) | def copy_unit_spike_features(self, sorting, unit_ids=None, start_frame...
method get_original_unit_ids (line 123) | def get_original_unit_ids(self, unit_ids):
FILE: spikeextractors/testing.py
function check_recordings_equal (line 12) | def check_recordings_equal(RX1, RX2, return_scaled=True, force_dtype=Non...
function check_recording_properties (line 58) | def check_recording_properties(RX1, RX2):
function check_recording_return_types (line 70) | def check_recording_return_types(RX):
function check_sorting_return_types (line 81) | def check_sorting_return_types(SX):
function check_sortings_equal (line 90) | def check_sortings_equal(SX1, SX2):
function check_sorting_properties_features (line 101) | def check_sorting_properties_features(SX1, SX2):
function check_dumping (line 123) | def check_dumping(extractor, test_relative=False):
function get_default_nwbfile_metadata (line 216) | def get_default_nwbfile_metadata():
FILE: tests/test_extractors.py
class TestExtractors (line 15) | class TestExtractors(unittest.TestCase):
method setUp (line 16) | def setUp(self):
method tearDown (line 21) | def tearDown(self):
method _create_example (line 27) | def _create_example(self, seed):
method test_example (line 112) | def test_example(self):
method test_allocate_arrays (line 185) | def test_allocate_arrays(self):
method test_cache_extractor (line 209) | def test_cache_extractor(self):
method test_not_dumpable_exception (line 270) | def test_not_dumpable_exception(self):
method test_mda_extractor (line 281) | def test_mda_extractor(self):
method test_hdsort_extractor (line 295) | def test_hdsort_extractor(self):
method test_npz_extractor (line 304) | def test_npz_extractor(self):
method test_biocam_extractor (line 318) | def test_biocam_extractor(self):
method test_mearec_extractors (line 326) | def test_mearec_extractors(self):
method test_hs2_extractor (line 342) | def test_hs2_extractor(self):
method test_exdir_extractors (line 351) | def test_exdir_extractors(self):
method test_spykingcircus_extractor (line 366) | def test_spykingcircus_extractor(self):
method test_multi_sub_recording_extractor (line 374) | def test_multi_sub_recording_extractor(self):
method test_ttl_frames_in_sub_multi (line 427) | def test_ttl_frames_in_sub_multi(self):
method test_multi_sub_sorting_extractor (line 447) | def test_multi_sub_sorting_extractor(self):
method test_dump_load_multi_sub_extractor (line 466) | def test_dump_load_multi_sub_extractor(self):
method test_nwb_extractor (line 487) | def test_nwb_extractor(self):
method test_nixio_extractor (line 584) | def test_nixio_extractor(self):
method test_shybrid_extractors (line 603) | def test_shybrid_extractors(self):
method test_neuroscope_extractors (line 622) | def test_neuroscope_extractors(self):
method test_cell_explorer_extractor (line 700) | def test_cell_explorer_extractor(self):
FILE: tests/test_gin_repo.py
class TestNwbConversions (line 18) | class TestNwbConversions(unittest.TestCase):
method setUp (line 20) | def setUp(self):
method test_convert_recording_extractor_to_nwb (line 141) | def test_convert_recording_extractor_to_nwb(self, se_class, dataset_pa...
method test_convert_sorting_extractor_to_nwb (line 235) | def test_convert_sorting_extractor_to_nwb(self, se_class, dataset_path...
FILE: tests/test_numpy_extractors.py
class TestNumpyExtractors (line 6) | class TestNumpyExtractors(unittest.TestCase):
method setUp (line 7) | def setUp(self):
method tearDown (line 28) | def tearDown(self):
method test_recording_extractor (line 31) | def test_recording_extractor(self):
method test_sorting_extractor (line 55) | def test_sorting_extractor(self):
FILE: tests/test_tools.py
class TestTools (line 13) | class TestTools(unittest.TestCase):
method setUp (line 14) | def setUp(self):
method tearDown (line 26) | def tearDown(self):
method test_load_save_probes (line 29) | def test_load_save_probes(self):
method test_write_dat_file (line 64) | def test_write_dat_file(self):
Condensed preview — 118 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (744K chars).
[
{
"path": ".github/workflows/python-package.yml",
"chars": 1620,
"preview": "name: Python Package using Conda\n\non: \n push: \n branches:\n - master\n pull_request:\n branches: [master]\n "
},
{
"path": ".github/workflows/python-publish.yml",
"chars": 1885,
"preview": "# This workflow will upload a Python Package using Twine when a release is created\n# For more information see: https://h"
},
{
"path": ".gitignore",
"chars": 84,
"preview": ".eggs\n*.egg-info\n.ipynb_checkpoints\n__pycache__\nsample_*_dataset\nephy_testing_data/\n"
},
{
"path": "LICENSE",
"chars": 1071,
"preview": "MIT License\n\nCopyright (c) 2019 SpikeInterface\n\nPermission is hereby granted, free of charge, to any person obtaining a "
},
{
"path": "MANIFEST.in",
"chars": 102,
"preview": "include spikeextractors/extractors/neuropixelsdatrecordingextractor/channel_positions_neuropixels.txt\n"
},
{
"path": "README.md",
"chars": 337,
"preview": "# SpikeExtractors (LEGACY)\nThe `spikeextractors` package has now been integrated into [spikeinterface](https://github.co"
},
{
"path": "environment-dev.yml",
"chars": 284,
"preview": "name: test\ndependencies:\n - python=3.8\n - pip\n - pip:\n - numpy==1.22.0\n - tqdm\n - lxml\n - h5py\n - shyb"
},
{
"path": "full_requirements.txt",
"chars": 214,
"preview": "h5py #>=3.2.1\nscipy>=1.6.3\npyintan>=0.3.0\npyopenephys>=1.1.4\nneo>=0.9.0\nMEArec<1.8\npynwb>=1.4\nlxml>=4.6.3\nnixio==1.5.0\ns"
},
{
"path": "requirements-dev.txt",
"chars": 31,
"preview": "datalad\nparameterized\nneo==0.10"
},
{
"path": "requirements.txt",
"chars": 28,
"preview": "numpy==1.22.0\ntqdm\npackaging"
},
{
"path": "setup.py",
"chars": 1239,
"preview": "import setuptools\n\nd = {}\nexec(open(\"spikeextractors/version.py\").read(), None, d)\nversion = d['version']\npkg_name = \"sp"
},
{
"path": "spikeextractors/__init__.py",
"chars": 979,
"preview": "from .recordingextractor import RecordingExtractor\nfrom .sortingextractor import SortingExtractor\nfrom .cacheextractors "
},
{
"path": "spikeextractors/baseextractor.py",
"chars": 24122,
"preview": "import json\nfrom pathlib import Path\nimport importlib\nimport numpy as np\nimport datetime\nfrom copy import deepcopy\nimpor"
},
{
"path": "spikeextractors/cacheextractors.py",
"chars": 9188,
"preview": "from spikeextractors.extractors.bindatrecordingextractor import BinDatRecordingExtractor\nfrom spikeextractors.extractors"
},
{
"path": "spikeextractors/example_datasets/__init__.py",
"chars": 37,
"preview": "from .toy_example import toy_example\n"
},
{
"path": "spikeextractors/example_datasets/synthesize_random_firings.py",
"chars": 2120,
"preview": "import numpy as np\n\n\ndef synthesize_random_firings(*, K=20, sampling_frequency=30000.0, duration=60, seed=None):\n if "
},
{
"path": "spikeextractors/example_datasets/synthesize_random_waveforms.py",
"chars": 2947,
"preview": "import numpy as np\nfrom .synthesize_single_waveform import synthesize_single_waveform\n\n\ndef synthesize_random_waveforms("
},
{
"path": "spikeextractors/example_datasets/synthesize_single_waveform.py",
"chars": 2611,
"preview": "import numpy as np\n\n\ndef exp_growth(amp1, amp2, dur1, dur2):\n t = np.arange(0, dur1)\n Y = np.exp(t / dur2)\n # W"
},
{
"path": "spikeextractors/example_datasets/synthesize_timeseries.py",
"chars": 1209,
"preview": "import numpy as np\n\n\ndef synthesize_timeseries(*, sorting, waveforms, noise_level=1, sampling_frequency=30000.0, duratio"
},
{
"path": "spikeextractors/example_datasets/toy_example.py",
"chars": 2732,
"preview": "import numpy as np\nfrom pathlib import Path\nfrom typing import Optional, Union\n\nimport spikeextractors as se\nfrom .synth"
},
{
"path": "spikeextractors/exceptions.py",
"chars": 105,
"preview": "class NotDumpableExtractorError(TypeError):\n \"\"\"Raised whenever current extractor cannot be dumped\"\"\"\n"
},
{
"path": "spikeextractors/extraction_tools.py",
"chars": 37266,
"preview": "import numpy as np\nimport csv\nimport os\nimport sys\nfrom pathlib import Path\nimport warnings\nimport datetime\nfrom functoo"
},
{
"path": "spikeextractors/extractorlist.py",
"chars": 6065,
"preview": "from .extractors.mdaextractors.mdaextractors import MdaRecordingExtractor, MdaSortingExtractor\nfrom .extractors.mearecex"
},
{
"path": "spikeextractors/extractors/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "spikeextractors/extractors/alfsortingextractor/__init__.py",
"chars": 52,
"preview": "from .alfsortingextractor import ALFSortingExtractor"
},
{
"path": "spikeextractors/extractors/alfsortingextractor/alfsortingextractor.py",
"chars": 8496,
"preview": "from abc import ABC\n\nfrom spikeextractors import SortingExtractor\nfrom pathlib import Path\nimport numpy as np\n\ntry:\n "
},
{
"path": "spikeextractors/extractors/axonaunitrecordingextractor/__init__.py",
"chars": 69,
"preview": "from .axonaunitrecordingextractor import AxonaUnitRecordingExtractor\n"
},
{
"path": "spikeextractors/extractors/axonaunitrecordingextractor/axonaunitrecordingextractor.py",
"chars": 9709,
"preview": "from spikeextractors.extraction_tools import check_get_traces_args\nfrom spikeextractors.extractors.neoextractors.neobase"
},
{
"path": "spikeextractors/extractors/bindatrecordingextractor/__init__.py",
"chars": 63,
"preview": "from .bindatrecordingextractor import BinDatRecordingExtractor\n"
},
{
"path": "spikeextractors/extractors/bindatrecordingextractor/bindatrecordingextractor.py",
"chars": 6434,
"preview": "import shutil\nimport numpy as np\nfrom pathlib import Path\nfrom typing import Union, Optional\n\nfrom spikeextractors impor"
},
{
"path": "spikeextractors/extractors/biocamrecordingextractor/__init__.py",
"chars": 63,
"preview": "from .biocamrecordingextractor import BiocamRecordingExtractor\n"
},
{
"path": "spikeextractors/extractors/biocamrecordingextractor/biocamrecordingextractor.py",
"chars": 7476,
"preview": "from spikeextractors import RecordingExtractor\nfrom spikeextractors.extraction_tools import check_get_traces_args\nimport"
},
{
"path": "spikeextractors/extractors/cedextractors/__init__.py",
"chars": 57,
"preview": "from .cedrecordingextractor import CEDRecordingExtractor\n"
},
{
"path": "spikeextractors/extractors/cedextractors/cedrecordingextractor.py",
"chars": 7445,
"preview": "from spikeextractors import RecordingExtractor\nfrom .utils import get_channel_info, get_channel_data\nfrom spikeextractor"
},
{
"path": "spikeextractors/extractors/cedextractors/utils.py",
"chars": 3676,
"preview": "import numpy as np\n\ntry:\n from sonpy import lib as sp\n\n # Data storage and function finder\n DataReadFunctions ="
},
{
"path": "spikeextractors/extractors/cellexplorersortingextractor/__init__.py",
"chars": 71,
"preview": "from .cellexplorersortingextractor import CellExplorerSortingExtractor\n"
},
{
"path": "spikeextractors/extractors/cellexplorersortingextractor/cellexplorersortingextractor.py",
"chars": 6414,
"preview": "from spikeextractors import SortingExtractor\nimport numpy as np\nfrom pathlib import Path\nfrom spikeextractors.extraction"
},
{
"path": "spikeextractors/extractors/combinatosortingextractor/__init__.py",
"chars": 64,
"preview": "from .combinatosortingextractor import CombinatoSortingExtractor"
},
{
"path": "spikeextractors/extractors/combinatosortingextractor/combinatosortingextractor.py",
"chars": 4091,
"preview": "from pathlib import Path\nimport numpy as np\nfrom spikeextractors import SortingExtractor\nfrom spikeextractors.extraction"
},
{
"path": "spikeextractors/extractors/exdirextractors/__init__.py",
"chars": 75,
"preview": "from .exdirextractors import ExdirRecordingExtractor, ExdirSortingExtractor"
},
{
"path": "spikeextractors/extractors/exdirextractors/exdirextractors.py",
"chars": 28325,
"preview": "from spikeextractors import RecordingExtractor\nfrom spikeextractors import SortingExtractor\nimport numpy as np\nfrom path"
},
{
"path": "spikeextractors/extractors/hdsortsortingextractor/__init__.py",
"chars": 58,
"preview": "from .hdsortsortingextractor import HDSortSortingExtractor"
},
{
"path": "spikeextractors/extractors/hdsortsortingextractor/hdsortsortingextractor.py",
"chars": 11206,
"preview": "from pathlib import Path\r\nfrom typing import Union\r\nimport numpy as np\r\nimport sys\r\nimport os\r\n\r\nfrom spikeextractors.ex"
},
{
"path": "spikeextractors/extractors/hs2sortingextractor/__init__.py",
"chars": 53,
"preview": "from .hs2sortingextractor import HS2SortingExtractor\n"
},
{
"path": "spikeextractors/extractors/hs2sortingextractor/hs2sortingextractor.py",
"chars": 4773,
"preview": "from spikeextractors import SortingExtractor\nimport numpy as np\nfrom pathlib import Path\nfrom spikeextractors.extraction"
},
{
"path": "spikeextractors/extractors/intanrecordingextractor/__init__.py",
"chars": 60,
"preview": "from .intanrecordingextractor import IntanRecordingExtractor"
},
{
"path": "spikeextractors/extractors/intanrecordingextractor/intanrecordingextractor.py",
"chars": 5638,
"preview": "import numpy as np\nfrom pathlib import Path\nfrom packaging.version import parse\nfrom typing import Union, Optional\n\nfrom"
},
{
"path": "spikeextractors/extractors/jrcsortingextractor/__init__.py",
"chars": 52,
"preview": "from .jrcsortingextractor import JRCSortingExtractor"
},
{
"path": "spikeextractors/extractors/jrcsortingextractor/jrcsortingextractor.py",
"chars": 7808,
"preview": "from pathlib import Path\r\nimport re\r\nfrom typing import Union\r\nimport numpy as np\r\n\r\nfrom spikeextractors.extractors.mat"
},
{
"path": "spikeextractors/extractors/kilosortextractors/__init__.py",
"chars": 85,
"preview": "from .kilosortextractors import KiloSortSortingExtractor, KiloSortRecordingExtractor\n"
},
{
"path": "spikeextractors/extractors/kilosortextractors/kilosortextractors.py",
"chars": 1544,
"preview": "from spikeextractors.extractors.phyextractors import PhyRecordingExtractor, PhySortingExtractor\nfrom pathlib import Path"
},
{
"path": "spikeextractors/extractors/klustaextractors/__init__.py",
"chars": 78,
"preview": "from .klustaextractors import KlustaSortingExtractor, KlustaRecordingExtractor"
},
{
"path": "spikeextractors/extractors/klustaextractors/klustaextractors.py",
"chars": 6919,
"preview": "\"\"\"\nkwik structure based on:\nhttps://github.com/kwikteam/phy-doc/blob/master/docs/kwik-format.md\n\ncluster_group defaults"
},
{
"path": "spikeextractors/extractors/matsortingextractor/__init__.py",
"chars": 52,
"preview": "from .matsortingextractor import MATSortingExtractor"
},
{
"path": "spikeextractors/extractors/matsortingextractor/matsortingextractor.py",
"chars": 3151,
"preview": "from collections import deque\nfrom pathlib import Path\nfrom typing import Union\nimport numpy as np\n\ntry:\n import h5py"
},
{
"path": "spikeextractors/extractors/maxwellextractors/__init__.py",
"chars": 136,
"preview": "from .maxwellextractors import MaxOneRecordingExtractor, MaxOneSortingExtractor, \\\n MaxTwoRecordingExtractor, MaxTwoS"
},
{
"path": "spikeextractors/extractors/maxwellextractors/maxwellextractors.py",
"chars": 19929,
"preview": "from spikeextractors import RecordingExtractor, SortingExtractor\nfrom pathlib import Path\nimport numpy as np\nfrom spikee"
},
{
"path": "spikeextractors/extractors/mcsh5recordingextractor/__init__.py",
"chars": 61,
"preview": "from .mcsh5recordingextractor import MCSH5RecordingExtractor\n"
},
{
"path": "spikeextractors/extractors/mcsh5recordingextractor/mcsh5recordingextractor.py",
"chars": 6457,
"preview": "from spikeextractors import RecordingExtractor\nimport numpy as np\nfrom pathlib import Path\nfrom spikeextractors.extracti"
},
{
"path": "spikeextractors/extractors/mdaextractors/__init__.py",
"chars": 70,
"preview": "from .mdaextractors import MdaRecordingExtractor, MdaSortingExtractor\n"
},
{
"path": "spikeextractors/extractors/mdaextractors/mdaextractors.py",
"chars": 10329,
"preview": "from spikeextractors import RecordingExtractor\nfrom spikeextractors import SortingExtractor\nfrom spikeextractors.extract"
},
{
"path": "spikeextractors/extractors/mdaextractors/mdaio.py",
"chars": 15309,
"preview": "import numpy as np\nimport struct\nimport os\nimport tempfile\nimport traceback\nfrom pathlib import Path\n\n\nclass MdaHeader:\n"
},
{
"path": "spikeextractors/extractors/mearecextractors/__init__.py",
"chars": 79,
"preview": "from .mearecextractors import MEArecRecordingExtractor, MEArecSortingExtractor\n"
},
{
"path": "spikeextractors/extractors/mearecextractors/mearecextractors.py",
"chars": 9328,
"preview": "from spikeextractors import RecordingExtractor\nfrom spikeextractors import SortingExtractor\nfrom spikeextractors.extract"
},
{
"path": "spikeextractors/extractors/neoextractors/__init__.py",
"chars": 433,
"preview": "from .plexonextractor import PlexonRecordingExtractor, PlexonSortingExtractor\nfrom .neuralynxextractor import NeuralynxR"
},
{
"path": "spikeextractors/extractors/neoextractors/axonaextractor.py",
"chars": 857,
"preview": "from .neobaseextractor import NeoBaseRecordingExtractor\n\ntry:\n import neo\n\n HAVE_NEO = True\nexcept ImportError:\n "
},
{
"path": "spikeextractors/extractors/neoextractors/blackrockextractor.py",
"chars": 2112,
"preview": "from .neobaseextractor import NeoBaseRecordingExtractor, NeoBaseSortingExtractor\nfrom pathlib import Path\nfrom typing im"
},
{
"path": "spikeextractors/extractors/neoextractors/mcsrawrecordingextractor.py",
"chars": 284,
"preview": "from .neobaseextractor import NeoBaseRecordingExtractor\n\ntry:\n import neo\n HAVE_NEO = True\nexcept ImportError:\n "
},
{
"path": "spikeextractors/extractors/neoextractors/neobaseextractor.py",
"chars": 13225,
"preview": "import numpy as np\nimport warnings\n\nfrom spikeextractors import RecordingExtractor\nfrom spikeextractors import SortingEx"
},
{
"path": "spikeextractors/extractors/neoextractors/neuralynxextractor.py",
"chars": 1455,
"preview": "from .neobaseextractor import NeoBaseRecordingExtractor, NeoBaseSortingExtractor\n\ntry:\n import neo\n HAVE_NEO = Tru"
},
{
"path": "spikeextractors/extractors/neoextractors/plexonextractor.py",
"chars": 912,
"preview": "from .neobaseextractor import NeoBaseRecordingExtractor, NeoBaseSortingExtractor\n\ntry:\n import neo\n HAVE_NEO = Tru"
},
{
"path": "spikeextractors/extractors/neoextractors/spikegadgetsextractor.py",
"chars": 1053,
"preview": "from .neobaseextractor import NeoBaseRecordingExtractor, NeoBaseSortingExtractor\n\ntry:\n import neo\n HAVE_NEO = Tru"
},
{
"path": "spikeextractors/extractors/neuropixelsdatrecordingextractor/__init__.py",
"chars": 79,
"preview": "from .neuropixelsdatrecordingextractor import NeuropixelsDatRecordingExtractor\n"
},
{
"path": "spikeextractors/extractors/neuropixelsdatrecordingextractor/channel_positions_neuropixels.txt",
"chars": 48000,
"preview": "4.300000000000000000e+01 1.100000000000000000e+01 5.900000000000000000e+01 2.700000000000000000e+01 4.300000000000000000"
},
{
"path": "spikeextractors/extractors/neuropixelsdatrecordingextractor/neuropixelsdatrecordingextractor.py",
"chars": 4082,
"preview": "from spikeextractors.extractors.bindatrecordingextractor import BinDatRecordingExtractor\nimport numpy as np\nfrom pathlib"
},
{
"path": "spikeextractors/extractors/neuroscopeextractors/__init__.py",
"chars": 196,
"preview": "from .neuroscopeextractors import NeuroscopeRecordingExtractor, NeuroscopeMultiRecordingTimeExtractor\nfrom .neuroscopeex"
},
{
"path": "spikeextractors/extractors/neuroscopeextractors/neuroscopeextractors.py",
"chars": 32185,
"preview": "from spikeextractors import RecordingExtractor, MultiRecordingTimeExtractor, SortingExtractor, MultiSortingExtractor\nfro"
},
{
"path": "spikeextractors/extractors/nixioextractors/__init__.py",
"chars": 76,
"preview": "from .nixioextractors import NIXIORecordingExtractor, NIXIOSortingExtractor\n"
},
{
"path": "spikeextractors/extractors/nixioextractors/nixioextractors.py",
"chars": 9315,
"preview": "import os\nimport numpy as np\nfrom collections.abc import Iterable\nfrom pathlib import Path\ntry:\n import nixio as nix\n"
},
{
"path": "spikeextractors/extractors/npzsortingextractor/__init__.py",
"chars": 52,
"preview": "from .npzsortingextractor import NpzSortingExtractor"
},
{
"path": "spikeextractors/extractors/npzsortingextractor/npzsortingextractor.py",
"chars": 2821,
"preview": "from spikeextractors import SortingExtractor\nfrom pathlib import Path\nfrom spikeextractors.extraction_tools import check"
},
{
"path": "spikeextractors/extractors/numpyextractors/__init__.py",
"chars": 76,
"preview": "from .numpyextractors import NumpyRecordingExtractor, NumpySortingExtractor\n"
},
{
"path": "spikeextractors/extractors/numpyextractors/numpyextractors.py",
"chars": 6108,
"preview": "from spikeextractors import RecordingExtractor\nfrom spikeextractors import SortingExtractor\nfrom pathlib import Path\nimp"
},
{
"path": "spikeextractors/extractors/nwbextractors/__init__.py",
"chars": 70,
"preview": "from .nwbextractors import NwbRecordingExtractor, NwbSortingExtractor\n"
},
{
"path": "spikeextractors/extractors/nwbextractors/nwbextractors.py",
"chars": 68930,
"preview": "import uuid\nfrom datetime import datetime\nfrom collections import abc\nfrom pathlib import Path\nimport numpy as np\nfrom p"
},
{
"path": "spikeextractors/extractors/openephysextractors/__init__.py",
"chars": 87,
"preview": "from .openephysextractors import OpenEphysRecordingExtractor, OpenEphysSortingExtractor"
},
{
"path": "spikeextractors/extractors/openephysextractors/openephysextractors.py",
"chars": 6780,
"preview": "from spikeextractors import RecordingExtractor, SortingExtractor\nfrom pathlib import Path\nimport numpy as np\nfrom spikee"
},
{
"path": "spikeextractors/extractors/phyextractors/__init__.py",
"chars": 69,
"preview": "from .phyextractors import PhyRecordingExtractor, PhySortingExtractor"
},
{
"path": "spikeextractors/extractors/phyextractors/phyextractors.py",
"chars": 8367,
"preview": "import numpy as np\nfrom pathlib import Path\nimport csv\nfrom typing import Union, Optional\n\nfrom spikeextractors import S"
},
{
"path": "spikeextractors/extractors/shybridextractors/__init__.py",
"chars": 81,
"preview": "from .shybridextractors import SHYBRIDRecordingExtractor, SHYBRIDSortingExtractor"
},
{
"path": "spikeextractors/extractors/shybridextractors/shybridextractors.py",
"chars": 7041,
"preview": "import os\nfrom pathlib import Path\nimport numpy as np\nfrom spikeextractors import RecordingExtractor, SortingExtractor\nf"
},
{
"path": "spikeextractors/extractors/spikeglxrecordingextractor/__init__.py",
"chars": 67,
"preview": "from .spikeglxrecordingextractor import SpikeGLXRecordingExtractor\n"
},
{
"path": "spikeextractors/extractors/spikeglxrecordingextractor/readSGLX.py",
"chars": 13354,
"preview": "# -*- coding: utf-8 -*-\n\"\"\"\n----------------------------------------------------------------\nThis is an adapted version "
},
{
"path": "spikeextractors/extractors/spikeglxrecordingextractor/spikeglxrecordingextractor.py",
"chars": 8160,
"preview": "from .readSGLX import readMeta, SampRate, makeMemMapRaw, GainCorrectIM, GainCorrectNI, ExtractDigital\nimport numpy as np"
},
{
"path": "spikeextractors/extractors/spykingcircusextractors/__init__.py",
"chars": 100,
"preview": "from .spykingcircusextractors import SpykingCircusSortingExtractor, SpykingCircusRecordingExtractor\n"
},
{
"path": "spikeextractors/extractors/spykingcircusextractors/spykingcircusextractors.py",
"chars": 11261,
"preview": "from spikeextractors import RecordingExtractor, SortingExtractor\nfrom spikeextractors.extractors.numpyextractors import "
},
{
"path": "spikeextractors/extractors/tridescloussortingextractor/__init__.py",
"chars": 69,
"preview": "from .tridescloussortingextractor import TridesclousSortingExtractor\n"
},
{
"path": "spikeextractors/extractors/tridescloussortingextractor/tridescloussortingextractor.py",
"chars": 2385,
"preview": "from spikeextractors import SortingExtractor\nfrom pathlib import Path\nfrom spikeextractors.extraction_tools import check"
},
{
"path": "spikeextractors/extractors/waveclussortingextractor/__init__.py",
"chars": 62,
"preview": "from .waveclussortingextractor import WaveClusSortingExtractor"
},
{
"path": "spikeextractors/extractors/waveclussortingextractor/waveclussortingextractor.py",
"chars": 1958,
"preview": "from pathlib import Path\nfrom typing import Union\n\nimport numpy as np\n\nfrom spikeextractors.extractors.matsortingextract"
},
{
"path": "spikeextractors/extractors/yassextractors/__init__.py",
"chars": 49,
"preview": "from .yassextractors import YassSortingExtractor\n"
},
{
"path": "spikeextractors/extractors/yassextractors/yassextractors.py",
"chars": 2682,
"preview": "import numpy as np\nfrom pathlib import Path\n\nfrom spikeextractors import SortingExtractor\nfrom spikeextractors.extractor"
},
{
"path": "spikeextractors/multirecordingchannelextractor.py",
"chars": 6255,
"preview": "from .recordingextractor import RecordingExtractor\nfrom .extraction_tools import check_get_traces_args\nimport numpy as n"
},
{
"path": "spikeextractors/multirecordingtimeextractor.py",
"chars": 8551,
"preview": "from .recordingextractor import RecordingExtractor\nfrom .extraction_tools import check_get_traces_args, check_get_ttl_ar"
},
{
"path": "spikeextractors/multisortingextractor.py",
"chars": 5459,
"preview": "from .sortingextractor import SortingExtractor\nimport numpy as np\nfrom .extraction_tools import check_get_unit_spike_tra"
},
{
"path": "spikeextractors/recordingextractor.py",
"chars": 41351,
"preview": "from abc import ABC, abstractmethod\nimport numpy as np\nfrom copy import deepcopy\n\nfrom .extraction_tools import load_pro"
},
{
"path": "spikeextractors/save_tools.py",
"chars": 2496,
"preview": "from pathlib import Path\n\nfrom .cacheextractors import CacheRecordingExtractor, CacheSortingExtractor\nfrom .recordingext"
},
{
"path": "spikeextractors/sortingextractor.py",
"chars": 30887,
"preview": "from abc import ABC, abstractmethod\nimport numpy as np\nfrom copy import deepcopy\n\nfrom .extraction_tools import get_sub_"
},
{
"path": "spikeextractors/subrecordingextractor.py",
"chars": 8396,
"preview": "from .recordingextractor import RecordingExtractor\nfrom .extraction_tools import check_get_traces_args, cast_start_end_f"
},
{
"path": "spikeextractors/subsortingextractor.py",
"chars": 7816,
"preview": "from .sortingextractor import SortingExtractor\nimport numpy as np\nfrom .extraction_tools import check_get_unit_spike_tra"
},
{
"path": "spikeextractors/testing.py",
"chars": 10881,
"preview": "import os\nimport shutil\nfrom pathlib import Path\nimport uuid\nfrom datetime import datetime\nimport numpy as np\n\nfrom .ext"
},
{
"path": "spikeextractors/version.py",
"chars": 19,
"preview": "version = '0.9.11'\n"
},
{
"path": "tests/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "tests/probe_test.prb",
"chars": 1147,
"preview": "channel_groups = {\n 1: {\n 'channels': list(range(16)),\n 'graph' : [],\n 'geometry': {\n "
},
{
"path": "tests/test_extractors.py",
"chars": 34473,
"preview": "import os\nimport shutil\nimport tempfile\nimport unittest\nfrom pathlib import Path\n\nimport numpy as np\n\nimport spikeextrac"
},
{
"path": "tests/test_gin_repo.py",
"chars": 11762,
"preview": "import tempfile\nimport unittest\nfrom pathlib import Path\nimport numpy as np\nimport sys\n\nfrom datalad.api import install,"
},
{
"path": "tests/test_numpy_extractors.py",
"chars": 2816,
"preview": "import numpy as np\nimport unittest\nimport spikeextractors as se\n\n\nclass TestNumpyExtractors(unittest.TestCase):\n def "
},
{
"path": "tests/test_tools.py",
"chars": 5234,
"preview": "import numpy as np\nimport unittest\nimport tempfile\nimport shutil\nimport spikeextractors as se\nimport os\nfrom copy import"
}
]
About this extraction
This page contains the full source code of the SpikeInterface/spikeextractors GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 118 files (697.7 KB), approximately 176.1k tokens, and a symbol index with 684 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.