Utility

Data Helper Methods

class brainlit.utils.NeuroglancerSession(url: str, mip: int = 0, url_segments: Optional[str] = None, fill_missing: bool = True, use_https: bool = False)[source]

Utility class which pulls and pushes data.

Parameters

  • url -- Precompued path either to a file URI or url URI. Defaults to mouselight brain1.

  • mip -- Resolution level to pull and push data at. Defaults to 0, the highest resolution.

  • url_segments -- Precomputed path to segmentation data. Optional, default None.

  • fill_missing -- Always passes directly into 'CloudVolume()' function to fill missing segent/image values with 0s.

  • use_https -- Always passes directly into 'CloudVolume()' function to set use_https to the desired value.

url

CloudVolumePrecomputedPath to image data.

url_segments

CloudVolumePrecomputedPath to segmentation data. Optional, default None. Automatically tries precomputed path url+"_segments" if None.

cv

CloudVolume object for image data.

Type

CloudVolumePrecomputed

cv_segments

CloudVolume object for segmentation data. Optional, default None.

Type

CloudVolumePrecomputed

cv_annotations

CloudVolume object for segmentation data. Optional, default None.

Type

CloudVolumePrecomputed

mip

Resolution level.

chunk_size

The chunk size of the volume at the specified mip, given as (x, y, z).

scales

The resolution of the volume at the specified mip, given as (x, y, z).

create_tubes(self, seg_id: Union[int, float], bbox: Bounds, radius: Optional[int] = None)[source]

Creates voxel-wise foreground/background labels associated with a particular neuron trace, within a given bounding box of voxel coordinates.

Parameters

  • seg_id -- The id of the .swc file.

  • bbox -- The bounding box to draw tubes within.

  • radius -- Euclidean distance threshold used to draw tubes, default None = 1 px thick.

  • rounding -- Optional, bool, default is True. False if no swc rounding.

Returns

A volume within the bounding box, with 1 on tubes and 0 elsewhere.

Return type

labels

get_segments(self, seg_id: int, bbox: Optional[Bounds] = None, rounding: Optional[bool] = True)[source]

Get a graph of a segmentation annotation within a bounding box.

Parameters

  • The segement to pull. (seg_id) --

  • bbox -- The bounding box object, default None. If None, uses entire volume.

  • rounding -- Optional, default True. Whether you want S3 file to be rounded or not.

Returns

A networkx subgraph from the specified segment and bounding box.

Return type

G

pull_bounds_img(self, bounds: Bounds)[source]

Pull a volume around a specified bounding box. Works on image channels.

Parameters

bounds -- Bounding box, or tuple containing (x0, y0, z0, x1, y1, z1) bounds.

Returns

Volume pulled according to the bounding box.

Return type

img

abstract pull_bounds_seg(self, bounds: Bounds)[source]

Pull a volume around a specified bounding box. Works on annotation channels.

Parameters

bounds -- Bounding box, or tuple containing (x0, y0, z0, x1, y1, z1) bounds.

Returns

Volume pulled according to the bounding box.

Return type

img

pull_chunk(self, seg_id: int, v_id: int, radius: int = 0)[source]

Pull a subvolume around a specified skeleton vertex according to chunk size. Each data set has a specified chunk size, which can be found by calling self.cv.info.

Parameters

  • seg_id -- ID of the segment to use, depends on data in s3.

  • v_id -- ID of the vertex to use, depends on the segment.

  • radius -- Radius of pulled volume around central chunk, in chunks. Optional, default is 0 (single chunk which contains the voxel).

Returns

A chunk_size[0]*2*nx X chunk_size[1]*2*ny X chunk_size[2]*2*nz volume. bounds: Bounding box object which contains the bounds of the volume. vox_in_img: List of coordinates which locate the initial point in the volume.

Return type

img

pull_vertex_list(self, seg_id: int, v_id_list: List[int], buffer: List[int] = [1, 1, 1], expand: bool = False)[source]

Pull a subvolume containing all listed vertices.

Parameters

  • seg_id -- ID of the segment to use, depends on data in s3.

  • v_id_list -- list of vertex IDs to use.

  • buffer -- Buffer around the bounding box (in voxels). Can be int or list of ints. Default [1, 1, 1], set to [0, 0, 0] if expand is True.

  • expand -- Flag whether to expand subvolume to closest set of chunks.

Returns

The image volume containing all vertices. bounds: Bounding box object which contains the bounds of the volume. vox_in_img_list: List of coordinates which locate the vertices in the volume.

Return type

img

pull_voxel(self, seg_id: int, v_id: int, radius: int = 1)[source]

Pull a subvolume around a specified skeleton vertex with of shape [2r+1, 2r+1, 2r+1], in voxels.

Parameters

  • seg_id -- ID of the segment to use, depends on data in s3.

  • v_id -- ID of the vertex to use, depends on the segment.

  • radius -- Radius of pulled volume around central voxel, in voxels. Optional, default is 1 (3x3 volume is pulled, centered at the vertex).

Returns

A 2*nx+1 X 2*ny+1 X 2*nz+1 volume. bounds: Bounding box object which contains the bounds of the volume. vox_in_img: List of coordinates which locate the initial point in the subvolume.

Return type

img

abstract push(self, img: np.ndarray, bounds: Bounds)[source]

Push a volume to an annotation channel.

Parameters

  • img -- Volume to push

  • bounds -- Bounding box or tuple containing (x0, y0, z0, x1, y1, z1) bounds.

set_url_segments(self, seg_url: str)[source]

Sets the url_segments and cv_segments attributes.

Parameters

seg_url -- CloudvolumePrecomputedPath to segmentation data.

class brainlit.utils.NeuronTrace(path: str, seg_id: int = None, mip: int = None, rounding: bool = True, read_offset: bool = False, fill_missing: bool = True, use_https: bool = False)[source]

Neuron Trace class to handle neuron traces as swcs and s3 skeletons

Parameters

  • path (str) -- Path to either s3 bucket (url) or swc file (filepath).

  • seg_id (int) -- If s3 bucket path is provided, the segment number to pull, default None.

  • mip (int) -- If s3 bucket path is provided, the resolution to use for scaling, default None.

  • rounding (bool) -- If s3 is provided, specifies if it should be rounded, default True

  • read_offset (bool) -- If swc is provided, whether offset should be read from file, default False.

  • fill_missing (bool) -- Always passes directly into 'CloudVolume()' function to fill missing skeleton values with 0s, default True.

  • use_https (bool) -- Always passes directly into 'CloudVolume()' function to set use_https to desired value, default True.

path

Path to either s3 bucket (url) or swc file (filepath)

Type

str

input_type

Specifies whether input file is 'swc' or 'skel'

Type

bool

df

Indices, coordinates, and parents of each node

Type

pandas.DataFrame

args

Stores arguments for df - offset, color, cc, branch

Type

tuple

seg_id

If s3 bucket path is provided, the segment number to pull

Type

int

mip

If s3 bucket path is provided, the resolution to use for scaling

Type

None,int

Example

>>> swc_path = "./data/data_octree/consensus-swcs/2018-08-01_G-002_consensus.swc"
>>> s3_path = "s3://open-neurodata/brainlit/brain1_segments"
>>> seg_id = 11
>>> mip = 2

>>> swc_trace = NeuronTrace(swc_path)
>>> s3_trace = NeuronTrace(s3_path,seg_id,mip)
generate_df_subset(self, vox_in_img_list: list, subneuron_start: int = None, subneuron_end: int = None)[source]

Read a new subset dataframe in coordinates in img spacing. Specify specific range of vertices from dataframe if desired

Parameters

  • vox_in_img_list (list) -- List of voxels

  • subneuron_start (None, int (default = None)) -- Provides start index, if specified, to apply function to a portion of the dataframe Default is None.

  • subneuron_end (None, int (default = None)) -- Provides end index, if specified, to apply function to a portion of the dataframe Default is None.

Returns

df -- Indicies, coordinates (in img spacing) and parents of each node. Coordinates are in spatial units.

Return type

pandas.DataFrame

Example

>>> #swc input, subneuron_start and subneuron_end specified

>>> subneuron_start = 5
>>> subneuron_end = 8

>>> #generate vox_in_img_list
>>> my_list = []
>>>for i in range(subneuron_end-subneuron_start):
    my_list.append(10)
>>> vox_in_img_list_2 = list([my_list,my_list,my_list])

>>>swc_trace.generate_df_subset(vox_in_img_list_2,subneuron_start,subneuron_end)

>>> sample    structure    x    y    z    r    parent
        5    6    0    10    10    10    1.0    5
        6    7    0    10    10    10    1.0    6
        7    8    0    10    10    10    1.0    7
get_bfs_subgraph(self, node_id: int, depth: int, df: pd.DataFrame = None, spacing: np.array = None, origin: np.array = None)[source]

Creates a spanning subgraph from a seed node and parent graph using BFS.

Returns
G_subnetworkx.classes.digraph.DiGraph

Subgraph

treeDiGraph

The tree returned by BFS.

pathslist

List of Nx3 numpy.array. Rows of the array are 3D coordinates in voxel units. Each array is one path.

Example ..

>>> #swc input, specify node_id and depth
    ..

>>> swc_trace.get_bfs_subgraph(node_id=11,depth=2)
    ..
>>>(<networkx.classes.digraph.DiGraph at 0x7f7f2ce65670>,

<networkx.classes.digraph.DiGraph at 0x7f7f2ce65370>, array([array([[4727, 4440, 3849],

[4732, 4442, 3850], [4739, 4455, 3849]]), array([[4732, 4442, 3850], [4749, 4439, 3856]])], dtype=object))

get_df(self)[source]

Gets the dataframe providing indices, coordinates, and parents of each node

Returns

self.df -- dataframe providing indices, coordinates, and parents of each node

Return type

pandas.DataFrame

Example

>>> swc_trace.get_df()
>>> sample    structure    x    y    z    r    parent
    0    1    0    -52.589700    -1.448032    -1.228827    1.0    -1
    1    2    0    -52.290940    -1.448032    -1.228827    1.0    1
    2    3    0    -51.992181    -1.143616    -0.240423    1.0    2
    3    4    0    -51.095903    -1.143616    -0.240423    1.0    3
    4    5    0    -50.797144    -0.839201    -0.240423    1.0    4
    ...    ...    ...    ...    ...    ...    ...    ...
    148    149    0    45.702088    14.381594    -7.159252    1.0    148
    149    150    0    46.000847    14.686010    -7.159252    1.0    149
    150    151    0    46.897125    14.686010    -7.159252    1.0    150
    151    152    0    47.494643    15.294842    -7.159252    1.0    151
    152    153    6    48.092162    15.294842    -7.159252    1.0    152
    53 rows × 7 columns
get_df_arguments(self)[source]

Gets arguments for df - offset, color, cc, branch

Returns

self.args -- list of arguments for df, if found - offset, color, cc, branch

Return type

list

Example

>>> swc_trace.get_df_arguments()
>>> [[73954.8686, 17489.532566, 34340.365689], [1.0, 1.0, 1.0], nan, nan]
get_df_voxel(self, spacing: np.array, origin: np.array = np.array([0, 0, 0]))[source]

Converts coordinates in pd.DataFrame from spatial units to voxel units

Parameters

  • spacing (numpy.array) -- Conversion factor (spatial units/voxel). Assumed to be np.array([x,y,z])

  • origin (numpy.array) -- Origin of the spatial coordinate. Default is (0,0,0). Assumed to be np.array([x,y,z])

Returns

df_voxel -- Indicies, coordinates, and parents of each node in the swc. Coordinates are in voxel units.

Return type

pandas.DataFrame

Example

>>> swc_trace.get_df_voxel(spacing=np.asarray([2,2,2]))
>>> sample    structure    x    y    z    r    parent
    0    1    0    -26    -1    -1    1.0    -1
    1    2    0    -26    -1    -1    1.0    1
    2    3    0    -26    -1    0    1.0    2
    3    4    0    -26    -1    0    1.0    3
    4    5    0    -25    0    0    1.0    4
    ...    ...    ...    ...    ...    ...    ...    ...
    148    149    0    23    7    -4    1.0    148
    149    150    0    23    7    -4    1.0    149
    150    151    0    23    7    -4    1.0    150
    151    152    0    24    8    -4    1.0    151
    152    153    6    24    8    -4    1.0    152
    153 rows × 7 columns
get_graph(self, spacing: np.array = None, origin: np.array = None)[source]

Converts dataframe in either spatial or voxel coordinates into a directed graph. Will convert to voxel coordinates if spacing is specified.

Parameters

  • spacing (None, numpy.array (default = None)) -- Conversion factor (spatial units/voxel). Assumed to be np.array([x,y,z]). Provided if graph should convert to voxel coordinates first. Default is None.

  • origin (None, numpy.array (default = None)) -- Origin of the spatial coordinate, if converting to voxels. Default is None. Assumed to be np.array([x,y,z])

Returns

G -- Neuron from swc represented as directed graph. Coordinates x,y,z are node attributes accessed by keys 'x','y','z' respectively.

Return type

networkx.classes.digraph.DiGraph

Example

>>> swc_trace.get_graph()
>>> <networkx.classes.digraph.DiGraph at 0x7f81a83937f0>
get_paths(self, spacing: np.array = None, origin: np.array = None)[source]

Converts dataframe in either spatial or voxel coordinates into a list of paths. Will convert to voxel coordinates if spacing is specified.

Parameters

  • spacing (None, numpy.array (default = None)) -- Conversion factor (spatial units/voxel). Assumed to be np.array([x,y,z]). Provided if graph should convert to voxel coordinates first. Default is None.

  • origin (None, numpy.array) -- Origin of the spatial coordinate, if converting to voxels. Default is None. Assumed to be np.array([x,y,z])

Returns

paths -- List of Nx3 numpy.array. Rows of the array are 3D coordinates in voxel units. Each array is one path.

Return type

list

Example

>>> swc_trace.get_paths()[0][1:10]
>>> array([[-52, -1, -1],
            [-51, -1, 0],
            [-51, -1, 0],
            [-50, 0, 0],
            [-50, 0, 0],
            [-49, 0, 0],
            [-48, 0, 0],
            [-46, 0, 0],
            [-46, 0, 0]], dtype=object)
get_skel(self, benchmarking: bool = False, origin: np.ndarray = None)[source]

Gets a skeleton version of dataframe, if swc input is provided

Parameters

  • origin (None, numpy array with shape (3,1) (default = None)) -- origin of coordinate frame in microns, (default: None assumes (0,0,0) origin)

  • benchmarking (bool) -- For swc files, specifies whether swc file is from benchmarking dataset, to obtain skeleton ID

Returns

skel -- Skeleton object of given SWC file

Return type

cloudvolume.Skeleton

Example

>>> swc_trace.get_skel(benchmarking=True)
>>> Skeleton(segid=, vertices=(shape=153, float32), edges=(shape=152, uint32), radius=(153, float32), vertex_types=(153, uint8), vertex_color=(153, float32), space='physical' transform=[[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0]])
get_sub_neuron(self, bounding_box: Union[tuple, list, None], spacing: np.array = None, origin: np.array = None)[source]

Returns sub-neuron with node coordinates bounded by start and end

Parameters

  • bounding_box (tuple or list or None) -- Defines a bounding box around a sub-region around the neuron. Length 2 tuple/list. First element is the coordinate of one corner (inclusive) and second element is the coordinate of the opposite corner (exclusive). Both coordinates are numpy.array([x,y,z])in voxel units.

  • spacing (None, numpy.array (default = None)) -- Conversion factor (spatial units/voxel). Assumed to be np.array([x,y,z]). Provided if graph should convert to voxel coordinates first. Default is None.

  • origin (numpy.array) -- Origin of the spatial coordinate, if converting to voxels. Default is None. Assumed to be np.array([x,y,z])

Returns

G_sub -- Neuron from swc represented as directed graph. Coordinates x,y,z are node attributes accessed by keys 'x','y','z' respectively.

Return type

networkx.classes.digraph.DiGraph

Example

>>> bounding_box=[[1,2,4],[1,2,3]]

>>> #swc input, no spacing and origin
>>> swc_trace.get_sub_neuron(bounding_box)
>>> <networkx.classes.digraph.DiGraph at 0x7f81a95d1e50>
get_sub_neuron_paths(self, bounding_box: Union[tuple, list, None], spacing: np.array = None, origin: np.array = None)[source]

Returns sub-neuron with node coordinates bounded by start and end

Parameters

  • bounding_box (tuple or list or None) -- Defines a bounding box around a sub-region around the neuron. Length 2 tuple/list. First element is the coordinate of one corner (inclusive) and second element is the coordinate of the opposite corner (exclusive). Both coordinates are numpy.array([x,y,z])in voxel units.

  • spacing (None, numpy.array (default = None)) -- Conversion factor (spatial units/voxel). Assumed to be np.array([x,y,z]). Provided if graph should convert to voxel coordinates first. Default is None.

  • origin (numpy.array) -- Origin of the spatial coordinate, if converting to voxels. Default is None. Assumed to be np.array([x,y,z])

Returns

paths -- List of Nx3 numpy.array. Rows of the array are 3D coordinates in voxel units. Each array is one path.

Return type

list

Example

>>> bounding_box=[[1,2,4],[1,2,3]]

>>> #swc input, no spacing and origin
>>> swc_trace.get_sub_neuron_paths(bounding_box)
>>> array([], dtype=object)
static ssd(pts1: np.array, pts2: np.array)[source]

Compute significant spatial distance metric between two traces as defined in APP1. :param pts1: array containing coordinates of points of trace 1. shape: npoints x ndims :type pts1: np.array :param pts2: array containing coordinates of points of trace 1. shape: npoints x ndims :type pts2: np.array

Returns

significant spatial distance as defined by APP1

Return type

[float]

Example

>>> pts1 = swc_trace.get_paths()[0][1:10]
>> pts2 = swc_trace.get_paths()[0][11:20]

>>> NeuronTrace.ssd(pts1,pts2)

>>>6.247937554557103

brainlit.utils.czi_to_zarr(czi_path: str, out_dir: str, fg_channel: int = 0, parallel: int = 1)[source]

Convert 4D czi image to a zarr file(s) at a given directory. Single channel image will produce a single zarr, two channels will produce two.

Parameters

  • czi_path (str) -- Path to czi image.

  • out_dir (str) -- Path to directory where zarr(s) will be written.

  • fg_channel (int) -- Index of foreground channel.

  • parallel (int) -- Number of cpus to use to write zarr.

Returns

paths to zarrs that were written

Return type

list

brainlit.utils.zarr_to_omezarr(zarr_path: str, out_path: str, res: list)[source]

Convert 3D zarr to ome-zarr.

Parameters

  • zarr_path (str) -- Path to zarr.

  • out_path (str) -- Path of ome-zarr to be created.

  • res (list) -- List of xyz resolution values in nanometers.

Raises

S3 Helper Methods

brainlit.utils.get_data_ranges(bin_path: List[List[str]], chunk_size: Tuple[int, int, int])[source]

Get ranges (x,y,z) for chunks to be stitched together in volume

Parameters

  • bin_path -- Binary paths to files.

  • chunk_size -- The size of chunk to get ranges over.

Returns

x-coord int bounds. y_range: y-coord int bounds. z_range: z-coord int bounds.

Return type

x_range