bmtk.utils.sonata package

Subpackages

• bmtk.utils.sonata.config package
  • Submodules
  • bmtk.utils.sonata.config.sonata_config module
    • ConfigParser
      • ConfigParser.parse()
      • ConfigParser.usr_vars
    • SonataConfig
      • SonataConfig.from_dict()
      • SonataConfig.from_json()
      • SonataConfig.from_yaml()
      • SonataConfig.load()
      • SonataConfig.validate()
      • SonataConfig.validator
    • copy_config()
    • from_dict()
    • from_json()
    • from_yaml()
  • Module contents

Submodules

bmtk.utils.sonata.column_property module

class bmtk.utils.sonata.column_property.ColumnProperty(name, dtype, dimension, nrows=0, attrs=None)

Bases: object

Representation of a column name and metadata from a hdf5 dataset, csv column, etc.

property attributes

property dimension

property dtype

classmethod from_csv(pd_obj, name=None)

classmethod from_h5(hf_obj, name=None)

property name

property nrows

bmtk.utils.sonata.edge module

class bmtk.utils.sonata.edge.Edge(src_node_id, trg_node_id, source_pop, target_pop, group_id, group_props, edge_types_props)

Bases: object

property dynamics_params

property edge_type_id

property group_id

property source_node_id

property source_population

property target_node_id

property target_population

class bmtk.utils.sonata.edge.EdgeSet(edge_ids, population)

Bases: object

next()

bmtk.utils.sonata.edge_stats module

bmtk.utils.sonata.edge_stats.chisquare(edges_orig, edges_new, edge_prop, raw_counts=False, **kwargs)

bmtk.utils.sonata.edge_stats.edge_props_distribution(edge_files, edge_prop, populations=None, edge_props_grouping=None, source_props_grouping=None, target_props_grouping=None, fill_val=1, operation='sum', population_columns=False)

Reads in one or more SONATA edges files and return a DataFrame consisting of the distribution of a given edge property across an arbitary grouping of cells. For example return the total number of synapses between each source/target node-type, or the mean syn_weights for edge edge-type, or the variance of connecting in-degrees across morphologies.

By default will return a DataFrame of each “source_node_id”, “target_node_id”, and “<edge_prop>” row found in the edge file(s). But you can group the rows by any property/column found in the edges, source or target populations; and apply an arbitary <operation> on the results.

The <edge_prop> can be any property/column found in the edges, target-nodes, or source-nodes populations. But if the property is not a numeric the <operation> applied to it may fail.

Parameters:

• edge_files – str, dict, list of str or dict. Is the edges (and optional nodes) files paths. It can be a SONATA h5 file, a dictionary of h5/csv files, or the location(s) of SONATA config files.

• edge_prop – string, property name (column if h5 or csv) file that is being investigated.

• populations – string or list of strings. If SONATA file(s) contains multiple edge populations you can specify .

• edge_props_grouping – str or list[str]. List of columns in edges file(s) to group results by.

• source_props_grouping – str or list[str]. List of columns in source-node file(s) to group results by.

• target_props_grouping – str or list[str]. List of columns in target-node file(s) to group results by.

• fill_val – If <edge_prop> has missing/None/NaN values will fill in with given value. set to None to Turn off.

• operation – Str or function: pandas or numpy function to apply to when doing the grouping, eg. ‘sum’, ‘mean’, np.std.

• population_columns – If set to true will return extra column describing the edges/nodes populations for each row.

bmtk.utils.sonata.edge_stats.edge_stats_table(edges_data)

bmtk.utils.sonata.edge_stats.kolmogorov_smirnov(edges_orig, edges_new, edge_prop, **kwargs)

bmtk.utils.sonata.edge_stats.nconnections_distributions(edge_files, populations=None, edge_props_grouping=None, source_props_grouping=None, target_props_grouping=None, **kwopts)

Reads in one or more SONATA edges files and return a DataFrame consisting of the total number of connection given any arbitary grouping of network properties. The property will be called “nconns” in the returned table. Similar to edge_props_distribution().

Note: A connection here just refers to whether-or-not two cells are connected (including autapses) and ignores the number of synapses between each cell. Use nsyns_distribution() to get distribution of the number of synapses.

Parameters:

• edge_files – str, dict, list of str or dict. Is the edges (and optional nodes) files paths. It can be a SONATA h5 file, a dictionary of h5/csv files, or the location(s) of SONATA config files.

• populations – string or list of strings. If SONATA file(s) contains multiple edge populations you can specify .

• source_props_grouping – str or list[str]. List of columns in source-node file(s) to group results by.

• target_props_grouping – str or list[str]. List of columns in target-node file(s) to group results by.

bmtk.utils.sonata.edge_stats.nsyns_distribution(edge_files, populations=None, edge_props_grouping=None, source_props_grouping=None, target_props_grouping=None)

Reads in one or more SONATA edges files and return a DataFrame consisting of the total number of synapses given any arbitary grouping of network properties. The property will be called “nsyns” in the returned table. Similar to edge_props_distribution().

Note: Each cell-to-cell connection may have mutiliple synapses. Use nconnections_distributions() to get distribution of the raw connectivity map.

Parameters:

• edge_files – str, dict, list of str or dict. Is the edges (and optional nodes) files paths. It can be a SONATA h5 file, a dictionary of h5/csv files, or the location(s) of SONATA config files.

• populations – string or list of strings. If SONATA file(s) contains multiple edge populations you can specify .

• source_props_grouping – str or list[str]. List of columns in source-node file(s) to group results by.

• target_props_grouping – str or list[str]. List of columns in target-node file(s) to group results by.

bmtk.utils.sonata.edge_stats.pearson_r(edges_orig, edges_new, edge_prop, **kwargs)

bmtk.utils.sonata.edge_stats.plot_correlation(edges_orig, edges_new, edge_prop, log_scale=False, ax=None, show=True)

bmtk.utils.sonata.edge_stats.plot_distribution(edges_data, edge_prop, names=None, log_scale=False, ax=None, show=True, **kwopts)

Plots the distribution of an edge property given an arbitary grouping of rows based on edge, target or source node columns.

Parameters:

• edge_files – str, dict, list of str or dict. Is the edges (and optional nodes) files paths. It can be a SONATA h5 file, a dictionary of h5/csv files, or the location(s) of SONATA config files. Can also be a csv file containg results from edge_props_distribution() function

• edge_prop – string, property name (column if h5 or csv) file that is being investigated.

• names – A list of names/titles to use for the labeling of the distribution(s) curves. If None then will infer from data.

• log_scale – If set to True then use log-scale on the x-axis. Default: False.

• ax – If true will save distribution plot to pre-generated matplotlib axis, for merging into another figure. Default: None.

• show – If true will plot distribution. Default: True.

• kwargs – optional args that will be passed into edge_props_distribution() function.

bmtk.utils.sonata.edge_stats.to_edges_dataframe(edges_pop_h5, edge_types_path=None, with_properties=True)

bmtk.utils.sonata.edge_stats.to_nodes_dataframe(nodes_pop_h5, node_types_path=None, with_properties=True)

bmtk.utils.sonata.file module

class bmtk.utils.sonata.file.File(data_files, data_type_files, mode='r', gid_table=None, require_magic=True)

Bases: object

property edges

property has_edges

property has_nodes

property nodes

property version

bmtk.utils.sonata.file_root module

class bmtk.utils.sonata.file_root.EdgesRoot(edges, edge_types, mode='r')

Bases: FileRoot

property edge_types_table

get_populations(name=None, source=None, target=None)

Find all populations with matching criteria, either using the population name (which will return a list of size 0 or 1) or based on the source/target population.

To return a list of all populations just use populations() method

Parameters:

• name – (str) name of population

• source – (str or NodePopulation) returns edges with nodes coming from matching source-population

• target – (str or NodePopulation) returns edges with nodes coming from matching target-population

Returns:

A (potential empty) list of EdgePopulation objects filter by criteria.

class bmtk.utils.sonata.file_root.FileRoot(root_name, h5_files, h5_mode, csv_files)

Bases: object

Base class for both /nodes and /edges root group in h5 file

check_format()

get_population(population_name, default=None)

Return a population group object based on population’s name

property population_names

property populations

property root_name

property types_table

class bmtk.utils.sonata.file_root.NodesRoot(nodes, node_types, mode='r', gid_table=None)

Bases: FileRoot

generate_gids(file_name, gids=None, force=False)

Creates a gid <–> (node_id, population) table based on sonnet specifications.

Generating gids will take some time and so not recommend to call this during the simulation. Instead save the file to the disk and pass in h5 file during the simulation (using gid_table parameter). In fact if you’re worried about efficeny don’t use this method.

Parameters:

• file_name – Name of h5 file to save gid map to.

• gids – rule/list of gids to use

• force – set to true to overwrite existing gid map (default False).

property has_gids

property node_types_table

set_gid_table(gid_table, force=False)

Adds a map from a gids <–> (node_id, population) based on specification.

Parameters:

• gid_table – An h5 file/group containing map specifications

• force – Set to true to have it overwrite any exsiting gid table (default False)

bmtk.utils.sonata.group module

class bmtk.utils.sonata.group.EdgeGroup(group_id, h5_group, parent)

Bases: Group

build_indicies(force=False)

property edge_type_ids

filter(**filter_props)

get_values(property_name, all_rows=False)

Returns all values for a group property.

Note that a row within a group may not have a corresponding node/edge, or they may have a different order or multiple node/edges may share the same group row. Setting all_rows=False will return all the values as you see if you iterated through all the population’s items. Setting all_rows=True just returns the data as a list as they appear in the dataset (will be faster).

Parameters:

• property_name – Name of dataset property/column to fetch.

• all_rows – Set false to return order in which they appear in population, false to return entire dataset

Returns:

A list of values for the given column name.

next()

property src_node_ids

to_dataframe()

property trg_node_ids

class bmtk.utils.sonata.group.Group(group_id, h5_group, parent)

Bases: object

A container containig a node/edge population groups.

A node or edge population will have one or more groups, each having a unique identifier. Each group shared the same columns and datatypes, thus each group is essentially a different model.

property all_columns

build_indicies(force=False)

check_format()

column(column_name, group_only=False)

property columns

get_dataset(column_name)

get_values(property_name, all_rows=False)

Returns all values for a group property.

Note that a row within a group may not have a corresponding node/edge, or they may have a different order or multiple node/edges may share the same group row. Setting all_rows=False will return all the values as you see if you iterated through all the population’s items. Setting all_rows=True just returns the data as a list as they appear in the dataset (will be faster).

Parameters:

• property_name – Name of dataset property/column to fetch.

• all_rows – Set false to return order in which they appear in population, false to return entire dataset

Returns:

A list of values for the given column name.

property group_columns

property group_id

property has_dynamics_params

property has_gids

property parent

to_dataframe()

class bmtk.utils.sonata.group.NodeGroup(group_id, h5_group, parent)

Bases: Group

build_indicies(force=False)

filter(**filter_props)

Filter all nodes in the group by key=value pairs.

The filter specifications may apply to either node_type or group column properties. Currently at the moment it only supports equivlency. An intersection (and operator) is done for every different filter pair. This will produce a generator of all nodes matching the the filters.

for node in filter(pop_name=’VIp’, depth=10.0):

assert(node[‘pop_name’] == ‘VIp’ and node[‘depth’] == 10.0)

Parameters:

filter_props – keys and their values to filter nodes on.

Returns:

A generator that produces all valid nodes within the group with matching key==value pairs.

get_values(property_name, filtered_indicies=True)

Returns all values for a group property.

Note that a row within a group may not have a corresponding node/edge, or they may have a different order or multiple node/edges may share the same group row. Setting all_rows=False will return all the values as you see if you iterated through all the population’s items. Setting all_rows=True just returns the data as a list as they appear in the dataset (will be faster).

Parameters:

• property_name – Name of dataset property/column to fetch.

• all_rows – Set false to return order in which they appear in population, false to return entire dataset

Returns:

A list of values for the given column name.

property gids

property node_ids

property node_type_ids

to_dataframe()

bmtk.utils.sonata.node module

class bmtk.utils.sonata.node.Node(node_id, pop_name, node_type_id, node_types_props, group_id, group_props, dynamics_params, gid=None)

Bases: object

property dynamics_params

property gid

property group_id

property group_props

property node_id

property node_type_id

property node_type_properties

property population_name

class bmtk.utils.sonata.node.NodeSet(node_indicies, population, **parameters)

Bases: object

get_properties(property_name)

property gids

next()

property node_ids

property node_type_ids

bmtk.utils.sonata.population module

class bmtk.utils.sonata.population.EdgePopulation(pop_name, pop_group, edge_types_tables)

Bases: Population

build_indicies()

property edge_types_table

filter(**filter_props)

get_row(index)

get_source(source_node_id)

static get_source_population(pop_group_h5)

get_sources(source_node_ids)

get_target(target_node_id)

static get_target_population(pop_group_h5)

get_targets(target_node_ids)

property group_id_column

property group_index_column

group_indicies(group_id, build_cache=False, as_list=False)

Returns a list of all the population row index that maps onto the given group.

Used for iterating or searching within a Group

Parameters:

• group_id – id of a given group

• build_cache – Will cache indicies for all groups. Will be faster if making multiple calls but requires more memory (default False)

Returns:

A (possibly empty) list of row indicies (non-contiguous, but unique)

next()

property source_population

property target_population

to_dataframe()

Convert Population to dataframe

property type_ids_column

class bmtk.utils.sonata.population.NodePopulation(pop_name, pop_group, node_types_tables)

Bases: Population

add_gids(gid_map_df, force=False)

filter(**filter_props)

get_gid(gid)

get_node_id(node_id)

get_row(row_indx)

get_rows(row_indicies)

Returns a set of all nodes based on list of row indicies.

Warning: currently due to the use of h5py, the list must be ordered and cannot contain duplicates.

Parameters:

row_indicies – A list of row indicies

Returns:

An iterable NodeSet of nodes in the specified indicies

property gids

property group_id_column

property group_index_column

property has_gids

igids(row_indicies)

property index_column_name

inode_ids(row_indicies)

inode_type_ids(row_indicies)

next()

property node_ids

property node_types_table

to_dataframe(index_by_id=True)

Convert Population to dataframe

property type_ids_column

class bmtk.utils.sonata.population.Population(pop_name, pop_group, types_table)

Bases: object

get_group(group_id)

property group_id_column

property group_id_ds

property group_ids

List of all group_ids belonging to population

property group_index_column

property group_index_ds

group_indicies(group_id, build_cache=False)

Returns a list of all the population row index that maps onto the given group.

Used for iterating or searching within a Group

Parameters:

• group_id – id of a given group

• build_cache – Will cache indicies for all groups. Will be faster if making multiple calls but requires more memory (default False)

Returns:

A (possibly empty) list of row indicies (non-contiguous, but unique)

property groups

Returns a list of sonata.Group objects

igroup_ids(row_indicies)

igroup_indicies(row_indicies)

property name

name of current population

to_dataframe()

Convert Population to dataframe

property type_ids

property type_ids_column

property types_table

bmtk.utils.sonata.types_table module

class bmtk.utils.sonata.types_table.EdgeTypesTable(parent=None)

Bases: TypesTable

property edge_type_ids

property index_column_name

class bmtk.utils.sonata.types_table.NodeTypesTable(parent=None)

Bases: TypesTable

property index_column_name

property node_type_ids

class bmtk.utils.sonata.types_table.TypesTable(parent=None)

Bases: object

add_table(nt_df)

column(column_name)

property columns

find(column_key, column_val, silent=False)

Returns a list of type_ids that contain column property column_key==column_val

Parameters:

• column_key – Name of column to search

• column_val – Value of column to select for

• silent – Set to true to prevent KeyError if column_key doesn’t exist (default=False)

Returns:

A (potentially empty) list of type_ids

property index_column_name

next()

to_dataframe(cache=False)

property type_ids

bmtk.utils.sonata.types_table.remove_nans(types_dict)

Convert nan values to None in type row (dict)

bmtk.utils.sonata.utils module

bmtk.utils.sonata.utils.add_hdf5_magic(hdf5_handle)

bmtk.utils.sonata.utils.add_hdf5_version(hdf5_handle)

bmtk.utils.sonata.utils.check_magic(hdf5_file)

Check the magic attribute exists according to the sonata format

bmtk.utils.sonata.utils.get_attribute_h5(h5obj, attribut_name, default=None)

bmtk.utils.sonata.utils.get_node_ids(nodes_path, population)

bmtk.utils.sonata.utils.get_version(hdf5_file)

bmtk.utils.sonata.utils.listify(files)

bmtk.utils.sonata.utils.load_csv(csvfile)

bmtk.utils.sonata.utils.load_h5(h5file, mode='r')

Module contents