bmtk.builder.network_adaptors package

Submodules

bmtk.builder.network_adaptors.dm_network module

class bmtk.builder.network_adaptors.dm_network.DenseNetwork(name, **network_props)

Bases: Network

edges_iter(trg_gids, src_network=None, trg_network=None)

Given a list of target gids, returns a generator for iteratoring over all possible edges.

It is preferable to use edges() method instead, it allows more flexibility in the input and can better indicate if their is a problem.

The order of the edges returned will be in the same order as the trg_gids list, but does not guarentee any secondary ordering by source-nodes and/or edge-type. If their isn’t a edge with a matching target-id then it will skip that gid in the list, the size of the generator can 0 to arbitrarly large.

Parameters:

• trg_gids – list of gids to match with an edge’s target.

• src_network – str, only returns edges coming from the specified source network.

• trg_network – str, only returns edges coming from the specified target network.

Returns:

iteration of bmtk.build.edge.Edge objects representing given edge.

edges_table()

import_nodes(nodes_file_name, node_types_file_name, population=None)

property nedges

Returns the total number of edges for this network.

property nnodes

Returns the number of nodes for this network.

nodes_iter(node_ids=None)

bmtk.builder.network_adaptors.dm_network.add_hdf5_attrs(hdf5_handle)

bmtk.builder.network_adaptors.dm_network.barrier(self) → None

Barrier

bmtk.builder.network_adaptors.dm_network_orig module

class bmtk.builder.network_adaptors.dm_network_orig.DenseNetworkOrig(name, **network_props)

Bases: Network

class EdgeTable(connection_map)

Bases: object

has_target(node_id)

property nsyn_table

property source_ids

property target_ids

trg_itr(trg_id)

class PropertyTable(nvalues, dtype)

Bases: object

itr_vals(src_id, trg_id)

edges_iter(trg_gids, src_network=None, trg_network=None)

Given a list of target gids, returns a generator for iteratoring over all possible edges.

It is preferable to use edges() method instead, it allows more flexibility in the input and can better indicate if their is a problem.

The order of the edges returned will be in the same order as the trg_gids list, but does not guarentee any secondary ordering by source-nodes and/or edge-type. If their isn’t a edge with a matching target-id then it will skip that gid in the list, the size of the generator can 0 to arbitrarly large.

Parameters:

• trg_gids – list of gids to match with an edge’s target.

• src_network – str, only returns edges coming from the specified source network.

• trg_network – str, only returns edges coming from the specified target network.

Returns:

iteration of bmtk.build.edge.Edge objects representing given edge.

edges_table()

import_nodes(nodes_file_name, node_types_file_name, population=None)

property nedges

Returns the total number of edges for this network.

property nnodes

Returns the number of nodes for this network.

nodes_iter(node_ids=None)

bmtk.builder.network_adaptors.dm_network_orig.add_hdf5_attrs(hdf5_handle)

bmtk.builder.network_adaptors.dm_network_orig.barrier(self) → None

Barrier

bmtk.builder.network_adaptors.edge_props_table module

class bmtk.builder.network_adaptors.edge_props_table.EdgeTypesTable(*args, **kwargs)

Bases: object

class bmtk.builder.network_adaptors.edge_props_table.EdgeTypesTableIFace

Bases: object

property n_edges

property n_syns

class bmtk.builder.network_adaptors.edge_props_table.EdgeTypesTableMPI(connection_map, network_name)

Bases: EdgeTypesTableMemory

Like parent used for storing actualized edges data, but designed for when using MPI and the edges-tables rules are split across different ranks/processors.

The data tables are saved in temporary files on the disk, so that the rank which is responsible for writing to the final hdf5 file. This could also be useful for very large networks built on one core.

TODO: Look into saving in memory, use MPI Gather/Send.

static get_tmp_table_path(rank=0, name=None)

save()

Saves edges data to hdf5 on the disk so that other ranks can read it (without MPISend).

class bmtk.builder.network_adaptors.edge_props_table.EdgeTypesTableMemory(connection_map, network_name)

Bases: object

A class for creating and storing the actual connectivity matrix plus all the possible (hdf5 bound) properties of an edge - unlike the ConnectionMap class which only stores the unevaluated rules each edge-types. There should be one EdgeTypesTable for each call to Network.add_edges(…)

By default edges in the SONATA edges.h5 table are stored in a (sparse) SxT table, S/T the num of source/target cells. If individual edge properties (syn_weight, syn_location, etc) and added then it must be stored in a SxTxN table, N the avg. number of synapses between each source/target pair. The actually number of edges (ie rows) saved in the SONATA file will vary.

create_property(prop_name, prop_type=None)

property edge_type_node_ids

Returns a table n_edges x 2, first column containing source_node_ids and second target_node_ids.

property edge_type_properties

free_data()

get_property_metatadata()

get_property_value(prop_name)

property hash_key

Creates a hash key for edge-types based on their (hdf5) properties, for grouping together properties of different edge-types. If two edge-types have the same (hdf5) properties they should have the same hash value.

iter_edges()

property n_edges

Number of unque edges/connections (eg rows in SONATA edges file). When multiple synapse can be safely represented with just one edge it will have n_edges < n_syns.

property n_syns

Number of synapses.

save()

set_nsyns(source_id, target_id, nsyns)

set_property_value(prop_name, edge_index, prop_value)

property source_nodes_map

property target_nodes_map

to_dataframe(**kwargs)

bmtk.builder.network_adaptors.edge_props_table.barrier(self) → None

Barrier

bmtk.builder.network_adaptors.edges_collator module

class bmtk.builder.network_adaptors.edges_collator.EdgesCollator(*args, **kwargs)

Bases: object

class bmtk.builder.network_adaptors.edges_collator.EdgesCollatorMPI(edge_types_tables, network_name)

Bases: object

For collecting all the different edge-types tables to make writing edges and iterating over the entire network easier. Similar to above but for when edge-rules data are split across multiple MPI ranks/processors. Can also be utlized for single core building when the network is too big to store in memory at once.

TODO: Consider saving tables to memory on each rank, and using MPI Gather/Send.

get_edge_group_ids(chunk_id)

get_edge_group_indices(chunk_id)

get_edge_type_ids(chunk_id)

get_group_data(chunk_id)

get_group_metadata(group_id)

for a given group_id return all the property dataset metadata; {name, type, size}, across all ranks.

get_group_property(prop_name, group_id, chunk_id)

get_source_node_ids(chunk_id)

get_target_node_ids(chunk_id)

static get_tmp_table_path(rank=0, name=None)

property group_ids

itr_chunks()

process()

sort(sort_by, sort_group_properties=True)

class bmtk.builder.network_adaptors.edges_collator.EdgesCollatorSingular(edge_types_tables, network_name)

Bases: object

Used to collect all the edges data-tables created and stored in the EdgeTypesTable to simplify the process of saving into a SONATA edges file. All the actual edges may be stored across diffrent edge-type-tables/mpi-ranks and needs to be merged together (and possibly sorted) before writing to HDF5 file.

assign_groups()

get_edge_group_ids(chunk_id)

get_edge_group_indices(chunk_id)

get_edge_type_ids(chunk_id)

get_group_data(chunk_id)

get_group_metadata(group_id)

get_group_property(group_name, group_id, chunk_id)

get_source_node_ids(chunk_id)

get_target_node_ids(chunk_id)

property group_ids

itr_chunks()

process()

sort(sort_by, sort_group_properties=True)

In memory sort of the dataset

Parameters:

• sort_by

• sort_group_properties

bmtk.builder.network_adaptors.edges_collator.barrier(self) → None

Barrier

bmtk.builder.network_adaptors.network module

class bmtk.builder.network_adaptors.network.Network(name, **network_props)

Bases: object

The Network class is used for building and saving a brain network/circuit. By default it will save to SONATA format for running network simulations using BioNet, PointNet, PopNet or FilterNet bmtk modules, however it can be generalized to build any time of network for any time of simulation.

Building the network:

For the general use-case building a network consists of 4 steps, each with a corresponding method.

1. Initialize the network:

   net = Network("network_name")
   

2. Create nodes (ie cells) using the add_nodes() method:

   net.add_nodes(N=80, model_type='Biophysical', ei='exc')
   net.add_nodes(N=20, model_type='IntFire', ei='inh')
   ...
   

3. Create Connection rules between different subsets of nodes using add_edges() method:

   net.add_edges(source={'ei': 'exc'}, target={'ei': 'inh'},
                 connection_rule=my_conn_func, synaptic_model='e2i')
   ...
   

4. Finally build the network and save the files:

   net.build()
   net.save(output_dir='network_path')
   

See the bmtk documentation, or the method doc-strings for more advanced functionality

Network Accessor methods:

nodes()

Will return a iterable of Node objects for each node created. The Node objects can be used like dictionaries to fetch their properties. By default returns all nodes in the network, but you can filter out a given subset by passing in property/values pairs:

for node in net.nodes(model_type='Biophysical', ei='exc'):
    assert(node['ei'] == 'exc')
    ...

edges()

Like the nodes() methods, but insteads returns a list of Edge type objects. It too can be filtered by an edge property:

for edge in net.edges(synaptic_model='exp2'):
    ...

One can also pass in a list of source (or target) to filter out only those edges which belong to a specific subset of cells:

for edge in net.edges(target_nodes=net.nodes(ei='exc')):
...

Network Properties:

• name - name of the network

• nnodes - number of nodes (cells) in the network.

• nedges - number of edges. Will be zero if build() method hasn’t been called

add_edges(source=None, target=None, connection_rule=1, connection_params=None, iterator='one_to_one', **edge_type_properties)

Used to create the connectivity matrix between subsets of nodes. The actually connections will not be created until the build() method is called, using the ‘connection_rule.

Node Selection:

To specify what subset of nodes will be used for the pre- and post-synaptic one can use a dictionary to filter the nodes. In the following all inh nodes will be used for the pre-synaptic neurons, but only exc fast-spiking neurons will be used in the post-synaptic neurons (If target or source is not specified all neurons will be used):

net.add_edges(source={'ei': 'inh'}, target={'ei': 'exc', 'etype': 'fast-spiking'},
              dynamic_params='i2e.json', synaptic_model='alpha', ...)

In the above code there is one connection between each source/target pair of nodes, but to create a multi-graph with N connections between each pair use ‘connection_rule’ parameter with an integer value:

net.add_edges(
    source={'ei': 'inh'},
    target={'ei': 'exc', 'etype': 'fast-spiking'},
    connection_rule=M,
    ...
)

Connection rules:

Usually the ‘connection_rule’ parameter will be the name of a function that takes in source-node and target-node object (which can be treated like dictionaries, and returns the number of connections (ie synapses, 0 or None if no synapses should exists) between the source and target cell:

def my_conn_fnc(source_node, target_node):
    src_pos = source_node['position']
    trg_pos = target_node['position']
    ...
    return N_syns

net.add_edges(source={'ei': 'exc'}, target={'ei': 'inh'}, connection_rule=my_conn_fnc, **opt_edge_attrs)

If the connection_rule function requires addition arguments use the ‘connection_params’ option:

def my_conn_fnc(source_node, target_node, min_edges, max_edges)
    ...

net.add_edges(connection_rule=my_conn_fnc, connection_params={'min_edges': 0, 'max_edges': 20}, ...)

Sometimes it may be more efficient or even a requirement that multiple connections are created at the same time. For example a post-synaptic neuron may only be targeted by a limited number of sources which couldn’t be done by the previous connection_rule function. But by setting property ‘iterator’ to value ‘all_to_one’ the connection_rule function now takes in as a value a list of N source neurons, a single target, and should return a list of size N:

def bulk_conn_fnc(sources, target):
    syn_list = np.zeros(len(sources))
    for source in sources:
        ....
    return syn_list

net.add_edges(connection_rule=bulk_conn_fnc, iterator='all_to_one', ...)

There is also a ‘all_to_one’ iterator option that will pair each source node with a list of all available target nodes.

Edge Properties:

Normally the properties used when creating a given type of edge will be shared by all the indvidual connections. To create unique values for each edge, the add_edges() method returns a ConnectionMap object:

def set_syn_weight_by_dist(source, target):
    src_pos, trg_pos = source['position'], target['position']
    ....
    return syn_weight

cm = net.add_edges(connection_rule=my_conn_fnc, model_template='Exp2Syn', ...)
                delay=2.0)
cm.add_properties('syn_weight', rule=set_syn_weight_by_dist)
cm.add_properties('delay', rule=lambda *_: np.random.rand(0.01, 0.50))

In this case the ‘model_template’ property has a value for all connections of this given type of edge. The ‘syn_weight’ and ‘delay’ properties will (most likely) be unique values. See ConnectionMap documentation for more info.

Parameters:

• source – A dictionary or list of Node objects (see nodes() method). Used to filter out pre-synaptic subset of nodes.

• target – A dictionary or list of Node objects). Used to filter out post-synaptic subset of nodes

• connection_rule – Integer or a function that returns integer(s). Rule to determine number of connections between each source and target node

• connection_params – A dictionary, used when the ‘connection_rule’ is a function that requires additional argments

• iterator – ‘one_to_one’, ‘all_to_one’, ‘one_to_all’. When ‘connection_rule’ is a function this sets how the subsets of source/target nodes are passed in. By default (one-to-one) the connection_rule is called for every source/target pair. ‘all-to-one’ will pass in a list of all possible source nodes for each target, and ‘all-to-one’ will pass in a list of all possible targets for each source.

• edge_type_properties – properties/attributes of the given edge type

Returns:

A ConnectionMap object

add_gap_junctions(source=None, target=None, resistance=1.0, conductance=None, distance_range=[0.0, 300.0], target_sections=['somatic'], connection_rule=1, iterator='one_to_one', **edge_type_properties)

A special function for marking a edge group as gap junctions. Just a wrapper for add_edges.

Parameters:

• resistance – gap junction resistance (megaohm)

• conductance – gap junction conductance (microsiemens). If specified, resistance is ignored.

add_nodes(N=1, **properties)

Used to add nodes (eg cells) to a network. User should specify the number of Nodes (N) and can use any properties/attributes they require to define the nodes. By default all individual cells will be assigned a unique ‘node_id’ to identify each node in the network and a ‘node_type_id’ to identify each group of nodes.

If a property is a singular value then said property will be shared by all the nodes in the group. If a value is a list of length N then each property will be uniquely assigned to each node. In the below example a group of 100 nodes is created, all share the same ‘model_type’ parameter but the pos_x values will be different for each node:

net.add_nodes(N=100, pos_x=np.random.rand(100), model_type='intfire1', ...)

You can use a tuple to store property values (in which the SONATA hdf5 will save it as a dataset with multiple columns). For example to have one property ‘positions’ which keeps track of the x/y/z coordinates of each cell:

net.add_nodes(N=100, positions=[(rand(), rand(), rand()) for _ in range(100)], ...)

Parameters:

• N – number of nodes in this group

• properties – Individual and group properties of given nodes

build(force=False)

Builds nodes and edges.

Parameters:

force – set true to force complete rebuilding of nodes and edges, if nodes() or save_nodes() has been called before then forcing a rebuild may change gids of each node.

clear()

Resets the network removing the nodes and edges created.

edges(target_nodes=None, source_nodes=None, target_network=None, source_network=None, **properties)

Returns a list of dictionary-like Edge objects, given filter parameters.

To get all edges from a network:

edges = net.edges()

To specify the target and/or source node-set:

edges = net.edges(target_nodes=net.nodes(type='biophysical'), source_nodes=net.nodes(ei='i'))

To only get edges with a given edge_property:

edges = net.edges(weight=100, syn_type='AMPA_Exc2Exc')

Parameters:

• target_nodes – gid, list of gid, dict or node-pool. Set of target nodes for a given edge.

• source_nodes – gid, list of gid, dict or node-pool. Set of source nodes for a given edge.

• target_network – name of network containing target nodes.

• source_network – name of network containing source nodes.

• properties – edge-properties used to filter out only certain edges.

Returns:

list of bmtk.builder.edge.Edge properties.

property edges_built

Returns True if the connectivity matrix has been instantiated for this network.

edges_iter(trg_gids, src_network=None, trg_network=None)

Given a list of target gids, returns a generator for iteratoring over all possible edges.

It is preferable to use edges() method instead, it allows more flexibility in the input and can better indicate if their is a problem.

The order of the edges returned will be in the same order as the trg_gids list, but does not guarentee any secondary ordering by source-nodes and/or edge-type. If their isn’t a edge with a matching target-id then it will skip that gid in the list, the size of the generator can 0 to arbitrarly large.

Parameters:

• trg_gids – list of gids to match with an edge’s target.

• src_network – str, only returns edges coming from the specified source network.

• trg_network – str, only returns edges coming from the specified target network.

Returns:

iteration of bmtk.build.edge.Edge objects representing given edge.

get_connections()

Returns a list of all bmtk.builder.connection_map.ConnectionMap objects representing all edge-types for this network.

import_nodes(nodes_file_name, node_types_file_name)

property name

Get the name (string) of this network.

property nedges

Returns the total number of edges for this network.

property nnodes

Returns the number of nodes for this network.

nodes(**properties)

Returns an iterator of Node (glorified dictionary) objects, filtered by parameters.

To get all nodes on a network:

for node in net.nodes():
    ...

To only get those nodes with properties that match a given list of parameter values:

for nod in net.nodes(param1=value1, param2=value2, ...):
    ...

Parameters:

properties – key-value pair of node attributes to filter returned nodes

Returns:

An iterator of Node objects

property nodes_built

Returns True if nodes has been instantiated for this network.

nodes_iter(nids=None)

save(output_dir='.', force_overwrite=True, compression='gzip')

Used to save the network files in the appropriate (eg SONATA) format into the output_dir directory. The file names will be automatically generated based on the network names.

To have more control over the output and file names use the save_nodes() and save_edges() methods.

Parameters:

• output_dir – string, directory where network files will be generated. Default, current working directory.

• force_overwrite – Overwrites existing network files.

save_edges(edges_file_name=None, edge_types_file_name=None, output_dir='.', src_network=None, trg_network=None, name=None, force_build=True, force_overwrite=False, compression='gzip')

Save the instantiated edges in SONATA format files.

Parameters:

• edges_file_name – file-name of hdf5 edges file. By default will use <src_network>_<trg_network>_edges.h5.

• edge_types_file_name – file-name of csv edge-types file. By default will use <src_network>_<trg_network>_edges.h5.

• output_dir – Directory where network files will be generated. Default, current working directory.

• src_network – Name of the source-node populations.

• trg_network – Name of the target-node populations.

• name – Name of file.

• force_build – Force to (re)build the connection matrix if it hasn’t already been built.

• force_overwrite – Overwrites existing network files.

save_nodes(nodes_file_name=None, node_types_file_name=None, output_dir='.', force_overwrite=True, compression='gzip')

Save the instantiated nodes in SONATA format files.

Parameters:

• nodes_file_name – file-name of hdf5 nodes file. By default will use <network.name>_nodes.h5.

• node_types_file_name – file-name of the csv node-types file. By default will use <network.name>_node_types.csv

• output_dir – Directory where network files will be generated. Default, current working directory.

• force_overwrite – Overwrites existing network files.

bmtk.builder.network_adaptors.network.barrier(self) → None

Barrier

bmtk.builder.network_adaptors.nxnetwork module

class bmtk.builder.network_adaptors.nxnetwork.NxNetwork(name, **network_props)

Bases: Network

property nedges

Returns the total number of edges for this network.

property nnodes

Returns the number of nodes for this network.

Module contents