Downloading Images

The ImageDownloadApi class in the AllenSDK provides a Python interface for accessing image data produced by the Allen Institute for Brain Science.

Contents:

Reference Documentation:

Download this file in .ipynb format here.

In [1]:
from allensdk.api.queries.image_download_api import ImageDownloadApi
from allensdk.api.queries.svg_api import SvgApi
from allensdk.config.manifest import Manifest

import matplotlib.pyplot as plt
from skimage.io import imread
import pandas as pd

import logging
import os
from base64 import b64encode

from IPython.display import HTML, display
%matplotlib inline

We will want to look at the images we download in this notebook. Here are a couple of functions that do this:

In [2]:
def verify_image(file_path, figsize=(18, 22)):
    image = imread(file_path)

    fig, ax = plt.subplots(figsize=figsize)
    ax.imshow(image)
    
    
def verify_svg(file_path, width_scale, height_scale):
    # we're using this function to display scaled svg in the rendered notebook.
    # we suggest that in your own work you use a tool such as inkscape or illustrator to view svg
    
    with open(file_path, 'rb') as svg_file:
        svg = svg_file.read()
    encoded_svg = b64encode(svg)
    decoded_svg = encoded_svg.decode('ascii')
    
    st = r'<img class="figure" src="data:image/svg+xml;base64,{}" width={}% height={}%></img>'.format(decoded_svg, width_scale, height_scale)
    display(HTML(st))

Finally, we will need an instance of ImageDownloadApi and an instance of SvgApi:

In [3]:
image_api = ImageDownloadApi()
svg_api = SvgApi()

Downloading a single Mouse Brain Section image

In this example, we will download a single section image from Mouse Brain Atlas experiment 71210895.

For now, we'll assume that we already know the id (unique integer identifier) of the image that we wish to download. In a later section, we'll use our ImageDownloadApi instance to find the ids of all the images from a particular experiment.

In [4]:
section_image_id = 70945123
file_path = '70945123.jpg'

The raw images can be quite large, so we will download downsampled versions. The downsample keyword argument reduces the size of the obtained image by a factor of $2^{\text{downsample}}$ along each axis.

In [5]:
downsample = 3

Now we download the image using our api object and view it in the notebook.

In [6]:
image_api.download_section_image(section_image_id, file_path, downsample=downsample)

verify_image(file_path)

2018-07-27 13:51:39,103 allensdk.api.api.retrieve_file_over_http INFO     Downloading URL: http://api.brain-map.org/api/v2/section_image_download/70945123?downsample=3

Downloading a single Mouse Connectivity image

In this example, we will download a single section image from Mouse Brain Connectivity experiment 297225422.

In [7]:
section_image_id = 297225716
file_path = '297225716_connectivity.jpg'
downsample = 3

In order to clearly distinguish projection signal from background structure, it helps to window and rescale each channel. We have precomputed channelwise windows for each section data set in the connectivity atlas. You can obtain these window parameters using the ImageDownloadApi

In [8]:
ranges = image_api.get_section_image_ranges([section_image_id])[0]

Then simply pass them in via the range keyword argument.

In [9]:
image_api.download_projection_image(section_image_id, file_path, downsample=downsample, range=ranges)

verify_image(file_path)

2018-07-27 13:51:42,496 allensdk.api.api.retrieve_file_over_http INFO     Downloading URL: http://api.brain-map.org/api/v2/projection_image_download/297225716?downsample=3&range=0,825,0,943,0,4095

If we set the projection keyword argument to True, we can download a mask which labels important features of the image, such as the injection site, detected projection signal, and the boundary of detectable tissue.

In [10]:
section_image_id = 297225716
file_path = '297225716_projection.jpg'
downsample = 3
projection=True

image_api.download_projection_image(section_image_id, file_path, downsample=downsample, 
                                    projection=projection)

verify_image(file_path)

2018-07-27 13:51:53,716 allensdk.api.api.retrieve_file_over_http INFO     Downloading URL: http://api.brain-map.org/api/v2/projection_image_download/297225716?downsample=3&projection=true

Downloading a single atlas image

If we know the id of the atlas image we want, we can download it directly. We'll use an image from the Adult Human Brain Reference Atlas for this example.

In [11]:
atlas_image_id = 112282603
downsample = 6
file_path = '112282603_nissl.jpg'

Atlas images consist of an annotation image layered over a biological specimen image. You can control which one you download via the annotation keyword argument.

Setting annotation to False will download the biological specimen (in this case, a Nissl section).

In [12]:
annotation = False
In [13]:
image_api.download_atlas_image(atlas_image_id, file_path, annotation=annotation, downsample=downsample)

verify_image(file_path)

2018-07-27 13:52:01,659 allensdk.api.api.retrieve_file_over_http INFO     Downloading URL: http://api.brain-map.org/api/v2/atlas_image_download/112282603?downsample=6&annotation=false

We can download the structure drawings and labels (instead of the nissl) by setting annotation=True

In [14]:
atlas_image_id = 112282603
annotation = True
downsample = 6
file_path = '112282603_annotation.jpg'

image_api.download_atlas_image(atlas_image_id, file_path, annotation=annotation, downsample=downsample)
verify_image(file_path)

2018-07-27 13:52:06,570 allensdk.api.api.retrieve_file_over_http INFO     Downloading URL: http://api.brain-map.org/api/v2/atlas_image_download/112282603?downsample=6&annotation=true

Downloading a single atlas svg

We make the atlas drawings available (sans textual labels) as SVG. We'll download and display the SVG for the above atlas image:

In [15]:
from allensdk.api.queries.svg_api import SvgApi

svg_api = SvgApi()
svg_api.download_svg(atlas_image_id, file_path='112282603.svg')

2018-07-27 13:52:10,197 allensdk.api.api.retrieve_file_over_http INFO     Downloading URL: http://api.brain-map.org/api/v2/svg_download/112282603
In [16]:
verify_svg('112282603.svg', 35, 35)

Listing images from an atlas (or section data set)

The ImageDownloadApi class can list the images in an atlas or section dataset. For an atlas (the same Human Brain reference atlas as above), this looks like:

In [17]:
atlas_id = 138322605

# image_api.section_image_query(section_data_set_id) is the analogous method for section data sets
atlas_image_records = image_api.atlas_image_query(atlas_id)

# this returns a list of dictionaries. Let's convert it to a pandas dataframe
atlas_image_dataframe = pd.DataFrame(atlas_image_records)

# and use the .head() method to display the first few rows
atlas_image_dataframe.head()
Out[17]:
annotated axes bits_per_component data_set_id expression_path failed height id image_height image_type ... path projection_function resolution section_number specimen_id structure_id tier_count width x y
0 True None 8 100149965 None False 43392 102339919 47703 Primary ... /external/devhuman/prod14/3760000240/376000024... None 0.971 111 None 10155 9 29056 2176 896
1 True None 8 100149965 None False 50176 102339937 53159 Primary ... /external/devhuman/prod14/3760000257/376000025... None 0.971 179 None 10155 9 36992 1280 2816
2 True None 8 100149965 None False 58080 102339949 58118 Primary ... /external/devhuman/prod14/3760000274/376000027... None 0.971 247 None 10155 9 49760 0 0
3 True None 8 100149965 None False 61824 102339958 61872 Primary ... /external/devhuman/prod14/3760000289/376000028... None 0.971 307 None 10155 9 47904 0 0
4 True None 8 100149965 None False 68447 102291506 68470 Primary ... /external/devhuman/prod14/3760000363/376000036... None 0.971 370 None 10155 10 49791 0 0

5 rows × 25 columns

The id column of this dataframe contains the ids of the images in this atlas - these are the same integer identifiers that we have been using to download single images.

In [18]:
atlas_image_dataframe['id'].head()
Out[18]:
0    102339919
1    102339937
2    102339949
3    102339958
4    102291506
Name: id, dtype: int64

Downloading all of the images from a section data set

Specifically, Brainspan ISH experiment 100147602.

We'll start with some parameters:

In [19]:
section_data_set_id = 100147602
downsample=6

section_image_directory = '75214738_section_images'
format_str = '.jpg'

Now we need to get the image ids for all of the images in this data set:

In [20]:
section_images = image_api.section_image_query(section_data_set_id)
section_image_ids = [si['id'] for si in section_images]

print(len(section_image_ids))

287

and then iterate:

In [21]:
# You have probably noticed that the AllenSDK has a logger which notifies you of file downloads. 
# Since we are downloading ~300 images, we don't want to see messages for each one.
# The following line will temporarily disable the download logger.
logging.getLogger('allensdk.api.api.retrieve_file_over_http').disabled = True

for section_image_id in section_image_ids:
    
    file_name = str(section_image_id) + format_str
    file_path = os.path.join(section_image_directory, file_name)
    
    Manifest.safe_make_parent_dirs(file_path)
    image_api.download_section_image(section_image_id, file_path=file_path, downsample=downsample)
    
# re-enable the logger
logging.getLogger('allensdk.api.api.retrieve_file_over_http').disabled = False

We should now have a directory containing the requested images.

In [22]:
file_names = os.listdir(section_image_directory)
print(len(file_names))

287
In [23]:
display_file_paths = [os.path.join(section_image_directory, dfn) for dfn in file_names[::50]]

fig, ax = plt.subplots(ncols=len(display_file_paths), figsize=(18, 22))
for ii, dfp in enumerate(display_file_paths):
    image = imread(dfp)
    ax[ii].imshow(image)
    ax[ii].get_xaxis().set_visible(False)
    ax[ii].get_yaxis().set_visible(False)

Listing the section data sets in a product

Related experiments are grouped into products. The ImageDownloadApi has a method for listing the section data sets in a particular product.

Here we will use this method to list section data sets from the Human Brain ISH SubCortex Study (id=10 per the above link).

In [24]:
human_subcortex_datasets = image_api.get_section_data_sets_by_product([10])
In [25]:
print('there are {} section data sets in the Human Brain ISH SubCortex Study'.format(len(human_subcortex_datasets)))

there are 519 section data sets in the Human Brain ISH SubCortex Study
In [26]:
pd.DataFrame(human_subcortex_datasets).head()
Out[26]:
blue_channel delegate expression failed failed_facet green_channel id name plane_of_section_id qc_date red_channel reference_space_id rnaseq_design_id section_thickness specimen_id sphinx_id storage_directory weight
0 None False True False 734881840 None 100095749 None 1 2010-04-08T15:45:52Z None None None 20.0 704524 127968 None 5240
1 None False False False 734881840 None 100126177 None 1 2010-09-24T20:07:06Z None None None 20.0 706491 40102 None 5240
2 None False True False 734881840 None 100096328 None 1 2010-05-03T18:08:19Z None None None 20.0 704521 48691 None 5240
3 None False True False 734881840 None 100095573 None 1 2010-04-07T19:28:54Z None None None 20.0 704523 154805 None 5240
4 None False True False 734881840 None 100095557 None 1 2010-04-21T18:12:44Z None None None 20.0 704523 162480 None 5240