Documentation

Mapping

If you have a single neuron, how can you find other neurons of the same or similar type within or between data sources?

27 Dec 2021

Superseded by the workshop

This page is kept for reference from the 2021 Hacking the connectome workshop. The current material is Session 2 · Bridging & Identity on the VFB Self-led Learning Sessions — the same ground against live VFB data, worked through in Python, R, an MCP-capable assistant, VFB Chat and the 3D browser.

!pip install vfb-connect --upgrade
# Import libs and initialise API objects
from vfb_connect.cross_server_tools import VfbConnect
import pandas as pd

vc = VfbConnect()

import pymaid
import navis

navis.set_pbars(jupyter=False)
pymaid.set_pbars(jupyter=False)

# Connect to the VFB CATMAID server hosting the FAFB data
rm = pymaid.connect_catmaid(server="https://fafb.catmaid.virtualflybrain.org/", api_token=None, max_threads=10)

# Test call to see if connection works 
print(f'Server is running CATMAID version {rm.catmaid_version}')

# Many functions return JSON-compatible nested data structures. This function coverts them to DataFrame.
def summary_2_df(summary, sort=None):
    """Convert summary to DataFrame.  Optionally specify a set of columns to sort as a list of strings"""
    if sort:
        return pd.DataFrame.from_records(summary).sort_values(sort)
    else:
        return pd.DataFrame.from_records(summary)
WARNING: Could not load OpenGL library.
INFO  : Global CATMAID instance set. Caching is ON. (pymaid)
Server is running CATMAID version 2020.02.15-925-gf56795c9c

Mapping

Use case: If I have a single neuron, how can I find other neurons of the same or similar type within or between data sources?

Mapping via common parent type

# lets take some examples from a discovery query on the previous spreadhseet
sayin_tab = pd.DataFrame.from_records(vc.get_instances_by_dataset('Sayin2019', summary=True))
sayin_tab

labelsymbolidtagsparents_labelparents_iddata_sourceaccessiontemplatesdatasetlicense
0OA-VPM3 (FAFB:1329078)VFB_001001drEntity|Octopaminergic|Adult|Anatomy|has_image|...octopaminergic VPM3 neuronFBbt_00110151catmaid_fafb1329078JRC2018UnisexSayin2019https://creativecommons.org/licenses/by-sa/4.0...
1OA-VPM4 (FAFB:1191261)VFB_001001dqEntity|Octopaminergic|Adult|Anatomy|has_image|...octopaminergic VPM4 neuronFBbt_00110152catmaid_fafb1191261JRC2018UnisexSayin2019https://creativecommons.org/licenses/by-sa/4.0...
oct_VPM3 = summary_2_df(vc.get_instances('octopaminergic VPM3 neuron', summary=True))
oct_VPM3

labelsymbolidtagsparents_labelparents_iddata_sourceaccessiontemplatesdatasetlicense
0OA-VPM3(NO2/NO3)_R - 329566174VFB_jrchk10dEntity|Octopaminergic|Adult|Anatomy|has_image|...octopaminergic VPM3 neuronFBbt_00110151neuprint_JRC_Hemibrain_1point1329566174JRC2018Unisex|JRC_FlyEM_HemibrainXu2020NeuronsV1point1https://creativecommons.org/licenses/by/4.0/le...
1OA-VPM3(NO2/NO3)_L - 5813061260VFB_jrchk10eEntity|Octopaminergic|Adult|Anatomy|has_image|...octopaminergic VPM3 neuronFBbt_00110151neuprint_JRC_Hemibrain_1point15813061260JRC_FlyEM_Hemibrain|JRC2018UnisexXu2020NeuronsV1point1https://creativecommons.org/licenses/by/4.0/le...
2OA-VPM3 (FAFB:1329078)VFB_001001drEntity|Octopaminergic|Adult|Anatomy|has_image|...octopaminergic VPM3 neuronFBbt_00110151catmaid_fafb1329078JRC2018UnisexSayin2019https://creativecommons.org/licenses/by-sa/4.0...
oct_VPM3_images = vc.neo_query_wrapper.get_images(oct_VPM3['id'], stomp=True, template='JRC2018Unisex', image_folder = 'oct_VPM3b')
oct_VPM3_images

labelsymbolidtagsparents_labelparents_iddata_sourceaccessiontemplatesdatasetlicensefilename
0OA-VPM3 (FAFB:1329078)VFB_001001drEntity|Octopaminergic|Adult|Anatomy|has_image|...octopaminergic VPM3 neuronFBbt_00110151catmaid_fafb1329078JRC2018UnisexSayin2019https://creativecommons.org/licenses/by-sa/4.0...OA_VPM3__FAFB_1329078_.swc
1OA-VPM3(NO2/NO3)_L - 5813061260VFB_jrchk10eEntity|Octopaminergic|Adult|Anatomy|has_image|...octopaminergic VPM3 neuronFBbt_00110151neuprint_JRC_Hemibrain_1point15813061260JRC_FlyEM_Hemibrain|JRC2018UnisexXu2020NeuronsV1point1https://creativecommons.org/licenses/by/4.0/le...OA_VPM3_NO2_NO3__L___5813061260.swc
2OA-VPM3(NO2/NO3)_R - 329566174VFB_jrchk10dEntity|Octopaminergic|Adult|Anatomy|has_image|...octopaminergic VPM3 neuronFBbt_00110151neuprint_JRC_Hemibrain_1point1329566174JRC2018Unisex|JRC_FlyEM_HemibrainXu2020NeuronsV1point1https://creativecommons.org/licenses/by/4.0/le...OA_VPM3_NO2_NO3__R___329566174.swc
oct_VPM3_images = vc.get_images_by_type('octopaminergic VPM3 neuron', stomp=True, template='JRC2018Unisex', image_folder = 'oct_VPM3')
nl = navis.read_swc('./oct_VPM3')
navis.plot3d(nl)
Running query: FBbt:00110151
Query URL: http://owl.virtualflybrain.org/kbs/vfb/instances?object=FBbt%3A00110151&prefixes=%7B%22FBbt%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FFBbt_%22%2C+%22RO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FRO_%22%2C+%22BFO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FBFO_%22%7D&direct=False
Query results: 3

Mapping via NBLAST

VFB supports image based queries that find neurons with similar morphology to a query neuron. We use a matrix of NBLAST scores for this purpose. This currently covers mapping between FAFB & Hemibrain and internal mappings within the hemibrain and FLyCircuit neurons. More mappings will be added soon, as well as support for other similarity mappings.

summary_2_df(vc.neo_query_wrapper.get_anatomical_individual_TermInfo(['VFB_001012ay'], summary=True))

labelsymbolidtagsparents_labelparents_iddata_sourceaccessiontemplatesdatasetlicense
0EPG-6L#1 (FAFB:676911)VFB_001012ayEntity|has_image|Adult|Anatomy|has_neuron_conn...EB-PB glomerulus 6-Vgall neuronFBbt_00111421catmaid_fafb676911JRC2018UnisexTurner_Evans2020https://creativecommons.org/licenses/by-sa/4.0...
similar_to_EPG6L1 = vc.get_similar_neurons('VFB_001012ay')
similar_to_EPG6L1 

idNBLAST_scorelabeltypessource_idaccession_in_source
0VFB_jrchjtk60.525EPG(PB08)_L6 - 541870397[EB-PB 1 glomerulus-D/Vgall neuron]neuprint_JRC_Hemibrain_1point1541870397
1VFB_jrchjtk80.524EPG(PB08)_L6 - 912601268[EB-PB 1 glomerulus-D/Vgall neuron]neuprint_JRC_Hemibrain_1point1912601268
2VFB_jrchjtkb0.504EPG(PB08)_L6 - 788794171[EB-PB 1 glomerulus-D/Vgall neuron]neuprint_JRC_Hemibrain_1point1788794171
3VFB_jrchjtji0.481EL(EQ5)_L - 1036753721[EBw.AMP.s-Dga-s.b neuron]neuprint_JRC_Hemibrain_1point11036753721
EPG6L1_similar_images = vc.neo_query_wrapper.get_images(list(similar_to_EPG6L1['id']) + ['VFB_001012ay'], 
                                template='JRC2018Unisex',
                                image_folder='EPG6L1',
                                stomp=True) # Note - stomp allows an existing folder to be over-written


EPG6L1_similar_images

labelsymbolidtagsparents_labelparents_iddata_sourceaccessiontemplatesdatasetlicensefilename
0EL(EQ5)_L - 1036753721VFB_jrchjtjiEntity|has_image|Adult|Anatomy|has_neuron_conn...EBw.AMP.s-Dga-s.b neuronFBbt_00049180neuprint_JRC_Hemibrain_1point11036753721JRC_FlyEM_Hemibrain|JRC2018UnisexXu2020NeuronsV1point1https://creativecommons.org/licenses/by/4.0/le...EL_EQ5__L___1036753721.swc
1EPG(PB08)_L6 - 788794171VFB_jrchjtkbEntity|has_image|Adult|Anatomy|has_neuron_conn...EB-PB 1 glomerulus-D/Vgall neuronFBbt_00047030neuprint_JRC_Hemibrain_1point1788794171JRC_FlyEM_Hemibrain|JRC2018UnisexXu2020NeuronsV1point1https://creativecommons.org/licenses/by/4.0/le...EPG_PB08__L6___788794171.swc
2EPG-6L#1 (FAFB:676911)VFB_001012ayEntity|has_image|Adult|Anatomy|has_neuron_conn...EB-PB glomerulus 6-Vgall neuronFBbt_00111421catmaid_fafb676911JRC2018UnisexTurner_Evans2020https://creativecommons.org/licenses/by-sa/4.0...EPG_6L_1__FAFB_676911_.swc
3EPG(PB08)_L6 - 912601268VFB_jrchjtk8Entity|has_image|Adult|Anatomy|has_neuron_conn...EB-PB 1 glomerulus-D/Vgall neuronFBbt_00047030neuprint_JRC_Hemibrain_1point1912601268JRC2018Unisex|JRC_FlyEM_HemibrainXu2020NeuronsV1point1https://creativecommons.org/licenses/by/4.0/le...EPG_PB08__L6___912601268.swc
4EPG(PB08)_L6 - 541870397VFB_jrchjtk6Entity|has_image|Adult|Anatomy|has_neuron_conn...EB-PB 1 glomerulus-D/Vgall neuronFBbt_00047030neuprint_JRC_Hemibrain_1point1541870397JRC_FlyEM_Hemibrain|JRC2018UnisexXu2020NeuronsV1point1https://creativecommons.org/licenses/by/4.0/le...EPG_PB08__L6___541870397.swc
list(EPG6L1_similar_images['parents_label'])
['EBw.AMP.s-Dga-s.b neuron',
 'EB-PB 1 glomerulus-D/Vgall neuron',
 'EB-PB glomerulus 6-Vgall neuron',
 'EB-PB 1 glomerulus-D/Vgall neuron',
 'EB-PB 1 glomerulus-D/Vgall neuron']
import trimesh

jrc2018u = trimesh.load_remote('https://github.com/schlegelp/navis-flybrains/raw/main/flybrains/meshes/JRC2018U.ply')
jrc2018u = navis.Volume(jrc2018u, color=(.9, .9, .9, .1), name='JRC2018U')
jrc2018u
<navis.Volume(name=JRC2018U, color=(0.9, 0.9, 0.9, 0.1), vertices.shape=(11943, 3), faces.shape=(24558, 3))>
import matplotlib.pyplot as plt

# nl = navis.read_swc('./EPG6L1')
fig, ax = navis.plot2d([nl, jrc2018u], figsize=(12, 12), method='3d_complex')

ax.elev = -90
ax.azim = -90
ax.dist = 5

#plt.savefig('EPG6L1.png', transparent=True, dpi=400)

png

## Generate a link to display these neurons on VFB

print(vc.get_vfb_link(list(similar_to_EPG6L1['id']) + ['VFB_001012ay'], template='JRC2018Unisex'))
https://v2.virtualflybrain.org/org.geppetto.frontend/geppetto?id=VFB_001012ay&i=VFB_00101567,VFB_jrchjtk6,VFB_jrchjtk8,VFB_jrchjtkb,VFB_jrchjtji

Note: to clean up ‘spiky’ neurons with obvious alignment issues you can use the navis function despike