NeuroBlocks -
Visual Tracking of Segmentation and Proofreading for Large Connectomics Projects
IEEE SciVis 2015
Ali K.
Al‑Awami
Johanna Beyer Daniel Haehn Narayanan Kasthuri Jeff W. Lichtman Hanspeter Pfister Markus Hadwiger
KAUST Harvard University Harvard University Boston University Harvard University Harvard University KAUST
teaser
In the field of connectomics, neuroscientists acquire electron microscopy volumes at nanometer resolution in order to reconstruct a detailed wiring diagram of the neurons in the brain. The resulting image volumes, which often are hundreds of terabytes in size, need to be segmented to identify cell boundaries, synapses, and important cell organelles. However, the segmentation process of a single volume is very complex, time-intensive, and usually performed using a diverse set of tools and many users. To tackle the associated challenges, this paper presents NeuroBlocks, which is a novel visualization system for tracking the state, progress, and evolution of very large volumetric segmentation data in neuroscience. NeuroBlocks is a multi-user web-based application that seamlessly integrates the diverse set of tools that neuroscientists currently use for manual and semi-automatic segmentation, proofreading, visualization, and analysis. NeuroBlocks is the first system that integrates this heterogeneous tool set, providing crucial support for the management, provenance, accountability, and auditing of large-scale segmentations. We describe the design of NeuroBlocks, starting with an analysis of the domain-specific tasks, their inherent challenges, and our subsequent task abstraction and visual representation. We demonstrate the utility of our design based on two case studies that focus on different user roles and their respective requirements for performing and tracking the progress of segmentation and proofreading in a large real-world connectomics project.

bibtex

@ARTICLE{awami_neuroblocks_2015,
author = {A.K. Al-Awami and J. Beyer and D. Haehn and
N. Kasthuri and J.W. Lichtman and H. Pfister and M. Hadwiger},
title = {NeuroBlocks - Visual Tracking of Segmentation
and Proofreading for Large Connectomics Projects},
journal = {IEEE Transactions on Visualization and
Computer Graphics (Proceedings IEEE SciVis 2015)},
year = {2015},
volume = {22},
number = {1},
pages = {738--746},
}

youtube

figures

NeuroBlocks Visualizing large-scale segmentation projects in connectomics. Left: The main view of NeuroBlocks, comprising a scalable “pixel-based” visualization of the current segmentation state on top, and a timeline for exploring the project’s evolution at the bottom. Top right: Detailed view of a segmentation task assigned to a user, showing recent modifications and thumbnails of the corresponding tool states. Bottom right: Linked petascale volume rendering of the underlying microscopy data and the selected segmented object. connectomics workflow Connectomics workflow.Top: High-resolution electron microscope images are acquired and registered into a 3D volume before neuronal structures are segmented and labeled. Automatic segmentation approaches require a subsequent proofreading step before the data can be visualized and analyzed. Bottom: Visualization of the typical relative durations of these tasks in a medium- to large-scale neuroscience project. design iterations Design iterations of NeuroBlocks.(a) virtual lab notebook; (b) provenance-driven analysis; (c) data-centric view; (d) final design. multi-scale pixel view Multi-scale pixel view of the current segmentation state.“Downsampling” the full-resolution pixel view ensures overview visibility of all data. Here, as an example, elements are sorted by size, and color-coded according to the number of recent modifications. All individual segments comprising the selected super-pixel are shown in the top right. The bottom right shows a connectivity graph of the selected object. segmentation object detail view Segmentation object details and volume rendering integration.Details about the segmented object, all its segments, revision history, and collaborators are shown in this view. Recent changes to the segmentation can be examined in detail and at full resolution in the integrated volume renderer, and then be approved or discarded by the user. The view at the bottom right shows a “diff” of the most recent merge. system design NeuroBlocks system design.The NeuroBlocks server integrates user management, handling of meta data, and a provenance database. External applications can use the NeuroBlocks plugin API to communicate with NeuroBlocks and access the original volume and segmentation data, which is stored on a shared filesystem. case study Case study 2: Proofreader workflow.After task creation and assignment by a project manager (a), it is displayed in the segmenter’s task list, which allows switching to the specified segment in the proofreading tool (b). Progress there is sent back to NeuroBlocks (c). After task completion, the project manager can double-check the new segmentation using the integrated volume renderer (d), and set the task to complete if satisfied.