Building on the team’s expertise in developing algorithms and user-friendly software for use in biology under real-world conditions, the Center will focus on two Technology Research and Development (TR&D) projects: (1) deep learning-based image processing, and (2) workflows and accessibility of image processing algorithms for biologists. This work will not occur in isolation at the Center; rather, the Center will nucleate a larger community working on these two areas and serve as a catalyst and organizing force to create software and resources shared by all.
CellProfiler
CellProfiler is a versatile, open-source software tool for quantifying data from biological images, particularly in high-throughput experiments. CellProfiler is designed for modular, flexible, high-throughput analysis of images, measuring the size, shape, intensity, and texture of every cell (or other object) in every image. Using the point-and-click graphical user interface (GUI), users construct an image analysis “pipeline”, a sequential series of modules that each perform an image processing function such as illumination correction, object identification (segmentation), and object measurement. Users mix and match modules and adjust their settings to measure the phenotype of interest.
Piximi
Piximi is a no-code, local-first, open-source web application for performing image understanding tasks. It utilizes deep learning for the tasks of image classification and object segmentation, exposes model training and fine-tuning capabilities, and contains a library of models pretrained on biological images. Piximi contains an image viewer and annotation tool for labeling data and provides a variety of image-level and object-level measurements. Piximi is interoperable with other tools through the importing and exporting of common data formats.
Fiji
Fiji is an essential distribution of the ImageJ application, including the SciJava plugin framework and ImageJ2 data structures, as well as hundreds of curated community plugins for scientific image analysis. We place a continuous focus on maintenance and development to keep Fiji functional and relevant for the community as underlying technologies and image modalities change over time.
This involves maintaining hundreds of code repositories: responding to issues, community pull requests, and bug reports. Also, we must determine and implement essential architecture and engineering changes. The most significant recent direction has been modernizing the operating system-specific launchers shipped with Fiji to facilitate new versions of Java, in a manner we hope to be sustainably future-proof.
PyImageJ
PyImageJ is a modular Python framework for exchanging data between Python and Java. Powered by the JPype library, PyImageJ enables direct mapping of ImageJ data structures to Python, allowing interoperability without duplication. PyImageJ serves as an application entry point for users, with the capacity to create custom environments for any flavor of ImageJ, ImageJ2, or Fiji, or to wrap local installations - operating with a traditional GUI or headlessly. PyImageJ also acts as the foundation for many of our other projects.
- napari-imagej]
Napari-imagej (https://napari.imagej.net/) is a napari plugin built on PyImageJ, defining the necessary logic to adapt ImageJ data structures to those of napari. It also includes a GUI extension enabling napari users to search for and run ImageJ plugins. Plugins built to run headlessly will work automatically with napari layers. Classic ImageJ 1. x plugins which are tied to a UI are more limited, but can still be run via the option to open the ImageJ GUI from within napari, with data being freely transferrable between the two environments. - RunImageJScript
RunImageJScript is a CellProfiler module that uses PyImageJ to expose ImageJ functionality within CellProfiler. Creation of this module restored CellProfiler behavior that had been lost over time due to architectural changes, and provided the opportunity for direct collaboration between the Broad Institute and Eliceiri laboratory development teams, intending to increase shared architecture between CellProfiler and ImageJ.
SciJava Ops
SciJava Ops (https://ops.scijava.org/) is a framework for extensible algorithm retrieval and execution. The fundamental goal of SciJava Ops is to fit the “best” algorithm possible to each task. This is done by creating a declarative syntax that separates the "what" that a user wants to do from the "how" it should be done. The underlying machinery goes beyond just "matching" algorithms to requests: it converts and produces parameters on the fly, adapting existing algorithm implementations to fit the needs of the user.
In addition to the framework itself, this project includes SciJava Ops Image: a collection of hundreds of algorithms (Ops) for scientific image processing. We hope this effort provides immediate benefit to the scientific community as we continue to incorporate more algorithm libraries into the Ops framework.