SEGMENTATION AND QUANTIFICATION OF NUCLEI IN FLUORESCENT IMAGES USING DEEP LEARNING TECHNIQUE

Abstract

<p>In the study of bioimage analysis, the segmentation of cell nuclei is the pivotal stage. The identification of the nucleus aids researchers in comprehending the underlying processes for drug discovery and detection of cancerous cells. Nucleus segmentation is a challenging problem due to the presence of overlapping nuclei, image intensity heterogeneities, and image noise. Furthermore, performing manual segmentation at the pixel level poses a major obstacle due to its time-consuming nature, the need for expert professionals, and its susceptibility to errors. To overcome these shortcomings, we provide a pipeline to perform automatic nucleus segmentation and quantification for fluorescent images using deep learning. In this study, to perform nucleus segmentation, we modify the U-Net model by substituting its encoder with the EfficientNet model while retaining the U-shaped structure unaltered. This adaptation in feature extraction encoder path has shown U-Net to perform better in creating fine-grained segmentation map. The proposed pipeline delivers notable results, with an F1-score of 87% and an Intersection over Union (IoU) of 80%. A post-processing step was employed to enable morphological quantification for each segmented nucleus. Additionally, we present novel datasets with annotated boundaries for over 3000 nuclei, tailored for binary segmentation tasks. Therefore, the work presented in this study demonstrates the potential for automatic nuclei extraction from fluorescent images.</p>