Aleksandar (Alex) Vakanski

 

Machine Learning-based Analysis of Breast Ultrasound Images

The project employs machine learning approaches for identification and segmentation of breast tumors in ultrasound images. This is a collaborative research project led by Dr. Min Xian from the Department of Computer Science at the University of Idaho. In general, medical image processing for automated cancer detection is challenging. In particular, breast ultrasound images are characterized by low contrast and low signal-to-noise ratio resulting in fuzzy borders of the tissue, substantial presence of image artifacts, and poor depth perception.

One challenge that the project addresses is tumor segmentation in breast ultrasound images. Toward this goal, we considered incorporating topological and anatomical prior information into a deep learning model for image segmentation. Incorporating human expertise and domain knowledge is particularly important for medical image processing applications, marked with small datasets, and objects of interest in the form of organs or lesions not typically seen in traditional datasets. However, the incorporation of prior knowledge for breast tumor diagnosis is challenging, since shape, boundary, curvature, intensity, or other common medical priors vary significantly across patients and cannot be employed. In our work, we employed maps of visual saliency as prior image topology knowledge. Visual saliency refers to image maps that assign values to the pixels based on the ability to stand out and differentiate from the neighboring pixels. For instance, tumor regions in medical images have higher visual saliency and attract radiologists’ attention when examining images for breast cancer screening. We proposed a deep learning model that integrates visual saliency by using the attention mechanism. A U-Net architecture consisting of fully convolutional encoder and decoder sub-networks with skip connections was used for incorporating the prior knowledge in the form of a pyramid of visual saliency maps. The attention blocks are integrated with the layers of the encoder to force the network to learn feature representations that place spatial attention to target regions with high saliency values. Unlike similar deep learning models that introduce attention blocks by merging internal feature representations from different layers, the proposed approach employs external auxiliary inputs in the form of visual saliency maps for training the model parameters.

Figure: A breast ultrasound image, the segmented tumor in the image, and a visual saliency map indicating the most salient regions in the image.

In other related works, we designed models for segmentation of small-size tumors in breast ultrasound images. Namely, detecting small tumors has been particularly difficult for recent machine learning models that have otherwise exhibited impressing performance in detecting large-size tumors. On the other hand, small tumors are critically important for finding early-stage cancers using computer-aided diagnosis, since early detection is the key to improving the survival rate. Toward this goal, our team developed STAN (Small Tumor-Aware Network), which fuses image context information at different scales to achieve improved performance.

Figure: STAN architecture. The block sizes do not represent the actual feature maps.

Our team also conducted research on models for classification of breast ultrasound images. We studied the correlation between complexity measures and the generalization abilities of deep learning classifiers, and we found that PAC-Bayes flatness and path norm measures produce the most consistent explanation for the combination of models and data. Similarly, our lightweight multitask model EMT-Net is designed for simultaneous classification and segmentation of breast ultrasound images. The model employs a new numerically stable loss function that easily controls the balance between the sensitivity and specificity of cancer detection.

We are also interested in enhancing the explainability of machine learning models for breast cancer detection. BI-RADS-Net incorporates tasks for explaining and classifying breast tumors, by learning feature representations relevant to clinical diagnosis. Explanations of the predictions (benign or malignant) are provided in terms of the BIRADS descriptors of shape, orientation, margin, echo pattern, and posterior features. Additionally, our approach predicts the likelihood of malignancy of the findings, which relates to the BI-RADS assessment category reported by clinicians.

Figure: Network architecture of the proposed BI-RADS-Net for breast ultrasound computer-aided diagnosis

Another line of research of our group is directed toward analysis and segmentation of histopathology images. We proposed an approach that employs a bending loss regularization for separating overlapped nuclei in histopathology images. The bending loss applies high penalties to contour points with large curvature, to improve the segmentation of overlapped nuclei. Our team also designed TA-Net, which is a multitask learning model for gland segmentation, based on learning shared representation from two tasks: instance segmentation and gland topology estimation. Similarly, in another article we applied a GAN network to generate synthetic histopathology images. The model is dubbed Sharp-GAN, and it uses a sharpness loss regularized GAN for image synthesis.

Figure: TA-Net takes image patches as input and outputs the gland instance (INST) map and medial axis (MA) distance map. The Marker Map is generated using the MA map.

 

Publications

1. B. Zhang, A. Vakanski and M. Xian, "BI-RADS-NET-V2: A composite multi-task neural network for computer-aided diagnosis of breast cancer in ultrasound images with semantic and quantitative explanations," IEEE Access, vol. 11, pp. 79480–79494, Jul. 2023. [DOI: 10.1109/ACCESS.2023.3298569] [Bibtex] [IEEE Explore

2. M. Karimzadeh, A. Vakanski, M. Xian, and B. Zhang, "Post-hoc explainability of BI-RADS descriptors in a multi-task framework for breast cancer detection and segmentation," in Proceedings of the 33rd IEEE International Workshop on Machine Learning and Signal Processing (MLSP 2023), Rome, Italy, pp. 1-6, 2023. [DOI: 10.1109/MLSP55844.2023.10286006] [BibTex] [IEEE Explore

3. B. Shareef, M. Xian, A. Vakanski, and H. Wang, "Breast ultrasound tumor classification using a hybrid multitask CNN-transformer network," in Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2023), Vancouver, Canada, pp. 344-353, 2023. [DOI: 10.1007/978-3-031-43901-8_33] [BibTex] [Springer Link

4. H. Wang, M. Xian, A. Vakanski, and B. Shareef, "SIAN: Style-guided instance-adaptive normalization for multi-organ histopathology image synthesis," in Proceedings of the International Symposium on Biomedical Imaging (ISBI 2023), Cartagena de Indias, Colombia, pp. 1-5, 2023. [DOI: 10.1109/ISBI53787.2023.10230507]

5. S. Butte, H. Wang, A. Vakanski, and M. Xian, "Enhanced Sharp-GAN for histopathology image synthesis," in Proceedings of the International Symposium on Biomedical Imaging (ISBI 2023), Cartagena de Indias, Colombia, pp. 1-5, 2023. [DOI: 10.1109/ISBI53787.2023.10230516]

6. B. Shareef, A. Vakanski, P. E. Freer, and M. Xian, "ESTAN: Enhanced small tumor-aware network for breast ultrasound image segmentation," Healthcare, vol. 10, no. 11, pp. 1–14, Nov. 2022. [DOI: 10.3390/healthcare10112262] [Bibtex] [MDPI Healthcare

7. S. Sun, M. Xian, A. Vakanski, and H. Ghanem, "MIRST-DM: Multi-instance RST with Drop-Max Layer for Robust Classification of Breast Cancer," in Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2022), Singapore, Singapore, pp. 401-410, 2022. [DOI: 10.1007/978-3-031-16440-8_39] [BibTex] [Springer Link

8. J. Shi, A. Vakanski, M. Xian, J. Ding, and C. Ning, "EMT-Net: Efficient multitask network for computer-aided diagnosis of breast cancer," in Proceedings of the International Symposium on Biomedical Imaging (ISBI 2022), Kolkata, India, pp. 1-5, 2022. [DOI: 10.1109/ISBI52829.2022.9761438] [PMCID: PMC9074851] [BibTex] [IEEE Explore

9. S. Butte, H. Wang, M. Xian, and A. Vakanski, "Sharp-GAN: Sharpness loss regularized GAN for histopathology image synthesis," in Proceedings of the International Symposium on Biomedical Imaging (ISBI 2022), Kolkata, India, pp. 1-5, 2022. [DOI: 10.1109/ISBI52829.2022.9761534] [PMCID: PMC9074846] [BibTex] [IEEE Explore

10. H. Wang, M. Xian, and A. Vakanski, "TA-Net: Topology-aware network for gland segmentation," in Proceedings of the Winter Conference on Applications of Computer Vision (WACV 2022), Waikoloa, USA, pp. 1556-1564, 2022. [PMCID: PMC9063467] [BibTex] [WACV 2022 Open Access

11. B. Zhang, A. Vakanski, and M. Xian, "BI-RADS-Net: An explainable multitask learning approach for cancer diagnosis in breast ultrasound images," in Proceedings of the 31st IEEE International Workshop on Machine Learning and Signal Processing (MLSP 2021), Gold Coast, Australia, pp. 1-6, 2021. [DOI: 10.1109/MLSP52302.2021.9596314] [BibTex] [IEEE Explore

12. A. Vakanski, and M. Xian, "Evaluation of complexity measures for deep learning generalization in medical image analysis," in Proceedings of the 31st IEEE International Workshop on Machine Learning and Signal Processing (MLSP 2021), Gold Coast, Australia, pp. 1-6, 2021. [DOI: 10.1109/MLSP52302.2021.9596501] [BibTex] [IEEE Explore

13. A. Vakanski, M. Xian, and P. Freer, "Attention enriched deep learning model for breast tumor segmentation in ultrasound images," Ultrasound in Medicine and Biology, vol. 46, no. 10, pp. 2819–2833, Oct. 2020. [Bibtex] [Elsevier Science Direct

14. H. Wang, M. Xian, and A. Vakanski, "Bending loss regularized network for nuclei segmentation in histopathology images," in Proceedings of the 17th IEEE International Symposium on Biomedical Imaging (ISBI 2020), Iowa City, USA, pp. 1-5, 2020. [DOI: 10.1109/ISBI45749.2020.9098611] [PMCID: PMC7733529] [BibTex] [IEEE Explore]  

15. B. Shareef, M. Xian, and A. Vakanski, "STAN: Small tumor-aware network for breast ultrasound image segmentation," in Proceedings of 17th IEEE the International Symposium on Biomedical Imaging (ISBI 2020), Iowa City, USA, pp. 1-5, 2020. [DOI: 10.1109/ISBI45749.2020.9098691] [PMCID: PMC7733528] [BibTex] [IEEE Explore]