• Conference paper

Endoscopic Ultrasound Image Synthesis Using a Cycle-Consistent Adversarial Network

Alexander Grimwood, Joao Ramalhinho, Zachary M. C. Baum, Nina Montaña-Brown, Gavin J. Johnson, Yipeng Hu, Matthew J. Clarkson, Stephen P. Pereira, Dean C. Barratt and Ester Bonmati Coll 2021. Endoscopic Ultrasound Image Synthesis Using a Cycle-Consistent Adversarial Network. Second International Workshop, ASMUS 2021, Held in Conjunction with MICCAI 2021. Strasbourg, France 27 Sep 2021 Springer. https://doi.org/10.1007/978-3-030-87583-1_17

TitleEndoscopic Ultrasound Image Synthesis Using a Cycle-Consistent Adversarial Network
AuthorsAlexander Grimwood, Joao Ramalhinho, Zachary M. C. Baum, Nina Montaña-Brown, Gavin J. Johnson, Yipeng Hu, Matthew J. Clarkson, Stephen P. Pereira, Dean C. Barratt and Ester Bonmati Coll
TypeConference paper
Abstract

Endoscopic ultrasound (EUS) is a challenging procedure that requires skill, both in endoscopy and ultrasound image interpretation. Classification of key anatomical landmarks visible on EUS images can assist the gastroenterologist during navigation. Current applications of deep learning have shown the ability to automatically classify ultrasound images with high accuracy. However, these techniques require a large amount of labelled data which is time consuming to obtain, and in the case of EUS, is also a difficult task to perform retrospectively due to the lack of 3D context. In this paper, we propose the use of an image-to-image translation method to create synthetic EUS (sEUS) images from CT data, that can be used as a data augmentation strategy when EUS data is scarce. We train a cycle-consistent adversarial network with unpaired EUS images and CT slices extracted in a manner such that they mimic plausible EUS views, to generate sEUS images from the pancreas, aorta and liver. We quantitatively evaluate the use of sEUS images in a classification sub-task and assess the Fréchet Inception Distance. We show that synthetic data, obtained from CT data, imposes only a minor classification accuracy penalty and may help generalization to new unseen patients. The code and a dataset containing generated sEUS images are available at: https://ebonmati.github.io.

KeywordsEndoscopic ultrasound
Synthesis
Classification
Year2021
ConferenceSecond International Workshop, ASMUS 2021, Held in Conjunction with MICCAI 2021
PublisherSpringer
Accepted author manuscript
Publication dates
Published21 Sep 2021
JournalLecture Notes in Computer Science
Journal citation12967, pp. 169-178
ISSN0302-9743
1611-3349
Book titleSimplifying Medical Ultrasound: Second International Workshop, ASMUS 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27, 2021, Proceedings
ISBN9783030875824
9783030875831
Digital Object Identifier (DOI)https://doi.org/10.1007/978-3-030-87583-1_17

Related outputs

AI-assisted prostate cancer detection and localisation on biparametric MR by classifying radiologist-positives
Wu, X., Wang, Y., Yang, Q., Thorley, N., Punwani, S., Kasivisvanathan, V, Bonmati, E. and Hu, Y. 2025. AI-assisted prostate cancer detection and localisation on biparametric MR by classifying radiologist-positives. Medical Imaging 2025: Computer-Aided Diagnosis. San Diego 16 - 21 Feb 2025 SPIE. https://doi.org/10.1117/12.3046521

Guided ultrasound acquisition for nonrigid image registration using reinforcement learning
Shaheer U. Saeed, João Ramalhinho, Nina Montaña-Brown, Ester Bonmati, Stephen P. Pereira, Brian Davidson, Matthew J. Clarkson and Yipeng Hu 2025. Guided ultrasound acquisition for nonrigid image registration using reinforcement learning. Medical Image Analysis. 102 103555. https://doi.org/10.1016/j.media.2025.103555

Enhanced CATBraTS for Brain Tumour Semantic Segmentation
El Badaoui, R., Bonmati Coll, E., Psarrou, A., Asaturyan, H. and Villarini, B. 2025. Enhanced CATBraTS for Brain Tumour Semantic Segmentation. Journal of Imaging. 11 (1), p. 8. https://doi.org/10.3390/jimaging11010008

Competing for Pixels: A Self-Play Algorithm for Weakly-Supervised Semantic Segmentation
Shaheer U. Saeed, Shiqi Huang, João Ramalhinho, Iani J.M.B. Gayo, Nina Montaña-Brown, Ester Bonmati, Stephen P. Pereira, Brian Davidson, Dean C. Barratt, Matthew J. Clarkson and Yipeng Hu 2025. Competing for Pixels: A Self-Play Algorithm for Weakly-Supervised Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 47 (2), pp. 825-839. https://doi.org/10.1109/TPAMI.2024.3474094

Active learning using adaptable task-based prioritisation
Shaheer U. Saeed, João Ramalhinho, Mark Pinnock, Ziyi Shen, Yunguan Fu, Nina Montaña-Brown, Ester Bonmati, Dean C. Barratt, Stephen P. Pereira, Brian Davidson, Matthew J. Clarkson and Yipeng Hu 2024. Active learning using adaptable task-based prioritisation. Medical Image Analysis. 95 103181. https://doi.org/10.1016/j.media.2024.103181

The distinct roles of reinforcement learning between pre-procedure and intra-procedure planning for prostate biopsy.
Gayo, I., Saeed, Shaheer U, Bonmati Coll, E., Barratt, Dean C, Clarkson, Matthew J and Hu, Yipeng 2024. The distinct roles of reinforcement learning between pre-procedure and intra-procedure planning for prostate biopsy. International Journal of Computer Assisted Radiology and Surgery. 19, p. 1003–1012. https://doi.org/10.1007/s11548-024-03084-4

Fan-Slicer: A Pycuda Package for Fast Reslicing of Ultrasound Shaped Planes
João Ramalhinho, Thomas Dowrick, Bonmati Coll, E. and Matthew J. Clarkson 2023. Fan-Slicer: A Pycuda Package for Fast Reslicing of Ultrasound Shaped Planes. Journal of Open Research Software. 11 (1), p. 3. https://doi.org/10.5334/jors.422

Deep hashing for global registration of untracked 2D laparoscopic ultrasound to CT
Ramalhinho, J., Koo, B., Montaña-Brown, N., Saeed, S.U., Bonmati Coll, E., Gurusamy, K., Pereira, S.P., Davidson, B., Hu, Y. and Clarkson, M.J. 2022. Deep hashing for global registration of untracked 2D laparoscopic ultrasound to CT. International Journal of Computer Assisted Radiology and Surgery. 17, pp. 1461-1468. https://doi.org/10.1007/s11548-022-02605-3

Voice-Assisted Image Labeling for Endoscopic Ultrasound Classification Using Neural Networks
Bonmati Coll, E., Hu, Y., Grimwood, A., Johnson, G.J., Goodchild, G., Keane, M.G., Gurusamy, K., Davidson, B., Clarkson, M.J., Pereira, S.P. and Barratt, D.C. 2022. Voice-Assisted Image Labeling for Endoscopic Ultrasound Classification Using Neural Networks. IEEE Transactions on Medical Imaging. 41 (6), pp. 1311-1319. https://doi.org/10.1109/tmi.2021.3139023

Assisted Probe Positioning for Ultrasound Guided Radiotherapy Using Image Sequence Classification
Grimwood, A., McNair, H., Hu, Y., Bonmati Coll, E., Barratt, D. and Harris, E.J. 2020. Assisted Probe Positioning for Ultrasound Guided Radiotherapy Using Image Sequence Classification. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, 23rd International Conference. Lima, Peru 04 - 08 Oct 2020 Springer. https://doi.org/10.1007/978-3-030-59716-0_52

Novel Brain Complexity Measures Based on Information Theory
Bonmati Coll, E., Bardera, A., Feixas, M. and Boada, I. 2018. Novel Brain Complexity Measures Based on Information Theory. Entropy. 20 (7) 491. https://doi.org/10.3390/e20070491

Determination of optimal ultrasound planes for the initialisation of image registration during endoscopic ultrasound-guided procedures
Bonmati Coll, E., Hu, Y., Gibson, E., Uribarri, L., Keane, G., Gurusami, K., Davidson, B., Pereira, S.P., Clarkson, M.J. and Barratt, D.C. 2018. Determination of optimal ultrasound planes for the initialisation of image registration during endoscopic ultrasound-guided procedures. International Journal of Computer Assisted Radiology and Surgery. 13, pp. 875-883. https://doi.org/10.1007/s11548-018-1762-2

Automatic Multi-Organ Segmentation on Abdominal CT with Dense V-Networks
Gibson, E., Giganti, F., Hu, Y., Bonmati Coll, E., Bandula, S., Gurusamy, K., Davidson, B., Pereira, S.P., Clarkson, M.J. and Barratt, D.C. 2018. Automatic Multi-Organ Segmentation on Abdominal CT with Dense V-Networks. IEEE Transactions on Medical Imaging. 37 (8), pp. 1822-1834. https://doi.org/10.1109/tmi.2018.2806309

Automatic segmentation method of pelvic floor levator hiatus in ultrasound using a self-normalizing neural network
Bonmati Coll, E., Hu, Y., Sindhwani, N., Dietz, H.P., D'hooge, J., Barratt, D., Deprest, J. and Vercauteren, T. 2018. Automatic segmentation method of pelvic floor levator hiatus in ultrasound using a self-normalizing neural network. Journal of Medical Imaging. 5 (2) 021206. https://doi.org/10.1117/1.jmi.5.2.021206

Brain parcellation based on information theory
Bonmati Coll, E., Bardera, A. and Boada, I. 2017. Brain parcellation based on information theory. Computer Methods and Programs in Biomedicine. 151, pp. 203-212. https://doi.org/10.1016/j.cmpb.2017.07.012

Towards image-guided pancreas and biliary endoscopy: Automatic multi-organ segmentation on abdominal CT with dense dilated networks
Gibson, E., Giganti, F., Hu, Y., Bonmati Coll, E., Bandula, S., Gurusamy, K., Davidson, B.R., Pereira, S.P., Clarkson, M.J. and Barratt, D.C. 2017. Towards image-guided pancreas and biliary endoscopy: Automatic multi-organ segmentation on abdominal CT with dense dilated networks. Medical Image Computing and Computer Assisted Intervention − MICCAI 2017. Quebec City, QC, Canada 11 - 13 Sep 2017 Springer. https://doi.org/10.1007/978-3-319-66182-7_83

2D-3D Registration Accuracy Estimation for Optimised Planning of Image-Guided Pancreatobiliary Interventions
Hu, Y., Bonmati Coll, E., Gibson, E., Hipwell, J.H., Hawkes, D.J., Bandula, S., Pereira, S.P. and Barratt, D.C. 2016. 2D-3D Registration Accuracy Estimation for Optimised Planning of Image-Guided Pancreatobiliary Interventions. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016. MICCAI 2016. Athens, Greece 17 - 21 Oct 2016 Springer. https://doi.org/10.1007/978-3-319-46720-7_60

Assessment of Electromagnetic Tracking Accuracy for Endoscopic Ultrasound
Bonmati Coll, E., Hu, Y., Gurusamy, K., Davidson, B., Pereira, S.P., Clarkson, M.J. and Barratt, D.C. 2016. Assessment of Electromagnetic Tracking Accuracy for Endoscopic Ultrasound. Computer-Assisted and Robotic Endoscopy. CARE 2016. Athens, Greece 17 Oct 2016 Springer. https://doi.org/10.1007/978-3-319-54057-3_4

Measuring Complex Brain Networks Structure
Bonmati Ester, Bardera Anton, Boada Imma and Bonmati Coll, E. 2016. Measuring Complex Brain Networks Structure. Frontiers in Neuroinformatics. Conference Abstract: Neuroinformatics 2016. https://doi.org/10.3389/conf.fninf.2016.20.00012

Hierarchical clustering based on the information bottleneck method using a control process
Bonmati Coll, E., Bardera, A., Boada, I., Feixas, M. and Sbert, M. 2015. Hierarchical clustering based on the information bottleneck method using a control process. Pattern Analysis and Applications (PAA). 18, pp. 619-637. https://doi.org/10.1007/s10044-015-0467-1

Permalink - https://westminsterresearch.westminster.ac.uk/item/x0x0q/endoscopic-ultrasound-image-synthesis-using-a-cycle-consistent-adversarial-network


Share this

Tweet
Email

Usage statistics

0 total views
1 total downloads
These values cover views and downloads from WestminsterResearch and are for the period from September 2nd 2018, when this repository was created.