New research suggests AI image generation using DALL-E 2 has promising future in radiology – Medical Xpress

wp header logo 405


Forget Password?
Learn more
share this!
13
4
Share
Email
March 16, 2023
This article has been reviewed according to Science X’s editorial process and policies. Editors have highlighted the following attributes while ensuring the content’s credibility:
fact-checked
peer-reviewed publication
proofread
by JMIR Publications
A new paper published in the Journal of Medical Internet Research describes how generative models such as DALL-E 2, a novel deep learning model for text-to-image generation, could represent a promising future tool for image generation, augmentation, and manipulation in health care. Do generative models have sufficient medical domain knowledge to provide accurate and useful results? Dr. Lisa C Adams and colleagues explore this topic in their latest viewpoint titled “What Does DALL-E 2 Know About Radiology?”

First introduced by OpenAI in April 2022, DALL-E 2 is an artificial intelligence (AI) tool that has gained popularity for generating novel photorealistic images or artwork based on textual input. DALL-E 2’s generative capabilities are powerful, as it has been trained on billions of existing text-image pairs off the internet.
To understand whether these capabilities can be transferred to the medical domain to create or augment data, researchers from Germany and the United States examined DALL-E 2’s radiological knowledge in creating and manipulating X-ray, computed tomography (CT), magnetic resonance imaging (MRI), and ultrasound images.
The study’s authors found that DALL-E 2 has learned relevant representations of X-ray images and shows promising potential for text-to-image generation. Specifically, DALL-E 2 was able to create realistic X-ray images based on short text prompts, but it did not perform very well when given specific CT, MRI, or ultrasound image prompts.
It was also able to reasonably reconstruct missing aspects within a radiological image. It could do much more—for example, create a complete, full-body radiograph by using only one image of the knee as a starting point. However, DALL-E 2 was limited in its capabilities to generate images with pathological abnormalities.

New research suggests AI image generation using DALL-E 2 has promising future in radiology
Examples of text-to-image–generated anatomical structures in CT, MRI, and ultrasound images created with DALL-E 2. CT: computed tomography; MRI: magnetic resonance imaging. Credit: Journal of Medical Internet Research (2023). DOI: 10.2196/43110

Synthetic data generated by DALL-E 2 could greatly accelerate the development of new deep learning tools for radiology, as well as address privacy concerns related to data sharing between institutions. The study’s authors note that generated images should be subjected to quality control by domain experts to reduce the risk of incorrect information entering a generated data set. They also emphasize the need for further research to fine-tune these models to medical data and incorporate medical terminology to create powerful models for data generation and augmentation in radiology research.
Although DALL-E 2 is not available to the public to fine-tune, other generative models like Stable Diffusion are, which could be adapted to generate a variety of medical images.
Overall, this viewpoint provides a promising outlook for the future of AI image generation in radiology. Further research and development in this area could lead to exciting new tools for radiologists and medical professionals.

More information: Lisa C Adams et al, What Does DALL-E 2 Know About Radiology?, Journal of Medical Internet Research (2023). DOI: 10.2196/43110

Provided by JMIR Publications

Citation: New research suggests AI image generation using DALL-E 2 has promising future in radiology (2023, March 16) retrieved 2 January 2024 from https://medicalxpress.com/news/2023-03-ai-image-generation-dall-e-future.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

More information: Lisa C Adams et al, What Does DALL-E 2 Know About Radiology?, Journal of Medical Internet Research (2023). DOI: 10.2196/43110

Explore further
Facebook
Twitter
Email
Feedback to editors
4 hours ago
0
Dec 29, 2023
0
Dec 29, 2023
4
Dec 28, 2023
0
Dec 28, 2023
0
13 minutes ago
1 hour ago
1 hour ago
1 hour ago
2 hours ago
2 hours ago
2 hours ago
3 hours ago
3 hours ago
4 hours ago
Feb 2, 2023
Jul 21, 2022
Sep 13, 2022
Jun 7, 2022
Sep 13, 2022
Jan 6, 2021
Dec 29, 2023
Dec 29, 2023
Dec 27, 2023
Dec 27, 2023
Dec 27, 2023
Dec 22, 2023
Use this form if you have come across a typo, inaccuracy or would like to send an edit request for the content on this page. For general inquiries, please use our contact form. For general feedback, use the public comments section below (please adhere to guidelines).
Please select the most appropriate category to facilitate processing of your request
Thank you for taking time to provide your feedback to the editors.
Your feedback is important to us. However, we do not guarantee individual replies due to the high volume of messages.
Your email address is used only to let the recipient know who sent the email. Neither your address nor the recipient’s address will be used for any other purpose. The information you enter will appear in your e-mail message and is not retained by Medical Xpress in any form.

Get weekly and/or daily updates delivered to your inbox. You can unsubscribe at any time and we’ll never share your details to third parties.
More information Privacy policy
We keep our content available to everyone. Consider supporting Science X’s mission by getting a premium account.
Daily science news on research developments and the latest scientific innovations
The latest engineering, electronics and technology advances
The most comprehensive sci-tech news coverage on the web

source