Course: Artificial Intelligence (Elective) Code:

Year: 2024-25 Semester: 5 ‘Dʼ

Title of the Project: Text To Image Converter
Presented by Mentor : Dr.Rekha.K.S.Associate Professor,CSE,NIE
Batch - Y

Name USN

Chirag R Gowda 4NI22CS264

Yogeshwar R 4NI22CS263

Yadunandan K 4NI22CS251

Shravan H.R 4NI22CS201

Sujay Matur 4NI22CS225

Sanjay M.M 4NI22CS190

Objective:

The main goal of this project is to build a web-based tool that takes an text as input and generates a
image using AI-powered natural language processing.
Develop a web-based tool that generates images from uploaded textdescriptions using OpenAIʼs DALL-E model.
Leverage AI technologies including computer vision and natural language processing (NLP) to convert visual text into
accurate, images
DALL-E model is used in the text-to-images context, where it analyzes an image and generates a images based on its
content.
The tool allows users to upload text , which are then processed through the DALL-E API to generate relevant images
DALL-Eʼs transformer-based architecture is trained on large datasets of image-text pairs, enabling it to recognize objects,
scenes, and relationships in images.
The generated descriptions help bridge the gap between visual content and language, making images more accessible and
understandable.
Potential applications include:
Accessibility tools (e.g., helping visually impaired users understand images).
Content creation for generating captions or descriptions for social media, blogs, or websites.
Visual search engines for image categorization and retrieval.
Concept of AI Used in Project: Text-to-Image Generation with DALL-E
AI Model Used: The project leverages OpenAI's DALL-E model, which is a deep learning system designed for text-to-image
generation. This model interprets textual descriptions and generates corresponding images based on those inputs.
Core Functionality: DALL-E is capable of generating high-quality images from detailed text descriptions. The model
processes the input text, interpreting key concepts, objects, and relationships, then synthesizes this information to create
visually coherent and creative images that match the prompt.
Transformer-Based Architecture: DALL-E utilizes a transformer-based architecture, which is well-suited for handling large,
complex datasets. Transformers enable the model to learn patterns in both text and images, helping it generate relevant
visuals based on the given textual input.
Training and Datasets: The model has been trained on massive datasets of images paired with textual descriptions. This
training allows DALL-E to learn how specific words, phrases, and contexts correlate with visual elements, such as objects,
settings, and styles.
Applications: The ability to generate images from text has numerous applications, including in creative industries (such as
digital art, advertising, and entertainment), content creation (for social media, blogs, etc.), and design (e.g., concept art or
product prototypes).
Multimodal AI Capabilities: DALL-E represents a key advancement in multimodal AI, bridging the gap between language and
visual content. The model can generate realistic or imaginative images from a wide variety of textual prompts, whether they
describe everyday objects or entirely fantastical scenarios.
Software and Hardware Requirements:
Software:
Python: Used for backend development and integrating with the OpenAI API.
Flask: A lightweight web framework for building the web application.
OpenAI API: Provides access to the DALL-E model for text-to-image generation.
Replit: An online platform for hosting and deploying the web application.
HTML/CSS/JavaScript: For building and styling the frontend interface.
Jinja2: Templating engine used in Flask to render dynamic HTML content.
Requests Library: A Python library used for making HTTP requests to the OpenAI API.
Hardware:
Standard computer or laptop: Required for general development and web application deployment.
(Optional) GPU-enabled machine: Not strictly necessary, as DALL-E model processing is handled on OpenAIʼs cloud
infrastructure, but a GPU may be helpful for speeding up local computations if needed.
Design & Algorithm Details:
Frontend Design:
Web Page: A simple HTML/CSS interface allowing users to upload images for processing.
Submit Button: A button that triggers the image submission to the backend for processing.
Text Display Area: A section on the page to show the generated text description after processing the image.

Backend Design (Flask Application):

Flask Server: Manages HTTP requests, serves the frontend, and integrates with the OpenAI API.
Image Upload Handling: Flask captures the uploaded image from the frontend and sends it to the OpenAI DALL-E API for
processing.
Text Generation: The backend processes the image using the DALL-E model and retrieves a generated textual description.
Display Text: The backend sends the generated description back to the frontend for display on the web page.
Algorithm Explanation:
Text Upload: The user selects and uploads an text through the web interface. This triggers a file input event, allowing the
image to be submitted to the backend for processing via the Flask app.
API Call: Once the image is uploaded, the Flask app sends the text to the OpenAI DALL-E API using a POST request. The text is
sent in the request body, typically encoded in base64 format or as a multipart form-data payload.
Image Processing: The DALL-E API receives the text and uses its trained model to analyze the content. It processes the visual
elements, identifying objects, settings, and context, then generates a natural image description of the text's contents.
Return Description: After processing the image, DALL-E generates the image and sends it back to the Flask app in the API
response. This image is structured and contextualized to accurately reflect the visual elements from the input text.
Display Result: The Flask app receives the generated image and passes it to the frontend. The image is displayed on the
webpage in a designated area, allowing the user to view the result of the image-to-text transformation.
Implementation:

1. Set up OpenAI API

Sign up for OpenAI and obtain an API key.
Use the API to interact with DALL-Eʼs image generation capabilities.
2. Create Flask Web Application
Develop the Flask app to handle routes for uploading text and processing the request.
Integrate HTML forms for user input (image upload).
3. Integrate DALL-E Model
Send the uploaded text data to OpenAIʼs API endpoint for DALL-E.
Process the text and return image.
4. Deploy on Replit
Set up the project on Replit to host the Flask app and enable access from anywhere.
Use Replitʼs free hosting solution for the web application.
Results and Discussion:
Image Generation: The application successfully generates a image for any uploaded text, converting text into coherent
image. For example, eagle in iron man suit.
Example Output: If a user uploads a text of a cat in batman suit , the DALL-E model might produce a image like, “ cat in
batman suit.” The description matches the visible elements of the image.
Accuracy Dependence: The quality of the generated description depends on two factors: the clarity of the text and the
accuracy of DALL-Eʼs model in interpreting the text content. Clear, detailed text lead to better results.
Limitations: DALL-E might struggle with more abstract, complex, or ambiguous images. Its ability to generate meaningful
descriptions can vary based on the complexity of the visual content and how well it matches the model's training data.
Snapshots:

Image 1 Image2
Prompt for Image1: An anime character(gojo satoru) wearing black clothes in real life
Prompt for Image2: An eagle in iron man suit
Conclusion:
This project showcases how AI, particularly OpenAIʼs DALL-E, can be used to convert images into textual descriptions, making
visual content accessible in a new way. By using Flask for the backend and building a simple web interface, users can easily
upload images and receive descriptive text generated by DALL-E. The web application is hosted on Replit, allowing it to be
accessed from anywhere, making it both convenient and user-friendly.
While DALL-E performs well in generating descriptions, the project highlights areas where improvements can be made. Currently,
the model works best with clear and simple images, but it may struggle with more complex or abstract visuals. This means that
DALL-Eʼs ability to provide meaningful descriptions can vary depending on the content of the image. Despite these limitations, the
project demonstrates the potential of AI to bridge the gap between visual and textual data, offering exciting possibilities for
accessibility, content creation, and more.
In the future, enhancements could include refining the modelʼs ability to handle complex image contexts, as well as improving the
accuracy and relevance of the generated descriptions to provide more detailed, context-aware text.
References:
• OpenAI (DALL-E): https://openai.com/dall-e

• Flask Documentation:
https://flask.palletsprojects.com/
• Replit : https://replit.com/