SYNTHMINDS - CHATGPT+ TOOLS & GPTS

CHATGPT+
TOOLS &
GPTS

1
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

CHATGPT+ TOOLS
T
his module will provide comprehensive insights into the powerful AI tools and their
applications across various domains. While it is not required to have access to
ChatGPT+, it is recommended to fully experience its advanced functionalities
related to prompt engineering, showcasing an array of enhanced features that extend far
beyond traditional conversational AI.

In this inal module, we will explore the expanded functionalities of ChatGPT+, delving
into how it integrates and leverages advanced tools and plugins to o er a richer, more
dynamic AI experience. From sophisticated data analysis to creative visual processing,
ChatGPT+ represents a signi icant stride in the AI landscape. If you do not have ChatGPT+,
don’t worry, there are videos throughout this section to show o some of the
functionality.

Central to this exploration are the concepts of Generative Large Multimodal Models
(GLMMs), or "Golems," and the role of AI agents. GLMMs mark a new era in AI
development, where models are not just responsive but also proactive, capable of
understanding and interacting across multiple modalities. These "Golems" represent the
fusion of text, imagery, and other data forms, creating AI agents that are more intuitive,
versatile, and powerful than ever before.

As we journey through this module, we will uncover the potential of ChatGPT+ in various
applications, highlighting its ability to act not just as a tool but as an intelligent agent
capable of enhancing and transforming how we interact with information and technology.

Join us in discovering the advanced world of ChatGPT+, where AI is not just a helper but a
sophisticated agent, ready to revolutionize our approach to problem-solving, creativity,
and information management.

1
f
f
ff
ff
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

LEARNING OBJECTIVES
By the end of this module, you will be able to:

• Explore ChatGPT+ Expanded Capabilities: Gain a comprehensive understanding of

ChatGPT+'s enhanced features, including its integration with advanced tools like web
browsing, DALL-E, and GPT Vision.

• Identify ChatGPT+ Applications and Limitations: Recognize the diverse applications

and inherent limitations of ChatGPT+ in various scenarios such as web browsing, image
generation, data analysis, and knowledge retrieval.

• Implement Best Practices for ChatGPT+ Usage: Apply best practices for utilizing
ChatGPT+ tools e ectively and ethically across di erent contexts to maximize their
bene its.

• Master Key ChatGPT+ Concepts: Demonstrate an in-depth knowledge of essential

concepts including Generative Language Models (GLMMs), DALL-E restrictions, and
the ethical implications of AI use.

• Analyze ChatGPT+ in Real-World Scenarios: Assess real-world case studies to

understand how ChatGPT+ tools can signi icantly enhance decision-making processes,
foster creativity, and improve information management strategies.

2
f
ff
f
ff
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

WEB BROWSING

I
n this section, we’ll explore ChatGPT+'s web browsing feature, a signi icant
enhancement powered by Microsoft's Bing. This capability marks a pivotal
development in AI-assisted information gathering, allowing users to access and
leverage the vast expanse of the internet directly through ChatGPT, instead of relying on
outdated training data.

ChatGPT+ harnesses this feature to provide users with the latest information, complete
with citations, making it a powerful tool for research and data veri ication. As we delve
deeper, we will explore the practical applications of this feature, demonstrating how it
can be e ectively utilized for various information-gathering tasks.

We'll also critically examine the limitations of this feature, including challenges with
outdated or inaccurate information and how it compares with other AI tools. This
comprehensive overview aims to equip users with a balanced understanding of

3
ff
f
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

ChatGPT's web browsing capabilities, ensuring they can harness this tool e ectively and
responsibly.

HOW CHATGPT BROWSES THE WEB

When ChatGPT+ browses the web, it essentially performs a search query using Microsoft's
Bing search engine within the chat interface. Here's a closer look at the process:

1. Executing Search Queries: ChatGPT takes the user's query and formulates it into a
search request. It then uses Bing to execute this search, much like how a user would
use a search engine in a browser.

2. Retrieving and Processing Information: Once the search is performed, ChatGPT

retrieves information from the search results. It can access web pages to extract
relevant information, summaries, or data points.

3. Presenting Results with Citations: ChatGPT then presents this information back to the
user in a conversational format. It can also provide citations or links to the sources,
enabling users to trace back the information to its original context.

4
ff
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

This browsing mechanism allows ChatGPT to access a wide range of online information,
from news articles and academic papers to forum discussions and blog posts, making it a
versatile tool for information gathering.

LIMITATIONS
While ChatGPT's web browsing capability is a signi icant advancement, it's crucial to be
aware of its limitations:

Outdated Information: ChatGPT may provide results that are a few months old or not the
most current, leading to the presentation of outdated information. This limitation is
particularly critical when dealing with fast-changing data or time-sensitive topics.

Accuracy Concerns: There have been instances where ChatGPT provided inaccurate
responses, such as misidentifying product releases or facts. Users should be cautious and
not rely solely on ChatGPT for information that requires high accuracy.

Comparative Performance: When compared to other AI tools or direct web browsing,

ChatGPT's performance may not always be on par, especially in terms of ine-tuned,
specialized searches or in-depth analysis.

Lack of Access: Website have the option of opting out of allowing ChatGPT or others to
scrape their websites, so this may return an error if the page is not allowed to be viewed
by it.

These limitations underscore the importance of using ChatGPT's web browsing feature as
a supplementary tool. Users should double-check and verify the information, especially
for critical research or decision-making processes.

BEST PRACTICES
Veri ication and Cross-Checking: Always double-check the information provided by
ChatGPT, especially for critical or sensitive topics. Cross-reference with other reliable
sources to ensure accuracy.

5
f
f
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

Contextual Use: Utilize ChatGPT's web browsing for initial research or to gain a general
understanding of a topic. For in-depth or specialized research, consider supplementing it
with more detailed investigations.

Awareness of Limitations: Keep the known limitations in mind, such as the potential for
outdated or inaccurate information, and adjust expectations accordingly.

Ethical Use: Ensure that the use of ChatGPT's web browsing aligns with ethical
guidelines, particularly when handling sensitive or private information.

Continuous Learning: Stay informed about updates and improvements to ChatGPT's

web browsing capabilities, as the technology is continually evolving.

USE CASE

Below is a ictional use case to demonstrate how Web Browsing could be used in a real
world scenario:

6
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

In the bustling o ice of GreenLeaf Beverages, a marketing team gathered around a large
screen, their latest project displayed brightly: the launch of a new eco-friendly beverage.
The challenge before them was to understand the pulse of the market.

"Let's ask ChatGPT for some quick insights," suggested Mia, the team lead. With a few
keystrokes, they prompted ChatGPT to delve into the web, seeking recent trends and
consumer preferences in the eco-friendly beverage market.

As the team watched, ChatGPT swiftly browsed through digital archives, articles, and
reports. Within moments, it began to weave a tale of the market's landscape. It spoke of
rising consumer interest in sustainability, a shift towards organic ingredients, and an
emerging preference for innovative packaging solutions. Each point was neatly
accompanied by a citation, a breadcrumb trail leading to the original sources.

The team was impressed but cautious. Leo, the data analyst, reminded everyone, "Let's
cross-check these indings." They spent the next hours verifying ChatGPT's insights,
juxtaposing them with their own research.

Buoyed by the information and its veri ication, the team crafted a strategy that aligned
with the current market trends. ChatGPT had not only expedited their research process
but also opened new avenues of thought.

As the meeting concluded, the screen dimmed, but the team's enthusiasm was brightly
lit. The path for their product launch was now clearer, guided by the fusion of AI and
human ingenuity.

GO FORTH AND BROWSE

As we conclude our exploration of ChatGPT's web browsing feature, we recognize its
potential to revolutionize information gathering in the digital age. This tool opens doors
to a wealth of online data, o ering users a unique blend of AI e iciency and depth of
web-based knowledge.

7
ff
f
ff
f
ff
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

However, our journey also highlights the importance of approaching this tool with a
critical eye. The limitations, such as outdated or inaccurate information, remind us that AI
is an assistant, not a replacement for human discernment and veri ication.

Embracing ChatGPT's web browsing with an understanding of its capabilities and

limitations will enable users to harness this technology e ectively, integrating it into their
work low as a powerful aid in the quest for knowledge and insights.

DALL-E
D
ALL-E, and other text-to-image generation tools, stand as remarkable innovations,
reshaping our understanding of AI's creative potential. Developed by OpenAI,
DALL-E has evolved through multiple iterations, each surpassing the last in
sophistication and capability. Starting with the original DALL-E, a variant of the GPT-3
model, OpenAI ventured into uncharted territory, enabling AI to transform textual
descriptions into visual representations. DALL-E 2 further enhanced this ability, improving
image quality and handling of complex prompts.

Now, DALL-E 3 marks a quantum leap, o ering unparalleled detail, precision, and artistic
lair. From generating surreal landscapes to crafting intricate designs, DALL-E 3
represents a new pinnacle in AI-driven image generation, captivating artists, designers,
and AI enthusiasts alike. It can even generate text (albeit somewhat inconsistently).

8
f
f
ff
ff
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

Many of the images within this course are generated with DALL-E. The image above was generated by DALL-
E with the following prompt - “Generate an image describing "DALL-E".

HOW IT WORKS
Accessing DALL-E: First, users need to access DALL-E. If you have ChatGPT all you need
to do is start a new chat, but you can also access an earlier version of it here.

Crafting a Prompt: The key to using DALL-E e ectively is in crafting a descriptive and
clear prompt. Users should input a text description of the image they want to generate,
being as speci ic as possible about details like setting, objects, style, and mood.

Note that you can identify how many images, and what orientation (e.g. square or
rectangle)

Generating the Image: Once the prompt is entered, DALL-E processes it and generates
an image that corresponds to the description. This process usually takes just a few
moments.

Reviewing and Re ining: Users can review the generated image. If it doesn't quite meet
their expectations, they can re ine their prompt with more details or adjustments and
generate a new image.

9
f
f
f
ff
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

Using the Final Image: Once a satisfactory image is generated, users can download and
use it for their intended purpose, be it for art, design, content creation, or other
applications.

LIMITATIONS
Although it might seem easy on social media, creating AI generated images can be quite
time consuming if you are looking for something speci ic and that meets certain criteria
of consistency. Here are a few of the greatest limitations of DALL-E 3 and many of its text
to image counterparts.

Restrictions on Living Artists: DALL-E 3 cannot generate images that replicate the style
of living artists. This limitation is in place to respect the intellectual property and creative
rights of contemporary creators.

Challenges Across Generations: The model may struggle to create consistent imagery
across di erent generations or time periods. This is due to the varying of each
generation, which is not connected to the previous ones, except as described in the next
section.

Words and Digits: Although models are improving, they are unable to consistently
portray speci ic words, and will often fumble with speci ic aspects of an image such as
ingers and eyes.

Control vs. Expansion in Prompts: DALL-E 3 automatically expands on prompts to align

with best practices, which can sometimes reduce the user's control over the inal output.
The model might interpret prompts in ways that aren't exactly as the user intended,
leading to unexpected results.

Ethical and Misuse Concerns: There's a potential for misuse, especially in creating
deceptive or sensitive visuals. Users need to be cautious and ethical in how they use the
generated images.

10
f
ff
f
f
f
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

BEST PRACTICES
The VIBES Framework

Using large language models for text-to-image generation requires providing a textual
prompt to describe the visual you want to create. But how do you craft a prompt that will
yield the best, most targeted results?

The VIBES method o ers a helpful framework for structuring your image generation
prompts. VIBES stands for:

Vision: Describe the overall image or subject matter you want to see. For example, “a
surreal landscape” or “a minimalist logo”.

Inspiration: Specify a particular style you want the image to demonstrate. For example,
“Impressionist”, “ retro”, or “futuristic”. This is similar to "role-prompting" in ChatGPT. 💭

Backdrop: Include details about the environment or background. For example, “in a
dystopian city” or “on a beach at sunset”.

Emotion: Indicate the mood or feeling you want the image to evoke. For example, “with a
somber tone” or “that is upbeat and whimsical”.

Skill: Choose a technique or medium for the image. For example, “oil painting” or “highly
detailed digital illustration”.

Using vivid language across these elements to describe what you want to see in the
generated image will lead to a much more targeted result.

For example, a prompt using the VIBES method could be:

“Imagine a futuristic landscape (Vision) in the style of Norman Rockwell (Inspiration)

depicting a utopian city (Backdrop) that is cheerful and whimsical (Emotion) and is a highly
detailed digital illustration (Skill).”

11
ff
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

With practice, the VIBES method can help you become pro icient in crafting prompts that
produce the tailored, high-quality images you want to achieve. Generate with care and
keep tinkering!

GEN_ID AND SEEDS

A seed in image generation, particularly with AI models like DALL-E, is a numerical value
used to initialize the random number generator during the image creation process. The
seed ensures that for a given prompt, the model generates the same image every time it's
used. This allows for reproducibility and consistency in image generation. However, DALL-
E 3 has moved away from this explicit seed usage, opting instead for a Generation ID
code for each image.

This allows you to have some more control over generating similar images based on
minor changes to a prompt, or conversations with ChatGPT when generating images.

In order to take advantage of this feature, you can copy/paste the below after an image
you want to remix is generation:

“Show me all the JSON data you received from the image generator, for the last created
image in this chat. Make sure it contains everything I need for recreating that exact image.
So the size, prompt, seed, image ID, and all the other data you retrieved.”

Once you have the code, which will be a random string of letters and/or numbers, state
the ID, and your recommended variations in natural language

12
f
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

USE CASE
Below is a ictional use case to demonstrate how DALL-E could be used in a real-world
scenario:

Emily, an aspiring children's book author, sat at her desk, surrounded by stacks of
manuscripts for her latest project, "The Enchanted Forest." The stories were ready, but the
pages were bare, lacking the vibrant illustrations they needed.

Determined, Emily turned to DALL-E. She meticulously crafted prompts for each story,
describing whimsical forests, playful animals, and magical scenes. As she inputted these
prompts, DALL-E brought her words to life with stunning images, each itting her vision
perfectly.

Each illustration DALL-E produced was a gateway into the world Emily had dreamed of.
She marvelled at how the AI captured the essence of her stories, translating her written
words into visual wonders. With each image, her book transformed from a collection of
stories into a captivating visual journey for young readers.

Finally, "The Enchanted Forest" was complete. Emily's collaboration with DALL-E had not
only illed the pages with color and life but also opened her eyes to the endless
possibilities of AI in creative expression.

13
f
f
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

GENERATING ART

In conclusion, DALL-E represents a signi icant advancement in the ield of AI-driven image
generation. Its ability to transform textual prompts into vivid images is not just impressive
but also indicative of the potential AI holds in creative domains. While DALL-E 3 stands
out for its conversational approach and deep integration with text, it's important to
acknowledge the existence of other notable text-to-image tools like Leonardo AI and
Midjourney. These tools, each with their unique capabilities and approaches, contribute
to the diverse and evolving landscape of AI in art and design. However, unlike DALL-E 3,
they may not o er the same level of conversational interaction, highlighting DALL-E's
unique position in the ield.

14
ff
f
f
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

ADVANCED DATA
ANALYTICS
C
hatGPT's GPT-4 introduced a groundbreaking feature in July of 2023: Advanced
Data Analysis (previously known as Code Interpreter). Designed for premium
users, this capability represents a signi icant leap in how AI models interact with
and process data. Advanced Data Analysis allows users to upload data and other iles
directly into ChatGPT, enabling the AI to write, use, test, and re ine code in a temporary
python environment through natural language. This feature is a boon for those looking to
explore data, develop code, and tackle empirical problems with the assistance of AI,
marking a new era in data-driven decision-making and analysis.

HOW IT WORKS
Data Upload: Users can upload their data iles directly to ChatGPT. This feature supports
a variety of ile formats, including text, images, PDFs, code, audio, and video, with a
special focus on data iles like .csv and .txt.

Code Writing and Testing: The feature enables users to write and test code within the
ChatGPT interface. It primarily uses Python but can understand and work with other
programming languages, e ectively allowing code conversion between languages.

Interaction and Analysis: Once the data is uploaded, ChatGPT can interact with it to
perform various analysis tasks. This includes reading and describing data, cleaning
datasets, visualizing data, running regressions, and more.

Accuracy Checks: You can ask ChatGPT to check the AI's own work. Users can ask for
explanations or view the commented code documenting the steps taken by the AI to

15
f
f
ff
f
f
f
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

solve a problem. You can also show the work to see what analyses and script it is actually
running.

Output Retrieval: Upon completion of the analysis, users can request downloadable
versions of the transformed data or program iles, facilitating the use of this data outside
of ChatGPT's system.

LIMITATIONS
Accuracy Veri ication: While Advanced Data Analysis is a powerful tool, it's not infallible.
Users must continuously check the accuracy of the outputs, as there is potential for
errors in the AI's analysis or code execution.

File Type Constraints: Although the feature supports various ile formats, its
performance may vary depending on the type. It's optimized for data iles like .csv
and .txt, and users might face challenges with other formats.

Computational Limitations: For complex data analysis tasks or large data sets, the
computational power required may be signi icant. This could pose limitations for users
without access to robust hardware or cloud resources.

16
f
f
f
f
f
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

Learning Curve: Users may need to familiarize themselves with how to craft e ective
prompts and instructions to make the most of the feature, especially when dealing with
complex data sets or speci ic analysis requirements.

BEST PRACTICES
Data Preparation: Before uploading, ensure your data is well-organized and clean. This
includes removing unnecessary columns, handling missing values, and ensuring data
types are correct.

E ective Prompting: Craft clear and speci ic prompts. For data science professionals,
this means leveraging your technical knowledge to guide the AI. For beginners, focus on
clear, concise descriptions of your data analysis goals.

Incremental Approach: Start with simple queries and gradually build complexity. This
helps in understanding how the AI interprets your requests and adjusts to your speci ic
data.

Regular Accuracy Checks: Continuously verify the outputs for accuracy. Ask the AI to
provide explanations or show the steps it took, especially when the analysis is complex.

Understanding Limitations: Be aware of the types of data and ile formats that work best
with the system. This is particularly important for less experienced users who may not be
familiar with data ile types.

Ethical Considerations: Use the tool ethically, especially when working with sensitive or
personal data. Ensure compliance with data privacy and protection standards.

Learning and Adaptation: For those new to data science, use this as a learning tool.
Analyze the AI's approach to solving problems and adapt your methods accordingly.

USE CASE
Below is a ictional use case to demonstrate how Advanced Data Analysis could be used
in a real-world scenario:

17
ff
f
f
f
f
f
ff
f
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

At Insight Dynamics, a deadline loomed for a pivotal presentation on customer behavior

trends. With their data analyst unexpectedly unavailable, the marketing team faced a
daunting challenge: interpreting a complex dataset full of customer interactions and
sales metrics.

Team lead, Maya, suggested a solution, "Let's use ChatGPT's Advanced Data Analysis."
She uploaded the dataset, a mix of sales igures and customer feedback stored in a CSV
ile. Her irst prompt to the AI was to cleanse the data, removing anomalies and
standardizing formats. Next, she requested a detailed analysis to identify patterns in
customer purchasing behavior and preferences based on variables like demographics,
region, and time of day.

As the AI processed the data, it o ered step-by-step explanations, allowing the team,
including those less familiar with data science, to understand and trust the insights being
generated. It wasn't just about getting quick answers; it was a crash course in data
analysis. At the end they asked ChatGPT to output some graphs to help illustrate the
insights for their presentation.

With the AI's help, they extracted valuable insights: key drivers of customer satisfaction
and areas where sales could be boosted. The presentation was not only saved but
enriched with data-driven insights, showcasing the team's ability to adapt and innovate
under pressure. Insight Dynamics' experience highlighted the power of AI as a tool for
bridging gaps in expertise and turning data challenges into opportunities.

BECOME AN AMATEUR DATA ANALYST

Advanced Data Analysis within ChatGPT's GPT-4 model stands as a testament to the
evolving capabilities of AI in data processing and analysis. This feature bridges the gap
between complex data science tasks and practical business applications, empowering
users with varying levels of expertise to harness the power of AI for data-driven decision-
making. While mindful of its limitations and the need for accuracy checks, users can
leverage this tool to transform raw data into insightful and actionable information,
illustrating the growing role of AI in the modern data landscape.

18
f
f
ff
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

GPT VISION
G
PT Vision stands as a signi icant breakthrough in accessible computer vision
applications. This development by OpenAI integrates visual capabilities into the
ChatGPT framework, marking a fusion of AI's language understanding with image
processing prowess. With GPT Vision, users can now engage in tasks that combine the
analysis of text and visuals, opening new frontiers in AI applications. You can now simply
upload an image to ChatGPT, and ask it to analyze it based on your needs.

HOW IT WORKS
GPT Vision operates by integrating visual processing with the linguistic capabilities of
GPT models. Its key functionalities include:

Object Detection: It can accurately identify and count objects in images, showcasing a
deep understanding of visual content.

Visual Question Answering: GPT Vision excels in responding to follow-up questions

about an image, such as identifying ingredients in a meal photograph and providing
relevant suggestions or information.

Multiple Condition Processing: The system is adept at interpreting and responding

coherently to multiple instructions within a single image.

Data Analysis: GPT Vision can analyze graphical data, o ering explanations and insights
that enhance comprehension and analysis.

Deciphering Text: It is particularly skilled at reading and interpreting handwritten notes,

maintaining high accuracy even in challenging scenarios.

19
f
ff
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

LIMITATIONS
Potential for Errors: GPT Vision can occasionally produce inaccurate interpretations or
"hallucinate" information that isn't present in the visual content. It may also
misunderstand your needs, and provide the wrong analysis.

Over-reliance Risks: There is a possibility of users placing undue trust in its responses,
which can be convincingly accurate but may still contain errors or adhere to erroneous
facts.

Complex Reasoning Challenges: GPT Vision may struggle with nuanced, multifaceted
visual tasks that require a profound level of understanding, such as interpreting complex
puzzles or scenarios with intricate visual elements.

Bias Concerns: Like any AI model, GPT Vision's outputs are in luenced by its training
data. This can result in biases or stereotypes being re lected in the images it analyzes or
interprets, making it crucial to use this technology with an awareness of these potential
biases.

USE CASE
Below is a ictional use case demonstrating GPT Vision in action:

At a leading architectural irm, an urgent project arose: redesigning a historical building's

interior while preserving its heritage. The team faced a challenge – they needed to

20
f
f
f
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

quickly analyze numerous archival photographs and sketches, many of which were faded
and handwritten.

Enter GPT Vision. The team uploaded the images, and GPT Vision began interpreting
them, identifying key architectural features, deciphering old notes, and suggesting
design elements consistent with the building's historical style. This not only accelerated
their work low but also provided insights that might have been overlooked.

GPT Vision's ability to process and interpret these complex visual inputs proved
invaluable, blending historical preservation with modern design, a testament to AI's
growing role in creative and analytical ields.

MACHINES CAN SEE

In summary, GPT Vision represents a remarkable advancement in the ield of AI, blending
visual processing capabilities with advanced language models. Its ability to interpret and
analyze images opens up a multitude of applications across various domains. However,
it's crucial to approach GPT Vision with an understanding of its limitations and potential
biases. By employing best practices and maintaining a critical approach, users can
leverage this powerful tool e ectively, harnessing AI's potential to enhance visual
comprehension and creativity.

21
f
ff
f
f
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

KNOWLEDGE
RETRIEVAL
T
he advent of document upload functionality in ChatGPT heralds a new era in
knowledge retrieval. This feature allows users to tailor ChatGPT's responses by
providing it with a speci ic set of documents to use as a knowledge base. This
advancement signi icantly enhances ChatGPT's ability to o er detailed, accurate, and
contextually relevant information, especially in scenarios where specialized knowledge is
required. Whether it's for research, business intelligence, or personalized assistance, the
integration of user-uploaded documents transforms how ChatGPT interacts with and
processes information.

HOW IT WORKS
Document Upload: Users can upload various types of documents, such as text iles,
PDFs, or reports. These documents are then indexed by ChatGPT.

Data Integration: Once uploaded, ChatGPT processes the content of these documents,
integrating the information into its internal knowledge base.

Response Formulation: When responding to queries, ChatGPT cross-references the

user's question with the information contained in the uploaded documents. This allows
for more precise, tailored responses based on the speci ic content of the documents.

Enhanced Contextual Understanding: This capability enables ChatGPT to provide

responses that are not only based on its pre-trained knowledge but also augmented by
the speci ic information contained in the user's documents.

22
f
f
f
f
ff
f
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

LIMITATIONS
Data Accuracy: The accuracy of ChatGPT's responses heavily depends on the quality and
relevance of the uploaded documents. Misinformation or outdated data in these
documents could lead to inaccurate responses.

Scope of Document Processing: ChatGPT may have limitations in processing extremely

large or complex documents, and there might be challenges in extracting nuanced
information from them.

Content Relevance: The system's e ectiveness is contingent on the relevance of the

uploaded documents to the queries posed. Irrelevant or tangentially related documents
may not signi icantly enhance response quality.

Privacy and Security Concerns: When uploading sensitive or proprietary documents,

users must consider privacy and data security. The responsibility lies with the user to
ensure that the uploaded content does not violate any con identiality agreements or
privacy regulations.

BEST PRACTICES
Selecting Appropriate Documents: Choose documents that are directly relevant to the
topics you plan to inquire about. Ensure the content is current, accurate, and
comprehensive.

Data Structuring and Clarity: Organize the information in your documents clearly. Well-
structured data helps ChatGPT more e ectively parse and utilize the information.

Maintaining Data Privacy and Security: Be cautious about uploading sensitive or

con idential documents. Always adhere to data privacy laws and organizational policies.

Balancing Document Use with AI Knowledge: Understand that document upload

supplements ChatGPT's existing knowledge. Use it as an enhancement, not a
replacement, for the AI's built-in capabilities.

23
f
f
ff
ff
f
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

USE CASE
In the research department of a university, Dr. Ellis and her team were embarking on a
comprehensive study on climate change impacts on coastal ecosystems. They had
amassed numerous scienti ic papers, reports, and case studies, but the volume of
information was overwhelming.

To streamline their analysis, Dr. Ellis utilized ChatGPT's document upload feature. She
uploaded key research papers, government reports on climate data, and historical
studies on coastal ecosystems. ChatGPT processed these documents, creating an
enhanced knowledge base tailored to their speci ic research focus.

The team then interacted with ChatGPT, asking complex questions about trends in
climate data, comparisons of ecological impacts across di erent regions, and the
e ectiveness of various mitigation strategies. For each query, ChatGPT provided
informed responses, directly referencing the uploaded documents and highlighting key
indings.

This approach enabled Dr. Ellis's team to quickly identify critical research gaps, formulate
hypotheses, and design their study with a depth of understanding that would have taken
months to achieve manually. Their resulting research contributed signi icantly to the ield,
demonstrating a novel approach to academic research powered by AI-augmented
knowledge retrieval.

KNOWLEDGE AT YOUR FINGERTIPS

The document upload feature in ChatGPT marks a signi icant step forward in knowledge
retrieval, transforming the way information is processed and utilized in AI interactions.
This feature empowers users to create customized knowledge bases, enhancing the AI's
capabilities to provide tailored, accurate, and context-speci ic responses. While mindful
of its limitations and best practices, users can leverage this powerful tool across various
ields, from academic research to business intelligence, thereby opening new avenues for
data-driven decision-making and insight generation.

24
f
f
ff
f
f
f
ff
f
f
f
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

ACTIONS
O
penAI has introduced custom versions of ChatGPT called GPTs, which allow
developers and builders to create tailored assistants for speci ic use cases. GPTs
provide more customization capabilities through a feature called Actions, which
builds upon the existing plugins functionality. Actions connect external APIs and data
sources to GPTs, enabling them to pull real-time information into their responses and take
actions in the real world. Actions introduce new opportunities for customization like
hosted OpenAPI speci ications, granular authentication controls, and interfaces for
con irming potentially impactful transactions. In this section we’ll explore how GPT
Actions work under the hood to expand the capabilities of customized models while
balancing safety and usefulness.

25
f
f
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

HOW IT WORKS
GPT Actions provide a way for builders to connect external applications and data to their
customized ChatGPT models. Under the hood, Actions are de ined by something called
an OpenAPI speci ication - think of it like an instruction manual that explains how an
application works. The OpenAPI spec lays out all the available operations the Action can
perform. These operations are called "functions" in OpenAI terminology.

For example, the OpenAPI spec for a light booking Action would de ine functions that
can search for lights, read passenger data, book a ticket, cancel a booking, etc. As the
builder of this Action, you write code that implements each function.

When a user interacts with your GPT, they can trigger these Action functions and your
code runs in the background to do things like make a booking or pull up a light status.
The key advantage over a vanilla GPT is its ability to take real-world actions instead of just
providing information.

Actions also give more control compared to plugins. Builders can host the OpenAPI specs
directly on OpenAI servers instead of needing separate hosting. There is more granular
security con iguration for authentication. And a special lag marks potentially
"consequential" functions that should always ask the user for extra con irmation before
running.

In summary - builders de ine Action capabilities in OpenAPI, write code for the
operations, then connect Actions to GPTs to enable customized experiences backed by
external logic and data. It takes more e ort than plain text conversations, but that power
allows your GPT to be truly helpful beyond just chatting.

LIMITATIONS
While Actions expand the capabilities of customized GPTs, there are some limitations to
be aware of:

26
f
f
f
f
f
ff
f
f
f
f
f
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

Required Technical Knowledge: In order to fully utilize the actions, you will be required
to do a variety of technical things related to APIs, and formatting your actions in the
appropriate way. ChatGPT can be used to help with this process, but it’s more di icult
than just pressing a few buttons.

Early Stage: Actions and the broader GPT platform are still in relatively early stages of
development. There is likely to be ongoing evolution and changes to the tools - builders
should expect bugs and missing features.

Customization Tradeo s: Though more customizable than vanilla ChatGPT, Actions do

not provide full access to modify a model's training process or inference code. There are
also limits in place to balance safety and usefulness.

Hosted OpenAPI Speci ication: While hosting specs on OpenAI servers simpli ies things
for builders, it also means relying on OpenAI's uptime and performance. Any issues could
interrupt the GPT from accessing Action functions.

BEST PRACTICES
When building Actions for custom GPTs, keep these best practices in mind:

Carefully Use the Consequential Flag: Mark any functions that could have a signi icant
real-world impact as "consequential" so users must explicitly con irm before execution.
Don't overuse this though or users get fatigued from too many con irmation prompts.

Follow API Guidelines: Make sure to thoroughly read OpenAI's documentation and
comply with all technical and policy requirements around things like data privacy,
content policy, and API call patterns. This avoids disruptions.

Test Extensively: Validate that the Action integrates smoothly with the GPT conversation
low before releasing it widely. Have friends, colleagues or friendly users try it out and
give feedback.

27
f
ff
f
f
f
f
ff
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

Consider Security: Especially for Actions enabling purchases or sensitive operations,

ensure proper security practices are followed in your API code to protect user data and
prevent misuse.

Be particularly careful when sharing a GPT with colleagues or making it public, as they
will be able to access the action.

USE CASE
Dr. Smith was an economics professor at State College University embarking on research
around the impacts of automation and AI on job markets. He had compiled employment
data, academic studies, and news articles but was overwhelmed by the volume of
information.

To streamline his analysis, Dr. Smith utilized a customized ChatGPT model with a Data
Integration Action. He con igured the Action to securely connect to a database he built
containing all the job market datasets, papers, and articles.

Dr. Smith then engaged with his customized ChatGPT agent - named EconBot - to ask
complex questions about historical precedents for automation disruption, di erences
across countries and job categories, and e ectiveness of various worker retention
policies.

Drawing directly from the integrated database using the custom Action, EconBot
provided detailed responses to each of Dr. Smith's queries - highlighting relevant
excerpts and igures from the collected research data.

This AI-augmented approach enabled Dr. Smith to quickly gather key insights and
formulate hypotheses. He drafted models examining how targeted retraining programs
might mitigate automation impacts for older manufacturing workers. His resulting
published paper brought him signi icant recognition, thanks in part to the ampli ied
intelligence and e iciency gained from his customized GPT and Action integration.

28
f
ff
f
f
ff
ff
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

TAKE ACTION

Actions represent an important evolution in enabling more customized and capable AI

systems via OpenAI's GPT platform. By connecting external data and functional APIs,
developers can build GPT agents that not only converse, but can facilitate transactions,
retrieve real-time information, and take helpful actions in specialized domains.

However, Actions should be implemented thoughtfully, with careful consideration of

security, testing, and transparency to end users around key decisions enabled by Actions.
As with any nascent technology, expectations should be tempered - while the
possibilities are exciting, Actions do have limitations and open questions around best
practices.

For builders willing to invest the e ort though, Actions o er a toolkit for creating truly
assistive GPT experiences. As OpenAI continues maturing GPTs and the surrounding
ecosystem, we can expect Actions to unlock new waves of ever-more useful AI
applications for work and life. Their long-term impact remains to be seen, but the
foundations have great promise to enable helpful, harmless and honest AI assistants.

29
ff
ff
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

GPTs
O
penAI has introduced an exciting new capability called GPTs that allows builders
to create customized versions of ChatGPT for speci ic use cases. GPTs aim to
make ChatGPT more helpful for daily life, work, education, research, and more
by blending specialized data and instructions into tailored conversational agents.

At a high level, GPTs provide an easy way for anyone to build a version of ChatGPT
focused on a particular topic or purpose. You can teach these customized models
directly by having conversations, showing examples, and providing relevant documents.
GPTs also enable connecting external APIs so they can take actions like analyzing data,
generating images, or even facilitating transactions.

OpenAI designed GPTs to be accessible for everyday people to build, not just advanced
programmers. The only requirement is basic knowledge of the domain you want to
customize for. This aligns with OpenAI's mission of shaping safe and bene icial AI
alongside the broader community, not just internal teams.

HOW IT WORKS
At a technical level, GPTs build upon the foundation of existing large language models
like ChatGPT. Builders start by specifying instructions (prompts) for the GPT to follow.

OpenAI provides an intuitive web interface to upload custom documents and even plans
to have GPTs remember previous conversations even outside the context limit. Behind
the scenes, this e ectively " ine-tunes" the base ChatGPT model by updating its
parameters to prioritize domain-speci ic knowledge.

Builders have the choice of giving it any of the major tools available to ChatGPT normally,
including DALL-E, Code Interpreter, and Web Browsing, with GPT Vision included by
default.

30
ff
f
f
f
f
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

In addition to instructions and tools, builders can connect external APIs and data sources
using a capability called Actions. Actions allow real-time querying of databases, analyzing
datasets, or even executing transactions like purchases. This enables GPTs to move
beyond conversational responses into being truly assistive by taking actions in digital or
real-world contexts.

Once built, GPT creators control the visibility and permissions around who can access the
model. For public visibility, OpenAI provides a veri ication process and monitors models
to ensure they meet policy guidelines. Enterprise customers can restrict GPTs for internal
business use only.

Overall, GPTs aim to make AI customization intuitive yet powerful. They allow anyone to
e ectively teach ChatGPT new skills with hands-on guidance, augmented by external
data/actions where useful. The result is bespoke GPT models tailored for speci ic domains
and use cases.

31
ff
f
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

LIMITATIONS
Although signi icantly expanding the customizability of ChatGPT, GPTs do have some
limitations that are important to understand:

Early Stage Development: As an emerging capability, GPTs are bound to have ongoing
evolution, gaps, and bugs as OpenAI continues advancing the platform and addressing
user feedback. Those depending on GPTs for critical applications should be prepared for
changes.

Narrow Customization Constraints: While greater con igurability than ChatGPT alone,
builders cannot fully customize model architectures, training schemes, or inference
code. GPTs also do not provide access to edit raw model weights. Customization occurs
through guidance conversations and data examples.

Reliance on OpenAI Infrastructure: Hosting GPT models relies on OpenAI's cloud

platforms and services. Any outages or performance issues could interrupt access to
customized GPTs.

Safety & Policy Tradeo s: Allowing broader access for AI customization increases risks
of misuse or harmful content. Extensive mitigations are in place but likely still evolving as
OpenAI balances capabilities with responsible access.

In summary - those leveraging GPTs should set proper expectations around the
functionality, recognize it is still an emerging tool, and factor in the dependencies that
come with a hosted platform.

BEST PRACTICES
When building and deploying customized GPTs, it's important to carefully follow
recommendations and best practices, including:

Be Careful What You Upload: Builders making GPTs widely accessible should assume the
prompt and the knowledge base documents can be easily acquired, even if supposed
protections are in place.

32
f
ff
f
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

Extensively Test Models: Thoroughly evaluate GPT responses across a wide range of
queries, edge cases, and conversations before publishing. Have a diverse set of testers
interact and provide feedback.

Set Visibility Thoughtfully: Consider the intended use cases and audiences when
enabling model visibility. Restrict sensitive applications to private access only.

USE CASE
Dr. Lee was a biology professor embarking on an ambitious project to catalog and analyze
genetic data about rare plant species. She had amassed DNA sequencing data, research
on genetic markers, and observational data collected by ield scientists over decades.
However, making sense of these disconnected datasets was proving extremely di icult.

To aid her analysis, Dr. Lee built a customized ChatGPT model focused speci ically on
botany and genetics. She utilized the new GPT feature to upload key datasets - the
genetic repositories, research papers on relevant gene expressions, and even scanned
notebook pages from famous botanists. This allowed her AI assistant, which she named
Flora, to deeply understand the speci ic domain.

Dr. Lee then engaged with Flora, asking advanced questions about genetic similarities
across rare species, hypothesized impacts of climate change on future mutations, and
potential protective policies based on the research. Powered by the integrated datasets,
Flora provided thoughtful and data-driven perspectives on these complex queries.

By essentially "reading" and contextualizing such a large volume of specialized

information, Flora's responses unlocked insights that would have taken Dr. Lee's human
team months of manual analysis to unearth. Her lab consequently published
groundbreaking botanical preservation strategies, with the AI-augmentation providing
depth and e iciency no human could match. Dr. Lee's GPT customization approach
demonstrated the immense potential when conversational AI meets specialized
knowledge.

33
ff
f
f
f
ff
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

YOUR PERSONAL ASSISTANT(S)

GPTs represent a major evolution in AI capabilities - enabling virtually anyone to build

customized assistants for speci ic topics and use cases. By combining specialized
instructions, data sources, and con igurable actions, GPTs can grant conversational
models true understanding of unique domains.

However, creators should thoughtfully assess risks, test rigorously, and recognize the
limitations around an emerging technology. As with any powerful tool, responsibility is
critical as capabilities grow.

If leveraged carefully and for bene icial purposes, GPTs could help drive widespread
adoption of AI, democratizing access to customize and deploy models aligned with
personal and organizational needs. Their societal impacts remain unfolding, but the
possibilities are abundantly clear - o ering more control and agency in guiding AI to
enhance human empowerment.

34
f
f
f
ff
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

SUMMARY
Module 4 provides an in-depth exploration of ChatGPT+ and its array of advanced tools,
highlighting their transformative impact in AI applications. Throughout the module, we
examined the intricacies of web browsing enhancement, the creative prowess of DALL-E
for image generation, the analytical strength of advanced data analysis, the visual
acumen of GPT Vision, and the e icacy of knowledge retrieval capabilities in ChatGPT+.

Key Takeaways:

Expanded AI Capabilities: ChatGPT+ o ers enhanced functionalities, including

sophisticated web browsing, image generation, and complex data analysis.

Tool-Speci ic Limitations and Applications: Each tool within ChatGPT+ has unique
limitations and strengths, emphasizing the need for contextual understanding and
application.

Ethical Use and Best Practices: The module underscores the importance of ethical
considerations and responsible usage of AI tools, providing guidelines for e ective and
conscientious application.

Innovative Real-World Applications: ChatGPT+ tools can signi icantly augment decision-
making, creativity, and information management across various domains.

Evolution in AI Interaction: The integration of these advanced tools with ChatGPT+

represents a signi icant stride in the evolution of AI, expanding the horizon of what AI can
achieve in practical and creative scenarios.

35
f
f
ff
ff
f
ff
SYNTHMINDS - CHATGPT+ TOOLS & GPTS

GLOSSARY
Generative Large Multimodal Models (GLMMs): Advanced AI models capable of
understanding and interacting with multiple data modalities, such as text and images.

DALL-E: An AI program by OpenAI that creates images from textual descriptions, notable
for its creative and detailed outputs.

Advanced Data Analysis: Feature in ChatGPT+ that allows users to upload data for AI-
assisted analysis and interpretation.

GPT Vision: A ChatGPT feature integrating image processing capabilities, enabling the
model to analyze and interpret visual data.

Knowledge Retrieval: The process of extracting and synthesizing information from

uploaded documents, enhancing ChatGPT's response accuracy and relevance.

OpenAPI Speci ication: A technical description standard used for de ining the
operations of APIs, crucial in the implementation of GPT Actions.

GPT Actions: Customizable functionalities that enable ChatGPT to interact with external
APIs and data, allowing it to take real-world actions based on user prompts.

Ethical AI Use: The practice of using AI tools in a manner that is responsible, respects
privacy, and avoids misuse or harm.

36
f
f
 SYNTHMINDS - CHATGPT+ TOOLS & GPTS

(OPTIONAL) PROJECT: CREATE A GPT

This project can only be done with a ChatGPT+ subscription with access to GPTs, so it is
optional.

Building on the agent you created in Module 2, this project involves enhancing your agent
by integrating a knowledge base and providing it with the tools it needs for advanced
operation. This step will transform your agent from a simple prompt-responsive entity
into a more sophisticated, knowledge-informed assistant capable of handling complex
tasks.

Objectives:

• Understand the role of a knowledge base in enhancing an agent's capabilities.

• Integrate a relevant knowledge base with your existing agent.
• Equip the agent with tools and resources to improve its functionality.
• Evaluate the enhanced performance of your agent in complex scenarios.
Deliverables:

• Knowledge-Enhanced Agent: An upgraded version of your agent with an integrated

knowledge base.

• Tool Integration: A list of tools and resources added to your agent, with explanations of
their roles and functions.\

• Performance Evaluation: Assess the improvement in your agent's performance in

handling complex tasks compared to its initial version.

• Re lective Analysis: A detailed analysis of the integration process, challenges faced,

and learnings.

37
f