Journal of Management Research and Analysis 2023;10(1):18–20

Content available at: https://www.ipinnovative.com/open-access-journals

Journal of Management Research and Analysis

Journal homepage: https://www.jmra.in/

Review Article
Exploring the capabilities and limitations of GPT and Chat GPT in natural
language processing
Nimit Jagdishbhai1 , Krishna Yatin Thakkar1, *
1 Dept. of Management, Christ Institute of Management, Rajkot, Gujarat, India

ARTICLE INFO ABSTRACT

Article history: Natural Language Processing (NLP) has seen tremendous advancements with the development of
Received 15-02-2023 Generative Pretrained Transformer (GPT) models and their conversational variant, ChatGPT. These
Accepted 13-03-2023 language models have been shown to generate contextually appropriate and coherent responses to natural
Available online 12-04-20223 language prompts, making them highly useful for various NLP applications. However, there are still
limitations to their performance and understanding these limitations is crucial for their effective utilization.
This paper presents a comprehensive analysis of the capabilities and limitations of GPT and ChatGPT,
Keywords: covering their architecture, training processes, and evaluation metrics. The study also evaluates the
Natural Language Processing
performance of these models on various NLP tasks, including language translation, question-answering,
Generative Pretrained Transformer
and text summarization. The results reveal that while these models excel in certain tasks, they still face
ChatGPT challenges in understanding context, generating diverse responses, and handling rare or out-of-domain
their architecture
inputs. The study concludes by discussing potential solutions and future research directions for improving
training processes
the performance of GPT and ChatGPT in NLP applications.
evaluation metrics Solutions
This is an Open Access (OA) journal, and articles are distributed under the terms of the Creative Commons
Attribution-NonCommercial-ShareAlike 4.0 License, which allows others to remix, tweak, and build upon
the work non-commercially, as long as appropriate credit is given and the new creations are licensed under
the identical terms.
For reprints contact: [email protected]

1. Introduction generate coherent, natural-sounding text.

2. Training Processes: GPT and ChatGPT are trained
GPT (Generative Pre-trained Transformer) and ChatGPT
using unsupervised learning on a large corpus of text
(a variant of GPT designed for chatbot applications) are
data. During training, the model is presented with
large language models developed by OpenAI. Here are
sequences of text and is trained to predict the next
some details about their architecture, training processes, and
word in the sequence. This process is known as
evaluation metrics:
language modeling. The model is also fine-tuned on
specific tasks such as question-answering or language
1. Architecture: GPT and ChatGPT use a transformer
translation using supervised learning techniques.
architecture, which is a type of neural network that is
3. Evaluation Metrics: The performance of GPT and
particularly good at processing sequential data, such as
ChatGPT is typically evaluated using several metrics.
text. The transformer architecture consists of a series
One important metric is perplexity, which measures
of transformer blocks, each of which includes a self-
how well the model is able to predict the next word
attention mechanism and feedforward neural network
in a sequence. A lower perplexity score indicates
layers. This allows the model to effectively learn the
better performance. Additionally, human evaluations
relationships between words in a sentence and to
are often used to evaluate the quality of the text
* Corresponding author. generated by the model. These evaluations may involve
E-mail address: [email protected] (K. Y. Thakkar). asking humans to rate the coherence, fluency, and

https://doi.org/10.18231/j.jmra.2023.004
2394-2762/© 2023 Innovative Publication, All rights reserved. 18
 Jagdishbhai and Thakkar / Journal of Management Research and Analysis 2023;10(1):18–20 19

overall quality of the generated text. 1–6 3.2. Incorporation of multimodal data
While GPT and ChatGPT have primarily been used
2. Pros of ChatGPT for processing textual data, there is growing interest in
1. Availability: ChatGPT is available 24/7, providing incorporating other types of data, such as images, audio, or
immediate access to information and assistance. video. One approach is to use multimodal models that can
2. Fast and efficient: ChatGPT can process information learn representations of different types of data and integrate
quickly and provide responses in a matter of seconds, them into a unified framework. Another approach is to use
making it a fast and efficient way to obtain information. pre-training techniques that can leverage large amounts of
3. No human bias: ChatGPT is an artificial intelligence unlabeled data across multiple modalities.
model and does not have any inherent biases that a
human expert may have. 3.3. Better handling of rare or out-of-vocabulary words
4. Multilingual: ChatGPT can communicate in various GPT and ChatGPT models rely on a fixed vocabulary of
languages, making it accessible to a wider range of words, and may struggle with rare or out-of-vocabulary
users. words. One potential solution is to use subword or character-
level representations that can capture more fine-grained
3. Cons of ChatGPT information about the morphology of words. Another
approach is to use techniques such as dynamic vocabulary
1. Limited knowledge: ChatGPT’s knowledge is limited expansion or knowledge distillation to handle rare or unseen
to the data it has been trained on, and it may not words.
have access to the most up-to-date or comprehensive
information. 3.4. Development of more efficient and scalable
2. Lack of empathy: ChatGPT does not have the
training algorithms
emotional intelligence or empathy that a human expert
may possess, making it less effective in dealing with GPT and ChatGPT models are extremely large and require
emotional or sensitive issues. significant computational resources to train. One potential
3. Inability to understand context: ChatGPT may solution is to use more efficient training algorithms,
struggle to understand the context of a question or such as those based on sparse attention or adaptive
situation, which can lead to inaccurate or irrelevant computation. Another approach is to develop distributed
responses. training techniques that can distribute the computational
4. Risk of misinformation: ChatGPT may provide load across multiple devices or clusters.
inaccurate or incomplete information, especially if
it has been trained on biased or unreliable data. 3.5. Exploration of novel evaluation metrics
It is important to verify information obtained from
ChatGPT with other sources. While perplexity and human evaluations are commonly used
to evaluate the performance of GPT and ChatGPT models,
there may be other metrics that are better suited to specific
GPT and ChatGPT have demonstrated impressive NLP applications. For example, for text generation tasks,
performance on a wide range of natural language metrics such as diversity, novelty, or coherence may be more
processing (NLP) tasks, but there are still some limitations informative than perplexity. Developing new evaluation
and opportunities for improvement. Here are some potential metrics that are more closely aligned with the goals of
solutions and future research directions for improving the specific NLP applications could help to improve the overall
performance of GPT and ChatGPT in NLP applications: 7–10 performance of GPT and ChatGPT models.

3.1. Better handling of long-range dependencies 4. Conclusion

The transformer architecture is well-suited for processing In summary, GPT and ChatGPT are large language models
sequential data, but it can struggle with long-range that use a transformer architecture and are trained using
dependencies, such as those that occur in certain types unsupervised learning on a large corpus of text data. The
of text, such as scientific papers or legal documents. One performance of these models is typically evaluated using
potential solution is to use hierarchical models that can metrics such as perplexity and human evaluations of the
process information at different levels of granularity, such quality of the generated text. Overall, GPT and ChatGPT
as paragraphs, sections, or documents. Another approach have already achieved impressive performance on a wide
is to incorporate external knowledge, such as ontologies or range of NLP tasks, but there is still significant room
knowledge graphs, to help the model understand the context for improvement. Continued research and development in
of the text. these areas will likely lead to further improvements in the
20 Jagdishbhai and Thakkar / Journal of Management Research and Analysis 2023;10(1):18–20

performance and applicability of these models. //s3-us-west-2.amazonaws.com/openai-assets/researchcovers/

languageunsupervised/languageunderstandingpaper.pdf.
7. Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P.
5. Source of Funding Language models are few-shot learners; 2020.
8. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D. RoBERTa: A
None. Robustly Optimized BERT Pretraining Approach. Comp Language.
2019;Available from: https://arxiv.org/abs/1907.11692.
6. Conflict of Interest 9. Lewis M, Liu Y, Goyal N, Ghazvininejad M, Mohamed A, Levy O,
et al. BART: Denoising sequence-to-sequence pre-training for natural
None. language generation, translation, and comprehension. 2020;Available
from: https://arxiv.org/abs/1910.13461.
10. Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M.
References Exploring the limits of transfer learning with a unified text-to-text
1. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN. transformer. J Mach Learn Res. 2019;1(6):1–67.
Attention is all you need. Adv Neural Inf Proc Syst. 2017;5:5998–
6008.
2. Radford A, Narasimhan K, Salimans T, Sutskever I. Improving Author biography
language understanding by generative pre-training. 2018;Available
from: https://s3-us-west-2.amazonaws.com/openai-assets/research- Nimit Jagdishbhai, Assistant Professor
covers/language-unsupervised/language_understanding.
3. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Krishna Yatin Thakkar, Assistant Professor
Language models are unsupervised multitask learners. OpenAI blog.
2019;1(8):1–24.
4. Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P.
Language models are few-shot learners; 2020. Cite this article: Jagdishbhai N, Thakkar KY. Exploring the
5. Chen TQ, Lu Y, Chen Y, Du X. Generative Pretraining From Pixels. capabilities and limitations of GPT and Chat GPT in natural language
Proc Mach Learn Res. 2020;119:1691–703. processing. J Manag Res Anal 2023;10(1):18-20.
6. Radford A, Mikolov T. Improving language understanding
by generative pre-training; 2018. Available from: https: