Does Prompt Formatting Have Any Impact on LLM Performance?

Jia He1 * , Mukund Rungta1 * , David Koleczek1 , Arshdeep Sekhon1 ,

Franklin X Wang2 , Sadid Hasan1
1
Microsoft, 2 MIT
{hejia, rungtamukund, dkoleczek, asekhon sadidhasan}@microsoft.com
{fxwang}@mit.edu

Abstract
In the realm of Large Language Models
arXiv:2411.10541v1 [cs.CL] 15 Nov 2024

(LLMs), prompt optimization is crucial for

model performance. Although previous re-
search has explored aspects like rephrasing
prompt contexts, using various prompting tech-
niques (like in-context learning and chain-of-
thought), and ordering few-shot examples, our
understanding of LLM sensitivity to prompt Figure 1: An example to demonstrate how prompt for-
templates remains limited. Therefore, this pa- matting impacts GPT-35-turbo-16k-0613 model’s per-
per examines the impact of different prompt formance based on our experiments on multiple choice
templates on LLM performance. We formatted questions related to international law from the MMLU
the same contexts into various human-readable benchmark ((Hendrycks et al., 2020)). Texts inside
templates, including plain text, Markdown, "<>" are replaced by actual contexts. Accuracy goes
JSON, and YAML, and evaluated their impact up by 42% for JSON compared to Markdown.
across tasks like natural language reasoning,
code generation, and translation using Ope-
nAI’s GPT models. Experiments show that
GPT-3.5-turbo’s performance varies by up to Often overlooked, prompt format can signifi-
40% in a code translation task depending on the cantly impact model performance, contrary to the
prompt template, while larger models like GPT- assumption that it remains stable across different
4 are more robust to these variations. Our anal- templates. There exist limited research and anecdo-
ysis highlights the need to reconsider the use
tal evidence ((Aghajanyan, June 2023; Sclar et al.,
of fixed prompt templates, as different formats
can significantly affect model performance. 2023; Voronov et al., 2024)), which suggest that
prompt format choices may lead to substantial per-
1 Introduction formance variations, raising concerns about current
evaluation standards that ignore this factor. For
The emergence of LLMs marks a significant ad- example, one study showed that LLMs are sensi-
vancement in AI, revolutionizing natural language tive to minor fine-grained prompt modifications,
processing, understanding, and generation ((Brown such as separators or capitalization changes ((Sclar
et al., 2020; Ouyang et al., 2022; Chowdhery et al., et al., 2023)). Also, existing evaluation approaches
2022; Achiam et al., 2023)). Prompt engineering typically use fixed templates, potentially leading to
has become crucial, focusing on crafting inputs that misleading conclusions ((Voronov et al., 2024)).
guide LLMs to produce desired outputs, leverag-
Inspired by these findings, our study investigates
ing a nuanced understanding of how these models
whether broader changes in prompt format affect
interpret and respond to prompts ((Sahoo et al.,
model efficacy. We evaluate the impact of prompt
2024)). Effective prompt design generally includes
templates on OpenAI’s four GPT models across six
clear instructions, Retrieval-Augmented Genera-
benchmarks, using plain text, Markdown, YAML,
tion (RAG) or other prompting approaches for en-
and JSON formats, as illustrated in Figure 1. This
hancing in-context learning (ICL), and appropriate
comprehensive approach contrasts with prior re-
formatting.
search that primarily examined minor template al-
* Equal Contribution terations. Our research focuses on the GPT model

1
series for two main reasons: the lack of compara- 2 Experimental Setup
tive analyses of behavioral patterns across different
2.1 Datasets
GPT model iterations, especially the latest GPT-4-
turbo, and the need to identify effective interaction Our experiments span various tasks and datasets,
methods and optimal input formats for these mod- categorized into three main groups:
els, which do not disclose their training methodolo-
• Natural Language to Natural Language
gies or data.
(NL2NL): Includes Massive Multitask Lan-
Our study is designed to investigate the follow-
guage Understanding (MMLU) (Hendrycks
ing key questions:
et al., 2020) and NER Finance from OpenAI
• Sensitivity: To what extent does the perfor- Evals (OpenAI, 2023).
mance of GPT models vary with different
prompt formats? • Natural Language to Code (NL2Code): In-
cludes HumanEval (Chen et al., 2021) and
• Consistency: Are GPT models capable of pro- FIND (Schwettmann et al., 2023).
ducing uniform responses to identical queries
when presented with varying prompt struc- • Code to Code (Code2Code): Includes
tures? CODEXGLUE (Lu et al., 2021) and
HumanEval-X (Zheng et al., 2023).
• Transferability: Is there an optimal prompt
format that is universally effective across di- We initially assess model performance using
verse GPT models, thereby ensuring peak per- task-specific scalar scoring functions, followed by
formance? metrics from Sections 3 to 5 to address our research
questions. Detailed dataset descriptions and met-
In addition to our primary questions, we explore rics are in Appendix B.
the correlation between prompt format efficacy and
task-specific competencies, as well as the impact of 2.2 Prompt Design
model size on performance. OpenAI’s GPT models We use various input formats: plain text, mark-
including GPT-35-turbo and GPT-4 (Achiam et al., down, YAML, and JSON. Prompts include five
2023) show unpredictable sensitivity to prompt for- components: persona, task instructions, examples,
mat changes, with significant performance discrep- output format instructions, and user ask. We en-
ancies across all models and benchmarks. Notably, sure the content of each placeholder stays the same
there is no universally optimal format, even within across different prompt formats. The only differ-
the same generational lineage. However, GPT-4- ences are in structure and syntax. To avoid con-
turbo demonstrates greater resilience to prompt founding variables, we design the prompts so that
format changes compared to its predecessors and the context and meaning remain consistent, regard-
contemporaries. In summary, our key contributions less of the format. Examples are in Appendix C.
are as follows:
2.3 Models
• This study is the first to compare the impact
of different prompt formats on GPT models’ Experiments were conducted on OpenAI’s GPT-
performance across various tasks, examining 3.5 and GPT-4 models via Azure (Microsoft, 2024).
plain text, Markdown, YAML, and JSON. For GPT-3.5, we used “gpt-35-turbo-0613” and
“gpt-35-turbo-16k-0613” to compare context win-
• Our research provides an extensive analysis dow sizes (4k vs. 16k). For GPT-4, we used “gpt-
of prompt formatting effects on GPT mod- 4-32k-0613” and “gpt-4-1106-preview” to test the
els across a wide range of tasks, including newer, faster variant with a 128k context window.
multiple-choice questions, code generation,
and translation. 3 Sensitivity

• We present an evaluation of the GPT model 3.1 Metrics Definition

iterations via Azure OpenAI, revealing that Sensitivity. To evaluate how much the choice of
GPT-4-turbo is less susceptible to prompt prompt template impacts a model’s performance
structure variations compared to earlier mod- on a task T, we look at a variety of templates
els. {p1 , p2 , . . . , pn } and measure their performance

2
 GPT-35-turbo-0613 GPT-35-turbo-16k-0613 GPT-4-1106-preview GPT-4-32k-0613
Max Min p-value Max Min p-value Max Min p-value Max Min p-value
59.7 50.0 59.4 50.7 81.2 73.9 81.3 77.8
MMLU < 0.001 < 0.001 < 0.001 < 0.001
(JSON) (Markdown) (JSON) (Markdown) (Markdown) (JSON) (Markdown) (JSON)

59.8 40.2 57.9 37.20 86.6 82.9 76.2 21.95

HumanEval < 0.001 < 0.001 0.055 < 0.001
(JSON) (Plain text) (JSON) (Plain text) (Markdown) (Plain text) (Plain text) (JSON)

37.2 24.6 36.80 21.8 53.8 49.3 53.2 47.2

NER Finance < 0.001 < 0.001 0.001 < 0.001
(Plain text) (YAML) (Plain text) (YAML) (YAML) (Plain text) (YAML) (Plain text)

CODEXGLUE 78.4 66.5 78.4 66.5 77.0 68.2 74.2 67.2

< 0.001 < 0.001 < 0.001 < 0.001
(Java2CS) (JSON) (Plain text) (JSON) (Plain text) (Markdown) (Plain text) (Markdown) (Plain text)

CODEXGLUE 77.5 68.8 77.5 68.9 83.1 68.1 75.0 67.9

< 0.001 < 0.001 < 0.001 < 0.001
(Cs2Java) (JSON) (Plain text) (JSON) (Plain text) (JSON) (Plain text) (JSON) (Plain text)

69.8 63.0 69.8 62.9 72.4 63.9 72.3 65.0

HumanEval-X < 0.001 < 0.001 < 0.001 < 0.001
(YAML) (Plain text) (YAML) (Plain text) (JSON) (Plain text) (JSON) (Plain text)

15.9 5.2 15.8 5.0 20.7 20.08 21.9 17.4

FIND < 0.001 < 0.001 < 0.0269 < 0.001
(Plain text) (Markdown) (Plain text) (Markdown) (Markdown) (Plain text) (plaintext) (markdown)

Table 1: Sensitivity of model performance to prompt format assessed using one-sided matched pair t-tests. Table
displays metrics for top and bottom formats (Max/Min) and p-values for each dataset/model. All p-values are below
0.05, except for GPT-4-1104-preview on HumanEval, confirming widespread prompt format sensitivity.

using a scoring function s. We identify the high- 4 Consistency

est score M ax(s(pi , T)) and the lowest score
M in(s(pi , T)) achieved by the templates for task 4.1 Metrics Definition
T. Then, we use a matched pairs t-test to determine Following the sensitivity measurement, we quan-
if the performance difference is statistically signif- tify the extent of answer variation due to prompt
icant. If the test shows significance, the prompt changes using the consistency metric from (Shu
format matters; if not, the model’s performance is et al., 2023). This metric calculates the proportion
relatively insensitive to the prompt used. of test samples that yield identical responses for
two prompt templates. The consistency C(Pa , Pb )
for templates Pa and Pb is defined as:
3.2 Does prompt format impact the
performance of language models and how N
1 X
significant is the performance variation C(Pa , Pb ) = 1 (APa (xi ) = APb (xi ))
when switching prompt formats? N
i=1

We begin by analyzing if model performance is sen- where N is the test set size and A represents the
sitive to any changes in the prompt format at all. To model’s answer. A higher score indicates greater
assess this, we conducted a one-sided matched pair answer consistency between prompts.
t-test, comparing the best and worst performing
formats for each model across various benchmarks. 4.2 Are larger models more consistent in
The resulting p-values, which are shown in Table generated outputs between templates?
1, are mostly below 0.01. This suggests that the Our study assessed the consistency of model out-
differences in model performance due to format puts using the MMLU and FIND datasets, as shown
changes are statistically significant. in Figures 2 and 8. For MMLU, we set the temper-
Figure 4 visualizes how the models fare across ature to zero to eliminate response variability. The
all benchmarks, highlighting a considerable range GPT-3.5-turbo series displayed low consistency,
in performance. For instance, in the FIND dataset, with scores below 0.5, and only 16% identical re-
both GPT-35-turbo-0613 and GPT-35-turbo-16k- sponses between Markdown and JSON formats.
0613 show a dramatic 200% improvement when In contrast, GPT-4’s consistency scores surpassed
prompts are switched from Markdown to plain text. 0.5, indicating better reliability across different
Similarly, for the HumanEval benchmark, the GPT- prompts. For the FIND dataset, following the set-
4 model with a 32k-0613 configuration exhibits an tings from (Schwettmann et al., 2023), GPT-4 again
impressive performance boost of over 300% when outperformed the GPT-3.5-turbo series in consis-
the prompt format is changed from JSON to plain tency. These findings suggest that larger models
text. This suggests, LLM performance may not like GPT-4 are more consistent, but there is still a
be robust to the choice of prompt format. need for model improvements to achieve reliable

3
 (a) GPT-35-Turbo-0613 (b) GPT-35-turbo-16k-0613

Figure 3: Intersection over Union (IoU) scores for top

templates on the NER Finance benchmark across mod-
els. Higher IoU is observed within same-version model
(c) GPT-4-1106-preview (d) GPT-4-32k-0613 pairs, whereas cross-version pairs exhibit lower IoU.

Figure 2: Consistency comparison for MMLU dataset:

GPT-3.5 models show consistency scores below 0.5 over-Union (IoU) metrics (Figure 3 and Appendix
across format pairs, whereas GPT-4 consistently ex- B), we found low compatibility between different
ceeds 0.5, indicating greater reliability. model series, with IoU often below 0.2. How-
ever, models from the same sub-series, like GPT-
performance across various formats. In summary, 35-turbo-16k-0613 and GPT-35-turbo-0613, show
the consistency of model responses varies with high IoU over 0.7.
size, with larger models like GPT-4 providing These insights highlight that even with com-
more uniform outputs across different prompts. mon architectures and training goals, GPT-models
react differently to identical prompts. Optimal
5 Transferability performance requires model-specific prompt en-
gineering, as no single format works universally
5.1 Metrics Definition across various GPT models, even within the same
Intersection-over-Union. To assess the transfer- family. This underscores the necessity for
ability of prompt templates between models, we tailored prompt engineering due to the non-
calculate the Intersection-over-Union (IoU) for the transferability of prompt formats across differ-
sets of top-performing templates between model ent GPT models.
pairs. Top-performing templates are those with
statistically indistinguishable performance, deter- 6 Conclusion
mined by a matched pairs t-test. The IoU is defined Our study reveals that the way prompts are for-
as: matted significantly impacts GPT-based models’
|Pm1 ∩ Pm2 |
IoU = performance, with no single format excelling uni-
|Pm1 ∪ Pm2 | versally. This finding questions current evaluation
where Pm1 and Pm2 represent the sets of top tem- methods that often ignore prompt structure, po-
plates for models m1 and m2, respectively. An IoU tentially misjudging a model’s true abilities. We
threshold of 0.5 is common, but a higher threshold advocate for diverse prompt formats in future LLM
like 0.7 indicates greater overlap. testing to accurately gauge and enhance their per-
formance.
5.2 Do models from same family exhibit Regarding explainability, we observe that model
similar trend across prompt formats? size affects model’s responses to prompt variations.
Our research into Large Language Models (LLMs), For instance, GPT-4’s performance is less influ-
GPT-based models in particular, reveals that enced by prompt changes compared to GPT-3.5,
prompt formatting preferences vary by model. As suggesting that larger models may process prompts
demonstrated in Figure 5, GPT-3.5-turbo prefers more consistently. This discovery prompts further
JSON, whereas GPT-4 favors Markdown. When ex- research into LLM interpretability, aiming to refine
amining prompt transferability using Intersection- AI adaptability and human-AI interaction.

4
7 Limitations Chess, Jack Clark, Christopher Berner, Sam Mc-
Candlish, Alec Radford, Ilya Sutskever, and Dario
This study was focused on GPT-based models, how- Amodei. 2020. Language models are few-shot learn-
ever, we plan to examine the impact of prompt for- ers. Preprint, arXiv:2005.14165.
mats on other models, such as LLaMA (Touvron
Sébastien Bubeck, Varun Chandrasekaran, Ronen El-
et al., 2023), Gemini (Team et al., 2023), PaLM
dan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Pe-
(Chowdhery et al., 2022), or smaller models like ter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg,
Phi (Li et al., 2023) in the future. This would pro- et al. 2023. Sparks of artificial general intelligence:
vide a more holistic understanding of the influence Early experiments with gpt-4. arxiv. arXiv preprint
that prompt formatting exerts across different LLM arXiv:2303.12712.
families.
Nicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvi-
Moreover, there is an opportunity to enhance jotham, Thomas Steinke, Jonathan Hayase, A Feder
the breadth of template exploration in subsequent Cooper, Katherine Lee, Matthew Jagielski, Milad
studies. Our research did not include formats like Nasr, Arthur Conmy, et al. 2024. Stealing part
HTML or XML, which are prevalent in the train- of a production language model. arXiv preprint
arXiv:2403.06634.
ing datasets of many models. Incorporating these
formats could yield a more exhaustive examination Mark Chen, Jerry Tworek, Heewoo Jun, Qiming
of prompt format effects. Yuan, Henrique Ponde de Oliveira Pinto, Jared Ka-
Lastly, our experimental design maintained all plan, Harri Edwards, Yuri Burda, Nicholas Joseph,
other prompt design elements constant, isolating Greg Brockman, Alex Ray, Raul Puri, Gretchen
Krueger, Michael Petrov, Heidy Khlaaf, Girish Sas-
prompt format as the sole variable. It would be try, Pamela Mishkin, Brooke Chan, Scott Gray,
intriguing for future work to investigate how the Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz
sensitivity of models to prompt format might shift Kaiser, Mohammad Bavarian, Clemens Winter,
when other prompt engineering techniques are mod- Philippe Tillet, Felipe Petroski Such, Dave Cum-
mings, Matthias Plappert, Fotios Chantzis, Eliza-
ified. This includes varying the number of few- beth Barnes, Ariel Herbert-Voss, William Hebgen
shot examples provided or refining the precision Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie
of prompt instructions. Such research could offer Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain,
valuable insights into the interplay between prompt William Saunders, Christopher Hesse, Andrew N.
Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan
structure and model responsiveness, potentially in-
Morikawa, Alec Radford, Matthew Knight, Miles
forming more effective prompt engineering prac- Brundage, Mira Murati, Katie Mayer, Peter Welinder,
tices. Bob McGrew, Dario Amodei, Sam McCandlish, Ilya
Sutskever, and Wojciech Zaremba. 2021. Evaluating
large language models trained on code.
References
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin,
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Maarten Bosma, Gaurav Mishra, Adam Roberts,
Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Paul Barham, Hyung Won Chung, Charles Sutton,
Diogo Almeida, Janko Altenschmidt, Sam Altman, Sebastian Gehrmann, Parker Schuh, Kensen Shi,
Shyamal Anadkat, et al. 2023. Gpt-4 technical report. Sasha Tsvyashchenko, Joshua Maynez, Abhishek
arXiv preprint arXiv:2303.08774. Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vin-
odkumar Prabhakaran, Emily Reif, Nan Du, Ben
Armen Aghajanyan. June 2023. Tweet: Susan and i Hutchinson, Reiner Pope, James Bradbury, Jacob
found mmlu performance jump 6-10 points in the Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin,
40s by formatting multiple choice as (a) not a in Toju Duke, Anselm Levskaya, Sanjay Ghemawat,
mmlu (for internal model). all evaluation of llm’s Sunipa Dev, Henryk Michalewski, Xavier Garcia,
are broken. evaluating a task requires marginalizing Vedant Misra, Kevin Robinson, Liam Fedus, Denny
across all prompts that describe the task, not point Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim,
estimate of one. Barret Zoph, Alexander Spiridonov, Ryan Sepassi,
David Dohan, Shivani Agrawal, Mark Omernick, An-
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie drew M. Dai, Thanumalayan Sankaranarayana Pil-
Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind lai, Marie Pellat, Aitor Lewkowycz, Erica Moreira,
Neelakantan, Pranav Shyam, Girish Sastry, Amanda Rewon Child, Oleksandr Polozov, Katherine Lee,
Askell, Sandhini Agarwal, Ariel Herbert-Voss, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark
Gretchen Krueger, Tom Henighan, Rewon Child, Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy
Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov,
Clemens Winter, Christopher Hesse, Mark Chen, and Noah Fiedel. 2022. Palm: Scaling language mod-
Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin eling with pathways. Preprint, arXiv:2204.02311.

5
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha,
Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Vinija Jain, Samrat Mondal, and Aman Chadha.
2020. Measuring massive multitask language under- 2024. A systematic survey of prompt engineering in
standing. arXiv preprint arXiv:2009.03300. large language models: Techniques and applications.
Preprint, arXiv:2402.07927.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio
Petroni, Vladimir Karpukhin, Naman Goyal, Hein- Sarah Schwettmann, Tamar Rott Shaham, Joanna
rich Küttler, Mike Lewis, Wen tau Yih, Tim Rock- Materzynska, Neil Chowdhury, Shuang Li, Jacob An-
täschel, Sebastian Riedel, and Douwe Kiela. 2021. dreas, David Bau, and Antonio Torralba. 2023. Find:
Retrieval-augmented generation for knowledge- A function description benchmark for evaluating in-
intensive nlp tasks. Preprint, arXiv:2005.11401. terpretability methods. Preprint, arXiv:2309.03886.

Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane
Del Giorno, Suriya Gunasekar, and Yin Tat Lee. 2023. Suhr. 2023. Quantifying language models’ sensitiv-
Textbooks are all you need ii: phi-1.5 technical report. ity to spurious features in prompt design or: How i
arXiv preprint arXiv:2309.05463. learned to start worrying about prompt formatting.
arXiv preprint arXiv:2310.11324.
Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paran-
jape, Michele Bevilacqua, Fabio Petroni, and Percy Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia
Liang. 2023. Lost in the middle: How language mod- Dunagan, Dallas Card, and David Jurgens. 2023. You
els use long contexts. Preprint, arXiv:2307.03172. don’t need a personality test to know these models are
unreliable: Assessing the reliability of large language
Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey models on psychometric instruments. arXiv preprint
Svyatkovskiy, Ambrosio Blanco, Colin Clement, arXiv:2311.09718.
Dawn Drain, Daxin Jiang, Duyu Tang, et al. 2021.
Yuan Sui, Mengyu Zhou, Mingjie Zhou, Shi Han, and
Codexglue: A machine learning benchmark dataset
Dongmei Zhang. 2024. Table meets llm: Can large
for code understanding and generation. arXiv
language models understand structured table data? a
preprint arXiv:2102.04664.
benchmark and empirical study. In The 17th ACM
Yao Lu, Max Bartolo, Alastair Moore, Sebastian International Conference on Web Search and Data
Riedel, and Pontus Stenetorp. 2022. Fantastically Mining (WSDM ’24).
ordered prompts and where to find them: Over- Gemini Team, Rohan Anil, Sebastian Borgeaud,
coming few-shot prompt order sensitivity. Preprint, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu,
arXiv:2104.08786. Radu Soricut, Johan Schalkwyk, Andrew M Dai,
Anja Hauth, et al. 2023. Gemini: a family of
Microsoft. guidance. https://github.com/guidance-ai.
highly capable multimodal models. arXiv preprint
Microsoft. 2024. Azure openai service mod- arXiv:2312.11805.
els. https://learn.microsoft.com/en-us/ Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier
azure/ai-services/openai/concepts/models# Martinet, Marie-Anne Lachaux, Timothée Lacroix,
gpt-4-and-gpt-4-turbo-preview. Accessed: Baptiste Rozière, Naman Goyal, Eric Hambro,
2024-03-26. Faisal Azhar, et al. 2023. Llama: Open and effi-
cient foundation language models. arXiv preprint
OpenAI. 2023. Evals. https://github.com/openai/
arXiv:2302.13971.
evals.
Anton Voronov, Lena Wolf, and Max Ryabinin. 2024.
OpenAI. 2024. New embedding models and api updates. Mind your format: Towards consistent evaluation of
Accessed: 2024-03-26. in-context learning improvements. arXiv preprint
OpenAI. November 2023. Improved instruction follow- arXiv:2401.06766.
ing and json mode. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le,
Ed Chi, Sharan Narang, Aakanksha Chowdhery, and
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car- Denny Zhou. 2023. Self-consistency improves chain
roll L. Wainwright, Pamela Mishkin, Chong Zhang, of thought reasoning in language models. Preprint,
Sandhini Agarwal, Katarina Slama, Alex Ray, John arXiv:2203.11171.
Schulman, Jacob Hilton, Fraser Kelton, Luke Miller,
Maddie Simens, Amanda Askell, Peter Welinder, Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten
Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and
Training language models to follow instructions with Denny Zhou. 2023. Chain-of-thought prompting elic-
human feedback. Preprint, arXiv:2203.02155. its reasoning in large language models. Preprint,
arXiv:2201.11903.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-
Jing Zhu. 2002. Bleu: a method for automatic evalu- Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak
ation of machine translation. In Proceedings of the Shafran, Karthik Narasimhan, and Yuan Cao. 2023.
40th annual meeting of the Association for Computa- React: Synergizing reasoning and acting in language
tional Linguistics, pages 311–318. models. Preprint, arXiv:2210.03629.

6
Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex includes investigations into the sequential arrange-
Smola. 2022. Automatic chain of thought ment of context ((Liu et al., 2023; Zhao et al., 2021;
prompting in large language models. Preprint,
Lu et al., 2022)) and the design of prompt formats
arXiv:2210.03493.
((Sclar et al., 2023; Voronov et al., 2024; Shu et al.,
Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, 2023)). Our work contributes to this growing body
and Sameer Singh. 2021. Calibrate before use: Im- of literature by examining the impact of prompt
proving few-shot performance of language models.
Preprint, arXiv:2102.09690. formatting on the performance of LLMs.

Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Prompt Format The sensitivity of LLMs to
Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang, prompt construction is a well-documented phe-
Yang Li, Teng Su, Zhilin Yang, and Jie Tang. 2023. nomenon, yet research on the impact of prompt
Codegeex: A pre-trained model for code genera-
formats on model performance remains sparse. Pi-
tion with multilingual evaluations on humaneval-x.
Preprint, arXiv:2303.17568. oneering studies ( (Sclar et al., 2023; Voronov et al.,
2024; Shu et al., 2023)) have conducted rigorous
A Related Work investigations, revealing that widely used open-
source LLMs exhibit extreme sensitivity to vari-
Prompt Engineering The field of prompt engi-
ations in prompt format. These studies, however,
neering has garnered significant interest in recent
primarily focus on subtle, local changes to the for-
years, in parts due to the emergent capabilities of
mat—such as the number of colons following a
the most capable LLMs, while also trying to bet-
question, the insertion of newlines, or the selection
ter control their still unpredictable outcomes. A
of input/output verbalizers. Besides, their experi-
prominent strand of research within this domain
mental designs are confined to classification tasks,
concentrates on innovative prompting methodolo-
limiting the generalizability of findings across di-
gies. These include few-shot prompting ((Brown
verse tasks.
et al., 2020)), which enables models to adapt to
Our research diverges from these existing stud-
new tasks without extensive retraining, and Chain-
ies by examining the effects of global prompt for-
of-Thought prompting ((Wei et al., 2023)), both of
mat modifications on model performance, offering
which are designed to enhance the reasoning capa-
insights that are applicable to a broad spectrum
bilities of LLMs. Additionally, Automatic Chain-
of LLM-based tasks that necessitate prompt engi-
of-Thought (Auto-CoT) ((Zhang et al., 2022)) and
neering. The closest related work to ours is by
Self-Consistency ((Wang et al., 2023)) approaches
(Sui et al., 2024), which however only provides
have been developed to further refine these reason-
a cursory exploration of format influence and is
ing processes. To mitigate hallucinations in LLM
restricted to tabular data. To the best of our knowl-
outputs, techniques such as Retrieval Augmented
edge, our study is the first effort to systematically
Generation (RAG) ((Lewis et al., 2021)) and Re-
investigate the impact of global prompt format vari-
Act ((Yao et al., 2023)) have been introduced. A
ations - an inescapable aspect of prompt engineer-
thorough examination of these methodologies can
ing design decisions.
be found in the survey by (Sahoo et al., 2024).
In recent developments, a novel prompt program- B Datasets
ming framework ((Microsoft)) has been introduced,
which offers greater control and efficiency in gener- We evaluate six distinct benchmarks and classify
ating structured outputs. Our study diverges from them according to the nature of the task involved.
this approach by examining the effects of more NL2NL
prevalent and established prompt formats on LLMs,
as opposed to investigating formats that are newly • Massive Multitask Language Understand-
proposed and not widely adopted yet. Furthermore, ing (MMLU) (Hendrycks et al., 2020) covers
it is important to note that third-party tools are 57 subjects including 20 STEM subjects, 13
predominantly designed for integration with open- humanities subjects, 12 social sciences sub-
source models, which may not seamlessly extend jects and 12 other subjects. Each subject con-
to proprietary models such as GPT. Another similar tains at least 100 multiple choice questions,
vein of research is dedicated to the structural design which tests both world knowledge and prob-
of prompts, aiming to optimize task performance lem solving ability. We use the dev set which
without altering the inherent semantic content. This contains 5 questions per subjects as few-shot

7
 examples, and use test set containing 14,079 benchmark for CODE. It was originally intro-
questions with different levels of difficulty to duced to address the lack of diversified bench-
evaluation model performance. We use accu- marks in code intelligence by providing a di-
racy score to measure model performance. verse set of tasks, including code translation.
We use the parallel code for Java and C-Sharp
• NER Finance: OpenAI Evals (OpenAI,
and vice versa. We use the test set containing
2023) is a framework containing a registry of
1000 parallel code in Java and C-Sharp to ex-
evaluations to test LLMs where NER Finance
periment the capabilities of the LLMs in trans-
is one of those. This dataset contains samples
lating code from one programming language
between one sentence to one paragraph long
to another. The performance of the LLMs
from financial documents. The task is to ex-
is evaluated using the BLEU (Papineni et al.,
tract the all of the entities in the document,
2002) score, which compares the generated
with the evaluation being if the LLM outputs
code to the reference code.
each entity, in order. We randomly sample
500 examples from this dataset. • HumanEval-X (Zheng et al., 2023) dataset is
a benchmark designed to evaluate the multi-
NL2Code
lingual capabilities of code generative models.
• HumanEval (Chen et al., 2021) is a bench- It contains 820 high-quality, human-crafted
mark dataset consisting of a collection of data samples, each accompanied by test cases.
Python programming problems, each accom- The dataset supports a variety of program-
panied by a function signature, a docstring ming languages, including Python, C++, Java,
outlining the problem to be solved, and a set of JavaScript, and Go. In this we experiment
unit tests that the correct implementation must with one of the above dimension of code-
pass. We use the evaluation metric pass@1, translation focusing on Java to Python. To
which checks if the the generated code passes accomplish this task, we combine the "decla-
the unit given unit tests in one attempt. We ration" and "canonical-solution" together to fi-
use all 164 samples in this dataset. nally get the overall function in the respective
language. "Declaration" contains the function
• FIND (Schwettmann et al., 2023): The Func-
declaration for the respective language and
tion Interpretation and Description (FIND)
"canonical solution" has the human-crafted
benchmark dataset is a natural language-to-
example solution for the language. Similar to
code generation task. The LLM is given 5
CODEXGLUE, we use the BLEU (Papineni
example inputs and outputs to an unknown
et al., 2002) score for measuring the perfor-
Python function and is tasked with reverse en-
mance.
gineering the original Python code. We evalu-
ate the benchmark by comparing the output of
test cases on a ground truth function with the C Prompt Templates
output from LLM generated functions. We In this section we provide examples of the four
use the "strings" category of functions for the prompt templates we used for the NER Finance
task consisting of 500 functions. We provide task. Prompts for all other tasks followed identi-
the LLM with 5 pairs of input and output for cal formatting. Variables that are injected into the
each function. To select these examples, we prompt are denoted by blue text wrapped in braces.
randomly sample from a dataset provided by For example a user ask being injected is denoted as
(Schwettmann et al., 2023) that contains exam- {USER ASK}.
ple test strings for each function. To evaluate
the generated function code, we use the string D Additional Research Questions
indicator metric introduced by (Schwettmann
et al., 2023) that measures the number of test D.1 How does the format in which
cases passed by the function. information is structured and presented
influence the ability to solve problems
Code2Code that require different skill sets?
• CODEXGLUE (Lu et al., 2021) stands for We analyze whether model’s sensitivity to prompt
General Language Understanding Evaluation format changes is related to the skills required to

8
 Prompt Format Prompt Template
{persona} {instructions} {examples}
Plain text
{output format instructions} {user ask}
## Persona
{persona}
## Instructions
{instructions}
## Examples
Markdown
{examples}
## Output Format
{Output format instructions}
## User Question
{user ask}
Persona
- {persona}
Instructions
- {instructions}
Examples
YAML
- {examples}
Output format
- {output format instructions}
User question
- {user ask}
{
“Persona”: “{persona}”,
“Instructions”: ”{instructions}“,
“Examples”: “{examples}”,
JSON “Output format”: “{output format instructions}"
}
{
“User ask”: “{user ask}”
}

Table 2: Prompt templates considered in this paper. Placeholders are denoted with {variable name} and get replaced with task
specific context.

Plaintext
System:
You are a annotator working for large financial data company and are tasked with extracting named entities from
financial documents who follows strict guidelines for quality and formatting. The following sentence is from a financial
document. List the named entities in the order they appear in the sentence. If an entity appears multiple times, list
it multiples times. Entities should be stated in the format NAME - TYPE where TYPE can be PERSON, ORGANIZATION, or
LOCATION. State your final answer as a comma-separated list of entities enclosed in square brackets. Example: [Bank -
ORGANIZATION, Borrower - PERSON]. If there are no entities found, state your final answer as ’No entities found’. Provide
your chain of thought first and then respond with your final answer. Here is an example: {ICL EXAMPLE INPUT} {ICL EXAMPLE SOLUTION}

User:
{INPUT}

solve the task using the MMLU benchmark which formance spread exists across different tasks, and
comprises 57 subjects, categorized into four do- it’s not signified nor eliminated by specific skills
mains: humanities, social science, STEM, and oth- required. This suggests that the model’s sensitiv-
ers. Each domain encompasses various disciplines, ity to prompt formatting is a general characteristic,
necessitating distinct skill sets and knowledge for rather than being contingent on the specific skills
accurate question answering. or reasoning abilities required by different tasks.
Figure 5 breaks down the performance on Performance is influenced by how information is
MMLU dataset by domain. We observe the per- presented to it, regardless of the complexity or na-

9
 Markdown
System:
## Persona
- You are a annotator working for large financial data company are tasked with extracting named entities from financial
documents who follows strict guidelines for quality and formatting.

## Instructions
- You will be given a sentence from a financial document. - List the named entities in the order they appear in the sentence.
- If an entity appears multiple times, list it multiples times.
- Provide your chain of thought first and then respond with your final answer.

## Output Format
- Entities should be stated in the format NAME - TYPE where TYPE can be PERSON, ORGANIZATION, or LOCATION.
- State your final answer as a comma-separated list of entities enclosed in square brackets. Example: [Bank - ORGANIZATION,
Borrower - PERSON].
- If there are no entities found, state your final answer as ’No entities found’.

## Example
### DOCUMENT
{ICL EXAMPLE INPUT}

### Solution
{ICL EXAMPLE SOLUTION}

User:
### DOCUMENT
{INPUT}

YAML
System:
Persona:
Description: You are a annotator working for large financial data company are tasked with extracting named entities from
financial documents who follows strict guidelines for quality and formatting.

Instructions:
- You will be given a sentence from a financial document.
- List the named entities in the order they appear in the sentence.
- If an entity appears multiple times, list it multiples times.
- Provide your chain of thought first and then respond with your final answer.

Output_Format:
Entities should be stated in the format NAME - TYPE where TYPE can be PERSON, ORGANIZATION, or LOCATION. State your final
answer as a comma-separated list of entities enclosed in square brackets.

Examples:
- Document: {ICL EXAMPLE INPUT}
- Solution: {ICL EXAMPLE SOLUTION}

User:
Task:
- Document: {INPUT}

ture of the task, the way in which a problem is mat and compare if the degree of dispersion can be
framed and communicated to the model can sig- attributed to the size of the model. We compute the
nificantly impact its ability to process and respond Coefficient of Mean Deviation (CMD) across all
to the information. Model performance is consis- the prompt templates for every model.
tently influenced by prompt formatting across P
|s(pi ) − s̄|
various tasks, regardless of the specific skills or CM D =
n · s̄
knowledge required.

Here s(pi ) is the performance score for prompt pi ,

D.2 Is there a correlation between the model
s̄ is the average score across all prompt formats,
size and the robustness of the LLM for
and n is the number of formats. A lower CMD indi-
different prompt templates?
cates that the model’s performance is less affected
Coefficient of Mean Deviation. We measure the by prompt format changes, showing greater robust-
performance dispersion for a model caused by for- ness. A higher CMD suggests more variability and

10
 JSON
System:
{
"Persona": "You are a annotator working for large financial data company are tasked with extracting named entities from
financial documents who follows strict guidelines for quality and formatting.",
"Instructions": [
"You will be given a sentence from a financial document.",
"List the named entities in the order they appear in the sentence.",
"If an entity appears multiple times, list it multiples times.",
"Provide your chain of thought first and then respond with your final answer."
],
"OutputFormat": "Entities should be stated in the format NAME - TYPE where TYPE can be PERSON, ORGANIZATION, or LOCATION.
State your final answer as a comma-separated list of entities enclosed in square brackets. Example: [Bank - ORGANIZATION,
Borrower - PERSON]. If there are no entities found, state your final answer as ’No entities found’.",
"Example": "{ICL EXAMPLE INPUT}\n{ICL EXAMPLE SOLUTION}"
}

User:
{
"Task": "{INPUT}"
}

(a) MMLU (b) HumanEval (c) CODEXGLUE (Java2CS)

Figure 4: Model performance across prompt formats on MMLU, HumanEval and CODEXGLUE. Performance
measurement for MMLU is accuracy, for HumanEval is pass@1 to hecks if the the generated code passes the unit
given unit tests in one attempt, for CODEXGLUE(Java2CS) is BLEU score. Plots for the remaining datasets are in
Figure 9.

Figure 5: Performance spread across models on MMLU benchmark per domain. Wide performance spread is
observed across domains that required different skills.

sensitivity to prompt format. of GPT-4 are not published, it is assumed that

While the architectural details and exact size GPT-4 contains significantly more parameters, was

11
 Table 4 for results. Analyzing the model outputs,
we find the poor performance is because most of
the time the model would generate chain of thought
in plain text, but did not continue with actually gen-
erating the code. The other models did not exhibit
this behavior for the JSON template. We hypothe-
size that this may be related to the OpenAI’s claim
about fixing laziness in task completion in the 0125
version of GPT-4-turbo (OpenAI, 2024). In sum-
mary, larger models are more robust to template
Figure 6: Coefficient of mean deviation (CMD) of scalar variation.
metrics for all the prompt templates. Figure shows
the CMDs across models and datasets. GPT-3.5 series E Complete Results
exhibit larger CMD scores across benchmarks than GPT-
4 series, indicating higher sensitivity to the choice of E.1 Additional results on model performance
format. under all templates across benchamrks.
E.2 IoU scores on all benchmarks.
trained on more data than GPT-3.5, and is clearly E.3 Dotplot on all benchmark datasets
the overall more capable model ((Achiam et al.,
2023; Bubeck et al., 2023; Carlini et al., 2024)). In
this section, we aim to ascertain whether an expan-
sion in general capability translates to enhanced
stability in response to changes in templates. The
CMDs for all the models across benchmarks are
presented in Figure 6.
A lower value of CMD indicates more robust-
ness to template variation. The results indicate that
the GPT-4-1106-preview model exhibits superior
robustness to format changes, maintaining a perfor-
mance dispersion consistently below 0.036 across
all benchmarks. In contrast, the GPT-4-32k-0613
model demonstrates less robustness relative to the
GPT-4-1106-preview, yet it outperforms the GPT-
3.5 series, with CMDs not exceeding 0.043. The
GPT-3.5 series displays a broader range of CMDs,
from 0.035 to 0.176, signifying a higher degree
of performance variability under different prompt
formats. GPT-4’s observed improvements may be
attributed to its enhanced ability to process data in
diverse formats. Moreover, it is possible that the ro-
bustness of the model is not adversely impacted by
format variations at the level of the last hidden layer
of prompt embedding. Notably, the GPT-4-1106-
preview model achieves greater robustness com-
pared to the GPT-4-32k-0613, corroborating exist-
ing evidence that suggests the former has a height-
ened proficiency in comprehending and generating
content in specific formats as instructed (OpenAI,
November 2023). Further examining GPT-4-32k-
0613’s performance, we notice the CMD on Hu-
manEval benchmark is extremely high, this is due
the extremely low score using JSON format, see

12
 Model GPT-35-turbo-0613 GPT-35-turbo-16k-0613 GPT-4-1106-preview GPT-4-32k-0613
Plaintext 54.464 ± 18.300 54.184 ± 19.066 81.005 ± 12.979 80.638 ± 13.172
Markdown 50.021 ± 17.144 50.686 ± 17.436 81.252 ± 12.932 81.349 ± 13.158
YAML 56.355 ± 16.792 55.901 ± 16.347 80.758 ± 13.000 81.162 ± 13.110
JSON 59.705 ± 16.594 59.405 ± 17.092 73.918 ± 13.580 77.800 ± 13.725

Table 3: Model Performance on MMLU Benchmark. Accuracy is averaged over 57 different subjects.

Model GPT-35-turbo-0613 GPT-35-turbo-16k-0613 GPT-4-1106-preview GPT-4-32k-0613

Plaintext 40.24 ± 3.98 37.20 ± 3.77 82.93 ± 4.39 76.22 ± 4.76
Markdown 54.27 ± 4.70 48.17 ± 4.44 86.59 ± 4.06 75.61 ± 4.78
YAML 42.68 ± 4.14 37.20 ± 3.77 85.37 ± 4.18 68.29 ± 4.92
JSON 59.76 ± 4.85 57.93 ± 4.81 86.59 ± 4.06 21.95 ± 2.48

Table 4: Model performance on HumanEval benchmark. We used all 164 samples for testing.

Model GPT-35-turbo-0613 GPT-35-turbo-16k-0613 GPT-4-1106-preview GPT-4-32k-0613

Plaintext 37.20 ± 6.59 36.80 ± 6.54 49.40 ± 7.86 47.20 ± 7.67
Markdown 28.00 ± 5.31 30.00 ± 5.61 51.40 ± 8.01 51.60 ± 8.03
YAML 24.60 ± 4.78 21.80 ± 4.31 53.80 ± 8.18 53.20 ± 8.14
JSON 28.40 ± 5.37 31.00 ± 5.76 50.80 ± 7.97 52.40 ± 8.08

Table 5: Model performance on NER finance benchmark. We randomly sampled 500 samples for testing.

Model gpt-35-turbo-16k-0613 gpt-35-turbo-0613 gpt-4-32k-0613 gpt-4-1106-preview

plaintext 15.75±0.142 15.9±0.143 21.87±0.164 20.08±0.161
markdown 5.03±0.092 5.19±0.089 17.42±0.154 20.68±0.165
json 14.46±0.138 14.33±0.144 21.15±0.162 20.19±0.156
yaml 13.06±0.139 13.49±0.138 21.6±0.163 20.28±0.16

Table 6: Model performance on FIND benchmark. Test set includes 500 functions.

Model GPT-35-turbo-0613 GPT-35-turbo-16k-0613 GPT-4-1106-preview GPT-4-32k-0613

Plaintext 62.95 ± 15.64 62.92 ± 15.66 64.95 ± 15.52 63.86 ± 15.83
Markdown 69.82 ± 16.26 69.84 ± 16.23 70.70 ± 17.44 71.65 ± 18.10
YAML 69.05 ± 17.24 69.05 ± 17.24 71.16 ± 16.11 71.41 ± 17.96
JSON 68.85 ± 16.13 68.85 ± 16.13 72.30 ± 15.97 72.39 ± 16.46

Table 7: Model performance on HumanEval-X, a Java to Python translation task The test set contains 164 data
samples.

Dataset CODEXGLUE: Java to CS CODEXGLUE - CS to Java

Model GPT-35-turbo-0613 GPT-35-turbo-16k-0613 GPT-4-1106-preview GPT-4-32k-0613 GPT-35-turbo-0613 GPT-35-turbo-16k-0613 GPT-4-1106-preview GPT-4-32k-0613
Plaintext 66.46 ± 16.04 66.46 ± 16.04 67.16 ± 16.77 68.19 ± 13.14 68.81 ± 17.65 68.89 ± 17.64 67.93 ± 17.72 68.11 ± 16.29
Markdown 78.10 ± 18.75 78.10 ± 18.75 74.16 ± 16.77 76.95 ± 18.33 76.19 ± 18.40 76.12 ± 18.37 74.80 ± 20.14 80.36 ± 20.52
YAML 78.28 ± 18.92 78.30 ± 18.92 70.75 ± 16.08 76.41 ± 18.00 75.47 ± 20.16 75.39 ± 20.08 72.09 ± 17.98 78.49 ± 21.23
JSON 78.37 ± 18.93 78.40 ± 18.93 74.16 ± 16.77 76.86 ± 18.31 77.49 ± 19.51 77.49 ± 19.50 75.00 ± 17.66 83.05 ± 18.60

Table 8: Model performance on Java to C# and C# to Java translation task. The test set contains 1000 samples in
Java and C#.

13
(a) MMLU (b) FIND

Figure 7: Heatmap of IoU values for other benchmarks.

14
 (a) GPT-35-Turbo-0613 (b) GPT-35-turbo-16k-0613

(c) GPT-4-1106-preview (d) GPT-4-32k-0613

Figure 8: Performance of Consistency for FIND dataset

across models.

15
 (a) MMLU (b) NER Finance

(c) CODEXGLUE (Java2CS) (d) CODEXGLUE (CS2Java)

(e) HumanEval-X (Java2Python) (f) HumanEval

(g) FIND

Figure 9: Dotplot of model performance across prompt formats on all benchmarks.

16