The Generative AI

GLOSSARY
Discover the key technical terms associated with generative
AI and their meaning
A
Terms Description

Function used in neural networks to introduce non-linearity, such as ReLU,

Activation Function
sigmoid, and Tanh.

A software program that can interact with its environment, collect data, and
Agents
use the data to perform self-determined tasks to meet predetermined goals.

The branch of ethics that examines the moral implications and

AI Ethics
responsibilities associated with the creation and use of artificial intelligence.

Refer to Application Programme Interfaces that enable the flow of

API information between applications. An API would allow an AI chatbot to
connect to a LLM by a third-party service provider.

A technique used in machine learning and artificial intelligence to improve the

Attention
performance of models by focusing on relevant information.

A technique used to break the document into multiple chunks of text and
Auto Merging Retriever
further breaks the "parent" chunks into smaller "child" chunks.

B
Terms Description

An algorithm used to train neural networks by adjusting weights based on the

Backpropagation
error rate obtained in the previous epoch.

A process where a model learns to reverse the noise added during the
Backward Diffusion
forward diffusion process, reconstructing data from noise.

A conversational AI model developed by Google for generating human-like

BARD
text

BERT or Bidirectional Encoder Representations from Transformers, is based

on transformers, a deep learning model in which every output element is
BERT
connected to every input element, and the weightings between them are
dynamically calculated based upon their connection.
 Terms Description

Systematic error introduced into data or algorithms that leads to inaccurate

Bias
or unfair outcomes.

C
Terms Description

A prompt engineering technique that focuses on increasing the informational

Chain of Density
density of the generated text.

A method that structures prompts to utilize dictionary-like definitions for

Chain of Dictionary
precise and context-rich responses.

A technique that structures prompts to elicit emotionally resonant and

Chain of Emotion
contextually appropriate responses.

A prompt strategy that guides the model to provide detailed explanations or

Chain of Explanation
reasoning for a given topic or question.

A method that leverages structured knowledge prompts to elicit responses

Chain of Knowledge
based on specific facts and information.

A method that structures prompts to engage the model in logical and

Chain of Numerical Reasoning
mathematical reasoning for solving numerical problems.

A prompt engineering strategy that involves a series of related questions to

Chain of Question
guide the model through a logical sequence of responses.

A technique that employs symbolic representations within prompts to

Chain of symbol
generate structured and symbolically meaningful outputs.

A technique where a model reasons step-by-step through a problem,

breaking it down into smaller, more manageable parts. This method allows
Chain of Thought
the model to simulate human-like thinking patterns and improve its problem-
solving abilities.
D
Terms Description

Data Parallelism is a way of performing parallel execution of an application on

Data Parallelism multiple processors. focuising on distributing data across different nodes in
the parallel execution environment.

A DDPG is a reinforcement learning agent that searches for an optimal policy

DDPG
that maximizes the expected cumalative long-term reward.

Stands for Denoising Diffusion Probablistic Model, learns to generate data by

reversing a gradual diffusion process. Noise is added to the data in small
DDPM
increments until it becomes pure noise, and the model learns to denoise this
data step-by-step, ultimately generating realistic data samples.

A component of sequence-to-sequence models that processes input

Decoder
sequences into a fixed representation

Deep learning is a method in artificial intelligence (AI) that teaches computers

Deep Learning
to process data in a way that is inspired by the human brain.

An optimization library for deep learning applications, enhancing training

Deep Speed
efficiency and speed.

Denoising Autoencoder ( DAE A type of autoencoder used to remove noise from data, often utilized in the
) backward diffusion process.

A Diffusion model are generative models, meaning that they are used to
Diffusion Model
generate data similar to the data on which they are trained.

E
Terms Description

The layer in a neural network that converts categorical data, such as words,
Embedding Layer
into continuous vector representations.
 Terms Description

A component of sequence-to-sequence models that processes input

Encoder
sequences into a fixed representation.

A full pass through the entire training dataset during the training process of a
Epoch
neural network.

F
Terms Description

An algorithm used to train neural networks by adjusting weights based on the

Faithfulness
error rate obtained in the previous epoch.

A tool used to scan reports to provide AI-powered analysis of the business

Falcon AI
listings.

A technique in NLP where a model is given a few examples of a task within

Few Shot Prompting
the input prompt to guide its responses.

The process of taking a pre-trained model and making small adjustments to it

Fine-tuning
on a new, specific task or dataset to improve performance.

A process where noise is gradually added to data over a series of steps,

Forward Diffusion
transforming it into a noise distribution.

A form of generative artificial intelligence (generative AI). They generate

Foundation Model output from one or more inputs (prompts) in the form of human language
instructions.

A technique in distributed training of deep learning models where both the

Fully sharded data parallelism model's parameters and optimizer states are sharded (divided) across
multiple devices or nodes.
G
Terms Description

GANs, a generative adversarial network is a class of machine learning

GANs framework and a prominent framework for approaching generative AI. It is a
framework in which two neural networks compete to generate realistic data.

A type of statistical noise with a probability density function equal to that of

Gaussian Noise the normal distribution, commonly added during the forward diffusion
process.

A project or technology by Google DeepMind that aims to combine advanced

Gemini techniques in large language models and reinforcement learning, potentially
enhancing the capabilities and applications of AI systems.

A subset of artificial intelligence focused on generating new data that is

similar to existing data. This includes generating text, images, music, and
more, using models like Generative Adversarial Networks (GANs), Variational
Generative AI
Autoencoders (VAEs), and Transformer-based models such as GPT-3 and
BERT. Generative AI is widely used in applications like chatbots, content
creation, drug discovery, and creative arts.

Stands for Guided Language to Image Diffusion for Generation and Editing, a
GLIDE generative model developed by Open AI that uses diffusion processes to
generate and edit images based on textual descriptions.

Generative Pre-trained Transformer, a type of LLM developed by Open AI,

GPT
known for generating coherent and contextually relevant text.

An optimization algorithm used to minimize the cost function by iteratively

Gradient Descent
adjusting model parameters.

H
Terms Description

The state in a recurrent neural network (RNN) or transformer that contains

Hidden state
information about the input sequence seen so far.
 Terms Description

A metric used to measure the accuracy of a predictive model or

Hit rate recommendation system, defined as the ratio of the number of relevant items
successfully retrieved to the total number of relevant items.

Hugging Face is a machine learning (ML) and data science platform and
Hugging Face
community that helps users build, deploy and train machine learning models.

An advanced information retrieval technique that combines multiple retrieval

Hybrid Fusion Retriever methods, such as dense and sparse retrieval, to improve the accuracy and
relevance of search results.

Settings or configurations used to control the training process of a machine

Hyperparameter
learning model, such as learning rate or batch size.

I
Terms Description

Stands for Intel® Extension for PyTorch*, which is a library that optimizes
IPEX
PyTorch performance on Intel hardware, including CPUs and GPUs.

Image Recognisition Identifies which object or scene is in an image.

The process of organizing data to enable efficient retrieval of information. In

the context of databases and search engines, indexing involves creating a
Indexing
data structure, that allows for quick searches and access to relevant relevant
records or documents.

J
Terms Description

In models like BERT, the ability to attend to the left and right context of a
Joint attention
word simultaneously.
L
Terms Description

LangChain provides AI developers with tools to connect language models

Langchain with external data sources. It is open-source and supported by an active
community.

Refers to the original or earlier version of the syntax used in the LangChain
LangChain Legacy Syntax framework, which is designed for building applications with large language
models (LLMs).

A framework or tool designed to structure and visualize relationships and

dependencies between different language models and their components.
LangGraph
LangGraph helps in understanding and managing complex interactions within
multi-model systems

A service or platform designed to deploy and manage large language models

(LLMs) in production environments.LangServe provides infrastructure and
LangServe
tools to serve LLMs efficiently, ensuring scalability, reliability, and ease of
integration

A tool or platform designed to enhance the development and deployment of

applications using large language models (LLMs). LangSmith provides
LangSmith
features for optimizing, fine-tuning, and integrating LLMs into various
applications,

A model that assigns probabilities to sequnece of words and can generate

Large Language model
text based on learned patterns

A generative modeling technique where the diffusion process is applied in a

latent space rather than directly on the data. This approach involves
Latent Diffusion
encoding data into a lower dimension latent space to generate new data
points.

A lower-dimensional representation of data where similar data points are

Latent Space
closer together, often used in generative models.

A process or technique used to adapt pre-trained language models to

LIMA specific tasks or domains by fine-tuning them on a smaller, task-specific
dataset.

LlamaIndex is a simple, flexible data framework for connecting custom data

LlamaIndex
sources to large language models (LLMs).
 Terms Description

Stands for Large Language Model Operations and refers to the specialized
LLMops methods and processes meant to accelerate model creation, deployment,
and administration over its entire lifespan.

Low-Rank Adaption is a techniquew desinged to refine and optimize large

LoRA language model. It focuses on adapting only specific parts of the neural
network.

Stands for Long Short-Term Memory, a type of recurrent neural network

LSTMs (RNN) architecture designed to effectively capture long-term dependencies
in sequential data.

M
Terms Description

A stochastic process where the probability of each event depends only on

Markov Chain the state attained in the previous event, often used in modeling the steps of
diffusion proces

An AI tool used for generating images based on text descriptions. It

Midjourney leverages advanced neural networks and deep learning techniques to create
high-quality, photorealistic images from natural language prompts

A set of practices that aim to deploy and maintain machine learning models in
production reliably and efficiently. MLOps combines aspects of machine
MLops
learning (ML), data engineering, and DevOps to streamline the model
lifecycle, from development and training to deployment

The structure and design of a machine learning model, including the

Model Architecture arrangement and interactions of its components such as layers, nodes, and
connections.

A technique in deep learning where different parts of a model are distributed

Model Parallelism across multiple devices (such as GPUs or CPUs) to parallelize the
computation.
 Terms Description

Stands for Mean Reciporal Rank, a metric used to evaluate the effectiveness
of a search or recommendation system. It calculates the average of the
MRR
reciprocal ranks of the first relevant result for a set of queries, providing a
measure of how quickly the system retrieves relevant information.

AI systems or tools designed to handle, analyze, and generate responses

based on information from multiple documents. These agents integrate data
Multi-document agents
from various sources to provide comprehensive answers, insights, or
summaries,

N
Terms Description

A field of artificial intelligence that focuses on the interaction between

computers and humans through natural language. NLP involves the
NLP
development of algorithms and models to understand, interpret, and
generate human language

Empowers non-technical users to rapidally develop and deploy AI solutions

No-code AI
without extensive coding knowledge.

A predefined sequence of noise levels applied to data during the forward

Noise Schedule
diffusion process.

O
Terms Description

Shows the model one clear, descriptive example of what you'd like it to
One Shot Prompting imitate. When this prompt is run, the model's response will be to classify 'It
doesn't work' as positive or negative

Output parsers are responsible for taking the output of an LLM and
Output Parsers transforming it to a more suitable format. This is very useful when you are
using LLMs to generate any form of structured data.
P
Terms Description

In the data parallel paradigm, there are many different data and the same
operations (instructions in assembly language) are performed on these data
Parallel paradigm
at the same time. Parallelism is achieved by how many different data a single
operation can act on.

Stands for Parameter-efficient fine-tuning, used in a scenario where

PEFT computational resources are limited or where large pre-trained models are
involved.

Pipeline parallelism extends on simple task parallelism, breaking the task into
Pipeline Parallelism a sequence of processing stages. Each stage takes the result from the
previous stage as input, with results being passed downstream immediately.

Positional encoding is used to provide positional information to the model. In

Positional Encoding detail, a position-dependent signal is added to each word embedding for
each input sequence to help the model incorporate the order of words.

Prompt engineering is the practice of designing inputs for AI tools that will
Prompt Engineering produce optimal outputs. It involves experimenting with different prompts to
guide the model and achieve desired responses or outputs.

PySpark is the Python API for Apache Spark, an open source, distributed
Pyspark computing framework and set of libraries for real-time, large-scale data
processing.

Q
Terms Description

An efficient finetuning approach that reduces memory usage enough to

finetune a 65B parameter model on a single 48GB GPU while preserving full
QLoRA 16-bit finetuning task performance. QLoRA backpropagates gradients
through a frozen, 4-bit quantized pretrained language model into Low Rank
Adapters (LoRA).

Improves performance by reducing memory bandwith requirement and

increase cache utilization. With an LLM model, quantization process at
Quantization
different precision levels enables a model to be run on wider range of
devices.
 Terms Description

Query interface is a type-safe way to achieve a safe downcasting and to

Query Interface
allow interfaces to be aggregated to an object.

R
Terms Description

Stands for Retrieval Augmented Generation, is an architectural approach

RAG that can improve the efficacy of large language model (LLM) applications by
leveraging custom data.

A measure of the difference between the original data and the reconstructed
Reconstruction Loss
data, often used to train denoising models.

It helps in identifying relationships between document chunks as well as

Recursive Retriever
recursilvely retrieve related document chunks.

It is a machine learning (ML) technique that trains software to make decisions

Reinforcement Learning to achieve the most optimal results. It mimics the trial and error learning
process that humans use to achieve their goals.

Relevance AI is the home of the AI Workforce. Build and deploy AI Agents

and Tools to your workforce without code. With advanced customization,
Relevance AI magical deployment and multi-provider support, Relevance AI makes it easy
to integrate large language models into your workflows to create powerful
automations.

Responsible AI is an approach to developing and deploying artificial

Responsible AI intelligence (AI) from both an ethical and legal point of view. The goal of
responsible AI is to employ AI in a safe, trustworthy and ethical fashion.

Basic components of the majority of search systems. They're used in the

retrieval part of the retrieval-augmented generation (RAG) pipelines, they're
Retrievers
at the core of document retrieval pipelines, and they're paired up with a
Reader in extractive question answering pipelines.

RLHF is a specific technique that is used in training AI systems to appear

RLHF more human, alongside other techniques such as supervised and
unsupervised learning.
 Terms Description

RunPod is a cloud computing platform designed for AI, machine learning

Runpod
applications, and general compute.

S
Terms Description

The process of generating new data points from a learned distribution in

generative models. It is primarily used to generate detailed images
Sampling conditioned on text descriptions, though it can also be applied to other tasks
such as inpainting, outpainting, and generating image-to-image translations
guided by a text prompt.

It is an approach that simply asks a model the same prompt multiple times
Self-consistency Prompting
and takes the majority result as the final answer.

It separates the embedding and sythesis processes, allowing for more

granular and targeted information retrieval. Instead of embedding and
Sentence window Retriever
retrieving entire text chunks, this method focuses on individual sentences or
smaller units of text.

Spark's macine learning library is MLlib. Its goal is to make practical machine
learning scalable and easy. At a high level, it provides tools such as: ML
Spark ML
Algorithms: common learning algorithms such as classification, regression,
clustering, and collaborative filtering.

Stable Diffusion is a deep learning, text-to-image model released in 2022

Stable Diffusion
based on diffusion techniques.

Streamlit is an open-source Python framework for data scientists and AI/ML

Streamlit
engineers to deliver dynamic data apps with only a few lines of code.
T
Terms Description

A technique in distributed computing that splits the computation of large

neural network models across multiple devices by dividing the tensors (multi-
Tensor Parallelism
dimensional arrays) involved in the computations, enhancing training
efficiency and scalability.

A technology that converts textual descriptions into three-dimensional

Text to 3D models, leveraging natural language processing and computer graphics
techniques to create detailed 3D representations based on text input.

The process of breaking down text into smaller units called tokens, which
Tokenization can be words, subwords, or characters, to enable easier processing and
analysis by machine learning models in natural language processing tasks.

A type of deep learning model architecture that uses self-attention

Transforners mechanisms to process and generate sequences of data, such as text,
enabling advanced natural language processing tasks.

A hierarchical structure used in artificial intelligence to represent multiple

Tree of Thought
possible outcomes or pathways of a decision-making process.

U
Terms Description

A modeling error that occurs when a model is too simple to capture the
Underfitting
underlying patterns in the data.

A convolutional neural network architecture designed primarily for biomedical

image segmentation. It features an encoder-decoder structure with
UNet
symmetric skip connections, allowing for precise localization and context
utilization in segmentation tasks.

Unsupervised Learning A type of machine learning where the model is trained on data without labels.
V
Terms Description

Variational Autoencoders (VAEs) are generative models explicitly designed to

Variational Autoencoder
capture the underlying probability distribution of a given dataset and
(VAE )
generate novel samples.

A method of approximating complex probability distributions through

Variational Inference
optimization, often used in training generative models.

A vector database, vector store or vector search engine is a database that

Vector Database
can store vectors along with other data items.

A technique used in the development and fine-tuning of AI models,

particularly in natural language processing, where initial prompts or inputs
Verify and Edit Prompting are evaluated and modified to ensure they produce the desired output. This
iterative process helps improve the accuracy and relevance of the generated
responses.

W
Terms Description

Weight A parameter within a neural network that is adjusted during training

A learned representation for text where words that have the same meaning
Word Embedding
have a similar representation.

Z
Terms Description

A technique in natural language processing where a model is given a task it

has not been explicitly trained on, using a carefully designed prompt to guide
Zero-shot Prompting
the model in generating the correct output. This allows the model to perform
tasks without needing task-specific training data.