Scribd
Upload
3 views

AI Topics Summary

The document discusses key concepts in AI, including Active and Passive Reinforcement Learning, Information Extraction, and Early Image Processing Operations. It explains the differences between passive and active RL, methods of information extraction like Finite-State Automata and Probabilistic Models, and various early image processing techniques such as edge detection and image segmentation. Each section provides definitions, mathematical formulations, and examples to illustrate the concepts.

Uploaded by

gamingwithsameersyed
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
3 views

AI Topics Summary

The document discusses key concepts in AI, including Active and Passive Reinforcement Learning, Information Extraction, and Early Image Processing Operations. It explains the differences between passive and active RL, methods of information extraction like Finite-State Automata and Probabilistic Models, and various early image processing techniques such as edge detection and image segmentation. Each section provides definitions, mathematical formulations, and examples to illustrate the concepts.

Uploaded by

gamingwithsameersyed
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

AI Topics Summary

1. Active and Passive Reinforcement Learning


Concept of Reinforcement Learning (RL):
In RL, an agent learns to make decisions by receiving rewards or punishments for its
actions, aiming to maximize cumulative rewards. This setup can be imagined as a game
where the rules are unknown, and the agent learns by trial and error.

Passive Reinforcement Learning:


Definition: In passive RL, the agent follows a fixed policy π, meaning it executes a
predefined action for each state s. The main goal is to evaluate the utility of each state under
this fixed policy.
Learning Objective: The agent seeks to estimate a utility function U_π(s) that reflects the
cumulative expected rewards if the agent follows policy π.
Mathematics:
The utility of a state under policy π can be defined by a discounted sum of rewards:
U_π(s) = R(s) + γ Σ P(s' | s, π(s)) U_π(s')
Here, γ is a discount factor, R(s) is the immediate reward, and P(s' | s, π(s)) is the
probability of transitioning to state s' from s when action π(s) is taken.
Example: In a grid-world, a passive RL agent may learn how “good” it is to occupy specific
positions based on the rewards it encounters while following a pre-determined path.

Active Reinforcement Learning:


Definition: Unlike passive learning, active RL requires the agent to learn the optimal actions
to take in each state, developing a policy that maximizes cumulative rewards.
Exploration vs. Exploitation: A key challenge in active RL is balancing exploration (trying
new actions) and exploitation (sticking to actions with known rewards). This balance is
critical, as over-exploration or over-exploitation can hinder optimal learning.
Mathematics:
The Q-function, or action-utility function, Q(s, a), estimates the utility of taking action a in
state s:
Q(s, a) = R(s) + γ Σ P(s' | s, a) max_a' Q(s', a')
Example: In the grid-world scenario, an active RL agent might initially explore paths but will
eventually learn to select the shortest route to reach a goal with maximum reward.

2. Information Extraction
Concept: Information extraction (IE) is the task of identifying and structuring specific pieces
of information from unstructured text data, such as names, dates, and relationships
between entities. Common applications include extracting addresses from websites or
weather conditions from reports.
Methods in Information Extraction:

Finite-State Automata (FSA):


Definition: An FSA-based system extracts attributes using patterns or templates. Each
attribute is identified based on pre-defined rules, often using regular expressions.
Example: A pattern for price extraction may look for keywords like “price:” followed by a
currency symbol and a number.
Application: In web scraping, an FSA could extract product details, such as “$299” for prices
and “Model X” for product models.

Probabilistic Models:
Hidden Markov Models (HMM): Used for extracting sequences in noisy text data, such as
names or addresses. HMMs model a sequence of hidden states, allowing the system to
probabilistically “guess” the correct sequence based on observed data.
Conditional Random Fields (CRF): CRFs are advantageous for tasks that require contextual
awareness across multiple attributes, relaxing the independence assumptions in HMMs.
Example: In named entity recognition, HMMs might tag each word in a sentence as part of
an entity or not, while CRFs use the surrounding words to improve accuracy.

Applications of Information Extraction:


IE is widely used in natural language processing applications, from constructing ontologies
to building large knowledge bases, enabling automated understanding of vast corpora.
Techniques continue to evolve, leveraging machine learning models for higher accuracy in
more complex text structures.

3. Early Image Processing Operations


Early image processing operations serve as initial steps in analyzing data captured by
sensors. These operations are localized, focusing only on a few neighboring pixels at a time
without considering the broader context of objects within the scene. Such low-level
operations are highly suitable for implementation on parallel processing hardware like
GPUs. The main types include:

Edge Detection:
Concept: Detects points where image brightness changes significantly, corresponding to
potential object boundaries or depth changes.
Mathematics:
The brightness gradient across a cross-section perpendicular to an edge can be calculated
by differentiating the image intensity I(x). Locations where I'(x) (the derivative) is large
correspond to edges.
A Gaussian filter is applied to smooth the image, reducing noise. The Gaussian function is
defined as:
G(x) = 1/√(2πσ²) * exp(−x²/2σ²)
Applying the Gaussian filter at a pixel (x₀, y₀) smooths the image by computing a weighted
sum of neighboring pixel intensities:
I'(x₀, y₀) = Σ I(x, y) G(d)
Example: For an image of a desk with objects, edge detection identifies the object's outline
but may include noise edges. Later stages refine these detections.

Texture Analysis:
Concept: Texture is a visually repeating pattern, like stitches on fabric or spots on a leopard,
which can reveal surface characteristics. Some textures are periodic, while others, like
pebbles on a beach, follow only statistical regularity.
Application: Texture analysis is essential for detecting object boundaries in images with
intricate textures. For instance, the orientation histograms for a tiger’s fur and surrounding
grass differ, enabling the system to distinguish the animal from its background.

Optical Flow:
Concept: Optical flow describes the apparent motion of objects or the camera itself in a
video sequence. It measures pixel displacement over time, with motion recorded in terms of
pixels per second.
Mathematics:
To compute optical flow, a Sum of Squared Differences (SSD) method is often used. This
measures the similarity between pixel blocks centered at (x₀, y₀) at time t and candidate
pixels at (x₀ + Δx, y₀ + Δy) at time t + Δt:
SSD = Σ [I(x₀ + x, y₀ + y, t) - I(x₀ + x + Δx, y₀ + y + Δy, t + Δt)]²
Example: A moving race car in a video sequence shows high optical flow in the car and
slower flow in the background, useful for recognizing actions or estimating distances.

Image Segmentation:
Concept: Segmentation divides an image into regions of similar pixels, which helps in
identifying object boundaries by grouping regions based on brightness, color, or texture.
Example: Segmenting a landscape photo can separate regions of sky, water, and land.
Algorithms that create superpixels group pixels into homogeneous regions, simplifying
subsequent analysis.

You might also like