Integrating Chat Models with Spring AI

Integrating chat models with Spring AI is an important step in enhancing modern applications with advanced AI capabilities. By combining Spring Boot with OpenAI's ChatGPT APIs, developers can integrate powerful natural language processing and machine learning features into their Java applications. This integration enables the creation of interactive AI chatbots and intelligent customer support systems, making use of RESTful APIs for seamless communication. In this article, we will walk through the setup and configuration of a Spring Boot application that utilizes Spring AI to utilize the full potential of generative AI models.

We'll cover the complete setup, configuration, and implementation of a Spring Boot application that communicates with a chat model. By the end of this article, you'll be equipped with the knowledge to create a fully functional application capable of utilizing AI for conversational tasks.

Chat Models

Chat models are AI-driven systems designed to understand and generate human language. They use large-scale deep-learning models trained on vast amounts of text data. Popular examples include:

• GPT (Generative Pre-trained Transformer): Developed by OpenAI, this model generates human-like text based on input.
• BERT (Bidirectional Encoder Representations from Transformers): Developed by Google, this model excels in understanding the context of words in a sentence.

These models can be used for various tasks such as text generation, sentiment analysis, and translation, enhancing user interaction with personalized and intelligent responses.

ChatGPT

ChatGPT is a generative AI model that produces human-like text based on user prompts. It utilizes models such as GPT-3.5 and GPT-4 to generate responses.

Prompts in ChatGPT

Prompts are instructions or queries entered into the AI’s interface to receive responses. A prompt consists of keywords and phrases designed to elicit a reply. For example, asking "What is Spring Boot?" will prompt ChatGPT to provide an explanation.

What is Spring AI?

Spring AI is a module within the Spring ecosystem that simplifies integrating AI and machine learning capabilities into Java-based applications. It provides tools for connecting to AI models, handling data processing, and managing model inferences. Spring AI supports various machine learning frameworks and models, making it ideal for developers looking to incorporate AI into their applications.

Pre-requisites for Implementation

Before starting, ensure you have:

• Java Development Kit (JDK): JDK 11 or higher.
• Maven or Gradle: For dependency management and building the project.
• Spring Boot Setup: Basic knowledge of Spring Boot and its configuration.
• Understanding of RESTful APIs: Familiarity with APIs in web applications.
• API Key for Chat Model: Obtain from a provider like OpenAI.
• Basic Understanding of AI/ML Concepts: Beneficial but not mandatory.

OpenAI ChatGPT API

Create Chat Completion API is used to generate responses based on a given prompt.

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {
        "role": "user",
        "content": "What is Spring Boot?"
      }
    ],
    "temperature": 1,
    "max_tokens": 256
  }'

Request Parameters

• Model: Specifies the model version, e.g., "gpt-3.5-turbo".
• Messages: Contains the conversation context, with "role" and "content".
• Temperature: Controls response randomness; range from 0 to 2.
• Max_tokens: Limits the number of tokens in the response.

If you attempt the request mentioned earlier in Postman without passing the OpenAI API Key as a bearer token, you will encounter an authentication failure. It is important to include the OpenAI API Key as a bearer token to authenticate access to the OpenAI ChatGPT completion API.

Create OpenAI Key

Log in to your ChatGPT account and create your OpenAI API key.

Creating a New Spring Boot Project

1. Set Up Project: Use Spring Initializr to create a new Spring Boot project.
   • Group: com.example
   • Artifact: chat-integration
   • Dependencies: Spring Web, Spring Boot DevTools, Spring AI
2. Download and Unzip: Extract the project and open it in your IDE (e.g., IntelliJ IDEA, Eclipse).

OpenAI ChatGPT API Integration with Spring Boot

Project Structure

After successfully creating the project, the folder structure will be like below:

Configuring the Application

Open the pom.xml (for Maven) or build.gradle (for Gradle) file and ensure the following dependencies are included.

<dependencies>
    <!-- Spring Boot Starter Web for RESTful services -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <!-- Spring AI Starter for integrating AI models -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-ai</artifactId>
    </dependency>
    <!-- Jackson for JSON processing -->
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
    </dependency>
</dependencies>

Setting Up Configuration Properties

We need to configure application to interact with the chat model API. Add the following properties to the application.properties or application.yml file:

# API key for authentication with the chat model provider
spring.ai.api.key=your-api-key-here

# Endpoint URL for the chat model's API
spring.ai.endpoint=https://api.openai.com/v1/chat/completions

• spring.ai.api.key: This is where you'll place your API key from the chat model provider.
• spring.ai.endpoint: The endpoint URL for the chat model's API.

Implementing the Chat Model Integration

Creating a Service for Chat Integration:

Create a service class that handles communication with the chat model. This class will send user input to the model and process the response.

import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.*;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.core.JsonProcessingException;

import java.util.HashMap;
import java.util.Map;

@Service
public class ChatService {

    @Value("${spring.ai.api.key}")
    private String apiKey;  // API key for authenticating requests

    @Value("${spring.ai.endpoint}")
    private String apiEndpoint;  // Endpoint URL for the chat model API

    private final RestTemplate restTemplate;

    public ChatService(RestTemplateBuilder builder) {
        this.restTemplate = builder.build();  // Initialize RestTemplate
    }

    // Method to get a response from the chat model
    public String getChatResponse(String userInput) {
        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_JSON);  // Set content type to JSON
        headers.set("Authorization", "Bearer " + apiKey);  // Set API key in the Authorization header

        // Creating request body
        Map<String, Object> requestBody = new HashMap<>();
        requestBody.put("model", "gpt-3.5-turbo");  // Specify the model
        requestBody.put("messages", Collections.singletonList(
            Map.of("role", "user", "content", userInput)  // Set user input as message
        ));
        requestBody.put("temperature", 1);  // Control response randomness
        requestBody.put("max_tokens", 150);  // Limit the response length

        HttpEntity<Map<String, Object>> request = new HttpEntity<>(requestBody, headers);  // Create HTTP entity with request body and headers
        ResponseEntity<String> response = restTemplate.postForEntity(apiEndpoint, request, String.class);  // Send POST request to the API

        // Process and return the response
        return parseResponse(response.getBody());  // Extract and return response text
    }

    // Parse the response JSON to extract text
    private String parseResponse(String responseBody) {
        ObjectMapper mapper = new ObjectMapper();
        try {
            JsonNode root = mapper.readTree(responseBody);  // Parse JSON response
            return root.path("choices").get(0).path("text").asText();  // Extract response text
        } catch (JsonProcessingException e) {
            e.printStackTrace();
            return "Error processing response";  // Return error message if parsing fails
        }
    }
}

• RestTemplate: This is used to make HTTP requests to the chat model's API.
• Headers: The Authorization header contains your API key, required for authenticating with the chat model.
• Request Body: The body contains the user input and other parameters like max_tokens, which controls the length of the response.

Configuration for RestTemplate

import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.HttpHeaders;
import org.springframework.web.client.RestTemplate;

@Configuration
public class ChatGPTConfiguration {

    @Value("${spring.ai.api.key}")
    private String apiKey;  // API key for authentication

    // Bean configuration for RestTemplate
    @Bean
    public RestTemplate restTemplate() {
        RestTemplate restTemplate = new RestTemplate();
        restTemplate.getInterceptors().add((request, body, execution) -> {
            request.getHeaders().add("Authorization", "Bearer " + apiKey);  // Add API key to request headers
            return execution.execute(request, body);  // Proceed with the request
        });
        return restTemplate;  // Return configured RestTemplate
    }
}

This class configures RestTemplate with an interceptor that adds the API key to the request headers, ensuring secure communication with the chat model.

Creating a REST Controller

Create a REST controller to handle chat requests.

@RestController
@RequestMapping("/api/v1")
@Slf4j
public class ChatAPIController {

    @Autowired
    private RestTemplate restTemplate;

    @Value("${spring.ai.api.key}")
    private String apiKey;

    @Value("${spring.ai.endpoint}")
    private String apiUrl;

    @PostMapping("/chat")
    public String chat(@RequestParam("prompt") String prompt) {
        String response = null;
        try {
            Map<String, Object> requestBody = new HashMap<>();
            requestBody.put("model", "gpt-3.5-turbo");
            requestBody.put("messages", Collections.singletonList(
                Map.of("role", "user", "content", prompt)
            ));
            requestBody.put("temperature", 1);
            requestBody.put("max_tokens", 150);

            HttpHeaders headers = new HttpHeaders();
            headers.setContentType(MediaType.APPLICATION_JSON);
            headers.set("Authorization", "Bearer " + apiKey);

            HttpEntity<Map<String, Object>> request = new HttpEntity<>(requestBody, headers);
            ResponseEntity<String> chatResponse = restTemplate.postForEntity(apiUrl, request, String.class);

            response = new ObjectMapper().readTree(chatResponse.getBody())
                    .path("choices").get(0).path("text").asText();
        } catch (Exception e) {
            log.error("Error: " + e.getMessage());
        }
        return response;
    }
}

This class defines a REST endpoint /api/v1/chat to accept POST requests with user input and return the response from the chat model. The ChatService is injected to handle the communication with the chat model.

Output in Postman

The application should return a response from the chat model, displayed in the Postman.

• choices: Contains an array of responses from the AI model. Each item in the array has a text field that includes the generated response.
• content: The actual output text from the AI model, which answers the provided prompt.

Integrating Generative AI into Spring Boot Application with Spring AI

Features of Spring AI

• Support for Major AI Providers: Integrates with OpenAI, Microsoft, Amazon, Google, and Huggingface.
• Various Model Types: Supports chat and text-to-image models.
• Universal API: Provides a common API across different AI providers.
• Vector Database Support: Works with multiple vector database providers.
• Auto Configuration: Spring Boot auto-configuration and starters for AI models.

Note:

Spring-AI is very new and currently lacks a stable version. However, we anticipate releasing a stable version in the future. Spring AI introduces the AiClient interface with implementations for OpenAI and Azure’s OpenAI.

Conclusion

This article demonstrated how to integrate chat models with Spring AI in a Spring Boot application. We covered project setup, configuration, and implementation, enabling the creation of AI-powered conversational applications. By following these steps, developers can harness the power of AI for enhanced user interactions.

batraharshita12

Follow

Improve

Article Tags :

• Advance Java
• Spring AI