Create a resilient Spring Boot REST service that calls OpenAI models

If you want a Java REST API that actually talks to an OpenAI model and returns chat style replies without setting your hair on fire this guide will get you there. We will use Spring Boot and a lightweight HTTP client to keep things pragmatic and testable. Expect DTOs, a controller, a service that hides API details, and a few sanity checks so your key does not wander off into version control like a bad idea.

What you need before you start

• Java 11 or newer and Maven or Gradle
• Spring Boot web starter and a reactive client such as WebClient
• An OpenAI API key stored in the environment or a secret manager
• Basic unit test setup to mock the OpenAI calls

Project and dependencies

Create a standard Spring Boot project and add spring-boot-starter-web. Add spring-boot-starter-webflux if you plan to use WebClient which is great for non blocking calls and timeouts. If you prefer an official or community OpenAI SDK you can add that but it is fine to use plain WebClient so you know exactly what is being sent and received.

Why WebClient

WebClient gives you fine grained control over timeouts, retries and backoff without turning your service into a spaghetti stack. It plays nicely with Reactor for async flows and keeps the OpenAI calls isolated so tests can stub them easily.

Configure your API key and properties

Store the OpenAI API key in environment variables or a vault. Reference it from application properties for base URL and timeout values. Do not commit the key to git unless you need a new career in security incident response.

Keep configuration minimal and explicit. Example settings you will want include base URL, request timeout and a polite retry policy to handle rate limits. Use sensible defaults and let the environment override values at runtime.

Design DTOs for requests and responses

Make a ChatRequest with fields such as model and messages where messages is a list of role and content pairs. Make a ChatResponse that includes the model reply text and a status or error block for the client to check. Keep shapes explicit so client code does not guess the schema.

Implement the controller

Create a REST controller that accepts a JSON chat request and returns a JSON chat response. Keep controller methods small. They should validate input, call the OpenAiService and map the service output to your response DTO. Lean controllers are easier to test and easier to explain at stand ups.

Implement the OpenAiService

Put all API interaction logic behind a service named OpenAiService. Map your ChatRequest into the model payload that OpenAI expects and parse the model output into your ChatResponse DTO. Handle HTTP status codes and surface meaningful errors to the controller rather than letting low level exceptions bubble up to clients.

• Use WebClient for HTTP calls so you can configure timeouts and backoff
• Retry on transient failures with an exponential backoff and a max attempts limit
• Respect rate limit headers and fail gracefully when limits are hit

Testing locally and in CI

Run the app and POST simple prompts to the /chat endpoint. For unit tests mock the OpenAiService so you are not charged for every CI run. For one or two integration tests use a sandbox key or a carefully controlled test key to validate the end to end flow. Keep integration tests isolated and rate friendly.

Error handling and resilience

Wrap API calls with retry logic and circuit breaking where appropriate. Validate inputs carefully so the external API does not reject obvious mistakes. Return clear error objects from your API so front end and mobile clients can show helpful messages instead of a cryptic server stack trace.

Practical tips

• Log request identifiers so you can trace a prompt from client to model and back
• Throttle or queue heavy workloads to avoid surprise bills
• Keep request and response DTOs versioned if clients will be long lived

Security and best practices

Never bake API keys into your source. Prefer an environment secret or a dedicated secret manager. Limit the permissions of the key and rotate keys regularly. If you expose a public endpoint add rate limiting and an API key for your clients so one user does not burn your monthly quota.

Wrap up

In short here is the architecture to remember. Use Spring Boot for quick endpoints. Keep WebClient or a small SDK in a service to talk to OpenAI. Define clear DTOs. Add retries and rate limit handling. Test locally with mocks and run a small set of integration tests with a controlled key. You will end up with a maintainable Java REST API that can chat with OpenAI models and not make your weekends miserable.

If you want, I can sketch minimal class outlines and a sample WebClient call in plain pseudocode without any sensitive details. Say the word and I will draw the wiring diagram without the circus act.