CODE WITH SIBIN

Solving Real Problems with Real Code


Integrate Azure Event Hubs with Spring Boot: Producer & Consumer Guide
admin
April 14, 2025
Azure Event Hubs java Spring Boot

This comprehensive guide covers both Azure AD authentication and connection string approaches for integrating Azure Event Hubs with Spring Boot, including producers, consumers, and all required Azure configurations.

Table of Contents

  1. Prerequisites
  2. Azure Resource Setup
  3. Finding All Required Properties
  4. Spring Boot Project Setup
  5. Configuration Options
  6. Implementing Producer
  7. Implementing Consumer
  8. Testing & Monitoring
  9. Best Practices
  10. Troubleshooting

1. Prerequisites

  • Azure subscription
  • Java JDK 11+
  • Maven/Gradle
  • Spring Boot 2.7+
  • IDE (IntelliJ, VS Code, etc.)

2. Azure Resource Setup

A. Create Event Hubs Namespace & Hub

  1. In Azure Portal: Create Resource → Event Hubs
  2. Configure:
    • Namespace Name: your-eventhub-ns
    • Tier: Standard
    • Throughput Units: 1 (adjust for production)
  3. Create Event Hub:
    • Name: your-event-hub
    • Partitions: 4
    • Retention: 1 day

B. Create Storage Account (for Checkpointing)

  1. Create Resource → Storage Account
  2. Configure:
    • Name: yourstorageaccount
    • Performance: Standard
    • Redundancy: LRS
  3. Create Container:
    • Name: eventhub-checkpoints

C. Get Connection Strings (Alternative to Azure AD)

  1. Event Hubs Connection String:
    • Go to Event Hubs Namespace → Shared Access Policies
    • Click on policy (e.g., RootManageSharedAccessKey)
    • Copy Connection string-primary key
  2. Storage Account Connection String:
    • Go to Storage Account → Access Keys
    • Copy Connection string

3. Finding All Required Properties 

For Azure AD Authentication

PropertyWhere to Find
Tenant IDAzure AD → Properties → Tenant ID
Client IDAzure AD → App Registrations → Application ID
Client SecretAzure AD → App Registrations → Certificates & Secrets
Subscription IDSubscriptions → Overview
Resource GroupResource Groups → Name

For Connection String Approach

PropertyWhere to Find
Event Hubs Connection StringEvent Hubs Namespace → Shared Access Policies
Storage Connection StringStorage Account → Access Keys
Event Hub NameEvent Hubs Namespace → Event Hubs
Container NameStorage Account → Containers

4. Spring Boot Project Setup

A. Add Dependencies (pom.xml)

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.azure.spring</groupId>
            <artifactId>spring-cloud-azure-dependencies</artifactId>
            <version>5.21.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <dependency>
        <groupId>com.azure.spring</groupId>
        <artifactId>spring-cloud-azure-starter-eventhubs</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
</dependencies>

5. Configuration Options

Option 1: Azure AD Authentication (Recommended for Production)

# Azure AD Auth
spring.cloud.azure.credential.client-id=xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
spring.cloud.azure.credential.client-secret=your-client-secret
spring.cloud.azure.credential.tenant-id=xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
spring.cloud.azure.profile.subscription-id=your-subscription-id
spring.cloud.azure.profile.resource-group=your-resource-group

# Event Hubs Config
spring.cloud.azure.eventhubs.namespace=your-eventhub-ns
spring.cloud.azure.eventhubs.event-hub-name=your-event-hub

# Checkpointing
spring.cloud.azure.eventhubs.processor.checkpoint-store.account-name=yourstorageaccount
spring.cloud.azure.eventhubs.processor.checkpoint-store.container-name=eventhub-checkpoints
spring.cloud.azure.eventhubs.processor.consumer-group=$Default

Option 2: Connection Strings (For Development/Testing)

# Event Hubs Connection
spring.cloud.azure.eventhubs.connection-string=Endpoint=sb://your-eventhub-ns.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=your-key

# Storage Connection (for checkpointing)
spring.cloud.azure.storage.blob.connection-string=DefaultEndpointsProtocol=https;AccountName=yourstorageaccount;AccountKey=your-key;EndpointSuffix=core.windows.net

# Event Hub Name
spring.cloud.azure.eventhubs.event-hub-name=your-event-hub

# Checkpointing
spring.cloud.azure.eventhubs.processor.checkpoint-store.container-name=eventhub-checkpoints
spring.cloud.azure.eventhubs.processor.consumer-group=$Default

6. Implementing Producer 

import com.azure.spring.messaging.eventhubs.core.EventHubsTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class EventHubProducer {

    @Autowired
    private EventHubsTemplate eventHubsTemplate;

    @PostMapping("/messages")
    public String sendMessage(@RequestBody String message) {
        eventHubsTemplate.sendAsync("your-event-hub", message)
            .doOnSuccess(s -> System.out.println("Sent: " + message))
            .doOnError(e -> System.err.println("Error: " + e.getMessage()))
            .subscribe();
        return "Message sent: " + message;
    }
}

7. Implementing Consumer 

import com.azure.spring.messaging.checkpoint.Checkpointer;
import org.springframework.context.annotation.Bean;
import org.springframework.messaging.Message;
import java.util.function.Consumer;
import static com.azure.spring.messaging.AzureHeaders.CHECKPOINTER;
import org.springframework.context.annotation.Configuration;

@Configuration
public class EventHubConsumer {

    @Bean
    public Consumer<Message<String>> consume() {
        return message -> {
            Checkpointer checkpointer = (Checkpointer) message.getHeaders().get(CHECKPOINTER);
            System.out.println("Received: " + message.getPayload());
            checkpointer.success().block(); // Mark as processed
        };
    }
}

8. Testing & Monitoring

Test the Producer

curl -X POST -H "Content-Type: text/plain" -d "Test Message" http://localhost:8080/messages

Verify Consumer

Check application logs for:

Received: Test Message

Monitor in Azure Portal

  • Event Hubs: Metrics → Incoming Messages
  • Storage Account: Check container for checkpoints

9. Best Practices

  1. For Production:
    • Use Azure AD authentication
    • Store secrets in Azure Key Vault
    • Implement retry policies
  2. Performance:
    • Adjust partition count based on load
    • Use batch sending when possible
  3. Error Handling:
    • Implement dead-letter queues
    • Add circuit breakers

10. Troubleshooting

IssueSolution
Connection failuresVerify connection strings/Azure AD permissions
No messages receivedCheck consumer group and checkpointing
Checkpointing failsVerify storage account permissions
"AADSTS700016" errorValidate client-id and tenant-id

Conclusion

You now have a complete implementation of Azure Event Hubs with Spring Boot using:

  • Azure AD authentication (recommended for production)
  • Connection strings (for development/testing)
  • Producer and consumer components
  • Checkpointing for reliable message processing
🔖Tags: Azure Event Hubs Java Spring Boot
Share on Facebook
Share on Twitter

Leave a Reply

Your email address will not be published. Required fields are marked *