Choreography-Based Saga Architecture and Django Microservices Implementation

Choreography-Based Saga Architecture:

Here's a detailed walkthrough:

---

Steps in the Saga

1. Step 1: Initiating the Saga (POST /orders)

   • A client sends a POST request to the Order Service to create an order.
   • The Order Service processes this request, creates an Order entity (aggregate), and publishes an Order Created event to the Order Events Channel.

2. Step 2: Order Created Event

   • The Order Created event flows through the Order Events Channel to notify interested services (e.g., Customer Service) about the creation of the order.
   • The event contains details about the order, such as its ID, total amount, and customer ID.

3. Step 3: Reserve Credit (POST /customer)

   • The Customer Service receives the Order Created event and attempts to reserve the required credit for the order.
   • If the credit limit is sufficient:
     • It updates the Customer entity (aggregate) to reserve the credit.
     • It publishes a Credit Reserved event on the Customer Events Channel.

4. Step 4: Insufficient Credit

   • If the customer’s credit limit is exceeded:
     • The Customer Service publishes a Credit Limit Exceeded event on the Customer Events Channel.
     • No credit is reserved.

5. Step 5: Finalizing the Order

   • The Order Service listens for responses on the Customer Events Channel:
     • If Credit Reserved is received, the Order Service approves the order and updates the order state (e.g., approved).
     • If Credit Limit Exceeded is received, the Order Service rejects the order and updates the order state (e.g., rejected).

---

Key Features of the Architecture

Event Channels

• Order Events Channel: Handles events related to orders, such as Order Created.
• Customer Events Channel: Handles customer-related events, such as Credit Reserved and Credit Limit Exceeded.
• These channels decouple services, enabling asynchronous communication.

Aggregates

• Aggregates represent the core business logic and state for each service.
• Order Aggregate: Manages the lifecycle of orders.
• Customer Aggregate: Manages the customer’s credit state and reservations.

---

Transactional Guarantees in the Saga

Distributed Transactions

• Each service (e.g., Order Service, Customer Service) performs local transactions within its boundary.
• Changes are committed to the respective database independently.

Eventual Consistency

• While the saga executes, the system might be temporarily inconsistent.
• Consistency is achieved once the saga is completed, either successfully (order approved) or unsuccessfully (order rejected).

Compensations

• If the order is rejected, any reserved credit in the Customer Service must be released through a compensating action (e.g., removing the credit reservation).

---

Advantages of This Approach

1. Decoupled Services:

   • Services communicate indirectly via events, reducing dependencies and improving scalability.

2. Asynchronous Processing:

   • Services process events at their own pace, improving system responsiveness.

3. Fault Tolerance:

   • Failures in one service (e.g., credit reservation) do not directly impact others, and compensating actions can handle partial failures.

---

Challenges

1. Event Tracking:

   • Monitoring and debugging event flows across multiple services can be challenging without proper tooling.

2. Race Conditions:

   • Services must handle potential race conditions, such as duplicate or out-of-order events.

3. Error Handling:

• Designing compensating transactions for every failure scenario requires careful planning.

---

Here's a guide to implementing a Choreography-Based Saga Architecture using Django Microservices. This example assumes two Django microservices: Order Service and Customer Service, with event-driven communication via Kafka (or RabbitMQ).

1. Project Setup

1. Create two Django projects:

   • order_service
   • customer_service

2. Install dependencies:

   pip install django djangorestframework kafka-python

3. Create Kafka topics:

   kafka-topics --create --topic order-events --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1
   kafka-topics --create --topic customer-events --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1
   

---

2. Order Service Implementation

Models

Define an Order model in order_service/models.py:

from django.db import models

class Order(models.Model):
    order_id = models.CharField(max_length=255, unique=True)
    customer_id = models.CharField(max_length=255)
    total = models.FloatField()
    state = models.CharField(max_length=50, choices=[('PLACED', 'Placed'), ('APPROVED', 'Approved'), ('REJECTED', 'Rejected')])

    def __str__(self):
        return self.order_id

Views

Create a view for placing an order:

from rest_framework.views import APIView
from rest_framework.response import Response
from .models import Order
from kafka import KafkaProducer
import json

class PlaceOrderView(APIView):
    def post(self, request):
        data = request.data
        order = Order.objects.create(
            order_id=data['order_id'],
            customer_id=data['customer_id'],
            total=data['total'],
            state='PLACED'
        )
        # Publish event
        producer = KafkaProducer(bootstrap_servers='localhost:9092',
                                 value_serializer=lambda v: json.dumps(v).encode('utf-8'))
        producer.send('order-events', {
            'type': 'ORDER_CREATED',
            'order_id': order.order_id,
            'customer_id': order.customer_id,
            'total': order.total
        })
        return Response({'message': 'Order placed successfully!'})

Kafka Consumer

Consume events from the customer-events topic:

from kafka import KafkaConsumer
import threading
import json
from .models import Order

def handle_customer_events():
    consumer = KafkaConsumer(
        'customer-events',
        bootstrap_servers='localhost:9092',
        auto_offset_reset='earliest',
        group_id='order-group',
        value_deserializer=lambda x: json.loads(x.decode('utf-8'))
    )
    for message in consumer:
        event = message.value
        if event['type'] == 'CREDIT_RESERVED':
            order = Order.objects.get(order_id=event['order_id'])
            order.state = 'APPROVED'
            order.save()
        elif event['type'] == 'CREDIT_LIMIT_EXCEEDED':
            order = Order.objects.get(order_id=event['order_id'])
            order.state = 'REJECTED'
            order.save()

# Run Kafka consumer in a separate thread
threading.Thread(target=handle_customer_events, daemon=True).start()

---

3. Customer Service Implementation

Models

Define a Customer model in customer_service/models.py:

from django.db import models

class Customer(models.Model):
    customer_id = models.CharField(max_length=255, unique=True)
    credit_limit = models.FloatField()
    reserved_credit = models.FloatField(default=0.0)

    def __str__(self):
        return self.customer_id

Kafka Consumer

Consume events from the order-events topic:

from kafka import KafkaConsumer, KafkaProducer
import threading
import json
from .models import Customer

def handle_order_events():
    consumer = KafkaConsumer(
        'order-events',
        bootstrap_servers='localhost:9092',
        auto_offset_reset='earliest',
        group_id='customer-group',
        value_deserializer=lambda x: json.loads(x.decode('utf-8'))
    )
    producer = KafkaProducer(bootstrap_servers='localhost:9092',
                              value_serializer=lambda v: json.dumps(v).encode('utf-8'))

    for message in consumer:
        event = message.value
        if event['type'] == 'ORDER_CREATED':
            customer = Customer.objects.get(customer_id=event['customer_id'])
            if customer.credit_limit >= event['total']:
                customer.reserved_credit += event['total']
                customer.save()
                # Publish Credit Reserved event
                producer.send('customer-events', {
                    'type': 'CREDIT_RESERVED',
                    'order_id': event['order_id']
                })
            else:
                # Publish Credit Limit Exceeded event
                producer.send('customer-events', {
                    'type': 'CREDIT_LIMIT_EXCEEDED',
                    'order_id': event['order_id']
                })

# Run Kafka consumer in a separate thread
threading.Thread(target=handle_order_events, daemon=True).start()

---

4. Configure Kafka

Ensure the Kafka broker is running at localhost:9092. Update the Kafka settings in both services if required.

---

5. Testing the Saga

1. Run Kafka: Start Kafka and Zookeeper:

   zookeeper-server-start.sh config/zookeeper.properties
   kafka-server-start.sh config/server.properties

2. Run Services: Start both Django services:

   python manage.py runserver 8000  # Order Service
   python manage.py runserver 8001  # Customer Service

3. Place an Order: Use Postman or curl to create an order:

   curl -X POST http://localhost:8000/orders/ -H "Content-Type: application/json" \
   -d '{"order_id": "1", "customer_id": "123", "total": 500}'

4. Monitor Events: Check the logs of both services to see the saga choreography in action.

🚀 Master Django 4 with Real-World Projects! 🚀
Build powerful, scalable Python web apps effortlessly and level up your coding skills! 💻

✨ Grab your copy of 'Django 4 By Example' by Antonio Melé for only $10! ✨

✅ Learn through practical, hands-on examples
✅ Unlock the secrets of Django’s latest features
✅ Build and deploy real-world applications

Start your journey to becoming a Django expert today!
🔥 Buy Now and boost your web development skills! 🔥