Enterprise Cloud Architectural Patterns — Part3
Our previous Enterprise Cloud Architectural Patterns articles looked into Centralized Hybrid DNS Management, Centralized User Access, and Geo-distributed Applications. We continue exploring common architectural patterns for enterprise cloud systems with application integration.
Application Integration
As a business expands, delivering customer use cases requires an increasing number of applications. Effective application integration, whether internal or external, becomes a critical component of success. Organizations can choose from various options, such as event buses, notification systems, webhooks, queues, and workflow systems, to provide scalable and robust integrations.
Webhooks and event buses are commonly used to integrate external applications. Two such patterns are described below.
Choosing between Webhooks and Event Buses
Webhooks (also known as Reverse APIs) and event buses provide asynchronous, event-driven communication in cloud systems but differ in their approaches. Webhooks use a direct push model for real-time notifications to specific endpoints in response to a business event. On the other hand, event buses provide routers to bridge between sources and targets without direct endpoint knowledge. After the events are received, based on the routing rules, they are delivered to the endpoints. Event buses might imply fan-in and fan-out for many-to-many integration, while webhooks are point-to-point integration.
Scenario 1: External Processing System
In this use case, two applications owned by different companies must communicate as part of business logic. The consumer side submits a request for asynchronous processing, and a notification is sent back when the processing is completed. For example, a commerce application (webhook receiver) can coordinate payments with an external system (webhook provider).
The architecture below provides a webhook delivery system (company A) with control and data plane capabilities. The control plane registers endpoints and configures auth and delivery behavior, while the data plane delivers webhooks. Business events captured and transformed are sent to the destination API of the webhook receiver.
Note that we use EventBridge pipes to adapt to the control flow of CDC stream events. Pipes will pull these events and push them to the destination as per the pipeline.
On the receiving end (company B), the application needs to ensure the validity and authenticity of the webhook and process the event based on the business logic.
Webhook Considerations:
- Consider using queues and worker groups that scale based on load for the durability and scalability of webhook handling
- Ensure webhook delivery failures are handled and eventually sent to DLQ for investigation
- Webhook handlers might require calls coming from a specific IP to be allowed. In this case, custom send logic and infrastructure, e.g., from a configured VPC, are needed
- Ensure the incoming webhooks are validated using HMAC or other auth mechanisms
- Apply rate limiting and monitor for unusual activity
Scenario 2: Reacting to External Events
In our second use, we have an external app generating events that need further processing for business logic. The application on the “source” side submits them to a preconfigured, agreed-upon event bus. The destination application consumes these events based on the business logic and routes them to destination endpoints and systems. For example, events from an external CRM system can be received and routed to internal systems and workflows for further processing.
Event bus considerations:
- Provide minimum permissions needed to external senders
- Ensure the authenticity of the events and validate
- Audit events regularly
- Setup alerts on event bus metrics
- Designate a central processing logic for received events before any fanout to ensure admission control filtering of events that can cause downstream issues
Recap
This article examined considerations for application integration in enterprise cloud systems using webhooks and event buses for common scenarios. We hope you find this article helpful.
Other articles in the series:
