Arpio provides disaster recovery capabilities for Amazon MSK clusters, including both provisioned and serverless configurations. This documentation covers the resources Arpio replicates and important considerations for MSK disaster recovery.
Jump to:
Kafka Cluster Kafka Configuration
Kafka Cluster
Arpio replicates MSK clusters to your disaster recovery environment, preserving cluster configuration, authentication settings, and custom Kafka configurations. Both provisioned and serverless cluster types are supported.
Provisioned Clusters
Provisioned MSK clusters are replicated with their complete configuration including broker nodes, storage, networking, and authentication settings. Arpio creates an equivalent cluster in the target region with the same:
- Kafka version
- Number of broker nodes
- Instance type and storage configuration
- Networking configuration (subnets and security groups)
- Encryption settings
- Authentication methods
- Enhanced monitoring settings
- Custom Kafka configuration
|
Attribute |
Translation |
|
Cluster Name |
Preserved as-is in the target environment |
|
Kafka Version |
Preserved (e.g., 3.8.x) |
|
Number of Broker Nodes |
Preserved for provisioned clusters |
|
Instance Type |
Preserved (e.g., kafka.m5.large) |
|
Storage Volume Size |
Preserved per broker node |
|
Subnets |
Translated to equivalent subnets in the target VPC based on CIDR blocks |
|
Security Groups |
Translated to equivalent security groups in the target VPC |
|
Encryption at Rest |
KMS key reference translated to target region equivalent |
|
Encryption in Transit |
Client-broker and inter-broker encryption settings preserved |
|
Authentication |
IAM, TLS, and SCRAM authentication settings preserved |
|
Enhanced Monitoring |
Monitoring level preserved (DEFAULT, PER_BROKER, PER_TOPIC_PER_BROKER, PER_TOPIC_PER_PARTITION) |
|
Configuration |
References to custom Kafka configurations are translated to target equivalents |
|
Tags |
Replicated to target cluster |
Automatically Selected Resources
When a provisioned MSK cluster is included in an application, Arpio automatically discovers and replicates:
- VPC Subnets - All subnets used by broker nodes
- Security Groups - All security groups attached to the cluster
- KMS Keys - Encryption keys for data at rest (if configured)
- Kafka Configuration - Custom Kafka configuration and revisions (if configured)
- Secrets Manager Secrets - SCRAM authentication secrets (if SCRAM is enabled)
Serverless Clusters
Serverless MSK clusters provide automatic provisioning and scaling of compute and storage. Arpio replicates serverless clusters with:
- Cluster configuration
- VPC networking settings
- Authentication methods
- Encryption settings
|
Attribute |
Translation |
|
Cluster Name |
Preserved as-is in the target environment |
|
Kafka Version |
Preserved |
|
Subnets |
Translated to equivalent subnets in the target VPC based on CIDR blocks |
|
Security Groups |
Translated to equivalent security groups in the target VPC |
|
Authentication |
IAM authentication settings preserved |
|
Tags |
Replicated to target cluster |
Automatically Selected Resources
When a serverless MSK cluster is included in an application, Arpio automatically discovers and replicates:
- VPC Subnets - All subnets used by the cluster
- Security Groups - All security groups attached to the cluster
SCRAM Authentication
For MSK clusters using SCRAM (Salted Challenge Response Authentication Mechanism), Arpio handles the complete authentication setup including:
- Secret Discovery - Arpio automatically discovers all Secrets Manager secrets associated with your MSK cluster for SCRAM authentication
- Secret Replication - SCRAM secrets are replicated to the target region with the same username and password credentials
- Secret Association - After cluster restoration, Arpio automatically associates the replicated secrets with the target cluster
- KMS Encryption - If SCRAM secrets use KMS encryption, the KMS keys are translated to target region equivalents
SCRAM Prerequisites
For successful SCRAM authentication replication:
- Secrets must be stored in AWS Secrets Manager with the naming pattern AmazonMSK_*
- Secrets must contain username and password fields in JSON format
- If secrets use KMS encryption, the KMS key must be included in the application
Kafka Configuration
Arpio replicates custom Kafka configurations used by MSK clusters. A Kafka configuration contains Kafka server properties that control cluster behavior.
|
Attribute |
Translation |
|
Configuration Name |
Preserved as-is in the target environment |
|
Description |
Preserved |
|
Server Properties |
All custom Kafka broker settings are preserved (e.g., auto.create.topics.enable, log.retention.hours) |
|
Kafka Versions |
List of compatible Kafka versions preserved |
Configuration Revisions
Kafka configurations support multiple revisions. Arpio tracks which revision is used by each cluster and ensures the correct revision is applied during restoration. When a cluster is restored, it uses the same configuration revision as the source cluster.
Automatically Selected Resources
When a Kafka configuration is referenced by an MSK cluster, Arpio automatically:
- Discovers the configuration
- Identifies the specific revision in use
- Replicates the configuration to the target region
- Applies the correct revision during cluster restoration