Arpio replicates OpenSearch domains to your recovery environment when your application is in failover or failover test mode.
Jump to:
Arpio replicates OpenSearch domains to your recovery environment. OpenSearch domains can be expensive, so Arpio only replicates them when your application is in failover or failover test mode.
Arpio supports the following OpenSearch versions:
- ElasticSearch 6.8 and above
- OpenSearch 1.0.0 and above
OpenSearch Configuration
Data replication
Data replication is initiated during the backup phase. Arpio extracts OpenSearch data using native snapshots to an S3 bucket on the primary site. It enables S3 replication on that primary bucket to copy those snapshots to the recovery site. Arpio adds an additional layer of security from injection attacks to the recovery button with an “write once” bucket and associated Lambda function. The function prevents an object in the "write once" bucket from being updated after creation. Arpio then uses the snapshots in this write-once bucket to populate your recovery indexes during failover.
Enabling Arpio to access OpenSearch data for domains with fine grained access control
If you have fine-grained access control enabled for your domain, Arpio requires access to your OpenSearch domain so it can manage index snapshots and separately snapshot security data (users, roles, mappings, etc.). If you haven’t added that access, you’ll see an Issue similar to this one when we try to backup your domain:
Please add a mapping to the "security_manager" role for "arn:aws:iam::123456654321:role/ArpioPrimaryOSDel-appabc123-us-east-1"
To enable Arpio access to OpenSearch and ElasticSearch 7 domains, take the following steps:
- Log into the OpenSearch or ElasticSearch 7 dashboard for your domain
- Go to Security > Roles (on the left menu bar) and find the
security_managerrole - Click the link for the
security_managerrole > Mapped users tab > “Manage mapping” - Enter the role ARN as a Backend role and click the “Map” button at the bottom of the page
- Now go back to the list of roles (Security > Roles on the left menu bar) and find the
"manage_snapshots"role - Click the link, then activate the Mapped users tab, and "Manage mapping"
- Again, enter the role ARN as a Backend role, and click the "Map" button.
To enable access for ElasticSearch 6.8, do this:
- Log into the Kibana portal for the domain
- Go to Security > Role Mappings
- Select the "security_manager" role.
- Click the "Add Backend Role" button
- Enter the role ARN provided in the issue
- Submit the changes
- Go to Security > Role Mappings again
- Click the "+" button to add a new role mapping
- Select the "manage_snapshots" role from the Role dropdown menu
- Click the "Add Backend Role" button and enter the ARN of the role
- Submit the changes
It may take OpenSearch a few minutes to update the backend roles internally. If you click "Try Again" and Arpio says you still need to add the roles, wait a few minutes and then click "Try Again" another time.
Setting passwords for replicated OpenSearch users
OpenSearch does not return user passwords or password hashes during snapshotting, so Arpio can’t copy those to the recovery site. Instead you can set up a secret in Secrets Manager to set user passwords for the replicated domains.
- This secret should be placed in the recovery region and account. If this secret doesn't already exist on the recovery site Arpio will automatically create one for you
- It should have the name
/Arpio/OpenSearchDomain/$DOMAIN_NAME$/UserPasswords, where $DOMAIN_NAME$ is the name of the domain in which you want to populate the passwords. For example, if you domain name is "prodlogs", the secret should be named/Arpio/OpenSearchDomain/prodlogs/UserPasswords - The secret should have the value:
{
"user1": "password1",
"user2": "password2",
...etc..
}
Where user1 and user2 are the names of users in the domain, and password1 and password2 are the passwords that should be set for those users on the recovery domain.
Alternatively, when Arpio has finished replicating the domain, you can use the AWS console to set access for the master user, and then log into the domain dashboards and manually set the passwords for those users.
Giving Amazon OpenSearch Service permissions to access your recovery VPC
- Run this command if the domain is using ElasticSearch:
aws iam create-service-linked-role --aws-service-name es.amazonaws.com - Run this command if the domain is using OpenSearch:
aws iam create-service-linked-role --aws-service-name opensearchservice.amazonaws.com
Excluding OpenSearch indexes from backup and recovery
By default, all non-administrative OpenSearch indexes are replicated during a recovery event. If you'd like to exclude some or all of your indexes from Arpio's backup and recovery process, you can use our custom configuration tag. More details on that process can be found here.
Domain Replication & Translation
The following tables details how Arpio handles OpenSearch Domain replication and translation in the DR environment.
|
Attribute |
Translation |
|
Resource access policy |
For any resource we support, Arpio will update the ARNs and ids in the OpenSearch domain access policy to reference the mirrored resources |
|
CloudWatch LogGroup |
The ARNs for any Cloudwatch Log Groups used for log publishing are translated to the corresponding recovery log group ARN |
|
Domain endpoint certificate |
Arpio will change the replicated domain’s endpoint certificate ARN to point to the matching certificate in the recovery site. |
|
Encryption-at-rest KMS Key ARN |
The ARNs for the KMS key used for encryption-at-rest are switched to match the KMS key on the recovery site. |
|
IAM roles used in OpenSearch security role mappings |
Arpio will update the IAM roles in your domain’s security configuration to the corresponding role on the recovery site. |
|
VPC and subnets |
If a domain uses a VPC, the VPC and subnet ids are switched to the recovery ids. |
- IAM Roles used as backend roles in the OpenSearch domain security configuration
- The KMS key used for encryption-at-rest
- VPC and security groups used in the VPC if the cluster is configured to use a VPC
- Resource referenced in the access policy of the domain
- If you’ve enabled a custom endpoint for the domain, the certificate used for that custom endpoint is included
Domain Restrictions
- The following OpenSearch capabilities are not currently supported by Arpio:
- Custom domain plugins (customer-built plugins for specialized indexing)
- Cross-cluster searching
- Multi-AZ with Standby.
- Elasticsearch versions < 6.8
- OpenSearch Serverless resources
Pipeline Replication
|
Attribute |
Translation |
|
Encryption-at-rest KMS Key ARN |
The ARNs for the KMS key used for encryption-at-rest is switched to match the KMS key on the recovery site. |
|
Pipeline configuration body |
OpenSearch endpoint URLs, S3 bucket names, SQS queues, IAM STS roles, and DynamoDB table ARNs referenced in the pipeline configuration body are translated to references that match those which Arpio created in the recovery environment. |
|
VPC subnets, security groups, and CIDR block |
VPC subnets, security groups, and the CIDR block used in a pipeline's VPC options are translated to match the recovery environment. |
- Subnets and security groups if the pipeline is configured to use a VPC
- The KMS key used for encryption-at-rest
- S3 buckets named in sources or sinks in the pipeline configuration body
- DynamoDB tables use for sources in the pipeline configuration body
- OpenSearch domains used as sources or sinks in the pipeline configuration body
Pipeline Restrictions
- The following Pipeline capabilities are not currently supported by Arpio:
- Kafka sources, including Confluent, MSK, or self-managed Kafka
- DocumentDB sources
- Pipelines created outside of AWS using ElasticSearch APIs or the Kibana UI
- The AWS secrets extension to the pipeline configuration body for accessing data source credentials