OpenSearch

Arpio replicates OpenSearch domains to your recovery environment when your application is in failover or failover test mode. 

Jump to:


Arpio replicates OpenSearch domains to your recovery environment.  OpenSearch domains can be expensive, so Arpio only replicates them when your application is in failover or failover test mode.  


Arpio supports the following OpenSearch versions:

  • ElasticSearch 6.8 and above
  • OpenSearch 1.0.0 and above

OpenSearch Configuration

Data replication


Data replication is initiated during the backup phase.  Arpio extracts OpenSearch data using native snapshots to an S3 bucket on the primary site.  It enables S3 replication on that primary bucket to copy those snapshots to the recovery site.  Arpio adds an additional layer of security from injection attacks to the recovery button with an “write once” bucket and associated Lambda function. The function prevents an object in the "write once" bucket from being updated after creation.  Arpio then uses the snapshots in this write-once bucket to populate your recovery indexes during failover. 

Enabling Arpio to access OpenSearch data for domains with fine grained access control


If you have fine-grained access control enabled for your domain, Arpio requires access to your OpenSearch domain so it can manage index snapshots and separately snapshot security data (users, roles, mappings, etc.).  If you haven’t added that access, you’ll see an Issue similar to this one when we try to backup your domain:


Please add a mapping to the "security_manager" role for "arn:aws:iam::123456654321:role/ArpioPrimaryOSDel-appabc123-us-east-1"


To enable Arpio access to OpenSearch and ElasticSearch 7 domains, take the following steps:

  • Log into the OpenSearch or ElasticSearch 7 dashboard for your domain
  • Go to Security > Roles (on the left menu bar) and find the security_manager role
  • Click the link for the security_manager role > Mapped users tab > “Manage mapping”
  • Enter the role ARN as a Backend role and click the “Map” button at the bottom of the page
  • Now go back to the list of roles (Security > Roles on the left menu bar) and find the "manage_snapshots" role
  • Click the link, then activate the Mapped users tab, and "Manage mapping"
  • Again, enter the role ARN as a Backend role, and click the "Map" button.

To enable access for ElasticSearch 6.8, do this:

  • Log into the Kibana portal for the domain
  • Go to Security > Role Mappings
  • Select the "security_manager" role.
  • Click the "Add Backend Role" button
  • Enter the role ARN provided in the issue
  • Submit the changes
  • Go to Security > Role Mappings again
  • Click the "+" button to add a new role mapping
  • Select the "manage_snapshots" role from the Role dropdown menu
  • Click the "Add Backend Role" button and enter the ARN of the role
  • Submit the changes

It may take OpenSearch a few minutes to update the backend roles internally.  If you click "Try Again" and Arpio says you still need to add the roles, wait a few minutes and then click "Try Again" another time.


Setting passwords for replicated OpenSearch users


OpenSearch does not return user passwords or password hashes during snapshotting, so Arpio can’t copy those to the recovery site.  Instead you can set up a secret in Secrets Manager to set user passwords for the replicated domains.

  • This secret should be placed in the recovery region and account.  If this secret doesn't already exist on the recovery site Arpio will automatically create one for you 
  • It should have the name /Arpio/OpenSearchDomain/$DOMAIN_NAME$/UserPasswords, where $DOMAIN_NAME$ is the name of the domain in which you want to populate the passwords.  For example, if you domain name is "prodlogs", the secret should be named /Arpio/OpenSearchDomain/prodlogs/UserPasswords
  • The secret should have the value:

{

  "user1": "password1",

  "user2": "password2",

  ...etc..

}


Where user1 and user2 are the names of users in the domain, and password1 and password2 are the passwords that should be set for those users on the recovery domain.


Alternatively, when Arpio has finished replicating the domain, you can use the AWS console to set access for the master user, and then log into the domain dashboards and manually set the passwords for those users.

Excluding OpenSearch indexes from backup and recovery

By default, all non-administrative OpenSearch indexes are replicated during a recovery event. If you'd like to exclude some or all of your indexes from Arpio's backup and recovery process, you can use our custom configuration tag. More details on that process can be found here

 

Domain Replication & Translation

The following tables details how Arpio handles OpenSearch Domain replication and translation in the DR environment.


Attribute

Translation

Resource access policy

For any resource we support, Arpio will update the ARNs and ids in the OpenSearch domain access policy to reference the mirrored resources

CloudWatch LogGroup

The ARNs for any Cloudwatch Log Groups used for log publishing are translated to the corresponding recovery log group ARN

Domain endpoint certificate

Arpio will change the replicated domain’s endpoint certificate ARN to point to the matching certificate in the recovery site.

Encryption-at-rest KMS Key ARN

The ARNs for the KMS key used for encryption-at-rest are switched to match the KMS key on the recovery site.

IAM roles used in  OpenSearch security role mappings

Arpio will update the IAM roles in your domain’s security configuration to the corresponding role on the recovery site.

VPC and subnets

If a domain uses a VPC, the VPC and subnet ids are switched to the recovery ids.


Resources automatically discovered and included in recovery points when a domain is selected for replication:

  • IAM Roles used as backend roles in the OpenSearch domain security configuration
  • The KMS key used for encryption-at-rest
  • VPC and security groups used in the VPC if the cluster is configured to use a VPC
  • Resource referenced in the access policy of the domain
  • If you’ve enabled a custom endpoint for the domain, the certificate used for that custom endpoint is included

Domain Restrictions


The following OpenSearch capabilities are not currently supported by Arpio:

  • Custom domain plugins (customer-built plugins for specialized indexing)
  • Cross-cluster searching
  • Multi-AZ with Standby.
  • Elasticsearch versions < 6.8
  • OpenSearch Serverless resources

Pipeline Replication



Attribute

Translation

Encryption-at-rest KMS Key ARN

The ARNs for the KMS key used for encryption-at-rest is switched to match the KMS key on the recovery site.

Pipeline configuration body

OpenSearch endpoint URLs, S3 bucket names, SQS queues, IAM STS roles, and DynamoDB table ARNs referenced in the pipeline configuration body are translated to references that match those which Arpio created in the recovery environment.

VPC subnets, security groups, and CIDR block

VPC subnets, security groups, and the CIDR block used in a pipeline's VPC options are translated to match the recovery environment.

Resources automatically discovered and included in recovery points when a pipeline is selected for replication:

  • Subnets and security groups if the pipeline is configured to use a VPC
  • The KMS key used for encryption-at-rest
  • S3 buckets named in sources or sinks in the pipeline configuration body
  • DynamoDB tables use for sources in the pipeline configuration body
  • OpenSearch domains used as sources or sinks in the pipeline configuration body

Pipeline Restrictions

The following Pipeline capabilities are not currently supported by Arpio:

  • Kafka sources, including Confluent, MSK, or self-managed Kafka
  • DocumentDB sources
  • Pipelines created outside of AWS using ElasticSearch APIs or the Kibana UI
  • The AWS secrets extension to the pipeline configuration body for accessing data source credentials