OpenSearch

Arpio replicates OpenSearch domains to your recovery environment when your application is in failover or failover test mode. 

Jump to:

Arpio replicates OpenSearch domains to your recovery environment.  OpenSearch domains can be expensive, so Arpio only replicates them when your application is in failover or failover test mode.  


Arpio supports the following OpenSearch versions:

  • ElasticSearch 6.8 and above
  • OpenSearch 1.0.0 and above

OpenSearch Configuration

Data replication


Data replication is initiated during the backup phase.  Arpio extracts OpenSearch data using native snapshots to an S3 bucket on the primary site.  It enables S3 replication on that primary bucket to copy those snapshots to the recovery site.  Arpio adds an additional layer of security from injection attacks to the recovery button with an “write once” bucket and associated Lambda function. The function prevents an object in the "write once" bucket from being updated after creation.  Arpio then uses the snapshots in this write-once bucket to populate your recovery indexes during failover. 

Enabling Arpio to access OpenSearch data for domains with fine grained access control


If you have fine-grained access control enabled for your domain, Arpio requires access to your OpenSearch domain so it can manage index snapshots and separately snapshot security data (users, roles, mappings, etc.).  If you haven’t added that access, you’ll see an Issue similar to this one when we try to backup your domain:


Please add a mapping to the "security_manager" role for "arn:aws:iam::123456654321:role/ArpioPrimaryOSDel-appabc123-us-east-1"


To enable Arpio access to OpenSearch and ElasticSearch 7 domains, take the following steps:

  • Log into the OpenSearch or ElasticSearch 7 dashboard for your domain
  • Go to Security > Roles (on the left menu bar) and find the security_manager role
  • Click the link for the security_manager role > Mapped users tab > “Manage mapping”
  • Enter the role ARN as a Backend role and click the “Map” button at the bottom of the page
  • Now go back to the list of roles (Security > Roles on the left menu bar) and find the "manage_snapshots" role
  • Click the link, then activate the Mapped users tab, and "Manage mapping"
  • Again, enter the role ARN as a Backend role, and click the "Map" button.

To enable access for ElasticSearch 6.8, do this:

  • Log into the Kibana portal for the domain
  • Go to Security > Role Mappings
  • Select the "security_manager" role.
  • Click the "Add Backend Role" button
  • Enter the role ARN provided in the issue
  • Submit the changes
  • Go to Security > Role Mappings again
  • Click the "+" button to add a new role mapping
  • Select the "manage_snapshots" role from the Role dropdown menu
  • Click the "Add Backend Role" button and enter the ARN of the role
  • Submit the changes

It may take OpenSearch a few minutes to update the backend roles internally.  If you click "Try Again" and Arpio says you still need to add the roles, wait a few minutes and then click "Try Again" another time.


Setting passwords for replicated OpenSearch users


OpenSearch does not return user passwords or password hashes during snapshotting, so Arpio can’t copy those to the recovery site.  Instead you can set up a secret in Secrets Manager to set user passwords for the replicated domains.

  • This secret should be placed in the recovery region and account.  If this secret doesn't already exist on the recovery site Arpio will automatically create one for you 
  • It should have the name /Arpio/OpenSearchDomain/$DOMAIN_NAME$/UserPasswords, where $DOMAIN_NAME$ is the name of the domain in which you want to populate the passwords.  For example, if you domain name is "prodlogs", the secret should be named /Arpio/OpenSearchDomain/prodlogs/UserPasswords
  • The secret should have the value:

{

  "user1": "password1",

  "user2": "password2",

  ...etc..

}


Where user1 and user2 are the names of users in the domain, and password1 and password2 are the passwords that should be set for those users on the recovery domain.


Alternatively, when Arpio has finished replicating the domain, you can use the AWS console to set access for the master user, and then log into the domain dashboards and manually set the passwords for those users.


Domain


Attribute

Translation

Resource access policy

For any resource we support, Arpio will update the ARNs and ids in the OpenSearch domain access policy to reference the mirrored resources

CloudWatch LogGroup

The ARNs for any Cloudwatch Log Groups used for log publishing are translated to the corresponding recovery log group ARN

Domain endpoint certificate

Arpio will change the replicated domain’s endpoint certificate ARN to point to the matching certificate in the recovery site.

IAM roles used in  OpenSearch security role mappings

Arpio will update the IAM roles in your domain’s security configuration to the corresponding role on the recovery site.

VPC and subnets

If a domain uses a VPC, the VPC and subnet ids are switched to the recovery ids.


Resources automatically discovered and included in recovery points when a domain is selected for replication:

  • IAM Roles used as backend roles in the OpenSearch domain security configuration
  • VPC and security groups used in the VPC if the cluster is configured to use a VPC
  • Resource referenced in the access policy of the domain
  • If you’ve enabled a custom endpoint for the domain, the certificate used for that custom endpoint is included

Restrictions


The following OpenSearch capabilities are not currently supported by Arpio:

  • Custom domain plugins (customer-built plugins for specialized indexing)
  • Cross-cluster searching
  • Multi-AZ with Standby.
  • Elasticsearch versions < 6.8
  • OpenSearch Serverless resources