Designing centralized and distributed community connectivity patterns for Amazon OpenSearch Serverless

Amazon OpenSearch Serverless is a completely managed search and analytics service that mechanically provisions and scales infrastructure that can assist you run search and analytics workloads with out cluster administration. With OpenSearch Serverless, you possibly can rapidly construct search and analytics capabilities into your functions.

As organizations scale their use of OpenSearch Serverless, understanding community structure and DNS administration turns into more and more necessary. Constructing upon the connectivity patterns mentioned in our earlier put up Community connectivity patterns for Amazon OpenSearch Serverless, this put up covers superior deployment eventualities targeted on centralized and distributed entry patterns—particularly, how enterprises can simplify community connectivity throughout a number of AWS accounts and prolong entry to on-premises environments for his or her OpenSearch Serverless deployments.

We define two key deployment patterns:

• Sample 1 – A centralized endpoint mannequin the place interface digital non-public cloud (VPC) endpoints for OpenSearch Serverless are deployed in a shared providers VPC, permitting spoke VPCs from different AWS accounts and on premises to entry OpenSearch Serverless collections by way of these consolidated endpoints.
• Sample 2 – A distributed endpoint mannequin the place interface VPC endpoints are created in particular person spoke VPCs, with a number of shoppers (central account, on-premises networks, and different spoke accounts) accessing these endpoints by way of centralized DNS administration. This strategy supplies direct connectivity inside every spoke VPC whereas sustaining centralized DNS management and administration throughout the group.

Earlier than diving into superior deployment patterns, let’s assessment the DNS habits of OpenSearch Serverless when accessed by way of an interface VPC endpoint (AWS PrivateLink). Understanding this foundational side can assist make clear the connectivity patterns we discover on this put up.

OpenSearch Serverless interface VPC endpoint DNS decision

When creating an OpenSearch Serverless interface VPC endpoint, the service mechanically provisions three non-public hosted zones: one seen non-public hosted zone us-east-1.aoss.amazonaws.com that handles area decision for the OpenSearch Serverless assortment and dashboard, one other seen non-public hosted zone us-east-1.opensearch.amazonaws.com that manages decision for the OpenSearch UI (OpenSearch Dashboards), and one hidden inner non-public hosted zone that manages the ultimate DNS decision to personal IP addresses.

Our goal on this put up is to discover how the 2 non-public hosted zones for OpenSearch Serverless work collectively: the seen non-public hosted zone us-east-1.aoss.amazonaws.com for collections and dashboards, and the hidden non-public hosted zone for closing DNS decision to personal IP addresses. We study how these non-public hosted zones allow scalable DNS decision in each centralized and distributed architectures. The next workflow diagram exhibits the DNS decision circulate for the us-east-1 AWS Area. The identical sample applies to different Areas, with the Area identifiers within the DNS information altering accordingly.

The workflow consists of the next steps:

1. A consumer requests entry to a set URL (for instance, abc.us-east-1.aoss.amazonaws.com).
2. The DNS request is shipped to the Amazon Route 53 Resolver, which checks the seen non-public hosted zone us-east-1.aoss.amazonaws.com and finds a CNAME report pointing to the endpoint-specific area.
3. The Route 53 Resolver makes use of the hidden inner non-public hosted zone to resolve this endpoint-specific area to the VPC endpoint’s non-public IP deal with.
4. Visitors is allowed provided that it originates from the interface VPC endpoint authorised by OpenSearch Serverless community insurance policies.

Though this DNS Decision Course of supplies versatile and safe non-public entry, it turns into advanced if you want connectivity from a number of VPCs, totally different AWS accounts, or on-premises networks. The next patterns deal with these challenges and description methods to simplify community entry and DNS administration for OpenSearch Serverless in such environments.

Sample 1: Centralized interface VPC endpoint for OpenSearch Serverless

This sample makes use of a centralized strategy the place a shared providers AWS account with a shared providers VPC hosts the OpenSearch Serverless interface VPC endpoint and OpenSearch Serverless assortment. From there, different AWS accounts with Amazon VPCs (spoke VPCs) want to have the ability to entry OpenSearch Serverless collections by way of this central endpoint. Organizations generally implement this setup in hub-and-spoke community designs that join their VPCs utilizing both AWS Transit Gateway or AWS Cloud WAN. The next diagram illustrates this structure.

Problem

When accessing from on-premises networks, each community entry and DNS decision for the OpenSearch Serverless interface VPC endpoint work efficiently. Nonetheless, though the endpoint is network-accessible from spoke VPCs (for instance, by way of Transit Gateway or AWS Cloud WAN), DNS decision from these VPCs fail.

This occurs as a result of OpenSearch Serverless creates and makes use of a non-public hosted zone us-east-1.aoss.amazonaws.com that’s solely related to the VPC containing the endpoint, on this case, the Shared Companies VPC. Merely sharing this non-public hosted zone with the spoke VPCs doesn’t remedy the issue, as a result of the wildcard CNAME report references a DNS title privatelink.c0X.sgw.iad.prod.aoss.searchservices.aws.dev. This DNS title can’t be resolved from different VPCs with out further configuration, as a result of it belongs to a non-public hosted zone privatelink.c0X.sgw.iad.prod.aoss.searchservices.aws.dev that’s solely related to the shared providers VPC. This non-public hosted zone isn’t seen in your account and is managed by AWS.

Answer: Use Amazon Route 53 Profiles for cross-VPC DNS decision

To allow centralized DNS decision, you should utilize Amazon Route 53 Profiles. With Route 53 Profiles, you possibly can handle and apply DNS-related Amazon Route 53 configurations throughout a number of VPCs and AWS accounts. The next diagram illustrates the answer structure.

The answer consists of the next steps:

1. Create an OpenSearch Serverless interface VPC endpoint within the shared providers VPC. This mechanically creates and associates the next:
   • 1. Two default non-public hosted zones.
     2. One hidden non-public hosted zone with this VPC.
2. Create a Route 53 Profile within the shared providers account.
3. Affiliate the interface VPC endpoint for OpenSearch Serverless with the Route 53 Profile.
   1. The Route 53 Profile mechanically associates the hidden non-public hosted zone with the profile.
4. Affiliate the non-public hosted zone us-east-1.aoss.amazonaws.com that was mechanically created by OpenSearch Serverless with the Route 53 Profile.
5. Share the Route 53 Profile along with your different AWS accounts in your group utilizing AWS Useful resource Entry Supervisor (AWS RAM).
6. Affiliate the spoke VPCs (positioned in several accounts) with the Route 53 Profile.

When you have an present Route 53 Profile in your shared providers account that’s already related to spoke VPCs, you possibly can merely affiliate the OpenSearch Serverless interface VPC endpoint and the non-public hosted zone us-east-1.aoss.amazonaws.com to this profile.

After finishing these steps, the DNS decision for the OpenSearch Serverless assortment and dashboard endpoints works seamlessly from spoke VPCs related to the Route 53 Profile. Shoppers in spoke VPCs can resolve and entry OpenSearch Serverless collections and dashboards by way of the centralized VPC endpoint.

Sample 2: Distributed interface VPC endpoint for OpenSearch Serverless

Every spoke VPC, residing in its respective AWS account, hosts its personal OpenSearch Serverless assortment and interface VPC endpoint. We now need to obtain the next:

• Centralize DNS administration in a shared providers VPC to supply constant decision for OpenSearch Serverless collections deployed throughout a number of spoke accounts
• Present on-premises sources with DNS decision functionality for all OpenSearch Serverless collections throughout the group by way of a Route 53 Resolver inbound endpoint within the shared providers VPC

The next diagram illustrates this structure.

Problem

Managing DNS decision for OpenSearch Serverless collections and dashboards turns into advanced on this distributed mannequin as a result of every interface VPC endpoint creates its personal set of personal hosted zones which might be solely related to their respective VPCs. This creates a fragmented DNS panorama the place the shared providers VPC and on-premises networks want a consolidated method to resolve domains of OpenSearch Serverless collections and dashboards throughout a number of spoke accounts.

Answer: Use a self-managed non-public hosted zone within the shared providers VPC for on-prem DNS decision

To allow centralized DNS decision for distributed endpoints, create a self-managed non-public hosted zone within the shared providers account and affiliate it with the shared providers VPC. Inside this non-public hosted zone, you possibly can create CNAME information that map every OpenSearch Serverless assortment endpoint to its respective interface VPC endpoint DNS names within the spoke accounts. The next diagram illustrates this structure.

Implementation consists of the next steps:

1. Create a self-managed non-public hosted zone within the shared providers account with the area title us-east-1.aoss.amazonaws.com and affiliate it with the shared providers VPC. For every OpenSearch Serverless assortment, create a CNAME report that factors to the Regional DNS title of its corresponding interface VPC endpoint.

This configuration allows each on-premises sources and sources within the shared providers VPC to resolve OpenSearch Serverless endpoints which might be within the spoke accounts.

After you full these steps, every OpenSearch Serverless interface VPC endpoint stays inside its unique AWS account, sustaining safety boundaries and account-level autonomy. On-premises techniques can entry OpenSearch Serverless collections and dashboards utilizing unique assortment DNS names (for instance, {collection-name}.us-east-1.aoss.amazonaws.com) by way of DNS decision offered by the non-public hosted zone within the shared providers VPC.

Conclusion

As organizations scale their adoption of OpenSearch Serverless, establishing safe and centralized community entry turns into more and more necessary. On this put up, we explored two architectural patterns particularly round DNS administration:

• Centralized endpoint mannequin – This sample is right when a shared providers account manages the OpenSearch Serverless interface VPC endpoints, permitting a number of spoke accounts to entry OpenSearch Serverless collections and dashboards by way of a centralized set of community sources.
• Distributed endpoint mannequin with centralized DNS – This sample is appropriate for organizations that require account-level autonomy, the place every AWS account manages its personal OpenSearch Serverless interface VPC endpoints, whereas DNS decision is centralized by way of a shared self-managed non-public hosted zone in a shared providers account.

By understanding the DNS structure of OpenSearch Serverless and utilizing providers like Route 53 Profiles and AWS RAM, organizations can construct safe and sturdy entry patterns that align with their organizational construction and wishes.

Concerning the Authors

Ankush Goyal is a Enterprise Assist Lead in AWS Enterprise Assist who helps clients streamline their cloud operations on AWS. He’s a results-driven IT skilled with over 20 years of expertise.

Anvesh Koganti is a Options Architect at AWS specializing in Networking. He focuses on serving to clients construct networking architectures for extremely scalable and resilient AWS environments. Exterior of labor, Anvesh is enthusiastic about shopper know-how and enjoys listening to podcasts on tech and enterprise. When disconnecting from the digital world, Anvesh spends time outdoor mountaineering and biking.

Salman Ahmed is a Senior Technical Account Supervisor in AWS Enterprise Assist. He makes a speciality of guiding clients by way of the design, implementation, and assist of AWS options. Combining his networking experience with a drive to discover new applied sciences, he helps organizations efficiently navigate their cloud journey. Exterior of labor, he enjoys images, touring, and watching his favourite sports activities groups.