6 Deploying SmartSearch
This chapter describes how to deploy the SmartSearch service.
About SmartSearch
SmartSearch is a micronaut application, when integrated with OpenSearch, offers a powerful, flexible, and feature-rich search experience that can be tailored to specific business and user needs. Using OpenSearch as the underlying engine, SmartSearch can handle large volumes of data, perform real-time indexing, and support complex querying to enhance search relevancy. Features such as autocomplete, fuzzy matching, synonym recognition, and intelligent ranking make it easier for users to locate precise information, even if search terms are partially matched or misspelled.
Setting up Environment Variables
The SmartSearch service relies on access to certain environment variables to run seamlessly. Ensure the following variables are set in your environment:
- Path to your common cloud native toolkit
- Path to your specification directory
To set the environment variables:
- Set the COMMON_CNTK variable to the path of directory where common cloud native
toolkit is extracted as
follows:
$ export COMMON_CNTK=$WORKSPACEDIR/common-cntk
- Set
SPEC_PATHvariable to the location where applications, shape, configuration, and database yamls are copied. See "Assembling the Specifications" to copy specification files if not already copied.$ export SPEC_PATH=$WORKSPACEDIR/spec_dir
Creating Secrets
To create secrets:
- Run the following command to generate the
openSearchCredentialssecret and enter the basic auth credentials of OpenSearch as prompted:$COMMON_CNTK/scripts/manage-app-credentials.sh -p sr -i quick -a smartsearch -s $SPEC_PATH create openSearchCredentials #Enter the values appropriately against prompts: opensearch username: admin #opensearch basic oauth credentials opensearch password: xxxxx - Verify if the secrets are created as
follows:
sr-quick-open-search-credentials
Secret for Authentication Details
The SmartSearch service uses configuration values from Kubernetes secret
(<namespace>-<instance>-oauth-credentials) and Config Map
(<namespace>-<instance>-oauth-config-cm) objects from the
same namespace. This Secret and Configuration Map Kubernetes objects have to be
created before deploying the SmartSearch service for authentication. See "Adding Common OAuth Secret and ConfigMap" for creating the secret.
Secret to Pass Egress Certificates to SmartSearch
You have to create a commonTrust secret to provide SSL
certificates of IdP, OpenSearch, and so on to SmartSearch.
To create commonTrust secret:
- Generate truststore.jks that contains the required certificates as
follows:
#generate truststore with idp cert (applicable only if authentication is enabled ) keytool -importcert -v -alias <param> -file <path to IDP cert file> -keystore <truststorename>.jks -storepass <password> #add opensearch ssl trust certificate in truststore keytool -importcert -v -alias <param> -file <path to opensearch cert file> -keystore <truststorename>.jks -storepass <password> - Run the below command to generate the Kubernetes secret. The system
will prompt for the truststore file path and password of truststore. Provide the
details
accordingly.
$COMMON_CNTK/scripts/manage-app-credentials.sh -p $PROJECT -i $INSTANCE -s $SPEC_PATH create commonTrust
Configuring the Application Specification
In $SPEC_PATH/$PROJECT/$INSTANCE update the specification files:
app-smartsearch.yaml and applications-base.yaml to provide the values
of required configurations such as SmartSearch image, tag and ingress controller
configurations, and so on. The image tag can be updated
when a new version of the image is released. This allows easy upgrades and ensures that
the application runs the latest features and security patches.
To configure the application specification:
- Download and
pushthe SmartSearch image to your internal repository as follows:- Download the Oracle Communications Unified Inventory Management SmartSearch image from Oracle Software Delivery Cloud.
- Load the image from tar file and
pushthe image to a container repository. See "About Container Image Management" in UIM Cloud Native Deploy Guide.podman load -i <smart-search-latest.tar> podman tag smart-search:latest <conatiner repository>/smart-search:latest podman push <conatiner repository>/smart-search:latest
- Edit $SPEC_PATH/$PROJECT/$INSTANCE/applications-base.yaml
specification file as follows:
- Provide an image
pullsecret as follows to pull images from your private repository:# The image pull access credentials for Private repository, as a Kubernetes secret. # uncomment and set if required. # imagePullSecret: # imagePullSecrets: # - name: regcred - Update
loadbalancerportas follows:# For Generic Ingress Controllers: # If ssl is enabled this would be loadbalancer's ssl port. # IF ssl is disabled this would be loadblancer's non ssl port. # For examle ssl and non-ssl ports for external loadbalancer would be 443 and 80 respectively. # If loadbalancer is not created, provide nodePort of Nginx or any other Generic Ingress Controller loadBalancerPort: 30505 - For Generic ingress controller, update the following:
- Update the following to provide the annotations to
enable stickiness through
cookies:
# Valid values are GENERIC ingressController: "GENERIC" - Annotations for NGINX is as
follows:
ingress: className: nginx ##provide ingressClassName value annotations: nginx.ingress.kubernetes.io/affinity: "cookie" nginx.ingress.kubernetes.io/affinity-mode: "persistent" nginx.ingress.kubernetes.io/session-cookie-name: "nginxingresscookie" nginx.ingress.kubernetes.io/proxy-body-size: "50m"
- Update the following to provide the annotations to
enable stickiness through
cookies:
- Verify the shape name provided in applications-base.yaml
and the corresponding
$SPEC_PATH/$PROJECT/$INSTANCE/shapes/<shapeName>/smartsearch.yaml
file for resources. If you want to change shape according to
devorprod, you can change shape value as follows:shape: dev
- Provide an image
- Edit $SPEC_PATH/$PROJECT/$INSTANCE/app-smartsearch.yaml
specification file as follows:
- Provide the name and tag of the SmartSearch image from your
repository as
follows:
smartSearch: image: name: <conatiner repository>/smart-search tag: 0.4.9 - Set
pullPolicyfor SmartSearch image as follows:smartSearch: imagePullPolicy: IfNotPresent - Uncomment the annotations section for smartSearch specific
annotations as
follows:
smartSearch: ingress: annotations: nginx.ingress.kubernetes.io/rewrite-target: /$2
- Provide the name and tag of the SmartSearch image from your
repository as
follows:
Creating a SmartSearch Instance
This section describes how to create a SmartSearch service instance in your cloud native environment using the operational scripts and the configuration provided in the Common cloud native toolkit (Common CNTK).
Before you create a SmartSearch instance, you must validate cloud native environment. See "Planning UIM Installation" for details on prerequisites.
In this section, while creating a basic instance, the project name is considered as
sr and instance name is considered as
quick.
Note:
The project and instance names cannot contain any special characters.
To create a SmartSearch service instance in your environment using the scripts that are provided with the toolkit:
- Run the following command to create an SmartSearch service
instance:
$COMMON_CNTK/scripts/create-applications.sh -p sr -i quick -s $SPEC_PATH -a smartsearch
The
create-applicationsscript uses the Helm chart located in $COMMON_CNTK/charts/smartsearch-app to create and deploy a SmartSearch service. - Run the following command to create SmartSearch
Ingress:
$COMMON_CNTK/scripts/create-ingress.sh -p sr -i quick -s $SPEC_PATH -a smartsearchThe create-ingress script creates an Ingress object that defines hostname and path-based routing rules to direct incoming requests to SmartSearch services through the Ingress Controller. It is registered with the Ingress Controller based on the className property specified in the applications-base.yaml file.
- (Optional) If the scripts mentioned above fail, run the following
command to review the error message in the
log:
kubectl logs -n sr sr-quick-smartsearch
- Clear the failed pod by running the following
command:
helm uninstall sr-quick-smartsearch-n sr
- Fix the issues if any and run the script again to deploy the SmartSearch service.
Accessing the SmartSearch Service
To access SmartSearch service, use the SmartSearch endpoint in the format:
http://<topology-instance>.<topology-project>.topology.<hostSuffix>:<port>/<appVersion>/<uri-endpoint>
For example: SmartSearch API endpoint is http://quick.sr.topology.uim.org:30505/20240801/health
Make sure you have created SmartSearch ingress using create-ingress script before accessing the above URL.
Note:
Make sure you add entry in /etc/hosts: <k8s cluster ip
or external loadbalancer ip>
quick.sr.topology.uim.org.
For SmartSearch appVersion, see UIM Compatibility Matrix.
Creating SmartSearch Index and Metadata
The SmartSearch schema in OpenSearch is structured to optimize search performance and relevance, using six main indexes: Location, Logical Device, Physical Device, Equipment, Communication, and Events. Each index is defined by mappings that specify field types such as text for full-text search and keyword for exact matches and may include custom analyzers for handling text processing tasks such as tokenization and synonyms. Additionally, metadata is configured for each index to control search behavior, supporting features such as filters, sorting, and autocomplete to enhance user experience. All indexes are accessed using aliases, providing a flexible way to query and manage data. This setup ensures fast, accurate, and relevant search results tailored to user needs.
Prerequisites
Ensure the following environment variables are set:
SMARTSEARCHDOMAIN: Set this to the SmartSearch service endpoint (format:export SMARTSEARCHDOMAIN=<host:port/appVersion>), using the corresponding version .Note:
For SmartSearch appVersion, see UIM Compatibility Matrix.CLIENT_SECRET,CLIENT_ID,SCOPE, andIDP_TOKEN_URL: These are necessary for generating the access token for the SmartSearch service.- Ensure
SMARTSEARCHDOMAINandIDP_TOKEN_URLare accessible and included in theNO_PROXYsettings or that the proxy is disabled. - Ensure the relevant domain entries are present in the /etc/hosts file.
For example:
<instance>.<project>.topology.<hostSuffix>
Creating Index and Metadata
To create index and metadata:
- Navigate to the SmartSearch
directory:
cd $COMMON_CNTK/scripts/smartsearch - Run the script to create indexes and metadata for
NPD:
./refreshIndexes.sh --ssl-cert-smartsearch <path-to-cert> --ssl-cert-oauth <path-to-cert> - Run the following command to create indexes and metadata for Service
Impact Analysis related
indexes:
./indexConfig.sh --index smartsearch-event smartsearch-impact smartsearch-rejectedevent --action createIndex --ssl-cert-smartsearch <path-to-cert> --ssl-cert-oauth <path-to-cert> ./indexConfig.sh --index smartsearch-event smartsearch-impact smartsearch-rejectedevent --action updateMetadata --ssl-cert-smartsearch <path-to-cert> --ssl-cert-oauth <path-to-cert> - (Optional) For non-secure deployments, use the
--security-enablefalse flag.
Validating the Indexes before Creating
To validate the indexes before creating:
- Access the OpenSearch dashboard and navigate to DevTools.
- Run the following command to verify if the required indexes are
created:
GET /_cat/indices - Check if the following indexes with any suffixes are present:
smartsearch-location-{*}smartsearch-communication-{*}smartsearch-physicaldevice-{*}smartsearch-logicaldevice-{*}smartsearch-equipment-{*}smartsearch-event-{*}smartsearch-impact-{*}smartsearch-rejectedevent-{*}
Note:
If any of these indexes are missing, rerun the script. - If any of these indexes are already present, skip running the creation
scripts for such indexes.
Note:
In case you already have thesmartsearch-event-{*} index. Please check if you have any data located in that index by running following query in the OpenSearch DevTools:GET /smartsearch-event/_count - If the count returned is 0, do not include the existing index by running the
following in the $COMMON_CNTK/scripts/smartsearch directory. For non-secure
deployments, use the
--security-enable falseflag../indexConfig.sh --index smartsearch-event --action deleteIndex --ssl-cert-smartsearch <path-to-cert> --ssl-cert-oauth <path-to-cert>
Validating the Indexes after Creating:
- Access the OpenSearch dashboard and navigate to DevTools.
- Run the following command to verify if the required indexes are
created:
GET /_cat/indices - Check if the following indexes with any suffixes are present:
smartsearch-event-{*}smartsearch-impact-{*}smartsearch-rejectedevent-{*}
Note:
If any of these indexes are missing, rerun the script.
Creating Index and Metadata Required for Service Impact Analysis
Event and Impact Analysis service uses following indexes:
-
smartsearch-event -
smartsearch-impact -
smartsearch-rejectedevent
Validating the Indexes before Creating the Index and Metadata
To validate the indexes:
- Access the OpenSearch dashboard and navigate to DevTools.
- Run the following command to verify if the required indexes are already
present:
GET /_cat/indices - Check if the following indexes with any suffixes are present:
-
smartsearch-event-{*} -
smartsearch-impact-{*} -
smartsearch-rejectedevent-{*}
-
- If any of the indexes exists, skip running the create scripts for those indexes.
Note:
Existing Event Index Action:
- In case you already have the
smartsearch-event-{*}index. Check if you have any data located in that index by running the following command in the OpenSearch DevTools:GET /smartsearch-event/_count - If the count returned is 0, you can remove this existing
index by running the following command in
$COMMON_CNTK/scripts/smartsearch. For non-secure
deployments, use the
--security-enablefalse flag../indexConfig.sh --index smartsearch-event --action deleteIndex --ssl-cert-smartsearch <path-to-cert> --ssl-cert-oauth <path-to-cert> - If the count is not 0, do not proceed with the index creation and contact Oracle Support.
- In case you already have the
Upgrading the SmartSearch Service
Upgrading SmartSearch is required when there are updates made to applications-base.yaml, app-smartsearch.yaml, or other configuration files.
To upgrade the SmartSearch instance:
- Run the following command to upgrade the SmartSearch
service:
$COMMON_CNTK/scripts/upgrade-applications.sh -p sr -i quick -s $SPEC_PATH -a smartsearch
- After script is run, validate the SmartSearch service by running the
application-statusscript. - If there are any changes in Ingress configurations, loadbalancer port, annotation,
and so on, delete and Create SmartSearch Ingress as
follows:
$COMMON_CNTK/scripts/delete-ingress.sh -p sr -i quick -s $SPEC_PATH -a smartsearch $COMMON_CNTK/scripts/create-ingress.sh -p sr -i quick -s $SPEC_PATH -a smartsearch
Validating the SmartSearch Instance
To validate the SmartSearch instance, run the following to the check the status of the deployed SmartSearch instance:
$COMMON_CNTK/scripts/application-status.sh -p sr -i quick -s $SPEC_PATH -a smartsearchThis script returns the status of the SmartSearch service deployments and pods status.
Restarting the SmartSearch Instance
To restart the SmartSearch instance:
- Run the following command to restart the SmartSearch
service:
$COMMON_CNTK/scripts/restart-applications.sh -p sr -i quick -s $SPEC_PATH -a smartsearch
- After running the script, validate the SmartSearch service by running
the
application-statusscript.
Deleting the SmartSearch Service
To delete the SmartSearch instance, run the following command to delete the SmartSearch service instance:
$COMMON_CNTK/scripts/delete-applications.sh -p <project-name> -i <instance-name> -s $SPEC_PATH -a smartsearchAlternate Configuration Options for SmartSearch
You can configure SmartSearch using the following alternate options.
Setting Up a Secure Communication using TLS
To set up a secure communication using TLS:
- Edit the
$SPEC_PATH/$PROJECT/$INSTANCE/applications-base.yaml and set the TLS
enabledtotrue.tls: # The enabled flag is to enable or disable the TLS support for the smart search m-s end points enabled: true - Specify the SSL port of loadbalancer or Ingress Controller in
applications-base.yaml as follows. You use this port to access the
application from outside:
loadbalancerport: 30543 -
Generate common certificates commoncert.pem and commonkey.pem. Refer "SSL Certificates" in Appendix.
-
Create IngressTLS secret to pass the generated certificate and key pem files:
$COMMON_CNTK/scripts/manage-app-credentials.sh -p sr -i quick -s $SPEC_PATH -a smartsearch create ingressTLSThe script prompts for the following detail:
- Ingress TLS Certificate Path (PEM file): <path_to_cert.pem>
- Ingress TLS Key file Path (PEM file): <path_to_key.pem>
- Verify that the following secrets are created
successfully:
sr-quick-smart-search-ingress-tls-cert-secret - Create a SmartSearch Instance and Ingress as usual and access the
SmartSearch endpoints using
hostname <instance>.<instance>.topology.uim.orgas follows:#If instance already running delete and create Ingress. #instance $COMMON_CNTK/scripts/upgrade-applications.sh -p sr -i quick -s $SPEC_PATH -a smartsearch $COMMON_CNTK/scripts/delete-ingress.sh -p sr -i quick -s $SPEC_PATH -a smartsearch $COMMON_CNTK/scripts/create-ingress.sh -p sr -i quick -s $SPEC_PATH -a smartsearch - Add entry in
/etc/hosts <k8s cluster ip or external loadbalancer ip> quick.sr.topology.uim.org.The SmartSearch API endpoint is
https://quick.sr.topology.uim.org:<LB Port>/<appVersion>/<uri-endpoint>
Enabling Authentication on SmartSearch
To enable Authentication on SmartSearch, create the OATUH 2.0 client on your IDP and use the same client with OpenSearch, ATA, Authorization and Message Bus Services. After creating the client, enable authentication on SmartSearch service as follows:
- Create
oauthConfigsecret with IDP OAUTH 2.0 client details. See "Adding Common OAuth Secret and ConfigMap" for information on creatingOauthConfigsecret. - Create
commonTrustsecret with trustStore file that contains IDP and OpenSearch SSL certificate. See "Common TrustStore Secret" for more information. - Set the authentication.enabled flag to
truein $SPEC_PATH/$PROJECT/$INSATNCE/applications-base.yaml to enable authentication as follows:# The enabled flag is to enable or disable authentication authentication: enabled: true - Upgrade the SmartSearch service.
Supporting Wildcard Certificates
SmartSearch supports wildcard certificates. You can generate the
wildcard certificates with the hostSuffix value
provided in applications-base.yaml. The default is uim.org.
Change the subDomainNameSeperator value from period(.) to
hyphen(-) so that incoming hostnames match the wildcard DNS pattern.
Make the following modifications to the $SPEC_PATH/$PROJECT/$INSTANCE/applications-base.yaml file.
Uncomment and provide the value of subDomainNameSeparator. The default is ".".
#Value can be changed as "-" to match wild-card pattern of ssl certificates.
#Example hostnames for "-" quick-sr-topology.uim.org
subDomainNameSeparator: "-"
Configure Logging for SmartSearch
To update log level for SmartSearch:
- Update the SmartSearch.env.LOGLEVEL value with the required Log level (ERROR, INFO, DEBUG, TRACE) in $SPEC_PATH/sr/quick/app-smartsearch.yaml
- Upgrade the SmartSearch instance as
follows:
$COMMON_CNTK/scripts/upgrade-applications.sh -p sr -i quick -s $SPEC_PATH -a smartsearch
Configuring Shape for SmartSearch
There are predefined shapes available: devsmall, dev, prodsmall, prod, and prodlarge. Review the corresponding configurations at location $SPEC_PATH/$PROJECT/$INSTANCE/shapes/. To use a specific shape, specify its name in the applications-base.yaml file.
You can also create and customize your own shapes. See "Customizing the Shapes" for more information. If you want to use a different shape for SmartSearch than for other services, specify the shape value in the app-smartsearch.yaml file.
Managing SmartSearch SSL Certificates
To renew SmartSearch ingress SSL certificates or to import any new Egress certificate in SmartSearch truststore, see "Managing Certificate Expiry".
Scaling Up or Down the SmartSearch Service
To scale up or down the SmartSearch service:
-
Provide the replica count in shapes/<shape>/smartsearch.yaml to scale up or down the SmartSearch pods. Make sure the same shape value is configured in applications-base.yaml or app-smartsearch.yaml in which you update the replicas.
-
Update shapes/<shape>/smartsearch.yaml to increase replica count to 3 for SmartSearch deployment as follows:
smartSearch: replicas: 3 - Apply the change in replica count to the running Helm release by running
the upgrade-applications script as
follows:
$COMMON_CNTK/scripts/upgrade-applications.sh -p sr -i quick -s $SPEC_PATH -a smartsearch