6 Deploying SmartSearch
This chapter describes how to deploy the SmartSearch service.
About SmartSearch
SmartSearch is a micronaut application, when integrated with OpenSearch, offers a powerful, flexible, and feature-rich search experience that can be tailored to specific business and user needs. Using OpenSearch as the underlying engine, SmartSearch can handle large volumes of data, perform real-time indexing, and support complex querying to enhance search relevancy. Features such as autocomplete, fuzzy matching, synonym recognition, and intelligent ranking make it easier for users to locate precise information, even if search terms are partially matched or misspelled.
Creating a SmartSearch Instance
This section describes how to create a SmartSearch service instance in your cloud native environment using the operational scripts and the configuration provided in the Common cloud native toolkit (Common CNTK).
Before you create a SmartSearch instance, you must validate cloud native environment. See "Planning UIM Installation" for details on prerequisites.
In this section, while creating a basic instance, the project name is considered as
sr
and instance name is considered as
quick
.
Note:
The project and instance names cannot contain any special characters.
Creating SmartSearch Schema
The SmartSearch schema in OpenSearch is structured to optimize search performance and relevance, using six main indexes: Location, Logical Device, Physical Device, Equipment, Communication, and Events. Each index is defined by mappings that specify field types such as text for full-text search and keyword for exact matches and may include custom analyzers for handling text processing tasks such as tokenization and synonyms. Additionally, metadata is configured for each index to control search behavior, supporting features such as filters, sorting, and autocomplete to enhance user experience. All indexes are accessed using aliases, providing a flexible way to query and manage data. This setup ensures fast, accurate, and relevant search results tailored to user needs.
Prerequisites
Ensure the following environment variables are set:
SMARTSEARCHDOMAIN
: Set this to the SmartSearch service endpoint (format:export SMARTSEARCHDOMAIN=<host:port/appVersion>
), using the corresponding version .Note:
For SmartSearch appVersion, see UIM Compatibility Matrix.CLIENT_SECRET
,CLIENT_ID
,SCOPE
, andIDP_TOKEN_URL
: These are necessary for generating the access token for the SmartSearch service.- Ensure
SMARTSEARCHDOMAIN
andIDP_TOKEN_URL
are accessible and included in theNO_PROXY
settings or that the proxy is disabled. - Ensure the relevant domain entries are present in the /etc/hosts file.
For example:
<instance>.<project>.topology.<hostSuffix>
Creating Index and Metadata
To create index and metadata:
- Navigate to the smartsearch
directory:
cd $COMMON_CNTK/scripts/smartsearch
- Run the script to create indexes and metadata for
NPD:
./refreshIndexes.sh --ssl-cert-smartsearch <path-to-cert> --ssl-cert-oauth <path-to-cert>
- (Optional) For non-secure deployments, use the
--security-enable
false flag.
Validating the Indexes
To validate the indexes:
- Access the OpenSearch dashboard and navigate to DevTools.
- Run the following command to verify if the required indexes are
created:
GET /_cat/indices
- Check if the following indexes with any suffixes are present:
smartsearch-location-{*}
smartsearch-communication-{*}
smartsearch-physicaldevice-{*}
smartsearch-logicaldevice-{*}
smartsearch-equipment-{*}
smartsearch-event-{*}
Note:
If any of these indexes are missing, rerun the script.
Accessing the SmartSearch Service
To access SmartSearch service, use the SmartSearch endpoint in the format:
http://<topology-instance>.<topology-project>.topology.<hostSuffix>:<port>/<appVersion>/<uri-endpoint>
For example: SmartSearch API endpoint is http://quick.sr.topology.uim.org:30309/20240801/health
Note:
Make sure you add entry in /etc/hosts: <k8s cluster ip
or external loadbalancer ip>
quick.sr.topology.uim.org
.
For SmartSearch appVersion, see UIM Compatibility Matrix.
Validating the SmartSearch Instance
To validate the SmartSearch instance, run the following to the check the status of the deployed SmartSearch instance:
$COMMON_CNTK/scripts/application-status.sh -p sr -i quick -f $SPEC_PATH/sr/quick/applications.yaml -a smartsearch
application-status
This script returns the status of the SmartSearch service deployments and pods status.
Monitoring the SmartSearch Health
Use the following command to monitor the SmartSearch endpoint health:
https://<loadbalancerhost>:<loadbalancerport>/<appVersion>/health
Configure Logging for SmartSearch
To update log level for SmartSearch, check the following steps:
- Update SmartSearch.env.LOGLEVEL value with the appropriate Log level (ERROR, INFO, DEBUG, TRACE ) in $SPEC_PATH/sr/quick/applications.yaml.
- Upgrade SmartSearch instance as
follows:
$COMMON_CNTK/scripts/upgrade-applications.sh -p sr -i quick -f $SPEC_PATH/sr/quick/applications.yaml -a smartsearch
Managing SmartSearch SSL Certificates
To renew SmartSearch ingress SSL certificates or to import any new Egress certificate in SmartSearch truststore, see "Managing Certificate Expiry".
Setting up Environment Variables
The SmartSearch service relies on access to certain environment variables to run seamlessly. Ensure the following variables are set in your environment:
- Path to your common cloud native toolkit
- Traefik namespace (Optional)
To set the environment variables:
- Set the COMMON_CNTK variable to the path of directory where common cloud native
toolkit is extracted as
follows:
$ export COMMON_CNTK=$WORKSPACEDIR/common-cntk
- In case Traefik is used as ingress, set the
TRAEFIK_NS
variable for Traefik namespace as follows:$ export TRAEFIK_NS=Treafik Namespace
- Set the variable for Traefik Helm chart version. For the corresponding version, see
UIM Compatibility Matrix. The following is a sample for
Traefik chart version 15.1.0.
$ export TRAEFIK_CHART_VERSION=15.1.0
- Set
SPEC_PATH
variable to the location where application and database yamls are copied. See "Assembling the Specifications" to copy specification files if not already copied.$ export SPEC_PATH=$WORKSPACEDIR/spec_dir
Registering the Namespace
After you set the environment variables, register the namespace if Traefik is used as Ingress Controller.
To register the namespace, run the following command:
$COMMON_CNTK/scripts/register-namespace.sh -p sr -t targets
# For example, $COMMON_CNTK/scripts/register-namespace.sh -p sr -t traefik
# Where the targets are separated by a comma without extra spaces
Note:
traefik
is the name of the target for
registration of the namespace sr
. The script uses
TRAEFIK_NS to find these targets. Do not provide the Traefik target if you are not
using Traefik.
For Generic Ingress Controller, you do not have to register the namespace. To select the ingress controller, provide the ingressClassName value under the ingress.className field in the applications.yaml file. For more information about ingressClassName, see https://kubernetes.io/docs/concepts/services-networking/ingress/
Creating Secrets
To create secrets:
- Run the following command to generate the
openSearchCredentials
secret and enter the basic auth credentials of OpenSearch as prompted:$COMMON_CNTK/scripts/manage-app-credentials.sh -p sr -i quick -a smartsearch -f $SPEC_PATH/samples/applications.yaml create openSearchCredentials #Enter the values appropriately against prompts: opensearch username: admin #opensearch basic oauth credentials opensearch password: xxxxx
- Verify if the secrets are created as
follows:
sr-quick-open-search-credentials
Secret for Authentication Details
The SmartSearch service uses configuration values from Kubernetes secret
(<namespace>-<instance>-oauth-credentials
) and Config Map
(<namespace>-<instance>-oauth-config-cm
) objects from the
same namespace. This Secret and Configuration Map Kubernetes objects have to be
created before deploying the SmartSearch service for authentication. See "Adding Common OAuth Secret and ConfigMap" for creating the secret.
Secret to Pass Egress Certificates to SmartSearch
You have to create a commonTrust
secret to provide SSL
certificates of IdP, OpenSearch, and so on to SmartSearch.
To create commonTrust
secret:
- Generate truststore.jks that contains the required certificates as
follows:
#generate truststore with idp cert (applicable only if authentication is enabled ) keytool -importcert -v -alias <param> -file <path to IDP cert file> -keystore <truststorename>.jks -storepass <password> #add opensearch ssl trust certificate in truststore keytool -importcert -v -alias <param> -file <path to opensearch cert file> -keystore <truststorename>.jks -storepass <password>
- Run the below command to generate the Kubernetes secret. The system will prompt
for the truststore file path and password of truststore. Provide the details
accordingly.
$COMMON_CNTK/scripts/manage-app-credentials.sh -p project -i instance -f $SPEC_PATH/project/instance/applications.yaml create commonTrust
Configuring applications.yaml
In $SPEC_PATH/sr/quick/applications.yml, the SmartSearch image and tag are
configured to specify the container image used for the SmartSearch service. The
image
tag can be updated as new version of the image are
released, allowing easy upgrades and ensuring the application runs the latest features
and security patches. To apply a new image version, simply change the tag in
applications.yml to the desired version, facilitating version control and
deployment management.
To configure the applications.yaml file:
- Download the Oracle Communications Unified Inventory Management SmartSearch image from Oracle Software Delivery Cloud.
- Load the image from tar file and push the image to container
repository. Refer "About Container Image Management" in UIM Cloud Native
Deploy
Guide.
podman load -I <smart-search-latest.tar> podman tag smart-search:latest <conatiner repository>/smart-search:latest podman push <conatiner repository>/smart-search:latest
- Update the corresponding values under the smartsearch section in $SPEC_PATH/<PROJECT>/<INSTANCE>/applications.yaml.
- Edit the applications.yaml file and enter the name and tag of the SmartSearch
image from your repository as
follows:
smartSearch: image: name: <conatiner repository>/smart-search tag: 0.3.11
- You can provide an image pull secret as follows to pull images from your private
repository:
# The image pull access credentials for Private repository, as a Kubernetes secret. # uncomment and set if required. # imagePullSecret: # imagePullSecrets: # - name: regcred
- Set
pullPolicy
for SmartSearch images in applications.yaml as follows:smartSearch: #Provide imagePullPolicy i.e Always, IfNotPresent, Never imagePullPolicy: IfNotPresent
- Update
loadbalancerport
in applications.yaml as follows:# For Generic and Traefik Ingress Controllers: # If ssl is enabled this would be loadbalancer's ssl port. # IF ssl is disabled this would be loadblancer's non ssl port. # For examle ssl and non-ssl ports for external loadbalancer would be 443 and 80 respectively. # If loadbalancer is not created, provide nodePort of Nginx/Traefik or any other Generic Ingress Controller loadBalancerPort: 30505
- For Generic ingress controller update as follows:
- Update applications.yaml as follows to provide the following
annotations to enable stickiness through
cookies:
# Valid values are TRAEFIK, GENERIC ingressController: "GENERIC"
- Annotations for NGINX is as follows:
ingress: className: nginx ##provide ingressClassName value, default value for nginx ingressController is nginx. # This annotation is required for nginx ingress controller. annotations: nginx.ingress.kubernetes.io/affinity: "cookie" nginx.ingress.kubernetes.io/affinity-mode: "persistent" nginx.ingress.kubernetes.io/session-cookie-name: "nginxingresscookie" nginx.ingress.kubernetes.io/proxy-body-size: "50m" smartSearch: #uncomment and provide applications specific annotations if required, these will get added to list of annotations specified in common section. ingress: annotations: nginx.ingress.kubernetes.io/rewrite-target: /$2
- Update applications.yaml as follows to provide the following
annotations to enable stickiness through
cookies:
- Set the authentication.enabled flag to true to enable authentication as
follows:
# The enabled flag is to enable or disable authentication authentication: enabled: true
Creating a SmartSearch Instance
To create an SmartSearch service instance in your environment using the scripts that are provided with the toolkit:
- Run the following command to create an SmartSearch service
instance:
$COMMON_CNTK/scripts/create-applications.sh -p sr -i quick -f $SPEC_PATH/sr/quick/applications.yaml -a smartsearch
The
create-applications
script uses the Helm chart located in $COMMON_CNTK/charts/smartsearch-app to create and deploy a SmartSearch service. - (Optional) If the scripts fail, run the following command to review the error
message in the log:
kubectl logs -n sr sr-quick-smartsearch
- Clear the failed pod by running the following
command:
helm uninstall sr-quick-smartsearch-n sr
- Fix the issues if any and run the script again to deploy the SmartSearch service.
Upgrading the SmartSearch Service
Upgrading SmartSearch is required when there are updates made to applications.yaml or other configuration files.
To upgrade the SmartSearch instance:
- Run the following command to upgrade the SmartSearch
service:
$COMMON_CNTK/scripts/upgrade-applications.sh -p sr -i quick -f $SPEC_PATH/sr/quick/applications.yaml -a smartsearch
- After script is run, validate the SmartSearch service by running the
application-status
script.
Restarting the SmartSearch Instance
To restart the SmartSearch instance:
- Run the following command to restart the SmartSearch
service:
$COMMON_CNTK/scripts/restart-applications.sh -p sr -i quick -f $SPEC_PATH/sr/quick/applications.yaml -a smartsearch
- After running the script, validate the SmartSearch service by running
the
application-status
script.
Deleting the SmartSearch Service
To delete the SmartSearch instance, run the following command to delete the SmartSearch service instance:
$COMMON_CNTK/scripts/delete-applications.sh -p <project-name> -i <instance-name> $SPEC_PATH/sr/quick/applications.yaml -a smartsearch
Alternate Configuration Options for SmartSearch
You can configure SmartSearch using the following alternate options.
Setting Up a Secure Communication using TLS
To set up a secure communication using TLS:
- Edit the $COMMON_CNTK/samples/applications.yaml and set the TLS
enabled
totrue
.tls: # The enabled flag is to enable or disable the TLS support for the smart search m-s end points enabled: true
Smartsearch API endpoint: https://quick.sr.topology.uim.org:<LB Port>/<appVersion>/<uri-endpoint>
-
Generate common certificates commoncert.pem and commonkey.pem. Refer "SSL Certificates" in Appendix.
-
Create IngressTLS secret to pass the generated certificate and key pem files:
$COMMON_CNTK/scripts/manage-app-credentials.sh -p sr -i quick -f $SPEC_PATH/sr/quick/applications.yaml -a smartsearch create ingressTLS
The script prompts for the following detail:
- Ingress TLS Certificate Path (PEM file): <path_to_cert.pem>
- Ingress TLS Key file Path (PEM file): <path_to_key.pem>
- Verify that the following secrets are created
successfully:
sr-quick-smart-search-ingress-tls-cert-secret
- Create a SmartSearch Instance as usual and access the SmartSearch endpoints
using
hostname <instance>.<instance>.topology.uim.org
- Add entry in
/etc/hosts <k8s cluster ip or external loadbalancer ip> quick.sr.topology.uim.org
.The SmartSearch API endpoint is
https://quick.sr.topology.uim.org:<LB Port>/<appVersion>/<uri-endpoint>
Supporting Wildcard Certificates
SmartSearch supports wildcard certificates. You can generate the wildcard
certificates with the hostSuffix
value provided in
applications.yaml. The default is uim.org
.
Change the subDomainNameSeperator
value from period(.) to
hyphen(-) so that incoming hostnames match the wildcard DNS pattern.
Make the following modifications to the $SPEC_PATH/project/instance/applications.yaml file.
Uncomment and provide the value of subDomainNameSeparator. The default is ".".
#Value can be changed as "-" to match wild-card pattern of ssl certificates.
#Example hostnames for "-" quick-sr-topology.uim.org
subDomainNameSeparator: "-"
Scaling Up or Down the SmartSearch Service
To scale up or down the SmartSearch service:
-
Provide the replica count in applications.yaml to scale up or down the SmartSearch pods.
-
Update applications.yaml to increase replica count to 3 for topology-api deployment as follows:
smartSearch: replicas: 3
- Apply the change in replica count to the running Helm release by running the
upgrade-applications script as
follows:
$COMMON_CNTK/scripts/upgrade-applications.sh -p sr -i quick -f $SPEC_PATH/sr/quick/applications.yaml -a smartsearch