Default Security Module

Ocivault Security Module

The ocivault security module type uses a key stored in an OCI vault. Authentication for calls to the OCI APIs uses either platform credentials or credentials configured with external properties. Stored credentials cannot be used here because the module can be used to decrypt passwords for the EDQ database schemas at startup time.

The following additional properties are used with the type in module.properties:

The primary use case for the OCI module is with compute instances which can use platform credentials for authentication for OCI API calls. If the OCI dynamic groups imply the required permissions, no additional configuration is required.

Secrets Storage

If the module is enabled for secrets use, credentials for external integrations can be stored in a vault rather than in the EDQ configuration files. The examples include JMS broker connections, SMTP authentication, and LDAP credentials.

To specify credentials using a secret, replace the username and password properties with the following:

cred.secret = secretname

The secret vault in the value should be a JSON string containing username and password attributes, as shown below:

{ "username": "user1", "password": "mysecret2" }

To override the default secret encryption setting for the module, use the following:

cred.secret.encrypt = true | false

To specify the username in plain and retrieve the password from the vault, use the following:

cred.secret.username = username

The secret value in the vault is then just the password.

If no username is required, add the following:

cred.secret.passwordonly = true

The secret value in the vault is then just the password.

Sample OCI Vault module.properties Configurations

Using platform credentials with local compartment and region:

type            = ocivault
vault           = edqvault
key             = key1
secrets         = true
secrets.encrypt = true

Using a remote vault with credentials from ~/.oci/config:

type            = ocivault
auth.profile    = default
region          = us-phoenix-1
compartmentid   = compartmentid
vault           = edqvault
key             = key1
secrets         = true

Script Security Module

The module definition for the script security module must include a script property which is the name of the script file. If the file is not absolute it is found relative to the EDQ local configuration directory. The script defines functions as described in the following table:

The properties passed to the init function include a type value set to "encryption". This allows the same script to be used as the security module and a credentials store.

base64 stuff

External encryption APIs require base64 encoding for plain and cipher text. EDQ scripts can use these base64 support functions:

var encoder = BASE64.getEncoder()
var enc     = encoder.encode(bytes)                 // Encode an ArrayBuffer to base64 string
var enc2    = encoder.encodeFromString(str)         // Encode a string to base 64 string
 
var urlenc  = BASE64.getUrlEncoder()                // Encoder using URL-safe alphabet
var mimeenc = BASE64.getMimeEncoder()               // Encoder with MIME line splitting
vat nopad   = BASE64.getEncoder().withoutPadding()  // Encoder which does not use = padding
 
var decoder = BASE64.getDecoder()
var dec     = decoder.decode(string)                // Decode a base64 string to ArrayBuffer
var dec2    = encoder.decodeToString(str)           // Decode a base 64 string to a string
 
var urldec  = BASE64.getUrlEncoder()                // Decoder using URL-safe alphabet
var mimdec  = BASE64.getMimeEncoder()               // Decoder with MIME line splitting

Sample Scripts

Using the AWS Key and Secrets Management Frameworks

module.properties

type         = script
keyid        = alias/rde1
region       = eu-west-1
script       = awsenc.js
secrets      = true
auth.profile = myprofile

awsenc.js

// Script security module sample for AWS KMS and secrets manager
// =============================================================
//
// Required properties:
//
//      region          AWS region
//      keyid           KMS keyid or alias/name
//
// Optional:
//
//      secrets         Set to true for secrets support
//
// Configure proxy server with https.proxyHost and https.proxyPort properties
 
addLibrary("http")
addLibrary("logging")
 
var client
var secclient
var encoder = BASE64.getEncoder()
var decoder = BASE64.getDecoder()
var kmsurl
var keyid
var secrets   = false
var secretenc = false
 
function init(props) {
 
  // Authentication properties
 
  var aprops = Object.keys(props).filter(k => k.startsWith("auth.")).reduce((o, k) => (o[k.substring(5)] = props[k], o), {})
 
  // Client for KMS
 
  client = HTTP.builder().withAWSAuthentication(aprops).build();
 
  // Properties
 
  var region   = props.region
 
  keyid = props.keyid
 
  if (!region || !keyid) {
    throw "awsenc: region or key not specified"
  }
 
  kmsurl  = "https://kms." + region + ".amazonaws.com"
 
  // Secrets handling
 
  secrets = props.secrets == "true"
   
  if (secrets) {
 
    // URL and client for secret queries
 
    securl    = "https://secretsmanager." + region + ".amazonaws.com"
    secclient = HTTP.builder().withAWSAuthentication(aprops).build();
  }
}
 
function encrypt(plain) {
  var req = client.requestbuilder().header("X-Amz-Target", "TrentService.Encrypt").build();
  var res = req.post(kmsurl, JSON.stringify({KeyId: keyid, Plaintext: encoder.encode(plain)}), "application/x-amz-json-1.1")
 
  return decoder.decode(JSON.parse(res.data).CiphertextBlob)
}
 
function decrypt(ctext) {
  var req = client.requestbuilder().header("X-Amz-Target", "TrentService.Decrypt").build();
  var res  = req.post(kmsurl, JSON.stringify({keyId: keyid, CiphertextBlob: encoder.encode(ctext)}), "application/x-amz-json-1.1")
 
  return decoder.decode(JSON.parse(res.data).Plaintext)
}
 
function getCredentials(props) {
  if (secrets) {
    var s = props.secret
 
    if (s) {
      var req = secclient.requestbuilder().failonerror(false).header("X-Amz-Target", "secretsmanager.GetSecretValue").build()
      var res = req.post(securl, JSON.stringify({SecretId: s}), "application/x-amz-json-1.1")
 
      if (res.code != 200) {
        logger.log(Level.WARNING, "AWS secrets manager call failed with code {0}", res.code)
      } else {
        var obj = JSON.parse(res.data)
        var str = obj.SecretString || decoder.decodeToString(obj.SecretBinary)
         
        if (props["secret.passwordonly"] == "true") {
          return {username: "", password: str}
        } else {
          var val = JSON.parse(str)
          return {username: val.username, password: val.password}
        }
      }
    }
  }
 
  return null
}

Using GCP Key and Secrets Frameworks

module.properties

type       = script
script     = gcpenc.js
project    = green-wombat-4252311
keyfile    = /opt/edq/gcp-encrypt.json
location   = europe-west2
keyring    = rde
key        = rde1
secrets    = true
seckeyfile = /opt/edq/gcp-secrets.json

gcpenc.js

// Script security module sample for GCP KMS and secrets manager
// =============================================================
//
// Required properties:
//
//      project         GCP project name
//      location        Key location (such as europe-west2)
//      keyring         Keyring name
//      key             Key name
//      keyfile         Service account key file location
//
// Optional:
//
//      version         Key version name
//
// Configure proxy server with https.proxyHost and https.proxyPort properties
 
addLibrary("http")
addLibrary("logging")
 
var client
var encoder = BASE64.getEncoder()
var decoder = BASE64.getDecoder()
var decurl
var encurl
var secrets   = false
 
function init(props) {
 
  // Need:
  //
  // project
  // location
  // keyring
  // key
  // keyfile
  //
  // Optional:
  //
  // version
 
  if (!props.project || !props.location || !props.keyring || !props.key || !props.keyfile) {
     throw "gcpenc: missing properties"
  }
 
  var project = props.project
  var location = props.location
 
  var base = "https://cloudkms.googleapis.com/v1"
        + "/projects/"   + props.project
        + "/locations/"  + props.location
        + "/keyRings/"   + props.keyring
        + "/cryptoKeys/" + props.key
 
  decurl = base + ":decrypt"
  encurl = base
 
  if (props.version) {
    encurl += "/cryptoKeyVersions/" + props.version
  }
 
  encurl += ":encrypt"
 
  // Client for KMS
 
  client = HTTP.builder().withAuthentication("GCP", {keyfile: props.keyfile, claim_scope: "https://www.googleapis.com/auth/cloud-platform"}).build();
 
  secrets = props.secrets == "true"
   
  if (secrets) {
 
    // URL and client for secret queries
 
    securl = "https://secretmanager.googleapis.com/v1/projects/" + props.project + "/secrets/"
  }
}
 
function encrypt(plain) {
  var req = client.requestbuilder().build();
  var res = req.post(encurl, JSON.stringify({"plaintext": encoder.encode(plain)}))
   
  return decoder.decode(JSON.parse(res.data).ciphertext)
}
 
function decrypt(ctext) {
  var req = client.requestbuilder().build();
  var res  = req.post(decurl, JSON.stringify({ciphertext: encoder.encode(ctext)}))
 
  return decoder.decode(JSON.parse(res.data).plaintext)
}
 
function getCredentials(props) {
  if (secrets) {
    var s = props.secret
 
    if (s) {
      var url = securl + s + "/versions/latest:access"
      var req = client.requestbuilder().failonerror(false).build()
      var res = req.get(url)
 
      if (res.code != 200) {
        logger.log(Level.WARNING, "GCP secrets manager call failed with code {0}", res.code)
      } else {
        var obj = JSON.parse(res.data)
        var str = decoder.decodeToString(obj.payload.data)
         
        if (props["secret.passwordonly"] == "true") {
          return {username: "", password: str}
        } else {
          var val = JSON.parse(str)
          return {username: val.username, password: val.password}
        }
      }
    }
  }
 
  return null
}

Encryption Migration

If the security module is replaced or reconfigured in an existing system, then encrypted data is no longer usable. If the system is a new install and encryption is not used in the director.properties, then this is not an issue. If there is an existing encrypted data, the new security module can be defined using a migration REST API.

There are two new system administration REST endpoints.

POST to /edq/admin/security/migrateencryption

The payload is a JSON object with the following structure:

{ "type" : "moduletype",
  "properties": {
 	... other settings for the module ...
  }
}

The result of the migration call is a report summarizing the items which were updated. Any items which could not be decrypted are listed.

If the URL contains the query parameter?dryrun=true then the new module is created and decryption of existing data tested, but no updates are made.

Example Payload for Migration to a Remote OCI Vault

{ "type" : "ocivault",
  "properties": {
    "auth.profile"    : "default",
    "vault"           : "rde",
    "key"             : "rtest",
    "compartmentid"   : "compartmentid",
    "region"          : "us-phoenix-1",
    "secrets"         : true,
    "secrets.encrypt" : true
  }
}

POST to /etc/admin/security/rotateencryption

The payload is an empty JSON object. Existing encrypted data is decrypted using the current security module and then encrypted using the same module. This call can be used with a vault key after a new version is created so that the old version can be deleted.

The result of the call is a summary for the migrate call.

If the URL contains the query parameter ?dryrun=true, then decryption of existing data is tested but no updates are made.

Using Existing Schemas for New OCI Instance Without Losing Encrypted Data

If you create a new OCI instance but select the Use Existing Schemas option, the new instance starts with the legacy (or default) security module and creates a new secret key, rendering the existing encrypted data inaccessible. These are the steps to retain the data:

1. Before shutting down the old system, ensure it is using some external encryption mechanism (OCI vaults or AWS).
2. Create and setup the new system.
3. Run encryption migration to setup the external module on the new system. Decryption fails for existing data because the current module is different but after migration is complete. The encrypted data is usable again.

Configuring Additional Credential Stores

EDQ 12.2.1.4.4 and later versions extend the encryption module to allow additional credentials stores to be defined. The stores are configured by adding properties files to the localhome/security/credstores directory. Each properties file must contain a type property defining the store type along with additional settings which are specific to the store type. The following types are supported:

• ocivault - The credentials are stored as secrets in an OCI vault and can be encrypted using a value key.
• script - A JavaScript script can be configured to call out to external services such as AWS and GCP Secrets Managers.
• custom - A custom Java class.

Credentials stored are queried in the alphabetical order of the associated file names.

Ocivault Credentials Store

The ocivault security module type queries secrets stored in an OCI vault. Authentication for calls to the OCI APIs uses either platform credentials or credentials configured with external properties. Stored credentials cannot be used here because the module can be used to query passwords for the EDQ database schemas at startup time.

The following additional properties are used with the type.

The primary use case for the OCI store is with compute instances which can use platform credentials for authentication for OCI API calls. As long as the OCI dynamic groups imply the require permissions no additional configuration is required.

Using Secrets

To specify credentials using a vault secret, replace the username and password properties with the following:

cred.secret = secretname

The secret vault in the value should be a JSON string containing username and passwordattributes as shown below:

{ "username": "user1", "password": "mysecret2" }

To override the default secret encryption setting for the module, use the following:

cred.secret.encrypt = true | false

To specify the username in plain and retrieve the password from the vault, use the following:

cred.secret.username = username

The secret value in the vault is then just the password.

If no username is required, add the following:

cred.secret.passwordonly = true

The secret value in the vault is then just the password.

Example

oci.properties

type            = ocivault
secrets.encrypt = true
vault           = vault1
key             = rtest
compartmentid   = ocid1.compartment.oc1..aaaaaaaaeq2s...
region          = us-phoenix-1

Script Credentials Store

The properties for a script credentials store must include a script property which is the name of the script file. If the file is not absolute, it is found relative to the EDQ local configuration directory. The script defines the following functions:

The properties passed to the init function include a type value set to "credentials". This allows the same script to be used as the security module and a credentials store.

Example Using the AWS Secrets Management Framework

For more information about base64 support functions, see EDQ Enhanced Encryption Overview.

aws.properties

type         = script
region       = eu-west-1
script       = awscreds.js
auth.profile = myprofile

awscreds.js

// Script credentials module sample for AWS secrets manager
// =========================================================
//
// Required properties:
//
//      region          AWS region
//
// Configure proxy server with https.proxyHost and https.proxyPort properties
 
addLibrary("http")
addLibrary("logging")
 
var secclient
var encoder = BASE64.getEncoder()
var decoder = BASE64.getDecoder()
 
function init(props) {
 
  // Authentication properties
 
  var aprops = Object.keys(props).filter(k => k.startsWith("auth.")).reduce((o, k) => (o[k.substring(5)] = props[k], o), {})
 
  // Properties
 
  var region   = props.region
 
  if (!region) {
    throw "awscreds: region not specified"
  }
 
  // URL and client for secret queries
 
  securl    = "https://secretsmanager." + region + ".amazonaws.com"
  secclient = HTTP.builder().withAWSAuthentication(aprops).build();
}
 
function getCredentials(props) {
  var s = props.secret
 
  if (s) {
    var req = secclient.requestbuilder().failonerror(false).header("X-Amz-Target", "secretsmanager.GetSecretValue").build()
    var res = req.post(securl, JSON.stringify({SecretId: s}), "application/x-amz-json-1.1")
 
    // Error 400 can mean secret not found, so don't report it
 
    if (res.code != 200) {
      if (res.code != 400) {
        logger.log(Level.WARNING, "AWS secrets manager call failed with code {0}", res.code)
      }
    } else {
      var obj = JSON.parse(res.data)
      var str = obj.SecretString || decoder.decodeToString(obj.SecretBinary)
 
      if (props["secret.passwordonly"] == "true" || props["secret.username"]) {
        return {username: props["secret.username"] || "", password: str}
      } else {
        var val = JSON.parse(str)
        return {username: val.username, password: val.password}
      }
    }
  }
 
  return null
}

type  = custom
class = java class name

Custom Encryption Module

A custom encryption module must implement the interface oracle.edq.security.module.custom.interfaces.CustomEncryptionModule as shown below:

CustomEncryptionModule

/*
 * Copyright (C) 2023, Oracle and/or its affiliates. All rights reserved.
 */
package oracle.edq.security.module.custom.interfaces;
 
import java.nio.file.Path;
import java.util.Properties;
 
/**
 * Interface implemented by custom encryption modules.
 */
 
public interface CustomEncryptionModule {
 
  /**
   * Initialize the encryption module.
   *
   * @param localconfig The local configuration area.  This can be used to locate or store extra files
   * required by the module.
   * @param props The module properties
   * @param persist If <code>true</code> the module data can be persisted for future use.
   * This flag will be <code>false</code> if the module is being initalized for a "dryrun" encryption migration.
   * In this case no permanent changes should be made in the file system.
   *
   * @throws Exception If initialization failed.
   */
 
  void initEncryption(Path localconfig, Properties props, boolean persist) throws Exception;
 
  /**
   * Encrypt some data.
   *
   * @param in The plain text
   *
   * @return The cipher text
   *
   * @throws Exception If encryption failed
   */
 
  byte [] encrypt(byte [] in) throws Exception;
 
  /**
   * Decrypt some data.
   *
   * @param in The cipher text
   *
   * @return The plain text
   *
   * @throws Exception If decryption failed
   */
 
  byte [] decrypt(byte [] in) throws Exception;
}

If the encryption module implements oracle.edq.security.module.custom.interfaces.CustomCredentialsModule also then it acts as a credentials store. In this case the initCredentials method is not called.

Custom Credentials Store

A custom encryption module must implement the interface oracle.edq.security.module.custom.interfaces.CustomCredentialsModule as shown below:

CustomCredentialsModule

/*
 * Copyright (C) 2023, Oracle and/or its affiliates. All rights reserved.
 */
package oracle.edq.security.module.custom.interfaces;
 
import java.nio.file.Path;
import java.util.Properties;
 
/**
 * Interface implemented by custom credentials modules.
 */
 
public interface CustomCredentialsModule {
 
  /**
   * Get the prefix used to extract properties recognized by the module.  The default value is "cred".
   *
   * @return The prefix
   */
 
  default String credsPrefix() {
    return "cred";
  }
 
  /**
   * Initialize the credentials provider for credentials-only use.
   *
   * @param localconfig The local configuration area
   * @param props The module properties
   *
   * @throws Exception If initialization failed.
   */
 
  void initCredentials(Path localconfig, Properties props) throws Exception;
 
  /**
   * Attempt to get credentials.
   *
   * @param props The filtered property set.
   *
   * @return The credentials, or <code>null</code> if not found
   *
   * @throws Exception If an error ocurred
   */
 
  Credentials getCredentials(Properties props) throws Exception;
 
  /**
   * Credentials result class.
   */
 
  final class Credentials {
 
    private String  username;
    private String  password;
 
    /**
     * Constructor.
     *
     * @param username The user name
     * @param password The password
     */
 
    public Credentials(String username, String password) {
      this.username = username;
      this.password = password;
    }
 
    /**
     * Get the user name.
     *
     * @return The user name
     */
 
    public String getUsername() {
      return username;
    }
 
    /**
     * Get the password.
     *
     * @return The password
     */
 
    public String getPassword() {
      return password;
    }
  }
}