10.5.3 Amazon Kinesis
- About Amazon Kinesis Connections
- Before You Begin
- Create and Validate an Amazon Kinesis Connection
- Use an Amazon Kinesis Connection in Replicat
Parent topic: Streaming and Messaging
10.5.3.1 About Amazon Kinesis Connections
An Amazon Kinesis connection stores the information that Oracle GoldenGate for Distributed Applications and Analytics uses to connect to Amazon Kinesis Data Streams.
You can use the connection with Replicat to publish transactional change data to Amazon Kinesis Data Streams. After you create and validate the connection, you can reuse it across multiple Oracle GoldenGate for Distributed Applications and Analytics processes.
The connection stores the Amazon Kinesis endpoint information, authentication settings, security configuration, and optional runtime settings independently of the process configuration.
Parent topic: Amazon Kinesis
10.5.3.2 Before You Begin
Before creating a Kafka connection, ensure that you have the following:
- An Oracle GoldenGate for Distributed Applications and Analytics deployment.
- Network connectivity between the Oracle GoldenGate for Distributed Applications and Analytics deployment and the Kafka cluster.
- One or more Kafka bootstrap servers.
- The security protocol and authentication mechanism required by the Kafka cluster.
- Authentication credentials for the selected SASL mechanism, if applicable.
- Truststore and keystore files if the deployment uses SSL or SASL over SSL.
- Optional
producer.propertiesorconsumer.propertiesfiles if additional Kafka client configuration is required. - The required Apache Kafka client libraries.
To download the required dependencies, use the Oracle GoldenGate for Distributed Applications and Analytics Dependency Downloader:
{GGforDAA install}/DependencyDownloader/kafka.sh <version>After downloading the dependencies, configure the gg.classpath property
in the Java Adapter properties file to reference the downloaded libraries before
starting the Extract or Replicat process.
Proxy information if the Oracle GoldenGate for Distributed Applications and Analytics deployment accesses the Kafka cluster through an HTTP proxy.
Parent topic: Amazon Kinesis
10.5.3.3 Create and Validate an Amazon Kinesis Connection
Use the Add Connection page to create an Amazon Kinesis connection.
Procedure
- In the navigation menu, select DB Connections.
- Click Add Connection.
- In Connection Alias, enter a unique name for the connection.
- From Technology Type, select Amazon Kinesis.
- In the Technology Settings section,
provide the Amazon Kinesis connection information.
Configure the following technology settings:
- Access Key ID – Specifies the AWS Access Key ID used to authenticate with Amazon Kinesis. This field is required and supports 16 to 128 characters.
- Secret Access Key – Specifies the AWS Secret Access Key associated with the Access Key ID. This field is treated as sensitive information.
- Region – Specifies the AWS Region that hosts the target Amazon Kinesis streams. This field is required.
- Endpoint – Specifies the Amazon Kinesis endpoint URL. Configure this field only if your deployment requires a specific endpoint.
Note:
If an Access Key ID and Secret Access Key are not specified, Oracle GoldenGate for Distributed Applications and Analytics can use the AWS Default Credentials Provider Chain when supported by the deployment. - Configure Common Settings,
if additional runtime configuration is required.Configure the following common
settings:
Classpath – Specifies the runtime classpath used to load the required AWS Kinesis client libraries. You can use Dependency Downloader to download client libraries.
Note:
Download the supported AWS Kinesis client libraries by using the Dependency Downloader for Amazon Kinesis and configure the runtime classpath to reference the downloaded libraries before starting the Replicat process. - (Optional) Configure Proxy Settings if
the Oracle GoldenGate for Distributed Applications
and Analytics deployment accesses Amazon Kinesis
through an HTTP proxy.
Configure the following proxy settings:
- Proxy URL – Specifies the URL or host name of the proxy server used to access Amazon Kinesis.
- Proxy Port – Specifies the port number of the proxy server.
- Proxy Username – Specifies the user name for proxy authentication, if required.
- Proxy Password – Specifies the password for proxy authentication. This field is treated as sensitive information.
- Truststore – Specifies the Java truststore file that contains the certificate authority (CA) certificates used to validate the proxy server certificate, when required.
- Truststore Password – Specifies the password associated with the truststore.
- Keystore – Specifies the Java keystore file containing the client certificate and private key used for client authentication, when required.
- Keystore Password – Specifies the password associated with the keystore.
- SSL Key Password – Specifies the password protecting the private key stored in the keystore when it differs from the keystore password.
Note:
Configure the proxy and SSL settings only if the Oracle GoldenGate for Distributed Applications and Analytics deployment accesses Amazon Kinesis through a proxy that requires authentication or TLS. If the deployment has direct network connectivity to Amazon Kinesis, leave the Proxy Settings blank. - Click Submit.
- On the DB Connections page, click DB
Login to validate the connection.
Note: A successful DB Login validates only the saved connection information. It does not validate the Replicat configuration or Kinesis Streams Handler configuration. After successful validation, the connection is available for selection during Replicat creation.
Important: Ensure that the AWS IAM user or role configured for the connection has the
kinesis:ListStreamspermission. Although the connection can be created successfully without this permission, DB Login requires it to retrieve the available Amazon Kinesis streams. If the permission is missing, DB Login returns an authorization error.
Parent topic: Amazon Kinesis
10.5.3.4 Use an Amazon Kinesis Connection in Replicat
After the Amazon Kinesis connection has been successfully validated:
- Create or edit a Replicat.
- Select Amazon Kinesis as the target technology.
- Select the validated Amazon Kinesis connection.
- Complete the remaining Replicat configuration.
- Configure the required Kinesis Streams Handler properties, including the runtime classpath, if necessary.
- Create and start the Replicat.
Related Topics
Parent topic: Amazon Kinesis