In order to create your own base cartridge, you need to establish a uniquely named directory structure for the source files, and create the skeleton properties file that will be used to generate the starting source files.

This step is an analysis of how the device performs the functions that the base cartridge is going to manage. You will need to gather information on login and logout sequences, prompts, and more.

Each cartridge you create with the SDK requires a skeleton properties file. In this file, you will edit the property values that control the generation of the cartridge source files. For complete details on all properties, refer to "Base Cartridge Generation Properties".

This step uses the SDK tools to read the skeleton properties file and create the skeleton cartridge source files.

This step is where most of your development effort will be spent. The key cartridge source components include:

• Device Model (DM) schema definition

• Service Model (SM) to DM transform

• DM validation

• Annotated DM to CLI transform

• Message (success/warning/error) pattern definitions

There are many other source file components you may need to create or modify including files that support audit services, options, capabilities, and pre- and post-checks. These are described later in this chapter.

This step uses the SDK tools to compile and package the cartridge.

Unit and device tests are created as part of the generated skeleton source files and are run using the SDK tools.

To perform end-to-end testing, you'll need to deploy your cartridge into a test IP Service Activator system.

A sample working base cartridge for Cisco IOS devices, called cisco, is provided with the SDK in the form of:

• A skeleton.properties file

  This file is used to generate the source files for the sample base cartridge. For more information, see "Generating the Sample Base Cartridge Source Files".

• Three pre-edited source files that demonstrate the edits required by the generated source files to produce a working cisco sample base cartridge.

  • ...\audit\auditTemplate.xml

  • ...\messages\successMessages.xml

  • ...\xquerylib\dm2cli-common.xq

The provided sample source files are located in

SDK_home\samples\baseCartridge\cisco\...

while the generated sample cartridge source files are placed in

SDK_home\baseCartridges\cisco\...

To complete the sample, you can copy the provided files over their generated counterparts; or you can edit the generated files.

The pre-completed skeleton properties file illustrates how to populate the fields in the file for a working base cartridge. The provided sample source files are based on those generated by the sample base cartridge skeleton.properties file, but are customized further for Cisco IOS. This illustrates the typical post-generation steps you need to perform.

It can be useful to perform a diff between the customized sample source files and a set of generated sample skeleton source files, and then examine the difference between them.

The main values provided by the base cartridge sample are:

• Shows how to fill out the skeleton properties for a base cartridge

• Provides material to increase your understanding of the roles of the various prompt-related regular expressions

• Demonstrates implementation of the config version check optional function

• Lets you inspect the generated source files to see how a simple, working, base cartridge is constructed.

• Lets you complete, compile and package the overwritten generated sample source files into a working base cartridge and deploy it in a test system.

• Lets you take a copy of the provided skeleton properties file, relocate and rename it, and use it as the starting point to generate your own skeleton cartridge source files.

The sample skeleton properties file that is used to create the source files for the cisco sample base cartridge is called

SDK_home\samples\baseCartridge\cisco\skeleton.properties

This properties file is pre-populated with the information needed to construct the starting source files for the cisco sample base cartridge - a working base cartridge that will support Cisco IOS devices.

Some of the generated source files will require editing or you can overwrite them with the provided source files.

As you read through the base cartridge creation steps, instructions are given on how to use the sample to test some of the SDK tools and commands.

Refer to "Base Cartridge Generation Properties" for details on all the properties implemented in the sample properties file which create the source files for the sample.

IP Service Activator primarily uses SNMP for discovery. The policy server component queries the device using SNMP MIBs and constructs the device topology, its interfaces and sub-interfaces in the IP Service Activator object model.

The amount of detail on the device retrieved through this basic SNMP discovery process may be sufficient to start construction of your base cartridge. You can test this by discovering your device as you normally would in the IP Service Activator client. Refer to the Service Activator online Help for more information about discovering one or more devices.

For additional information, see the discussion of network discovery and representation in IP Service Activator Concepts.

This section describes the two aspects of the access device characteristic.

Most devices have two modes of operation: user (read-only), and configuration (read-write) mode. User mode allows you to query the device about its hardware, configuration or statistics, but bars you from making changes. Configuration mode allows modification of the device's configuration. For example, Cisco devices use the command configure terminal (or conf t for short).

In the skeleton properties file, specify the exact commands to enter and exit configuration mode on the device:

• sdk_configMode_cmd

• sdk_configModeTerminate_cmd

The Network Processor determines when a command it has sent to a device has been received by the reception of a prompt from the device.

The response time varies by device and command. On a slow connection, sending and receiving the command, and receiving the prompt may take a number of seconds. The Network Processor waits until it has received the return prompt to be sure that the command was received correctly.

Care must be taken when configuring a regular expression to match a prompt because prompts are configurable on a per-device basis. Additionally, prompts can change throughout the communication session.

For example, when a device is being configured, the prompt may change to show which area of the configuration is currently being worked on.

In a session:

• The device presents the prompt mydevice> after successful login.

• The prompt changes to mydevice# when the user enters privileged mode.

• The prompt changes to mydevice(config)# when configuring.

• The prompt may change to variants of mydevice(config-*)# when configuring.

In order to match a prompt, the Network Processor attempts to discover the prompt upon logging in. It remembers the original prompt and uses it throughout the communication session.

When configuring your base cartridge, the sdk_getConfiguration_matchPattern property defines a regular expression to be used by the Network Processor to discover the prompt upon login by matching the pattern with the responses from the device. There are also properties to define text which precedes and text which follows the prompt.

Table 2-1 lists the prompt definition properties and their matched regular expression.

sdk_getConfiguration_matchPattern=([^>#\\n]*)[>#]

sdk_getConfiguration_prePendPattern=\\n

sdk_getConfiguration_appendPattern=(([>#])|(\\(.*\\)#))

    mydevice>
    mydevice#
    mydevice(config)#
    mydevice(config-pmap)#

For complete details on all properties, refer to "Base Cartridge Generation Properties".

An audit compares the configuration that the Network Processor has persisted for the device with the actual configuration that is retrieved from the device. Commands not administered by IP Service Activator are filtered out.

In order to perform an audit, the Network Processor must be able to parse all of the commands on the device. Most devices allow this by providing a command that returns the entire configuration of the device (e.g. show-running-configuration on Cisco).

The required properties to configure auditing for a base cartridge are:

• sdk_audit_supported

• sdk_showRunningConfig_supported

• sdk_showRunningConfig_cmd

To enable auditing, the base cartridge properties sdk_audit_supported and sdk_showRunningConfig_supported should be set to true. The property sdk_showRunningConfig_cmd should be set to the exact command to retrieve the current configuration of the device.

For complete details on these properties, refer to "Base Cartridge Generation Properties".

Some devices have both a running configuration — the configuration that is currently executing on the device, and a startup configuration — the configuration that is loaded upon restart of the device. The startup configuration is stored on disk whereas the running configuration is usually only stored in memory.

Devices with running and startup configurations typically support the copying of one configuration to the other. For example, when a new configuration is applied to a device it may then be backed up to the startup configuration.

The properties that are used by the SDK to support this are:

• sdk_saveRunningConfig_supported: set this to true to indicate support for copying the running configuration to the startup configuration

• sdk_saveRunningConfig_cmd: set this to the exact command to copy the running configuration to the startup configuration

IP Service Activator, when generating device models, generates a new configuration version statement that conforms to the following format:

YYYY-MM-DDTHH:MM:SS.sssZ

This configuration version is stored in the device model and is written to the device along with the general configuration commands.

The configuration version, read back from the device during audit and subsequent configuration updates, is used to determine if the last persisted device model is in sync with what is on the device. If the device and device model configuration versions match, then the last persisted device model should match with the commands on the device. If the configuration versions don't match, then either the last persisted device model is out of date or the device is out of sync with IP Service Activator.

The configuration version functionality consists of several sub-functions that an SDK developer needs to be aware of.

• Configuration versioning in the target and last device models

• Configuration versioning on the network devices

• Configuration version pre-check prior to committing configuration changes to a device

• Configuration version auditing

The first function is always performed by the cartridge framework. In other words, the target, and consequently, the last device models automatically contain configuration version information.

The last three functions are optional in the SDK. An SDK developer can enable them with the sdk_configurationversion_supported property. When set to true, the source file generator generates XQuery functions responsible for the following:

• Transforming the configuration version in the annotated device model into CLI commands.

• Retrieving the configuration version from the device, comparing it with the configuration version in the last device model and, when out-of-sync, raising faults that prevent making further configuration changes to a device that is out of sync.

To generate the sample base cartridge source files using the data from the sample skeleton properties file:

1. Set the cartridge version string variable. For example, if the cartridge version is 1.0, on a Windows host, type the command:

   set VERSION_STRING=1.0
   

2. In SDK_home, either:

   • Run the included batch file to run the cartridge generator script:

     genbc samples\baseCartridge\cisco\skeleton.properties
     

     or

   • Type in the command to run the cartridge generator script:

     ant -DtemplateType=baseCartridge -DpropFile=SDK_home\samples\baseCartridge\cisco\skeleton.properties

   Note:

   To use the batch file, you must first add SDK_home\bin to your PATH variable where SDK_home is the SDK directory.

When you create your own base cartridge, the cartridge name and the root folder for the generated source are based on the sdk_global_cartridgeName property value in the skeleton properties file. See "Creating a Base Cartridge Source Directory and Skeleton Properties File" for details. The property value is incorporated into the cartridge source files in place of sdk_global_CartridgeName.

To generate your skeleton base cartridge source files using your customized skeleton properties file:

1. Set the cartridge version string variable. For example, if the cartridge version is 1.0, on a Windows host, type the command:

   set VERSION_STRING=1.0
   

2. In SDK_home, either:

   • Run the included batch file to run the cartridge generator script:

     genbc \baseCartridges\<sdk_global_cartridgeName>\skeleton.properties
     

     or

   • Type in the command to run the cartridge generator script:

     ant -DtemplateType=baseCartridge -DpropFile=SDK_home\baseCartridges\sdk_global_cartridgeName\skeleton.properties

   Note:

   To use the batch file, you must first add SDK_home\bin to your PATH variable where SDK_home is the SDK directory.

This section discusses where to find information to help you resolve cartridge generation issues.

This device model extends the Network Processor's base device model to realize the new services being administered by the cartridge.

This XQuery or java transform transforms the device-independent service model to a device-specific device model.

It is essential that the service model Definition IDs, which identify policy definitions, and Association IDs, which identify the links between defined policies and their target objects and their representative concretes in IP Service Activator, flow through from the service model to the device model.

If the cartridge registry entry <dmValidation> contains a dmValidation entry, the Network Processor will invoke this function to validate the transformed device model. This would capture logical faults as opposed to syntax faults which would be caught by the device model validation using deviceModel.xsd.

The Network Processor compares the target device model with the last device model that was persisted to the database after the last successful push to the device. The Network Processor annotates the target device model. For each policy object, the annotation includes the smId, a dmId that is generated by the Network Processor and a changeType which indicates whether configuration is being added, deleted or modified on the device.

The Network Processor invokes the <dmToCli> entry in the cartridge instance. This transforms the annotated device model into a CLI document that is a list of native configuration commands to be sent to the device.

As part of the creation of a base cartridge, you will need to create success, error, and warning message pattern files. This section explains how to analyze device responses and create the appropriate entries in the message XML files.

For an overview of the concepts behind message files, refer to IP Service Activator SDK Developer Overview Guide.

As part of the base cartridge creation process, you must define the list of device command response patterns. Valid response pattern files are in XML format and conform to the cliModel.xsd schema.

Using the options framework in IP Service Activator, you can customize the configuration style for different device types and IOS combinations.

To do this, you define and document configuration options for a cartridge, and implement the variations in the service model to device model transform, and the annotated device model to CLI transform, based on the option values.

Option values for specific device type and IOS combinations are specified in option configuration files, which are registered by cartridge units in the Registry.xml file. The option configuration files, and the registry entries that reference them, may be customized by the system administrator once the cartridge is deployed.

When a cartridge is installed, by extracting it to the Service_Activator_home directory, it creates a sample registry configuration directory under it such as Service_Activator_home\Config\networkprocessor\ciscoSampleRegistry. The name of the sample directory is sdk_global_cartridgeNameSampleRegistry. This directory contains samples of the various configuration files that you may want or need to customize.

One of the main files that you will need to edit, to customize any SDK registry entry, is the customization registry file. A sample of this file exists in the sample directory. The name of the file is sdk_global_cartridgeName.xml. To use the customization registry file to customize the SDK registries, edit the file and then copy it to the directory Service_Activator_home\Config\networkprocessor\Custom\Registries. The next time the Network Processor is restarted, the customizations will override the cartridge's registry entries.

The name of the file, as it exists in the Service_Activator_home\Config\networkprocessor\Custom\Registries directory is not actually important. It is the name field within the file that indicates which SDK registry the customization will edit.

The customization file must conform to the cartridge.xsd schema.

You can have your cartridge raise a fault to the Network Processor using the method AddFaultByThreadAndAbort() which is part of the FaultCollector. This method raises a fault and causes transformation to abort immediately.

The AddFaultByThreadAndAbort() method throws an AbortTransformException exception.

Since this java exception is invoked in an XQuery, Saxon throws it to syserr. On UNIX, the Network Processor shell script is modified to discard all syserr output.

Compilation problems are caused by schema or XQuery errors. To debug these problems, load the schema into an XML schema aware editor. This makes it much easier to find and correct problems in the schema.

When an SDK cartridge is built, a manifest file is created listing all of the files that are packaged into the cartridge zip file. Installation of the cartridge places the manifest file into the uninstall directory of the IP Service Activator installation.

Pre- and post-checks provide the ability verify information on a device when the annotated DM to CLI transform executes, before the general configuration is sent. This allows you to confirm that prerequisites to the configuration are met.

After configuration is sent, you have the opportunity to have a post-check invoked to verify some aspect of the commands that were sent to the device.

For further information on pre- and post-checks, see IP Service Activator SDK Developer Overview Guide.

Once the Network Processor has started, it will raise information faults in the system indicating each cartridge registered. The new base cartridge should be indicated. If this does not happen, check the network processor log; it will contain the details on why the cartridge was not loaded. If the log does not indicate the problem, check that the cartridge was deployed to the correct location.

If the CTM module is installed, it can be used to send commands to the base cartridge.

To configure CTM:

1. Find TABLE in database called XTM_DRIVER_TYPE

2. Add the name of your new cartridge's driverType to the list

   For example, from the sqlplus command line:

   insert into xtm_driver_type values (‘cisco');

Problems with the base cartridge should fall into one of the following categories:

• Environmental/loading of the base cartridge

• Cartridge transform errors: if the transform has been customized, it must produce the correct results. If not, this can cause configuration to be rejected and faults to be raised.

• Device discovery: ensure that the core IP Service Activator system can discover the device. If the SNMP discovery works but the capabilities fetch fails, this can indicate the cartridge is not registered for the correct device type/OS combination.

• Device access: ensure the correct session type (SSH/telnet) has been selected as well as the correct userid and password. If problems persist, ensure the same prompts are used in the device tests. These can be used to help isolate the problem.

• Command problems:

  • The auditTrail log shows commands being sent to the device. Use the auditTrail log to identify commands begin sent and whether or not they are being accepted.

  • Attempt the same commands manually on the device. Take note of any responses from the device. These may need to be added to the success messages for the cartridges.

The default audit logging properties are defined in auditLogging.properties in the cartridge jar file.

Once the cartridge is deployed, the audit logging properties can be overridden by the system administrator. For this purpose, the cartridge .zip file includes a copy of the auditLogging.properties file in the vendor specific sampleRegistry directory located in:

Service_Activator_home\Config\networkProcessor\sdk_global_cartridgeNameSampleRegistry\auditLogging.properties

where sdk_global_cartridgeName is the name of the cartridge.

When the skeleton cartridge source files are generated, occurrences of sdk_global_deviceNameLowerCase, as shown below in the source auditLogging.properties template file, are replaced with value of the property sdk_global_deviceName converted to all lowercase characters. The property sdk_global_deviceName is a property in the skeleton.properties file. For example, if sdk_global_deviceName=Cisco then cisco will be substituted for sdk_global_deviceNameLowerCase in the output auditLogging.properties file.

To set different default values for the audit logging properties:

1. edit auditLogging.properties. For example:

   # Audit output
   #  There are alternatives for file rollover.
   #
   #  The current default is to rollover a log when it reaches 8MB.
   # log4j.appender.sdk_global_deviceNameLowerCaseAudit=org.apache.log4j.RollingFileAppender
   # log4j.appender.sdk_global_deviceNameLowerCaseAudit.MaxFileSize=8MB
   # log4j.appender.sdk_global_deviceNameLowerCaseAudit.MaxBackupIndex=1
   #
   #   An alternative would be to rollover at the end of each day.
   #   To do this, replace the 3 lines shown above with the following.
   # log4j.appender.sdk_global_deviceNameLowerCaseAudit=org.apache.log4j.DailyRollingFileAppender
   # log4j.appender.sdk_global_deviceNameLowerCaseAudit.DatePattern='.'yyyy-MM-dd
   #
   log4j.loggerFactory=com.metasolv.serviceactivator.util.logging.TraceLoggerFactory
   log4j.logger.com.metasolv.serviceactivator.networkprocessor.AuditLogger.sdk_global_deviceNameLowerCase=info, sdk_global_deviceNameLowerCaseAudit
   log4j.appender.sdk_global_deviceNameLowerCaseAudit=org.apache.log4j.RollingFileAppender
   log4j.appender.sdk_global_deviceNameLowerCaseAudit.File=AuditTrails/npsdk_global_cartridgeName.audit.log
   log4j.appender.sdk_global_cdeviceNameLowerCaseAudit.MaxFileSize=8MB
   log4j.appender.sdk_global_cdeviceNameLowerCaseAudit.MaxBackupIndex=1
   log4j.appender.sdk_global_deviceNameLowerCaseAudit.layout=org.apache.log4j.PatternLayout
   log4j.appender.sdk_global_deviceNameLowerCaseAudit.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss}%m%n
   

To override the default audit logging properties once the cartridge is deployed:

1. Append the contents of auditLogging.properties found in the install directory at

   Service_Activator_home\Config\networkProcessor\sdk_global_cartridgeNameSampleRegistry\auditLogging.properties

   to the logging.properties file located in

   Service_Activator_home\Config\networkProcessor\com\metasolv\serviceactivator\networkprocessor\logging.properties

2. Adjust audit logging properties as necessary.

To change the audit trail logging level from debug to info:

1. Modify the line:

   log4j.logger.com.metasolv.serviceactivator.networkprocessor.AuditLogger.sdk_global_cartridgeName=debug, ${sdk_global_cartridgeName}Audit
   

   to:

   log4j.logger.com.metasolv.serviceactivator.networkprocessor.AuditLogger.sdk_global_cartridgeName=info, sdk_global_cartridgeNameAudit

To change the audit log file size:

1. Modify the value of MaxFileSize. For Example:

    log4j.appender.sdk_global_cartridgeNameAudit.MaxFileSize=8MB
   

To change the number of previous versions of the audit log file:

1. Modify the value of MaxBackupIndex. For example:

   log4j.appender.sdk_global_cartridgeNameAudit.MaxBackupIndex=1

IP Service Activator uses a rollover strategy to prevent log files from becoming too large.

To configure a log file to roll over when it reaches a specified size:

1. Use the following settings in the auditLogging.properties file:

   log4j.appender.sdk_global_cartridgeNameAudit=org.apache.log4j.RollingFileAppender
   log4j.appender.sdk_global_cartridgeNameAudit.MaxFileSize=8MB
   log4j.appender.sdk_global_cartridgeNameAudit.MaxBackupIndex=1
   

To configure a log file to roll over at the end of each day:

1. Use the following settings in the auditLogging.properties file:

   log4j.appender.sdk_global_cartridgeNameAudit=org.apache.log4j.DailyRollingFileAppender
   log4j.appender.sdk_global_cartridgeNameAudit.DatePattern='.'yyyy-MM-dd

Once a cartridge is deployed in production, a non-trivial change to the devicemodel.xsd in the next version of the cartridge will require a custom upgrade transform.

Examples of non-trivial changes are:

• You introduce mandatory elements

• You remove elements

• You completely modify an existing elements

• You reorder elements

When you create a cartridge, within the skeleton.properties file, the cartridge version is set in the <sdk_cartridgeVersion> property. This cartridge version is used to identify the version of the device model as seen in the cartridge file:

SDK_Home\basecartridges\cisco\src\com\metasolv\serviceactivator\cartridges\cisco\xquerylib\dm-version.xq

Increment the cartridge version whenever a significant change to the device model has occurred which requires a non-trivial upgrade to an existing device model. For more information about Device Model upgrades, see IP Service Activator SDK Developer Overview Guide.

The cartridge version is set in the file dm-version.xq:

declare variable $dmver:version := "2.0";

The version format can be a, a.b, or a.b.c (i.e. major, major.minor, major.minor.sub-minor), where a, b, c represent numeric values.

When you use the cartridge skeleton generator tool to generate the base cartridge source files, a skeleton XQuery file called DmUpgrade.xq is automatically created.

To implement the upgrade transform:

1. Edit DmUpgrade.xq.

   For example, if the device model has an existing element named 'myelement' which requires an upgrade transform, implement the upgrade transform in DmUpgrade.xq as follows:

   (: -*- nxml -*- :)
     import module namespace dmver = "dm-<sdk_global_cartridgeName>-version" at "dm-version.xq";
    
   declare namespace dm="http://www.metasolv.com/serviceactivator/<sdk_global_cartridgeName>";
     declare namespace lib="http://www.metasolv.com/serviceactivator/devicemodel";
    
   <lib:device xsi:type="dm:<sdk_global_cartridgeName>Device" xmlns:dm="http://www.metasolv.com/serviceactivator/<sdk_global_cartridgeName>">
     {
       dmver:getUpgradeAppInfo(),
       for $element in /lib:device/*
       return
         if (fn:local-name($element) = 'appInfo') then ()
         else if (fn:local-name($element) = 'myelement') then
         (: implement transform for 'myelement' here :)
         else $element
     }
   </lib:device>

When you use the cartridge skeleton generator tool to generate the base cartridge source files, a skeleton java file named DmUpgradeTests.java is automatically created. This file contains the device model upgrade unit tests.

To test the upgrade transform for an existing element in the device model named 'myelement' that requires an upgrade transform implemented in DmUpgrade.xq:

1. Create a sample device model XML file that includes data requiring an upgrade. In the sample code below, the file is named sampleDeviceModel.xml.

2. Implement a test in DmUpgradeTests.java.

   The test will load sampleDeviceModel.xml, transform the XML using the upgrade transform, and use methods in BaseSdkTest to verify the resulting upgraded device model, thereby validating the upgrade transform.

   In the sample code below, the test method is named testUpgradeFromVersionX.

3. Run the unit tests using the following command syntax:

   ant unittests SDK_home\cartridges\sdk_global_cartridgeNamebuild.xml
   

A code sample follows:

  package ${sdk_global_package}.test;
 
  import com.metasolv.serviceactivator.sdk.test.TestUtilsInterface;
  import com.metasolv.serviceactivator.sdk.test.BaseSdkTest;
  import com.metasolv.serviceactivator.util.XmlUtil;
  import org.apache.xmlbeans.XmlObject;
  import java.io.File;
 
  public class DmUpgradeTests extends BaseSdkTest {
    static private final String testUpgradeFromPackageName = "${sdk_global_customPackage}/test/models/upgradeFrom";
    public static final String dmUpgradeTransform = "${sdk_global_customPackage}/xquerylib/DmUpgrade.xq";
    static private final String upgradeFromDeviceModelXmlFile = testUpgradeFromPackageName + "/sampleDeviceModel.xml";
    static private TestUtilsInterface testUtils = null;
 
    protected void setUp() throws Exception {
		testUtils = getTestUtils();
	    }
 
    public void testUpgradeFromVersionX() throws Exception {
      XmlObject oldDm = loadXml(DmUpgradeTests.upgradeFromDeviceModelXmlFile);
      XmlObject upgradedDm = transform(oldDm, null, DmUpgradeTests.dmUpgradeTransform, false);
  //    System.out.println("Transformed:\n" + XmlUtil.bean2xml(upgradedDm));
 
   // use methods in BaseSdkTest to verify upgradedDm here
    }
 }

DMUpgrade.xq is executed by the Network Processor when an upgrade procedure is invoked.

To remove a generated cartridge from the SDK installation:

1. Delete all contents under SDK_home\cartridges\sdk_global_cartridgeName.

To uninstall the SDK:

1. Delete all contents under SDK_home.