Synchronizing Application Data - Updated July 30, 2002


	BEA Home \| Events \| Solutions \| Partners \| Products \| Services \| Download \| Developer Center \| WebSUPPORT
	http://www.oracle.com/technology/documentation/index.html \| Search \| PDF Files \| Contact \| Glossary

WebLogic Portal Documentation \| Deployment Guide \| Previous Topic \| Next Topic \| Contents \| Index

Synchronizing Application Data

Application data expresses your business model and specifies how your enterprise application behaves to support your business goals. For example, a campaign is a collection of application data that specifies how your enterprise application helps you reach sales or advertising goals.

A Business Engineer (BE) uses the E-Business Control Center to create, modify, and save application data to a local file system. Then a developer uses either the E-Business Control Center or a Java command to synchronize the modified data with an active enterprise application.

A System Administrator can configure the enterprise application to synchronize the data amongst a collection or cluster of active servers and applications.

This topic includes the following sections:

How Application Data is Organized and Distributed

During the development cycle, a Business Engineer (BE) creates application data on a workstation and then synchronizes it with an active enterprise application. Each environment stores the data in different formats:

On the BE's workstation, the E-Business Control Center stores application data on the local file system as XML files. We recommend that you place these files in your content management system or source control system.
On the application server, the enterprise application stores the data in the RDBMS database and in data repositories, which provide runtime representations of the data.

This section includes the following subsections:

For recommendations on setting up a development environment to support concurrent development, refer to "Milestone 5: Set Up a Development Site" under "Workflow for Developing an E-Business Web Site" in Strategies for Developing E-Business Web Sites.

E-Business Control Center Data

The E-Business Control Center creates a separate directory tree of XML files for each application that it modifies. For example, a BE named Pat installs the E-Business Control Center on her workstation and uses it to develop campaigns, customer segments, and portlets for two different enterprise applications, Bank and Invest. (See Figure 7-1.)

Figure 7-1 One Directory Tree for Each Enterprise Application

The directory tree of XML files for each application conforms to the following structure (see Figure 7-2):

The root directory contains two subdirectories: application-sync, which contains application data and library.
The library directory stores data that the E-Business Control Center uses but does not create or synchronize. For example, library contains skin preview images and layout definitions for portals.
Under application-sync, the data for each product feature is in a separate directory. For example, campaigns and customer segments are saved in separate directories (named campaign and segments).
- Within each directory, each discrete business policy is saved in a separate file, which uses a specific file extension. For example, each campaign is saved to a separate file with the .cam file extension. For a complete list of directories and filename extensions, see Table 7-1.
- Each Web application within the enterprise application uses a separate folder to save its webflow data. For more information, refer to Scope of Application Data.
  Figure 7-2 Directory Structure of Client-Side Application Data

For example, for the Bank enterprise application Pat defines two campaigns and two customer segments. (See Figure 7-3.)

Figure 7-3 Example of Application Data on the Client

Table 7-1 describes directory and filename extensions for all application data types. All of the directories and files are located under root-dir/application-sync. Note that segments, contentselectors, discounts, and entitlements store all files in a subdirectory.

Table 7-1 Directories and Filename Extensions for Application Data

Feature	Directory Name	Filename Extension
Campaigns	campaigns	cam
Catalog property sets	catalog	clg
Content selectors	contentselectors/GlobalContentSelectors	sel
Discounts	discounts/DefaultDiscountSet	dis
Entitlements	entitlements/GlobalEntitlements	ent
Event and Behavior Tracking	events	evt
Pipelines	pipelines	pln
Ad placeholders	placeholders	pla
Portlets	portlets	portlet
Request property sets	request	req
Scenarios	scenarios	sce
Customer segments	segments/GlobalClassifications	seg
Session property sets	session	ses
User profiles	userprofiles	usr
Webflow	webapps/web-application	wf

Scope of Application Data

With the exception of Webflow data, all application data is scoped for an enterprise application. For example, when you synchronize campaign and placeholder data, all Web applications within the enterprise application can use it.

Webflow data, however, is scoped for a single Web application to prevent ambiguities with the Web applications' namespaces. For example, a typical Web application contains a file named index.jsp. By scoping Webflow for a single Web application, the filename index.jsp is unambiguous: it refers to index.jsp in the current Web application.

Synchronized Data

When developers are ready to synchronize data, they can use either the E-Business Control Center or a Java command to synchronize the local files with a deployed application.

This section contains the following subsections:

For instructions on starting a synchronize operation, refer to Synchronizing Application Data.

For information on setting up data synchronization in a clustered environment, refer to Deploy and Configure DataSync Web Applications for the Cluster.

The DataSync Web Application and Master Data Repository

Each enterprise application includes a DataSync Web application, which receives synchronization requests from the E-Business Control Center. After it receives a request, the DataSync Web application channels the application data from the E-Business Control Center to the following locations:

The Master Data Repository, which is a runtime representation of the application data.
The RDBMS repository, which is a persistent storage location for the application data.

The services in your enterprise application create their own Data Repositories, which contain subsets of the data that is in the Master Data Repository. (See Figure 7-4.)

Figure 7-4 The Master Data Repository

The services use the data in the Data Repositories to carry out business tasks. For example, the Rules service uses the Data Repositories to evaluate whether customers fit into a particular customer segment.

This architecture enables each service to parse a small, specific set of information. For example, the Placeholder Data Repository copies placeholder-related information from the master. When the Placeholder Service requests data about placeholders, it only needs to parse the subset of information in the Placeholder Data Repository.

Each time the Master Data Repository is synchronized, it notifies the Data Repositories. These Data Repositories then synchronize their data subsets with the data in the master.

Proxy Data Repositories

If you want to synchronize the same application data to multiple enterprise applications, you can set up a Proxy Data Repository. A Proxy Data Repository represents a notification link to a DataSync Web application that is located in another enterprise application. Usually, you use Proxy Data Repositories to synchronize multiple instances of the same enterprise applications that are deployed on different servers. (See Figure 7-5.)

Figure 7-5 Proxy Data Repository

In a cluster, you can use a Proxy Data Repository to synchronize the Managed Servers. To enhance performance, the Data Repositories on Managed Servers always refer to the Master Data Repository on the proxy source; they never access the persisted data, which is located in the database. Only the Master Data Repository on the proxy source refers to the database when it needs to retrieve data from persistent storage. For more information on setting up Data Repositories for a cluster, refer to Deploy and Configure DataSync Web Applications for the Cluster and Refresh Data Repositories on Managed Servers.

In a non-clustered environment, one practical application of a Proxy Server might be to keep an application in a test environment synchronized with an application in the development environment.

Synchronization Process

When a developer synchronizes data, the following process occurs (see Figure 7-6):

A developer uses the E-Business Control Center or a Java command to send data files from the local directory tree to a DataSync Web application. The developer determines whether the synchronization operation sends all files from the local directory tree or only the files and directories that have changed.
To send the XML files to the DataSync Web application, the synchronization operation uses the HTTP protocol.
The DataSync Web application updates the Master Data Repository and the RDBMS repository.
The Master Data Repository notifies its Data Repositories and any Proxy Data Repositories of the new or modified data.
The notified Data Repositories synchronize themselves with the data that is in the Master Data Repository.
The Proxy Data Repositories notify their remote DataSync Web applications.
The remote DataSync Web applications synchronize their Master Data Repositories. Then the remote Data Repositories synchronize themselves with the data in their Master Data Repositories.
The Proxy Data Repositories use XML over HTTP to forward any notification to their remote DataSync Web applications.

Figure 7-6 Synchronizing Data

Data Synchronization is Non-Transactional

The overall process of synchronizing data is non-transactional, though it includes safeguards to prevent data corruption and keeps track of its progress for you to monitor. For more information about monitoring the process of data synchronization, refer to Monitoring and Managing Data Repositories.

With the exception of updating the database, if a communications failure (or some other type of problem) prevents the E-Business Control Center or Java command from properly synchronizing one of its files, it does not abandon the synchronization process. Instead, after the process, it reports which files were successfully synchronized and which were not. To prevent data corruption, the synchronization process does not partially synchronize a file. If it cannot synchronize an entire file, it skips the file and moves to the next one in the queue.

Even after the E-Business Control Center transfers data to the DataSync Web application, other communications within and beyond the current application can fail. For example, a Proxy Data Repository might be unable to reach its remote application. If an error occurs during a given transmission, the synchronization continues for other services or machines.

With this process it is possible that one service or machine fails to be synchronized while the others are successfully synchronized. If you use a clustered environment, we recommend that you maintain at least one mirror cluster. When you want to synchronize application data to your production cluster, you can disable one mirror site and synchronize the data onto the disabled site. Before bringing the disabled site back online, you can monitor the data repositories to verify that all subordinate Data Repositories have synchronized. Then repeat the process for the other mirror.

Synchronizing Applications That Are Deployed on the Same Server

Each enterprise application that you deploy must contain its own DataSync Web application. If you deploy multiple enterprise applications onto the same server instance, you must modify the URIs of the DataSync Web applications so that each DataSync uses a unique address. The application.xml file includes XML elements that declare the URI for the DataSync Web application. For information on using application.xml to declare the DataSync Web application, refer to Adding DataSync.

Configuring a Proxy Data Repository

If you want to synchronize the same application data with multiple enterprise applications, you can set up Proxy Data Repositories. This document uses proxy source to identify the Master Data Repository that notifies a proxy of synchronization requests; it uses proxy destination to identify the remote Master Data Repository to which the proxy forwards the notification. (See Figure 7-5.)

While a typical use for Proxy Data Repositories is to synchronize applications on Managed Servers in a cluster, you can use them to synchronize independent servers that are not participating in a WebLogic Server cluster.

Note: Before you configure Proxy Data Repositories for a cluster, you must deploy two instances of the DataSync Web application. One instance will be used by Managed Servers for configuring their own Master Data Repositories; this instance contains no proxies. The other instance of the DataSync Web application is used to synchronize data across the cluster; this second instance is the Web application in which you configure proxies. For more information, refer to Deploy and Configure DataSync Web Applications for the Cluster.

Configuring Proxies

To create a Proxy Data Repository, do the following:

Start the WebLogic Server Administration Console by entering the following URL in a Web browser:
http://hostname:port-number/console
The WebLogic Server Administration Console prompts you to log in with a user account that has administrator privileges.
After you log in, in the left pane, click Deployments -> Applications -> MyApplication -> datasyncWebApp
where datasyncWebApp is the DataSync Web application that you deployed for the purpose of synchronizing data. The name of the DataSync Web application depends on how you configured your enterprise application. For information on configuring the DataSync Web applications, refer to Adding DataSync.

In the example in Deploy and Configure DataSync Web Applications for the Cluster, this server is named datasyncBankSync.
On the datasyncWebApp page click Edit Web Application Descriptor. (See Figure 7-7.)
The WebLogic Server Administration Console opens the Web Application Descriptor in an additional browser window.

Figure 7-7 Edit Web Application Descriptor
In the Data Synchronization Web Application window, in the left pane, click Web App Descriptor -> Servlets -> DataSyncServlet.

Complete the following steps for each Managed Server in the cluster:

Right click the Parameters folder and click Configure a New Parameter. (See Figure 7-8.)
Figure 7-8 Configure a New Parameter

On the Configure a New Parameter page, enter the following information:

Name	Value
Description	The parameter name that you want the WebLogic Server Administration Console to display. We recommend that you adopt a naming convention that includes the following information: Name of the target enterprise application An indication that this parameter configures a Proxy Data Repository The type of parameter. For example: Bank2ProxyDRnotifiedName
Param Name	notifiedName
Param Value	A name for the Proxy Data Repository.

Click Create.
Repeat steps a through c for each of the following parameters:

notifiedUrl

Name	Value
Description	The parameter name that you want the WebLogic Server Administration Console to display. We recommend that you adopt a naming convention that includes the following information: Name of the target enterprise application An indication that this parameter configures a Proxy Data Repository The type of parameter. For example: Bank2ProxyDRnotifiedUrl
Param Name	notifiedUrl
Param Value	The URL for the DataSyncServlet in the target enterprise application. Use the following syntax: http://remote-host:listen-port/datasyncApp/DataSyncServlet Where datasyncApp is the URI for the DataSync Web application. For more information, refer to Adding DataSync. For example, you want to synchronize the Bank2 enterprise application with the Master Data Repository of the current application. The Bank2 application is deployed on a host named bread, which listens on port 7201. The DataSync Web application is named datasync. Enter the following URL: http://bread:7201/Bank2DataSync/DataSyncServlet

notifiedLogin

Name	Value
Description	The parameter name that you want the WebLogic Server Administration Console to display. We recommend that you adopt a naming convention that includes the following information: Name of the target enterprise application An indication that this parameter configures a Proxy Data Repository The type of parameter. For example: Bank2ProxyDRnotifiedLogin
Param Name	notifiedLogin
Param Value	A user ID that has permission to deploy data.

notifiedPassword

Name	Value
Description	The parameter name that you want the WebLogic Server Administration Console to display. We recommend that you adopt a naming convention that includes the following information: Name of the target enterprise application An indication that this parameter configures a Proxy Data Repository The type of parameter. For example: Bank2ProxyDRnotifiedPassword
Param Name	notifiedPassword
Param Value	The password for the user ID.

notifiedRealm

Name	Value
Description	The parameter name that you want the WebLogic Server Administration Console to display. We recommend that you adopt a naming convention that includes the following information: Name of the target enterprise application An indication that this parameter configures a Proxy Data Repository The type of parameter. For example: Bank2ProxyDRnotifiedRealm
Param Name	notifiedRealm
Param Value	The security realm that defines the user ID for deploying data.

In the left pane, click Data Synchronization Web Application (the top node in the left pane). (See Figure 7-9.)
In the right pane, click Persist to save your changes.
Figure 7-9 Persist Your Changes

When you synchronize data with the Master Data Repository, it notifies the Proxy Data Repository that you created.

Creating and Modifying Application Data

Before you use the E-Business Control Center to create and modify application data, you must create a directory tree that organizes the data on your computer.

This section contains the following subsections:

For information about the directory tree for application data, refer to E-Business Control Center Data.

Create an Application Data Directory Tree

For a new application, you can create the directory tree either through the E-Business Control Center or by using standard commands from your operating system. After you create the directory tree, place it under source control.

If you want to create or modify application data for an existing application, use your source control system to copy the directory tree to your computer. Then open the data in the E-Business Control Center.

Using the E-Business Control Center to Create a Directory Tree

To use the E-Business Control Center to create a directory tree for saving application data, do the following:

Start the E-Business Control Center. For more information, refer to "Introduction to the BEA E-Business Control Center" in Guide to Using the E-Business Control Center.
In the E-Business Control Center, select File -> New Application. (See Figure 7-10.)
Figure 7-10 Select New Application
In the Browse Filesystem box, choose a parent directory for your application data. Then in the New Directory box, enter a name for your application-data directory. We recommend that you use the same name as the application.

The E-Business Control Center creates an empty directory tree that conforms to the schema as described in E-Business Control Center Data.

Place this directory tree under source control.

Using the Operating System to Create a Directory Tree

You can use standard operating-system commands to create an empty directory tree that conforms to the schema as described in E-Business Control Center Data.

Place this directory tree under source control.

Opening Data for an Existing Application

If another developer has already created and checked in an application-data directory tree, do the following to open it on your computer:

Use your source control system to copy or update the application directory tree on your computer.
If you plan to update existing files, use your source control system to check out the files.
Start the E-Business Control Center. For more information, refer to "Introduction to the BEA E-Business Control Center" in Guide to Using the E-Business Control Center.
In the E-Business Control Center, click File -> Open Application.
In the Browse Filesystem box, select the root directory of your application-data directory tree. Then click Select.
Do not select subdirectories within the directory tree.

Synchronizing Application Data

You can use either the E-Business Control Center or a Java command to synchronize application data. This section describes the following tasks:

Setting Up the E-Business Control Center to Synchronize Data

To connect to applications, the E-Business Control Center maintains connection settings. These settings provide such information as the URI of the application with which it exchanges data. Because the URIs differ for each application, create one connection setting for each application instance that you deploy.

This section describes how to modify a connection setting to specify the URI of your DataSync Web application.

In the E-Business Control Center, click Tools -> Connection Settings.
In the Connection Settings box, click. the Connections tab.
From the Connections list, select a connection and click Edit. (See Figure 7-11.)
If the Connections list is empty, click New.

Figure 7-11 Select an Existing Connection
In the Connection Details box, click Advanced.
In the Data Synchronization box, enter the following (see Figure 7-12):
/datasyncApp/DataSyncServlet

Where datasyncApp is the URI for the DataSync Web application. For more information, refer to Adding DataSync.

Figure 7-12 Enter URI for DataSync
Click OK.

For information on the other settings in the Connection Details and Connection Settings boxes, refer to the E-Business Control Center online help or to "Using the BEA E-Business Control Center" under "Introduction to the BEA E-Business Control Center" in the Guide to Using the E-Business Control Center.

Using the E-Business Control Center to Validate Data

Before you synchronize data to an active application, you can use the E-Business Control Center to scan (validate) the directory tree of data for the current application and report the files that have incomplete data items. For example, you might have saved a discount without defining the conditions under which the discount is available.

To validate an application, do the following:

Use your source control system to copy or update the application directory tree on your computer.
Start the E-Business Control Center and open the application that you want to validate. For more information, refer to "Introduction to the BEA E-Business Control Center" in Guide to Using the E-Business Control Center and Opening Data for an Existing Application.
When you are ready to start the validation process, click Tools -> Validate.

The E-Business Control Center scans the directory tree and reports any incomplete files in the box. You can synchronize data that includes incomplete data items, but you might encounter errors because of the incomplete data. (See Figure 7-13.)

Figure 7-13 Report of Validation Results

Using the E-Business Control Center to Synchronize Data

When you are ready to synchronize data with an application on an active server, do the following:

Start the E-Business Control Center and open the application that you want to synchronize. For more information, refer to "Introduction to the BEA E-Business Control Center" in Guide to Using the E-Business Control Center and Opening Data for an Existing Application.
Click Tools -> Synchronize.

In the Synchronization Setup box, click the Synchronization tab and enter the following information:

In this box...	Enter...
Realm	The name of the security realm that contains User IDs for granting permissions to synchronize data. To use the default RDBMS Realm, enter weblogic. To use the file realm, enter fileRealm.
Scope	Select one of the following: Update only modified files in the application. This option determines which files have been removed, updated or created on the local file system and sends only the changes to the DataSync Web application. Using this option can minimize the amount of data that is transmitted over HTTP between the E-Business Control Center host and the server. Update all files in the application. This option removes all the data from the Data Repositories and then sends all of the data from the local file system to the DataSync Web application. Use this option if there is no data currently synchronized to your enterprise application, if you suspect that the in-memory data is un-synchronized with the persistent data in the database, or to recover from a synchronization error. Before you use this option, make sure that your local file system contains the latest set of source files. (You might need to use your source control system to copy any files that other team members have checked in from their computers.)
Validation	Select any of the following: Check that all files are complete. Scans the directory tree of data for the current application and reports the files that have incomplete data items. This option activates the same validation check as selecting Tools -> Validate. You can synchronize data that includes incomplete data items, but you might encounter errors because of the incomplete data. Show reset options for active campaigns. Prompts you to reset such campaign data as the number of times ads have displayed in a particular placeholder or the number of times discounts have been offered.

Click the Connections tab and do the following:
1. From the Connections list, choose the connection that you created or modified in Setting Up the E-Business Control Center to Synchronize Data.
2. In the Username and Password boxes, enter the ID and password of a user who has permission to synchronize data. All users in the Administrators group of wlcsRealm can synchronize data.
  To add users or other groups to the Administrators group, use the WebLogic Portal Administration Tools. For more information, refer to "Using the User Management Tool" under "Creating and Managing Users" in the Guide to Building Personalized Applications.
Before you click Connect to start the synchronization process, note the following:
- If files are open in the E-Business Control Center Editor window with unsaved changes when you synchronize, those unsaved changes are not synchronized. Changes must be saved to be synchronized.
- If you selected any of the validation options, you can stop the process during the validation phase. However, once you start the synchronization process, you cannot cancel or stop it.
Click Connect.

Using the DirectoryDataSync Command to Synchronize Data

Before you synchronize data, consider using the E-Business Control Center to validate it. For more information, refer to Using the E-Business Control Center to Validate Data.

To synchronize application data to an active server from a command line, enter the following java command from an E-Business Control Center host:

java com.bea.p13n.management.data.DirectoryDataSync 
{ -root pathname-to-directory-where-EBCC-stores-data }
{ -url http://hostname-or-IP:listen-port/datasyncApp/DataSyncServlet } 
{ -user user-ID -password user-password }
{ -mode one-way-from-client | refresh-from-client }

where datasyncApp is the URI for the DataSync Web application. For more information, refer to Adding DataSync.

Synchronization Modes

You use the -mode argument to determine whether the DirectoryDataSync command synchronizes all of the files in the E-Business Control Center directory tree or only the files that have changed. To do so, supply one of the following parameters for -mode:

refresh-from-client. This value removes all the data from the Data Repositories and then sends all of the data from the local file system to the DataSync Web application. Use this mode if there is no data currently synchronized to your enterprise application, if you suspect that the in-memory data is un-synchronized with the persistent data in the database, or to recover from a synchronization error. Before you use this command, make sure that your local file system contains the latest set of source files. (You might need to use your source control system to copy any files that other team members have checked in from their computers.)
one-way-from-client. This value determines which files have been removed, updated or created on the local file system and sends only the changes to the DataSync Web application. This synchronization mode can minimize the amount of data that is transmitted over HTTP between the E-Business Control Center host and the server.

For example, the following command synchronizes only the data that you have modified for the Bank application:

java com.bea.p13n.management.data.DirectoryDataSync 
-root d:\application-data\bank 
-url http://255.251.255.01:7501/bankDataSync/DataSyncServlet 
-user system -password weblogic -mode one-way-from-client

Cancelling the Command

If you cancel the command (by entering Ctrl-C, closing the command shell, or using some other method to force the command to stop), the data that has been synchronized to the application on the server might be in an inconsistent state. The next time you synchronize data, you should use the -mode refresh-from-client option.

Monitoring and Managing Data Repositories

The DataSync Web application includes a set of JSPs that report the following information:

The contents of the Master Data Repository
The Data Repositories that synchronize themselves with the Master Data Repository
The contents of each Data Repository
The Proxy Data Repositories that you associate with the Master Data Repository

In addition, you can use the JSPs to clear and refresh the contents of each repository.

This section contains the following subsections:

Viewing the Contents of Data Repositories

To view the contents of Data Repositories, complete any of the following tasks:

View the Contents of the Master Data Repository

The Master Data Repository contains all of the data that was under the root of an application-data directory tree at the time of the last data synchronization operation. To verify that the Master Data Repository contains the same information as the RDBMS repository, refresh the contents. For more information, refer to Refreshing the Contents of Data Repositories.

To view the contents of a data repository, a DataSync Web application must be deployed on an active server. After activating the server, do the following: