|
Maintaining availability of your WebLogic Integration solution
requires the use of WebLogic Platform tools and the application
of best practices for your enterprise information system (EIS).
For an overview of how WebLogic Server provides high availability
in cluster environments, see Communications
in a Cluster in Using WebLogic Server Clusters.
For information about configuring your WebLogic Integration
solution for high availability, see Understanding
WebLogic Integration High Availability in Deploying
WebLogic Integration Solutions.
For more information, see the following topics:
High Availability Configurations
WebLogic Integration provides failure and recovery features
specific to functional areas. To configure a highly available
solution:
For a pre-deployment checklist
of high availability configuration requirements, see the "Recovery
Checklist" in WebLogic Integration
Application Recovery in the WebLogic Integration Solutions
Best Practices FAQ.
Important Recommendations
The following provide some important
recommendations regarding WebLogic Integration solution
availability:
- While developing your application...
- Configure
your retries and retry delays in the project JMS queue
so that the transaction retry count (number of retries
* retry delay) of the JPD exceeds the time it takes to
recover a managed server. (Note that configuring retries
and retry delay parameters in the project JMS queue will
override the retries and retry delay in the JPD.) Configuring
these properties of the project JMS queue will prevent
problems during recovery associated with certain asynchronous
callbacks (timer, process control) that have no retry
parameters.
- For
information about configuring JMS queue retries and retry
delays, see "Managing Rolled Back, Recovered, Redelivered,
or Expired Messages" in Developing
a WebLogic JMS Application in Programming WebLogic
JMS.
- When an XA-capable enterprise information system
fails...
- Transactions
are often left active until the WebLogic Server TransactionManager
can contact the EIS resource manager to complete the transactions
by either committing the transactions or rolling them
back. The following practices will allow the WebLogic
Server Connector container and TransactionManager to properly
re-establish communication with the EIS and recover the
active transactions:
- Verify that the adapter
instances for the unavailable EIS are suspended.
- Verify that any dependent
application views are also suspended.
- When
the EIS instance becomes available again:
- Resume the adapter instances
for that EIS instance and redeploy them.
- Resume any dependent application
views.
- To
learn how to use the WebLogic Integration Administration
Console to perform these tasks, see Application
Integration in Managing WebLogic Integration Solutions.
- Failure
to employ these practices can leave transactions against
the EIS instance active even after the EIS instance becomes
available. If the EIS is a database, this can leave row
or table locks in place. Regardless of the type of EIS,
in-doubt transactions can cause failures in executing
previously suspended and future application view services.
- For
information about configuring JMS queue retries and retry
delays, see "Managing Rolled Back, Recovered, Redelivered,
or Expired Messages" in Developing
a WebLogic JMS Application in Programming WebLogic
JMS.
See also:
Notes on Microsoft SQL Server
If your configuration includes Microsoft SQL Server:
- In cases where a DBMS failure occurs and the DBMS instance
has been used in any XA transaction on a running managed
server, pending XA transactions may be left on that server.
If pending transactions exist on a managed server, graceful
shutdown hangs waiting for those transactions to complete.
Due to recovery issues with Microsoft SQL Server drivers,
sometimes these transactions cannot be successfully completed
(committed or rolled back), making graceful shutdown impossible.
- To
recover normal operations, be sure to follow the recommended
practices described in When an XA-capable
enterprise information system fails...
- When sending multiple asynchronous
requests to a single stateful process, requests may rollback
if you have configured too few retries for the process.
When this occurs, an SQL deadlock message similar to the
following appears in the log:
Exception
in ejbRemote java.sql.SQLException: [BEA][SQLServer JDBC
Driver][SQLServer]Transaction (Process ID 67) was deadlocked
on lock resources with another process and has been chosen
as the deadlock victim. Rerun the transaction.- For
information on setting retries, see "Managing Rolled Back,
Recovered, Redelivered, or Expired Messages" in Developing
a WebLogic JMS Application in Programming WebLogic
JMS.
If your configuration includes Oracle:
- In cases where a DBMS failure occurs and the DBMS instance
has been used in any XA transaction on a running managed
server, pending XA transactions may be left on that server.
If pending transactions exist on a managed server, graceful
shutdown hangs waiting for those transactions to complete.
Due to recovery issues with Oracle thin drivers, sometimes
these transactions cannot be successfully completed (committed
or rolled back), making graceful shutdown impossible.
- To recover normal operations, be sure to follow the recommended practices described in When
an XA-capable enterprise information system fails...
- When a managed server fails and leaves Oracle with an
in-doubt transaction, the following situations may occur:
- You will see an exception
if you try to access the Process Instance Statistics or
Process Instance Summary pages. The ability of the WebLogic
Integration Administration Console to fulfill requests
for process instance information is limited until the
managed server recovers.
- The following Oracle exception
may be thrown:
ORA-01591:
lock held by in-doubt distributed transaction global_tran_id- Other record inserts may hit errors on the table with
the in-doubt transaction. This may prevent the start of
new java processes. Under these circumstances, you should
restart or migrate the failed JTA service. Make sure you
have sufficient retries on running processes to cover
the time that it takes to migrate.
- Other
record inserts may hit errors on the table with the in-doubt
transaction. This may prevent the start of new java processes.
Under these circumstances, you should restart or migrate
the failed JTA service. Make sure you have sufficient
retries on running processes to cover the time that it
takes to migrate.
- For
information about restarting and migrating JTA services,
see Server-->Control-->Start-Stop
and Server-->Control-->JTA
Migrate in the WebLogic Server Console Online
Help.
- For
information on setting retries, see "Managing Rolled Back,
Recovered, Redelivered, or Expired Messages" in Developing
a WebLogic JMS Application in Programming WebLogic
JMS.
- It may take Oracle several minutes to return the transaction
to a recovery service. Wait several minutes before starting
recovery. Initiating JTA recovery before Oracle is ready
may cause recovery to fail. Restart recovery, if it fails.
- You
can also administratively resolve the in-doubt transaction
(transaction_id) on
Oracle by executing either a COMMIT
or ROLLBACK command. For example:
COMMIT
FORCE 'local_tran_id'ROLLBACK
WORK FORCE 'local_tran_id'- Another
administrative approach to resolving this situation is
for a qualified Oracle system administrator to analyze
the database, locate the tables where in-doubt transactions
occur, and reduce the number of rows per block in those
tables. This tuning technique should be applied very selectively;
while it reduces the likelihood of overlapping transactions
occurring in those blocks, it also has performance ramifications
for the database.
Related Topics
- Using WebLogic Server Clusters
- Learn about the WebLogic Server cluster functionality that forms the basis of WebLogic Integration high availability features.
|
