DEPRECATED
Object
FilenameFilter
The reference document repository bulk loader application.
This class is mainly designed to run as a command-line application, via a "java com.bea.p13n.content.document.loader.BulkLoader" command-line. To see a usage, give it a -h flag or read the Usage.txt in this package.
Additionally, BulkLoader objects can be created and used to provide the functionality in other places. The lifecycle of a BulkLoader is as follows:
parseArgs()
if not passed in to constructor.
validateArgs()
to make sure the loader has
valid arguments.
initialize()
to setup internal state.
go()
to execute the load.
shutdown()
to cleanup everything.
If manually constructing and utilizing a BulkLoader object, be certain to synchronize all access to the object. Since the command-line program is single-threaded, BulkLoader objects are not thread-safe by design.
To load the default LoaderFilter
, the BulkLoader
looks for the com/bea/p13n/content/document/loader/loader.properties
file in the CLASSPATH. From that it reads the list of default LoaderFilter
class names from the "loader.defFilters" property. To not use any of the
default filters, specify +filters in the command-line args.
Related Topics
Object
BulkLoader
FilenameFilter
Nested Class Summary |
public static class | com.bea.p13n.content.document.ref.loader.BulkLoader.ShowUsageException
Quick inner exception thrown on parseArgs() to say we should just print a usage report. |
Field Summary |
public |
|
public |
|
public int |
|
protected |
|
public |
|
public static final |
|
public static final |
|
public static final |
|
public static final |
|
public static final |
|
public static final |
|
protected |
|
public static final |
|
protected |
|
public static |
|
public static |
|
public |
|
public boolean |
|
public boolean |
|
public boolean |
|
public static final |
|
public static final |
|
public |
|
public |
|
public |
|
public boolean |
|
public |
|
public boolean |
|
public boolean |
|
public static final |
|
protected |
|
public static final |
|
protected |
|
protected |
|
protected |
|
public |
|
public |
|
public |
|
protected |
|
protected |
|
protected |
|
protected long |
|
public boolean |
|
public |
|
public boolean |
|
public |
|
public boolean |
|
public boolean |
|
public static final |
|
protected |
|
public boolean |
|
public |
|
Constructor Summary |
|
|
Method Summary |
public boolean | |
public static void | |
public void |
|
public void | |
public int | |
public void |
|
public void |
|
public void |
|
public void | |
public void | |
protected void |
|
public | |
public | |
public static | |
public |
|
public |
|
public |
|
public void |
|
public void |
|
public int |
|
public static boolean | |
public boolean |
|
public static boolean |
|
public void |
|
public static int |
|
public static void | |
public void | |
public void |
|
public void |
|
public void |
|
protected void |
|
public boolean |
|
public boolean |
|
public void |
|
public static | |
public static |
|
public void |
|
public void |
|
public void |
|
public void | |
public void |
Methods from class java.lang. |
clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Methods from interface java.io. |
accept |
Field Detail |
public Map
addlColumnMap
public Collection
addlColumnNames
public int commitAfter
protected Connection
connection
public String
conPoolName
public static final String
DEF_MD_FILE_EXT
public static final String
DEF_MIME_TYPE
public static final String
DEF_SCHEMA_NAME
public static final String
DEF_SCHEMA_PATH
public static final String
DEF_WLS_PROPS_PATH
public static final String
deleteDocSql
protected PreparedStatement
deleteDocStmt
public static final String
deleteMDSql
protected PreparedStatement
deleteMDStmt
public static String
DOC_MD_TABLE
public static String
DOC_TABLE
public String
docBase
public boolean doCleanUp
public boolean doDelete
public boolean doMetaParse
public static final Collection
explicitAttrNames
public static final Collection
explicitColumnNames
public String
fileEncoding
public List
fileList
public List
htmlMatchList
public boolean ignoreErrors
public List
ignoreList
public boolean includeHidden
public boolean inheritProps
public static final String
insertDocSql
protected PreparedStatement
insertDocStmt
public static final String
insertMDSql
protected PreparedStatement
insertMDStmt
protected Properties
jdbcProps
protected String
jdbcUrl
public List
loaderFilters
public List
matchList
Empty to include all.
public String
mdFileExt
This should start with a ".".
protected Collection
metadataNames
protected String
myInsertDocSql
This will constructed in initialize() from with any additional column information.
protected String
myUpdateDocSql
This will constructed in initialize() from with any additional column information.
protected long numDocsLoaded
public boolean recurse
public String
schemaName
public boolean schemaOnly
public String
schemaPath
public boolean testOnly
public boolean truncate
public static final String
updateDocSql
protected PreparedStatement
updateDocStmt
public boolean verbose
public String
wlsPropsPath
Constructor Detail |
public BulkLoader()
public BulkLoader(String
[] args)
Related Topics
BulkLoader.parseArgs(String[])
Method Detail |
public boolean accept(Implement the FilenameFilter interface method to use our match and ignore lists.File
dir,String
name)
public static void close(Object
o)
Close an object which has a close() method, ignoring any exceptions.
public void commit()Commit the transaction.
public void debug(String
mesg)
Out put a debug message.
Subclasses can override this method to change where messages go.
public int deleteDoc(This will remove the document with id of path from the database, including all of its implicit metadata.String
path)
throwsSQLException
SQLException
Related Topics
BulkLoader.deleteDocMetadata(String)
public void deleteDocMetadata(This will remove the implicit metadata for the specified document.String
path)
throwsSQLException
SQLException
Related Topics
BulkLoader.deleteDocMetadata(String)
public void doLoad()Do the actual bulk load logic on the file list.
throwsSQLException
SQLException
public void doLoad(Load the given path into the database.File
baseDir,String
path,Properties
dirProps)
throwsSQLException
If path is a directory, all files underneath it that match our patterns will be included. If path is a file, it will be loaded.
SQLException
public void error(Output an error message.String
mesg,Throwable
ex)
Subclasses can override this method to change where messages go.
public void error(String
mesg)
Output an error message.
protected void finalize()Called when this to be finialized.
throwsThrowable
Object.finalize()
Throwable
publicFix up a path to be forward-slash style and to not have empty path parts.String
fixPath(String
path)
publicFix up strings to check potentionally truncate too large values.String
fixString(String
in,String
tableName,String
colName)
public staticFix empty strings to be nulls.String
fixString(String
in)
publicGet the properties from the BulkLoader's LoaderFilters for the given file.Properties
getLoaderFilterProperties(File
f,Properties
p)
publicGet the metadata properties for the given file or directory.Properties
getMetadataProperties(File
base,Properties
p)
throwsIOException
This does not do a META data parse.
IOException
publicGet the property value for the specified additional column name.String
getPropertyForColumn(String
colName,Properties
p)
This will loop over the property names mapped to the column name and return the first found value in the properties.
public void go()Let the bulkloader go.
throwsSQLException
,IOException
SQLException
IOException
public void initialize()Initialize the bulkloader from the current state.
throwsSQLException
,IllegalStateException
This calls setJDBCInfo()
and then creates a
JDBC connection. It additinal configures any known column size
limitations and constructs any SQL statements it needs to.
SQLException
IllegalStateException
Related Topics
public int insertDoc(Update or insert a document and metadata into the database.String
path,File
f,Properties
p,String
mimeType)
throwsSQLException
SQLException
public static boolean isHidden(File
f)
Check if the specified file is a hidden file.
Under UNIX, the File.isHidden() reports that "/weblogicCommerce/dmsBase/." is a hidden file, which it is not. So, this fixes that problem by getting canonicals paths for directories before calling isHidden(). That seems to do the trick.
public boolean isHtmlFile(String
name)
Tell if the specified file name is an HTML file to the loader.
public static boolean isReadableDirectory(String
name)
Check if the specified file name is a directory that we can get into.
public void loadPropertyColumnInfo(Properties
p)
Load the set of jdbc.column.<columnName>=propName,... properties
into our property to column information.
public static int main(The main method invoked on a BulkLoader instance.BulkLoader
loader,String
[] args)
This will take a BulkLoader through the bulk loading steps. Output
will be sent via the BulkLoader's debug()
,
warning()
, and
error()
methods.
This will not call System.exit().
Related Topics
BulkLoader.parseArgs(String[])
BulkLoader.validateArgs()
BulkLoader.initialize()
BulkLoader.go()
BulkLoader.commit()
public static void main(String
[] args)
Command-line entry point.
This will call System.exit() on invalid args or error. To invoke a bulk load from your own code, create and manipulate a BulkLoader object. You can use the other main method, which does not exit.
Related Topics
BulkLoader.main(BulkLoader, String[])
public void parseArgs(Parse the given input arguments.String
[] args)
throwsIllegalArgumentException
IllegalArgumentException
public void printSchema()Print the schema xml for all the metadata we've loaded so far.
throwsIOException
IOException
public void printSchema(Print the schema xml for all the metadata we've loaded so far to the given output stream.PrintWriter
out,String
enc)
throwsIOException
IOException
public void rollback()Rollback the transaction.
protected void setJDBCInfo()Set the jdbc connection url and properties from the current weblogic.properties file.
throwsIllegalStateException
IllegalStateException
public boolean shouldIgnore(String
name)
Tell if the loader should ignore the specified file name.
public boolean shouldInclude(String
name)
Tell if the loader should include the specified file name.
public void shutdown()Shutdown this bulk loader.
public staticSplit a delimited list into a List.List
split(String
str,String
on)
public staticSplit a WLS connection pool style string in a Properties object of the name=value pairs.Properties
splitToProperties(String
str,String
on)
public void usage(PrintStream
out)
Print the usage of the application.
public void usage(PrintWriter
out)
Print the usage of the application.
public void validateArgs()Validate that we have been passed correct arguments.
throwsIllegalStateException
This does not validate that the arguments are valid. That will be done in initialize().
IllegalStateException
public void warning(Output a warning message.String
mesg,Throwable
ex)
Subclasses can override this method to change where messages go.
public void warning(String
mesg)
Output a warning message.