...
▽

Package transfer

import "github.com/oracle/oci-go-sdk/objectstorage/transfer"
Overview
Index

Overview ▾

Package transfer simplifies interaction with the Object Storage service by abstracting away the method used to upload objects. Depending on the configuration parameters, UploadManager may choose to do a single put_object request, or break up the upload into multiple parts and utilize multi-part uploads.

An advantage of using multi-part uploads is the ability to retry individual failed parts, as well as being able to upload parts in parallel to reduce upload time.

To use this package, you must be authorized in an IAM policy. If you're not authorized, talk to an administrator.

type FileUploader

FileUploader is an interface to upload a file 

type FileUploader interface {
    // split file into multiple parts and uploads them to blob storage, then merge
    UploadFileMultiparts(ctx context.Context, request UploadFileRequest) (response UploadResponse, err error)

    // uploads a file to blob storage via PutObject API
    UploadFilePutObject(ctx context.Context, request UploadFileRequest) (response UploadResponse, err error)

    // resume a file upload, use it when UploadFile failed
    ResumeUploadFile(ctx context.Context, uploadID string) (response UploadResponse, err error)
}

type MultiPartUploadPart

MultiPartUploadPart holds the details of Part that is uploaded 

type MultiPartUploadPart struct {
    PartNum    int
    TotalParts int
    Size       int64
    Offset     int64
    Hash       *string
    OpcMD5     *string
    Etag       *string
    Err        error
}

type MultipartUploadResponse

MultipartUploadResponse is the response from commitMultipart API operation. 

type MultipartUploadResponse struct {
    objectstorage.CommitMultipartUploadResponse

    // The upload ID for a multipart upload.
    UploadID *string
    // contains filtered or unexported fields
}

type SinglepartUploadResponse

SinglepartUploadResponse is the response from putObject API operation. 

type SinglepartUploadResponse struct {
    objectstorage.PutObjectResponse
}

type StreamUploader

StreamUploader is an interface for upload a stream 

type StreamUploader interface {
    // uploads a stream to blob storage
    UploadStream(ctx context.Context, request UploadStreamRequest) (response UploadResponse, err error)
}

type UploadCallBack

UploadCallBack API that gets invoked after a Part is successuly uploaded 

type UploadCallBack func(multiPartUploadPart MultiPartUploadPart)

type UploadFileRequest

UploadFileRequest defines the input parameters for UploadFile method 

type UploadFileRequest struct {
    UploadRequest

    // The path of the file to be uploaded (includs file name)
    FilePath string
}

type UploadManager

UploadManager is the interface that groups the upload methods 

type UploadManager struct {
    FileUploader   FileUploader
    StreamUploader StreamUploader
}

func NewUploadManager 

func NewUploadManager() *UploadManager

NewUploadManager return a pointer to UploadManager

func (*UploadManager) ResumeUploadFile 

func (uploadManager *UploadManager) ResumeUploadFile(ctx context.Context, uploadID string) (response UploadResponse, err error)

ResumeUploadFile resumes a multipart file upload.

func (*UploadManager) UploadFile 

func (uploadManager *UploadManager) UploadFile(ctx context.Context, request UploadFileRequest) (response UploadResponse, err error)

UploadFile uploads an object to Object Storage. Depending on the options provided and the size of the object, the object may be uploaded in multiple parts or just an signle object.

func (*UploadManager) UploadStream 

func (uploadManager *UploadManager) UploadStream(ctx context.Context, request UploadStreamRequest) (response UploadResponse, err error)

UploadStream uploads streaming data to Object Storage. If the stream is non-empty, this will always perform a multipart upload, splitting parts based on the part size (10 MiB if none specified). If the stream is empty, this will upload a single empty object to Object Storage. Stream uploads are not currently resumable.

type UploadRequest

UploadRequest defines the input parameters for UploadFile method 

type UploadRequest struct {
    // The top-level namespace used for the request.
    NamespaceName *string `mandatory:"true"`

    // The name of the bucket. Avoid entering confidential information. Example: my-new-bucket1
    BucketName *string `mandatory:"true"`

    // The name of the object. Avoid entering confidential information. Example: test/object1.log
    ObjectName *string `mandatory:"true"`

    // [Optional] Override the default part size of 128 MiB, value is in bytes.
    // The max part size is 50GiB
    PartSize *int64 `mandatory:"false"`

    // [Optional] Whether or not this UploadManager supports performing mulitpart uploads. Defaults to True.
    AllowMultipartUploads *bool `mandatory:"false"`

    // [Optional] Whether or not this UploadManager supports uploading individual parts of a multipart upload in parallel.
    // This setting has no effect on uploads that are performed using a single put_object call. Defaults to True.
    AllowParrallelUploads *bool `mandatory:"false"`

    // The number of go routines for uploading individual parts of a multipart upload.
    // This setting is only used if allow_parallel_uploads is set to True. Defaults to 5.
    // The upper bounds of the number is 10,000.
    NumberOfGoroutines *int `mandatory:"false"`

    // A configured object storage client to use for interacting with the Object Storage service.
    // Default timeout is 60s which includes the time for reading the body.
    // Default timeout doesn't work for big file size and big part size(once upload each part longer than 60s), need to manually update timeout to support big file upload.
    ObjectStorageClient *objectstorage.ObjectStorageClient `mandatory:"false"`

    // [Optional] The entity tag of the object to match.
    IfMatch *string `mandatory:"false"`

    // [Optional] The entity tag of the object to avoid matching. The only valid value is ‘*’,
    // which indicates that the request should fail if the object already exists.
    IfNoneMatch *string `mandatory:"false"`

    // [Optional] The base-64 encoded MD5 hash of the body. This parameter is only used if the object is uploaded in a single part.
    ContentMD5 *string `mandatory:"false"`

    // [Optional] The content type of the object to upload.
    ContentType *string `mandatory:"false"`

    // [Optional] The content language of the object to upload.
    ContentLanguage *string `mandatory:"false"`

    // [Optional] The content encoding of the object to upload.
    ContentEncoding *string `mandatory:"false"`

    // [Optional] Arbitrary string keys and values for the user-defined metadata for the object.
    // Keys must be in "opc-meta-*" format.
    Metadata map[string]string `mandatory:"false"`

    // [Optional] The client request ID for tracing.
    OpcClientRequestID *string `mandatory:"false"`

    // Metadata about the request. This information will not be transmitted to the service, but
    // represents information that the SDK will consume to drive retry behavior.
    RequestMetadata common.RequestMetadata

    // [Optional] The storage tier of the object to upload. If not specified, the storage tier is
    // defaulted to 'Standard'
    StorageTier objectstorage.PutObjectStorageTierEnum `mandatory:"false"`

    // [Optional] Callback API that can be invoked during multiPartUploads
    CallBack UploadCallBack `mandatory:"false"`

    // [Optional] Whether or not this UploadManager supports performing multipart uploads md5 checksum verification. Defaults to False.
    EnableMultipartChecksumVerification *bool `mandatory:"false"`

    // The optional header that specifies "AES256" as the encryption algorithm. For more information, see
    // Using Your Own Keys for Server-Side Encryption (https://docs.oracle.com/iaas/Content/Object/Tasks/usingyourencryptionkeys.htm).
    OpcSseCustomerAlgorithm *string `mandatory:"false"`

    // The optional header that specifies the base64-encoded 256-bit encryption key to use to encrypt or
    // decrypt the data. For more information, see
    // Using Your Own Keys for Server-Side Encryption (https://docs.oracle.com/iaas/Content/Object/Tasks/usingyourencryptionkeys.htm).
    OpcSseCustomerKey *string `mandatory:"false"`

    // The optional header that specifies the base64-encoded SHA256 hash of the encryption key. This
    // value is used to check the integrity of the encryption key. For more information, see
    // Using Your Own Keys for Server-Side Encryption (https://docs.oracle.com/iaas/Content/Object/Tasks/usingyourencryptionkeys.htm).
    OpcSseCustomerKeySha256 *string `mandatory:"false"`

    // The OCID (https://docs.oracle.com/iaas/Content/General/Concepts/identifiers.htm) of a master encryption key used to call the Key
    // Management service to generate a data encryption key or to encrypt or decrypt a data encryption key.
    OpcSseKmsKeyId *string `mandatory:"false"`
}

func (UploadRequest) RetryPolicy 

func (request UploadRequest) RetryPolicy() *common.RetryPolicy

RetryPolicy implements the OCIRetryableRequest interface. This retrieves the specified retry policy.

type UploadResponse

UploadResponse is the response from commitMultipart or the putObject API operation. 

type UploadResponse struct {

    // Polymorphic response type indicates the response type
    Type UploadResponseType

    // response for putObject API response (single part upload), will be nil if the operation is multiPart upload
    *SinglepartUploadResponse

    // response for commitMultipart API response (multipart upload), will be nil if the operation is singlePart upload
    *MultipartUploadResponse
}

func (UploadResponse) IsResumable 

func (resp UploadResponse) IsResumable() bool

IsResumable is a function to check is previous failed upload resumable or not

type UploadResponseType

UploadResponseType with underlying type: string 

type UploadResponseType string

Set of constants representing the allowable values for VolumeAttachmentLifecycleState 

const (
    MultipartUpload  UploadResponseType = "MULTIPARTUPLOAD"
    SinglepartUpload UploadResponseType = "SINGLEPARTUPLOAD"
)

type UploadStreamRequest

UploadStreamRequest defines the input parameters for UploadFile method 

type UploadStreamRequest struct {
    UploadRequest

    // The reader of input stream
    StreamReader io.Reader
}