Package com.ibm.fhir.bulkdata.provider.impl

Class S3Provider

java.lang.Object

com.ibm.fhir.bulkdata.provider.impl.S3Provider

All Implemented Interfaces:

Provider

public class S3Provider
extends Object
implements Provider

S3Provider for ibm-cos and aws-s3

Constructor Summary

Constructors

Constructor	Description
S3Provider(String source)	The provider loads the Client based on the Source.

Method Summary

Modifier and Type	Method	Description
void	close()
void	createSource()	creates the base output location for the type.
boolean	exists()	checks to see if the bucket exists.
com.ibm.cloud.objectstorage.services.s3.AmazonS3	getClient()
com.ibm.cloud.objectstorage.services.s3.model.ListObjectsV2Result	getListObject(String prefix, String continuationToken)	get the list of objects based on the continuation token.
long	getNumberOfLoaded()
long	getNumberOfParseFailures()
SparkParquetWriter	getParquetWriter()
List<Resource>	getResources()	gets the read resources.
long	getSize(String workItem)	gets the size of the given work item.
void	listBuckets()	lists the buckets when logging out bucket details
void	pushOperationOutcomes()	Pushes the Operation Outcomes
void	readResources(long numOfLinesToSkip, String workItem)	reads from a given workitem (or file) in a source and skips a certain noumber of lines
void	registerTransient(long executionId, ExportTransientUserData transientUserData, String cosBucketPathPrefix, String fhirResourceType)
void	registerTransient(ImportTransientUserData transientUserData)
void	writeResources(String mediaType, List<ReadResultDTO> dtos)	wraps the complexity of writing FHIR Resources out to a target

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

S3Provider

public S3Provider(String source)
           throws FHIRException

The provider loads the Client based on the Source. The Source (storageProvider) must be s3 compatible.

Parameters:

source -

Throws:

FHIRException

Method Detail

exists

public boolean exists()

checks to see if the bucket exists.

Returns:

listBuckets

public void listBuckets()

lists the buckets when logging out bucket details

getListObject

public com.ibm.cloud.objectstorage.services.s3.model.ListObjectsV2Result getListObject(String prefix,
                                                                                       String continuationToken)
                                                                                throws FHIRException

get the list of objects based on the continuation token.

Parameters:

prefix - the search key

continuationToken - null or string indicating a continuation token.

Returns:

Throws:

FHIRException

getSize

public long getSize(String workItem)
             throws FHIRException

Description copied from interface: Provider

gets the size of the given work item.

Specified by:

getSize in interface Provider

Returns:

Throws:

FHIRException

getResources

public List<Resource> getResources()
                            throws FHIRException

Description copied from interface: Provider

gets the read resources.

Specified by:

getResources in interface Provider

Returns:

Throws:

FHIRException

getNumberOfParseFailures

public long getNumberOfParseFailures()
                              throws FHIRException

Specified by:

getNumberOfParseFailures in interface Provider

Throws:

FHIRException

registerTransient

public void registerTransient(ImportTransientUserData transientUserData)

Specified by:

registerTransient in interface Provider

getNumberOfLoaded

public long getNumberOfLoaded()
                       throws FHIRException

Specified by:

getNumberOfLoaded in interface Provider

Throws:

FHIRException

readResources

public void readResources(long numOfLinesToSkip,
                          String workItem)
                   throws FHIRException

Description copied from interface: Provider

reads from a given workitem (or file) in a source and skips a certain noumber of lines

Specified by:

readResources in interface Provider

Throws:

FHIRException

pushOperationOutcomes

public void pushOperationOutcomes()
                           throws FHIRException

Description copied from interface: Provider

Pushes the Operation Outcomes

Specified by:

pushOperationOutcomes in interface Provider

Throws:

FHIRException

createSource

public void createSource()
                  throws FHIRException

Description copied from interface: Provider

creates the base output location for the type.

Specified by:

createSource in interface Provider

Throws:

FHIRException

getClient

public com.ibm.cloud.objectstorage.services.s3.AmazonS3 getClient()

getParquetWriter

public SparkParquetWriter getParquetWriter()

close

public void close()
           throws Exception

Specified by:

close in interface Provider

Throws:

Exception

writeResources

public void writeResources(String mediaType,
                           List<ReadResultDTO> dtos)
                    throws Exception

Description copied from interface: Provider

wraps the complexity of writing FHIR Resources out to a target

Specified by:

writeResources in interface Provider

Throws:

Exception

registerTransient

public void registerTransient(long executionId,
                              ExportTransientUserData transientUserData,
                              String cosBucketPathPrefix,
                              String fhirResourceType)
                       throws Exception

Specified by:

registerTransient in interface Provider

Throws:

Exception