Package com.ibm.fhir.jbatch.bulkdata.common

Class BulkDataUtils

java.lang.Object

com.ibm.fhir.jbatch.bulkdata.common.BulkDataUtils

public class BulkDataUtils
extends Object

Utility functions for IBM COS.

Constructor Summary

Constructors

Constructor	Description
BulkDataUtils()

Method Summary

Modifier and Type	Method	Description
static void	cleanupTransientUserData(ImportTransientUserData transientUserData, boolean isAbort)
static void	finishMultiPartUpload(com.ibm.cloud.objectstorage.services.s3.AmazonS3 cosClient, String bucketName, String itemName, String uploadID, List<com.ibm.cloud.objectstorage.services.s3.model.PartETag> dataPacks)
static com.ibm.cloud.objectstorage.services.s3.AmazonS3	getCosClient(String cosCredentialIbm, String cosApiKeyProperty, String cosSrvinstId, String cosEndpointUrl, String cosLocation, boolean useFhirServerTrustStore)
static long	getCosFileSize(com.ibm.cloud.objectstorage.services.s3.AmazonS3 cosClient, String bucketName, String itemName)
static javax.json.JsonArray	getDataSourcesFromJobInput(String dataSourcesInfo)
static long	getHttpsFileSize(String dataUrl)
static long	getLocalFileSize(String filePath)
static Map<Class<? extends Resource>,List<Map<String,List<String>>>>	getSearchParemetersFromTypeFilters(String typeFilters)
static void	listBuckets(com.ibm.cloud.objectstorage.services.s3.AmazonS3 cosClient)
static com.ibm.cloud.objectstorage.services.s3.model.PartETag	multiPartUpload(com.ibm.cloud.objectstorage.services.s3.AmazonS3 cosClient, String bucketName, String itemName, String uploadID, InputStream dataStream, int partSize, int partNum)
static int	readFhirResourceFromHttps(String dataUrl, int numOfLinesToSkip, List<Resource> fhirResources, ImportTransientUserData transientUserData)
static int	readFhirResourceFromLocalFile(String filePath, int numOfLinesToSkip, List<Resource> fhirResources, ImportTransientUserData transientUserData)
static int	readFhirResourceFromObjectStore(com.ibm.cloud.objectstorage.services.s3.AmazonS3 cosClient, String bucketName, String itemName, int numOfLinesToSkip, List<Resource> fhirResources, ImportTransientUserData transientUserData)
static String	startPartUpload(com.ibm.cloud.objectstorage.services.s3.AmazonS3 cosClient, String bucketName, String itemName, boolean isPublicAccess)
static List<OperationOutcome.Issue>	validateInput(Resource resource)	Validate the input resource and throw if there are validation errors

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BulkDataUtils

public BulkDataUtils()

Method Detail

startPartUpload

public static String startPartUpload(com.ibm.cloud.objectstorage.services.s3.AmazonS3 cosClient,
                                     String bucketName,
                                     String itemName,
                                     boolean isPublicAccess)
                              throws Exception

Throws:

Exception

getCosClient

public static com.ibm.cloud.objectstorage.services.s3.AmazonS3 getCosClient(String cosCredentialIbm,
                                                                            String cosApiKeyProperty,
                                                                            String cosSrvinstId,
                                                                            String cosEndpointUrl,
                                                                            String cosLocation,
                                                                            boolean useFhirServerTrustStore)

multiPartUpload

public static com.ibm.cloud.objectstorage.services.s3.model.PartETag multiPartUpload(com.ibm.cloud.objectstorage.services.s3.AmazonS3 cosClient,
                                                                                     String bucketName,
                                                                                     String itemName,
                                                                                     String uploadID,
                                                                                     InputStream dataStream,
                                                                                     int partSize,
                                                                                     int partNum)
                                                                              throws Exception

Throws:

Exception

finishMultiPartUpload

public static void finishMultiPartUpload(com.ibm.cloud.objectstorage.services.s3.AmazonS3 cosClient,
                                         String bucketName,
                                         String itemName,
                                         String uploadID,
                                         List<com.ibm.cloud.objectstorage.services.s3.model.PartETag> dataPacks)
                                  throws Exception

Throws:

Exception

listBuckets

public static void listBuckets(com.ibm.cloud.objectstorage.services.s3.AmazonS3 cosClient)

cleanupTransientUserData

public static void cleanupTransientUserData(ImportTransientUserData transientUserData,
                                            boolean isAbort)
                                     throws Exception

Throws:

Exception

readFhirResourceFromObjectStore

public static int readFhirResourceFromObjectStore(com.ibm.cloud.objectstorage.services.s3.AmazonS3 cosClient,
                                                  String bucketName,
                                                  String itemName,
                                                  int numOfLinesToSkip,
                                                  List<Resource> fhirResources,
                                                  ImportTransientUserData transientUserData)
                                           throws Exception

Parameters:

cosClient - - COS/S3 client.

bucketName - - COS/S3 bucket name to read from.

itemName - - COS/S3 object name to read from.

numOfLinesToSkip - - number of lines to skip before read.

fhirResources - - List holds the FHIR resources.

transientUserData - - transient user data for the chunk.

Returns:

- number of parsing failures.

Throws:

Exception

getCosFileSize

public static long getCosFileSize(com.ibm.cloud.objectstorage.services.s3.AmazonS3 cosClient,
                                  String bucketName,
                                  String itemName)
                           throws Exception

Throws:

Exception

readFhirResourceFromLocalFile

public static int readFhirResourceFromLocalFile(String filePath,
                                                int numOfLinesToSkip,
                                                List<Resource> fhirResources,
                                                ImportTransientUserData transientUserData)
                                         throws Exception

Parameters:

filePath - - file path to the ndjson file.

numOfLinesToSkip - - number of lines to skip before read.

fhirResources - - List holds the FHIR resources.

transientUserData - - transient user data for the chunk.

Returns:

- number of parsing failures.

Throws:

Exception

getLocalFileSize

public static long getLocalFileSize(String filePath)
                             throws Exception

Throws:

Exception

readFhirResourceFromHttps

public static int readFhirResourceFromHttps(String dataUrl,
                                            int numOfLinesToSkip,
                                            List<Resource> fhirResources,
                                            ImportTransientUserData transientUserData)
                                     throws Exception

Parameters:

dataUrl - - URL to the ndjson file.

numOfLinesToSkip - - number of lines to skip before read.

fhirResources - - List holds the FHIR resources.

transientUserData - - transient user data for the chunk.

Returns:

- number of parsing failures.

Throws:

Exception

getHttpsFileSize

public static long getHttpsFileSize(String dataUrl)
                             throws Exception

Throws:

Exception

validateInput

public static List<OperationOutcome.Issue> validateInput(Resource resource)
                                                  throws FHIRValidationException,
                                                         FHIROperationException

Validate the input resource and throw if there are validation errors

Parameters:

resource -

Throws:

FHIRValidationException

FHIROperationException

getSearchParemetersFromTypeFilters

public static Map<Class<? extends Resource>,List<Map<String,List<String>>>> getSearchParemetersFromTypeFilters(String typeFilters)
                                                                                                                    throws Exception

Throws:

Exception

getDataSourcesFromJobInput

public static javax.json.JsonArray getDataSourcesFromJobInput(String dataSourcesInfo)