Class AzureProvider
- java.lang.Object
-
- org.linuxforhealth.fhir.bulkdata.provider.impl.AzureProvider
-
- All Implemented Interfaces:
Provider
public class AzureProvider extends java.lang.Object implements Provider
AzureProvider integrates the BulkData feature with Azure Blob Storage. The code uses the Blob Clients. $export: The AppendBlobClient is used during upload to append to an export file. $import: The BlobClient with BlobRange is used to retrieve windows of data.
-
-
Constructor Summary
Constructors Constructor Description AzureProvider(java.lang.String source)
Configures the Azure based on the storageProvider source
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
addParseFailure(int idx)
logs the parse failures.void
close()
Closes the StorageProvider wrapped resources.void
createSource()
creates the base output location for the type.long
getNumberOfLoaded()
long
getNumberOfParseFailures()
java.util.List<Resource>
getResources()
gets the read resources.long
getSize(java.lang.String workItem)
gets the size of the given work item.protected void
initializeBlobClient(java.lang.String workItem)
initializes the blob clientvoid
listBlobsForContainer()
lists the blobs on the container.boolean
processLines(org.linuxforhealth.fhir.bulkdata.provider.impl.AzureProvider.CountInputStreamReader counter, java.io.BufferedReader reader, int window, boolean endOfFile, java.lang.StringBuilder previous)
processes the lines in the ndjson file.void
pushEndOfJobOperationOutcomes(java.io.ByteArrayOutputStream baos, java.lang.String folder, java.lang.String fileName)
Pushes End of Job OperationOutcomes and closes the Outcomes StorageProvidervoid
pushOperationOutcomes()
Pushes the Operation Outcomesvoid
readResources(long numOfLinesToSkip, java.lang.String workItem)
reads from a given workitem (or file) in a source and skips a certain noumber of linesprotected void
registerOverride(java.lang.String connectionString, java.lang.String container, ExportTransientUserData chunkData, java.lang.String cosBucketPathPrefix, java.lang.String fhirResourceType, long currentBytes)
registers the overrides for testing.void
registerTransient(long executionId, ExportTransientUserData transientUserData, java.lang.String cosBucketPathPrefix, java.lang.String fhirResourceType)
void
registerTransient(ImportTransientUserData transientUserData)
void
writeDirectly(java.lang.String workItem, java.io.InputStream in, int size)
writes directly to Azure using AppendBlock clientvoid
writeResources(java.lang.String mediaType, java.util.List<ReadResultDTO> dtos)
wraps the complexity of writing FHIR Resources out to a target
-
-
-
Method Detail
-
registerOverride
protected void registerOverride(java.lang.String connectionString, java.lang.String container, ExportTransientUserData chunkData, java.lang.String cosBucketPathPrefix, java.lang.String fhirResourceType, long currentBytes)
registers the overrides for testing.- Parameters:
connectionString
-container
- the container (also known as a bucket)chunkData
- the export datacosBucketPathPrefix
- path prefixfhirResourceType
- resource typecurrentBytes
- the long value or location of the end of the last resource
-
initializeBlobClient
protected void initializeBlobClient(java.lang.String workItem)
initializes the blob client- Parameters:
workItem
-
-
createSource
public void createSource() throws FHIRException
Description copied from interface:Provider
creates the base output location for the type.- Specified by:
createSource
in interfaceProvider
- Throws:
FHIRException
-
getSize
public long getSize(java.lang.String workItem) throws FHIRException
Description copied from interface:Provider
gets the size of the given work item.- Specified by:
getSize
in interfaceProvider
- Returns:
- Throws:
FHIRException
-
listBlobsForContainer
public void listBlobsForContainer() throws FHIRException
lists the blobs on the container.- Throws:
FHIRException
-
writeDirectly
public void writeDirectly(java.lang.String workItem, java.io.InputStream in, int size) throws java.lang.Exception
writes directly to Azure using AppendBlock client- Parameters:
workItem
- the filein
-size
-- Throws:
java.lang.Exception
-
writeResources
public void writeResources(java.lang.String mediaType, java.util.List<ReadResultDTO> dtos) throws java.lang.Exception
Description copied from interface:Provider
wraps the complexity of writing FHIR Resources out to a target- Specified by:
writeResources
in interfaceProvider
- Throws:
java.lang.Exception
-
readResources
public void readResources(long numOfLinesToSkip, java.lang.String workItem) throws FHIRException
Description copied from interface:Provider
reads from a given workitem (or file) in a source and skips a certain noumber of lines- Specified by:
readResources
in interfaceProvider
- Throws:
FHIRException
-
processLines
public boolean processLines(org.linuxforhealth.fhir.bulkdata.provider.impl.AzureProvider.CountInputStreamReader counter, java.io.BufferedReader reader, int window, boolean endOfFile, java.lang.StringBuilder previous) throws FHIRGeneratorException, java.io.IOException
processes the lines in the ndjson file.- Parameters:
counter
-reader
-window
-endOfFile
-previous
-- Returns:
- Throws:
FHIRGeneratorException
java.io.IOException
-
addParseFailure
public void addParseFailure(int idx) throws FHIRGeneratorException, java.io.IOException
logs the parse failures.- Parameters:
idx
-- Throws:
FHIRGeneratorException
java.io.IOException
-
pushOperationOutcomes
public void pushOperationOutcomes() throws FHIRException
Description copied from interface:Provider
Pushes the Operation Outcomes- Specified by:
pushOperationOutcomes
in interfaceProvider
- Throws:
FHIRException
-
close
public void close() throws java.lang.Exception
Description copied from interface:Provider
Closes the StorageProvider wrapped resources.
-
getResources
public java.util.List<Resource> getResources() throws FHIRException
Description copied from interface:Provider
gets the read resources.- Specified by:
getResources
in interfaceProvider
- Returns:
- Throws:
FHIRException
-
getNumberOfParseFailures
public long getNumberOfParseFailures() throws FHIRException
- Specified by:
getNumberOfParseFailures
in interfaceProvider
- Throws:
FHIRException
-
getNumberOfLoaded
public long getNumberOfLoaded() throws FHIRException
- Specified by:
getNumberOfLoaded
in interfaceProvider
- Throws:
FHIRException
-
registerTransient
public void registerTransient(ImportTransientUserData transientUserData)
- Specified by:
registerTransient
in interfaceProvider
-
registerTransient
public void registerTransient(long executionId, ExportTransientUserData transientUserData, java.lang.String cosBucketPathPrefix, java.lang.String fhirResourceType) throws java.lang.Exception
- Specified by:
registerTransient
in interfaceProvider
- Throws:
java.lang.Exception
-
pushEndOfJobOperationOutcomes
public void pushEndOfJobOperationOutcomes(java.io.ByteArrayOutputStream baos, java.lang.String folder, java.lang.String fileName) throws FHIRException
Description copied from interface:Provider
Pushes End of Job OperationOutcomes and closes the Outcomes StorageProvider- Specified by:
pushEndOfJobOperationOutcomes
in interfaceProvider
- Parameters:
baos
- the byte array output stream that is passed infolder
- the folder where the file is to be storedfileName
- the output filename to be used- Throws:
FHIRException
-
-