Class AzureProvider
- java.lang.Object
-
- com.ibm.fhir.bulkdata.provider.impl.AzureProvider
-
- All Implemented Interfaces:
Provider
public class AzureProvider extends Object implements Provider
AzureProvider integrates the BulkData feature with Azure Blob Storage. The code uses the Blob Clients. $export: The AppendBlobClient is used during upload to append to an export file. $import: The BlobClient with BlobRange is used to retrieve windows of data.
-
-
Constructor Summary
Constructors Constructor Description AzureProvider(String source)
Configures the Azure based on the storageProvider source
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
addParseFailure(int idx)
logs the parse failures.void
close()
void
createSource()
creates the base output location for the type.long
getNumberOfLoaded()
long
getNumberOfParseFailures()
List<Resource>
getResources()
gets the read resources.long
getSize(String workItem)
gets the size of the given work item.protected void
initializeBlobClient(String workItem)
initializes the blob clientvoid
listBlobsForContainer()
lists the blobs on the container.boolean
processLines(com.ibm.fhir.bulkdata.provider.impl.AzureProvider.CountInputStreamReader counter, BufferedReader reader, int window, boolean endOfFile, StringBuilder previous)
processes the lines in the ndjson file.void
pushOperationOutcomes()
Pushes the Operation Outcomesvoid
readResources(long numOfLinesToSkip, String workItem)
reads from a given workitem (or file) in a source and skips a certain noumber of linesprotected void
registerOverride(String connectionString, String container, ExportTransientUserData chunkData, String cosBucketPathPrefix, String fhirResourceType, long currentBytes)
registers the overrides for testing.void
registerTransient(long executionId, ExportTransientUserData transientUserData, String cosBucketPathPrefix, String fhirResourceType)
void
registerTransient(ImportTransientUserData transientUserData)
void
writeDirectly(String workItem, InputStream in, int size)
writes directly to Azure using AppendBlock clientvoid
writeResources(String mediaType, List<ReadResultDTO> dtos)
wraps the complexity of writing FHIR Resources out to a target
-
-
-
Constructor Detail
-
AzureProvider
public AzureProvider(String source)
Configures the Azure based on the storageProvider source- Parameters:
source
-
-
-
Method Detail
-
registerOverride
protected void registerOverride(String connectionString, String container, ExportTransientUserData chunkData, String cosBucketPathPrefix, String fhirResourceType, long currentBytes)
registers the overrides for testing.- Parameters:
connectionString
-container
- the container (also known as a bucket)chunkData
- the export datacosBucketPathPrefix
- path prefixfhirResourceType
- resource typecurrentBytes
- the long value or location of the end of the last resource
-
initializeBlobClient
protected void initializeBlobClient(String workItem)
initializes the blob client- Parameters:
workItem
-
-
createSource
public void createSource() throws FHIRException
Description copied from interface:Provider
creates the base output location for the type.- Specified by:
createSource
in interfaceProvider
- Throws:
FHIRException
-
getSize
public long getSize(String workItem) throws FHIRException
Description copied from interface:Provider
gets the size of the given work item.- Specified by:
getSize
in interfaceProvider
- Returns:
- Throws:
FHIRException
-
listBlobsForContainer
public void listBlobsForContainer() throws FHIRException
lists the blobs on the container.- Throws:
FHIRException
-
writeDirectly
public void writeDirectly(String workItem, InputStream in, int size) throws Exception
writes directly to Azure using AppendBlock client- Parameters:
workItem
- the filein
-size
-- Throws:
Exception
-
writeResources
public void writeResources(String mediaType, List<ReadResultDTO> dtos) throws Exception
Description copied from interface:Provider
wraps the complexity of writing FHIR Resources out to a target- Specified by:
writeResources
in interfaceProvider
- Throws:
Exception
-
readResources
public void readResources(long numOfLinesToSkip, String workItem) throws FHIRException
Description copied from interface:Provider
reads from a given workitem (or file) in a source and skips a certain noumber of lines- Specified by:
readResources
in interfaceProvider
- Throws:
FHIRException
-
processLines
public boolean processLines(com.ibm.fhir.bulkdata.provider.impl.AzureProvider.CountInputStreamReader counter, BufferedReader reader, int window, boolean endOfFile, StringBuilder previous) throws FHIRGeneratorException, IOException
processes the lines in the ndjson file.- Parameters:
counter
-reader
-window
-endOfFile
-previous
-- Returns:
- Throws:
FHIRGeneratorException
IOException
-
addParseFailure
public void addParseFailure(int idx) throws FHIRGeneratorException, IOException
logs the parse failures.- Parameters:
idx
-- Throws:
FHIRGeneratorException
IOException
-
pushOperationOutcomes
public void pushOperationOutcomes() throws FHIRException
Description copied from interface:Provider
Pushes the Operation Outcomes- Specified by:
pushOperationOutcomes
in interfaceProvider
- Throws:
FHIRException
-
close
public void close() throws Exception
-
getResources
public List<Resource> getResources() throws FHIRException
Description copied from interface:Provider
gets the read resources.- Specified by:
getResources
in interfaceProvider
- Returns:
- Throws:
FHIRException
-
getNumberOfParseFailures
public long getNumberOfParseFailures() throws FHIRException
- Specified by:
getNumberOfParseFailures
in interfaceProvider
- Throws:
FHIRException
-
getNumberOfLoaded
public long getNumberOfLoaded() throws FHIRException
- Specified by:
getNumberOfLoaded
in interfaceProvider
- Throws:
FHIRException
-
registerTransient
public void registerTransient(ImportTransientUserData transientUserData)
- Specified by:
registerTransient
in interfaceProvider
-
registerTransient
public void registerTransient(long executionId, ExportTransientUserData transientUserData, String cosBucketPathPrefix, String fhirResourceType) throws Exception
- Specified by:
registerTransient
in interfaceProvider
- Throws:
Exception
-
-