Package org.linuxforhealth.fhir.bulkdata.provider.impl

Class AzureProvider

  • All Implemented Interfaces:
    Provider
    public class AzureProvider
extends java.lang.Object
implements Provider
    AzureProvider integrates the BulkData feature with Azure Blob Storage. The code uses the Blob Clients. $export: The AppendBlobClient is used during upload to append to an export file. $import: The BlobClient with BlobRange is used to retrieve windows of data.

    • Constructor Summary

      Constructors 
      Constructor Description
      AzureProvider​(java.lang.String source)
      Configures the Azure based on the storageProvider source

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void addParseFailure​(int idx)
      logs the parse failures.
      void close()
      Closes the StorageProvider wrapped resources.
      void createSource()
      creates the base output location for the type.
      long getNumberOfLoaded()  
      long getNumberOfParseFailures()  
      java.util.List<Resource> getResources()
      gets the read resources.
      long getSize​(java.lang.String workItem)
      gets the size of the given work item.
      protected void initializeBlobClient​(java.lang.String workItem)
      initializes the blob client
      void listBlobsForContainer()
      lists the blobs on the container.
      boolean processLines​(org.linuxforhealth.fhir.bulkdata.provider.impl.AzureProvider.CountInputStreamReader counter, java.io.BufferedReader reader, int window, boolean endOfFile, java.lang.StringBuilder previous)
      processes the lines in the ndjson file.
      void pushEndOfJobOperationOutcomes​(java.io.ByteArrayOutputStream baos, java.lang.String folder, java.lang.String fileName)
      Pushes End of Job OperationOutcomes and closes the Outcomes StorageProvider
      void pushOperationOutcomes()
      Pushes the Operation Outcomes
      void readResources​(long numOfLinesToSkip, java.lang.String workItem)
      reads from a given workitem (or file) in a source and skips a certain noumber of lines
      protected void registerOverride​(java.lang.String connectionString, java.lang.String container, ExportTransientUserData chunkData, java.lang.String cosBucketPathPrefix, java.lang.String fhirResourceType, long currentBytes)
      registers the overrides for testing.
      void registerTransient​(long executionId, ExportTransientUserData transientUserData, java.lang.String cosBucketPathPrefix, java.lang.String fhirResourceType)  
      void registerTransient​(ImportTransientUserData transientUserData)  
      void writeDirectly​(java.lang.String workItem, java.io.InputStream in, int size)
      writes directly to Azure using AppendBlock client
      void writeResources​(java.lang.String mediaType, java.util.List<ReadResultDTO> dtos)
      wraps the complexity of writing FHIR Resources out to a target

      • Methods inherited from class java.lang.Object

        clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • AzureProvider

        public AzureProvider​(java.lang.String source)
        Configures the Azure based on the storageProvider source
        Parameters:
        source -

    • Method Detail

      • registerOverride

        protected void registerOverride​(java.lang.String connectionString,
                                java.lang.String container,
                                ExportTransientUserData chunkData,
                                java.lang.String cosBucketPathPrefix,
                                java.lang.String fhirResourceType,
                                long currentBytes)
        registers the overrides for testing.
        Parameters:
        connectionString -
        container - the container (also known as a bucket)
        chunkData - the export data
        cosBucketPathPrefix - path prefix
        fhirResourceType - resource type
        currentBytes - the long value or location of the end of the last resource

      • initializeBlobClient

        protected void initializeBlobClient​(java.lang.String workItem)
        initializes the blob client
        Parameters:
        workItem -

      • listBlobsForContainer

        public void listBlobsForContainer()
                           throws FHIRException
        lists the blobs on the container.
        Throws:
        FHIRException

      • writeDirectly

        public void writeDirectly​(java.lang.String workItem,
                          java.io.InputStream in,
                          int size)
                   throws java.lang.Exception
        writes directly to Azure using AppendBlock client
        Parameters:
        workItem - the file
        in -
        size -
        Throws:
        java.lang.Exception

      • writeResources

        public void writeResources​(java.lang.String mediaType,
                           java.util.List<ReadResultDTO> dtos)
                    throws java.lang.Exception
        Description copied from interface: Provider
        wraps the complexity of writing FHIR Resources out to a target
        Specified by:
        writeResources in interface Provider
        Throws:
        java.lang.Exception

      • readResources

        public void readResources​(long numOfLinesToSkip,
                          java.lang.String workItem)
                   throws FHIRException
        Description copied from interface: Provider
        reads from a given workitem (or file) in a source and skips a certain noumber of lines
        Specified by:
        readResources in interface Provider
        Throws:
        FHIRException

      • processLines

        public boolean processLines​(org.linuxforhealth.fhir.bulkdata.provider.impl.AzureProvider.CountInputStreamReader counter,
                            java.io.BufferedReader reader,
                            int window,
                            boolean endOfFile,
                            java.lang.StringBuilder previous)
                     throws FHIRGeneratorException,
                            java.io.IOException
        processes the lines in the ndjson file.
        Parameters:
        counter -
        reader -
        window -
        endOfFile -
        previous -
        Returns:
        Throws:
        FHIRGeneratorException
        java.io.IOException

      • close

        public void close()
           throws java.lang.Exception
        Description copied from interface: Provider
        Closes the StorageProvider wrapped resources.
        Specified by:
        close in interface Provider
        Throws:
        java.lang.Exception

      • registerTransient

        public void registerTransient​(long executionId,
                              ExportTransientUserData transientUserData,
                              java.lang.String cosBucketPathPrefix,
                              java.lang.String fhirResourceType)
                       throws java.lang.Exception
        Specified by:
        registerTransient in interface Provider
        Throws:
        java.lang.Exception

      • pushEndOfJobOperationOutcomes

        public void pushEndOfJobOperationOutcomes​(java.io.ByteArrayOutputStream baos,
                                          java.lang.String folder,
                                          java.lang.String fileName)
                                   throws FHIRException
        Description copied from interface: Provider
        Pushes End of Job OperationOutcomes and closes the Outcomes StorageProvider
        Specified by:
        pushEndOfJobOperationOutcomes in interface Provider
        Parameters:
        baos - the byte array output stream that is passed in
        folder - the folder where the file is to be stored
        fileName - the output filename to be used
        Throws:
        FHIRException