Class AzureProvider

  • All Implemented Interfaces:
    Provider

    public class AzureProvider
    extends java.lang.Object
    implements Provider
    AzureProvider integrates the BulkData feature with Azure Blob Storage. The code uses the Blob Clients. $export: The AppendBlobClient is used during upload to append to an export file. $import: The BlobClient with BlobRange is used to retrieve windows of data.
    • Constructor Summary

      Constructors 
      Constructor Description
      AzureProvider​(java.lang.String source)
      Configures the Azure based on the storageProvider source
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void addParseFailure​(int idx)
      logs the parse failures.
      void close()
      Closes the StorageProvider wrapped resources.
      void createSource()
      creates the base output location for the type.
      long getNumberOfLoaded()  
      long getNumberOfParseFailures()  
      java.util.List<Resource> getResources()
      gets the read resources.
      long getSize​(java.lang.String workItem)
      gets the size of the given work item.
      protected void initializeBlobClient​(java.lang.String workItem)
      initializes the blob client
      void listBlobsForContainer()
      lists the blobs on the container.
      boolean processLines​(org.linuxforhealth.fhir.bulkdata.provider.impl.AzureProvider.CountInputStreamReader counter, java.io.BufferedReader reader, int window, boolean endOfFile, java.lang.StringBuilder previous)
      processes the lines in the ndjson file.
      void pushEndOfJobOperationOutcomes​(java.io.ByteArrayOutputStream baos, java.lang.String folder, java.lang.String fileName)
      Pushes End of Job OperationOutcomes and closes the Outcomes StorageProvider
      void pushOperationOutcomes()
      Pushes the Operation Outcomes
      void readResources​(long numOfLinesToSkip, java.lang.String workItem)
      reads from a given workitem (or file) in a source and skips a certain noumber of lines
      protected void registerOverride​(java.lang.String connectionString, java.lang.String container, ExportTransientUserData chunkData, java.lang.String cosBucketPathPrefix, java.lang.String fhirResourceType, long currentBytes)
      registers the overrides for testing.
      void registerTransient​(long executionId, ExportTransientUserData transientUserData, java.lang.String cosBucketPathPrefix, java.lang.String fhirResourceType)  
      void registerTransient​(ImportTransientUserData transientUserData)  
      void writeDirectly​(java.lang.String workItem, java.io.InputStream in, int size)
      writes directly to Azure using AppendBlock client
      void writeResources​(java.lang.String mediaType, java.util.List<ReadResultDTO> dtos)
      wraps the complexity of writing FHIR Resources out to a target
      • Methods inherited from class java.lang.Object

        clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Constructor Detail

      • AzureProvider

        public AzureProvider​(java.lang.String source)
        Configures the Azure based on the storageProvider source
        Parameters:
        source -
    • Method Detail

      • registerOverride

        protected void registerOverride​(java.lang.String connectionString,
                                        java.lang.String container,
                                        ExportTransientUserData chunkData,
                                        java.lang.String cosBucketPathPrefix,
                                        java.lang.String fhirResourceType,
                                        long currentBytes)
        registers the overrides for testing.
        Parameters:
        connectionString -
        container - the container (also known as a bucket)
        chunkData - the export data
        cosBucketPathPrefix - path prefix
        fhirResourceType - resource type
        currentBytes - the long value or location of the end of the last resource
      • initializeBlobClient

        protected void initializeBlobClient​(java.lang.String workItem)
        initializes the blob client
        Parameters:
        workItem -
      • listBlobsForContainer

        public void listBlobsForContainer()
                                   throws FHIRException
        lists the blobs on the container.
        Throws:
        FHIRException
      • writeDirectly

        public void writeDirectly​(java.lang.String workItem,
                                  java.io.InputStream in,
                                  int size)
                           throws java.lang.Exception
        writes directly to Azure using AppendBlock client
        Parameters:
        workItem - the file
        in -
        size -
        Throws:
        java.lang.Exception
      • writeResources

        public void writeResources​(java.lang.String mediaType,
                                   java.util.List<ReadResultDTO> dtos)
                            throws java.lang.Exception
        Description copied from interface: Provider
        wraps the complexity of writing FHIR Resources out to a target
        Specified by:
        writeResources in interface Provider
        Throws:
        java.lang.Exception
      • readResources

        public void readResources​(long numOfLinesToSkip,
                                  java.lang.String workItem)
                           throws FHIRException
        Description copied from interface: Provider
        reads from a given workitem (or file) in a source and skips a certain noumber of lines
        Specified by:
        readResources in interface Provider
        Throws:
        FHIRException
      • processLines

        public boolean processLines​(org.linuxforhealth.fhir.bulkdata.provider.impl.AzureProvider.CountInputStreamReader counter,
                                    java.io.BufferedReader reader,
                                    int window,
                                    boolean endOfFile,
                                    java.lang.StringBuilder previous)
                             throws FHIRGeneratorException,
                                    java.io.IOException
        processes the lines in the ndjson file.
        Parameters:
        counter -
        reader -
        window -
        endOfFile -
        previous -
        Returns:
        Throws:
        FHIRGeneratorException
        java.io.IOException
      • close

        public void close()
                   throws java.lang.Exception
        Description copied from interface: Provider
        Closes the StorageProvider wrapped resources.
        Specified by:
        close in interface Provider
        Throws:
        java.lang.Exception
      • registerTransient

        public void registerTransient​(long executionId,
                                      ExportTransientUserData transientUserData,
                                      java.lang.String cosBucketPathPrefix,
                                      java.lang.String fhirResourceType)
                               throws java.lang.Exception
        Specified by:
        registerTransient in interface Provider
        Throws:
        java.lang.Exception
      • pushEndOfJobOperationOutcomes

        public void pushEndOfJobOperationOutcomes​(java.io.ByteArrayOutputStream baos,
                                                  java.lang.String folder,
                                                  java.lang.String fileName)
                                           throws FHIRException
        Description copied from interface: Provider
        Pushes End of Job OperationOutcomes and closes the Outcomes StorageProvider
        Specified by:
        pushEndOfJobOperationOutcomes in interface Provider
        Parameters:
        baos - the byte array output stream that is passed in
        folder - the folder where the file is to be stored
        fileName - the output filename to be used
        Throws:
        FHIRException