Class CosReader


  • public class CosReader
    extends Object
    Grabs work from the bucket database and dispatches items to the thread pool.
    • Constructor Detail

      • CosReader

        public CosReader​(ExecutorService commonPool,
                         FileType fileType,
                         CosClient client,
                         Consumer<ResourceEntry> resourceHandler,
                         int maxInflight,
                         DataAccess da,
                         boolean incremental,
                         int recycleSeconds,
                         boolean incrementalExact,
                         int bundleCostFactor)
        Public constructor
        Parameters:
        commonPool - thread pool shared by the readers and request handler
        fileType - the file type this read is responsible for processing
        client -
        resourceHandler -
        poolSize -
        da -
        incremental -
        recycleSeconds -
    • Method Detail

      • signalStop

        public void signalStop()
        Tell the main thread of this active object that it should start shutting down
      • waitForStop

        public void waitForStop()
        Tell the main loop thread to stop
      • init

        public void init()
        start the allocation thread
      • mainAllocationLoop

        public void mainAllocationLoop()
        The main loop of this active object
      • safeSleep

        protected void safeSleep​(long millis)
        Sleep this thread for the given milliseconds
        Parameters:
        millis -
      • markJobDone

        protected void markJobDone​(BucketLoaderJob job)
        Callback when the last record in the job completes
        Parameters:
        job -
      • processNDJSON

        public void processNDJSON​(BucketLoaderJob job,
                                  BufferedReader br)
        Read the resources from the given reader
        Parameters:
        is -
      • process

        protected boolean process​(BucketLoaderJob job,
                                  Resource resource,
                                  int lineNumber,
                                  String line)
        Process the resource parsed from the input stream
        Parameters:
        details - of the job being processed
        resource - parsed from the resource_bundle being processed
        lineNumber - the line number of this resource in the source
        line - the original resource string, for logging if we need to