Class CosScanner


  • public class CosScanner
    extends Object
    Active object to periodically scan COS buckets looking for new objects to load
    • Field Detail

    • Constructor Detail

      • CosScanner

        public CosScanner​(CosClient client,
                          Collection<String> buckets,
                          DataAccess dataAccess,
                          Set<FileType> fileTypes,
                          String pathPrefix,
                          int scanIntervalMs)
        Public constructor
        Parameters:
        client -
        buckets - the COS buckets to scan
        dataAccess - the data access layer for persisting items discovered during the scan
        fileTypes - set of FileType values accepted for processing
        prefix - only scan items with this prefix if set
        scanIntervalMs - the number of milliseconds between scans. -1 for automatic
    • Method Detail

      • init

        public void init()
        Run the scanner thread
      • signalStop

        public void signalStop()
        Tell the active object to stop any new work, but existing work can complete
      • waitForStop

        public void waitForStop()
        Tell the main loop thread to stop if it hasn't already and wait a reasonable time for the main thread loop to terminate
      • mainLoop

        public void mainLoop()
        The main loop running inside this active object's thread
      • safeSleep

        protected void safeSleep​(long millis)
        Sleep this thread for the given milliseconds
        Parameters:
        millis -
      • scan

        protected void scan()
        Perform a scan for each of the configured buckets
      • fileTyper

        protected static FileType fileTyper​(String itemName)
        Determine the type of the file based on the suffix
        Parameters:
        itemName -
        Returns:
      • handle

        protected void handle​(CosItem item)
        Process the item returned by the scan
        Parameters:
        item -
      • heartbeat

        protected void heartbeat()
        Update the heartbeat on a (reasonably) regular basis to demonstrate this loader instance is still alive