Package org.apache.hadoop.hdfs.server.federation.store.driver.impl

Class StateStoreFileBaseImpl

java.lang.Object
org.apache.hadoop.hdfs.server.federation.store.driver.StateStoreDriver
org.apache.hadoop.hdfs.server.federation.store.driver.impl.StateStoreBaseImpl
org.apache.hadoop.hdfs.server.federation.store.driver.impl.StateStoreSerializableImpl
org.apache.hadoop.hdfs.server.federation.store.driver.impl.StateStoreFileBaseImpl
All Implemented Interfaces:
StateStoreRecordOperations
Direct Known Subclasses:
StateStoreFileImpl, StateStoreFileSystemImpl
public abstract class StateStoreFileBaseImpl extends StateStoreSerializableImpl
StateStoreDriver implementation based on files. In this approach, we use temporary files for the writes and renaming "atomically" to the final value. Instead of writing to the final location, it will go to a temporary one and then rename to the final destination.

  • Constructor Details

    • StateStoreFileBaseImpl

      public StateStoreFileBaseImpl()

  • Method Details

    • getReader

      protected abstract <T extends BaseRecord> BufferedReader getReader(String path)
      Get the reader of a record for the file system.
      Type Parameters:
      T - Type of the state store record.
      Parameters:
      path - Path of the record to read.
      Returns:
      Reader for the record.

    • getWriter

      @VisibleForTesting public abstract <T extends BaseRecord> BufferedWriter getWriter(String path)
      Get the writer of a record for the file system.
      Type Parameters:
      T - Type of the state store record.
      Parameters:
      path - Path of the record to write.
      Returns:
      Writer for the record.

    • exists

      protected abstract boolean exists(String path)
      Check if a path exists.
      Parameters:
      path - Path to check.
      Returns:
      If the path exists.

    • mkdir

      protected abstract boolean mkdir(String path)
      Make a directory.
      Parameters:
      path - Path of the directory to create.
      Returns:
      If the directory was created.

    • rename

      protected abstract boolean rename(String src, String dst)
      Rename a file. This should be atomic.
      Parameters:
      src - Source name.
      dst - Destination name.
      Returns:
      If the rename was successful.

    • remove

      protected abstract boolean remove(String path)
      Remove a file.
      Parameters:
      path - Path for the file to remove
      Returns:
      If the file was removed.

    • getChildren

      protected abstract List<String> getChildren(String path)
      Get the children for a path.
      Parameters:
      path - Path to check.
      Returns:
      List of children.

    • getRootDir

      protected abstract String getRootDir()
      Get root directory.
      Returns:
      Root directory.

    • getConcurrentFilesAccessNumThreads

      protected abstract int getConcurrentFilesAccessNumThreads()

    • setInitialized

      public void setInitialized(boolean ini)
      Set the driver as initialized.
      Parameters:
      ini - If the driver is initialized.

    • initDriver

      public boolean initDriver()
      Description copied from class: StateStoreDriver
      Prepare the driver to access data storage.
      Specified by:
      initDriver in class StateStoreDriver
      Returns:
      True if the driver was successfully initialized. If false is returned, the state store will periodically attempt to re-initialize the driver and the router will remain in safe mode until the driver is initialized.

    • close

      public void close() throws Exception
      Description copied from class: StateStoreDriver
      Close the State Store driver connection.
      Overrides:
      close in class StateStoreDriver
      Throws:
      Exception - if something goes wrong while closing the state store driver connection.

    • initRecordStorage

      public <T extends BaseRecord> boolean initRecordStorage(String className, Class<T> recordClass)
      Description copied from class: StateStoreDriver
      Initialize storage for a single record class.
      Specified by:
      initRecordStorage in class StateStoreDriver
      Type Parameters:
      T - Type of the state store record.
      Parameters:
      className - String reference of the record class to initialize, used to construct paths and file names for the record. Determined by configuration settings for the specific driver.
      recordClass - Record type corresponding to the provided name.
      Returns:
      True if successful, false otherwise.

    • get

      public <T extends BaseRecord> QueryResult<T> get(Class<T> clazz) throws IOException
      Description copied from interface: StateStoreRecordOperations
      Get all records of the requested record class from the data store. To use the default implementations in this class, getAll must return new instances of the records on each call. It is recommended to override the default implementations for better performance.
      Type Parameters:
      T - Record class of the records.
      Parameters:
      clazz - Class of record to fetch.
      Returns:
      List of all records that match the class.
      Throws:
      IOException - Throws exception if unable to query the data store.

    • isOldTempRecord

      @VisibleForTesting public static boolean isOldTempRecord(String pathRecord)
      Check if a record is temporary and old.
      Parameters:
      pathRecord - Path for the record to check.
      Returns:
      If the record is temporary and old.

    • isDriverReady

      public boolean isDriverReady()
      Description copied from class: StateStoreDriver
      Check if the driver is currently running and the data store connection is valid.
      Specified by:
      isDriverReady in class StateStoreDriver
      Returns:
      True if the driver is initialized and the data store is ready.

    • putAll

      public <T extends BaseRecord> StateStoreOperationResult putAll(List<T> records, boolean allowUpdate, boolean errorIfExists) throws StateStoreUnavailableException
      Description copied from interface: StateStoreRecordOperations
      Creates multiple records. Optionally updates existing records that have the same primary key.
      Type Parameters:
      T - Record class of the records.
      Parameters:
      records - List of data records to update or create. All records must be of class clazz.
      allowUpdate - True if update of exiting record is allowed.
      errorIfExists - True if an error should be returned when inserting an existing record. Only used if allowUpdate = false.
      Returns:
      The result of the putAll operation.
      Throws:
      StateStoreUnavailableException

    • remove

      public <T extends BaseRecord> int remove(Class<T> clazz, Query<T> query) throws StateStoreUnavailableException
      Description copied from interface: StateStoreRecordOperations
      Remove multiple records of a specific class that match a query. Requires the getAll implementation to fetch fresh records on each call.
      Type Parameters:
      T - Record class of the records.
      Parameters:
      clazz - The class to match the records with.
      query - Query to filter what to remove.
      Returns:
      The number of records removed.
      Throws:
      StateStoreUnavailableException

    • removeAll

      public <T extends BaseRecord> boolean removeAll(Class<T> clazz) throws StateStoreUnavailableException
      Description copied from interface: StateStoreRecordOperations
      Remove all records of this class from the store.
      Type Parameters:
      T - Record class of the records.
      Parameters:
      clazz - Class of records to remove.
      Returns:
      True if successful.
      Throws:
      StateStoreUnavailableException