Package org.apache.hadoop.util

Class WeakReferenceMap<K,V>

java.lang.Object
org.apache.hadoop.util.WeakReferenceMap<K,V>
Direct Known Subclasses:
WeakReferenceThreadMap
@Private public class WeakReferenceMap<K,V> extends Object
A map of keys type K to objects of type V which uses weak references, so does lot leak memory through long-lived references at the expense of losing references when GC takes place.. This class is intended be used instead of ThreadLocal storage when references are to be cleaned up when the instance holding. In this use case, the key is the Long key. Concurrency. The class assumes that map entries are rarely contended for when writing, and that not blocking other threads is more important than atomicity. - a ConcurrentHashMap is used to map keys to weak references, with all its guarantees. - there is no automatic pruning. - see create(Object) for the concurrency semantics on entry creation.

  • Constructor Details

    • WeakReferenceMap

      public WeakReferenceMap(Function<? super K,? extends V> factory, @Nullable Consumer<? super K> referenceLost)
      instantiate.
      Parameters:
      factory - supplier of new instances
      referenceLost - optional callback on lost references.

  • Method Details

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • size

      public int size()
      Map size.
      Returns:
      the current map size.

    • clear

      public void clear()
      Clear all entries.

    • lookup

      public WeakReference<V> lookup(K key)
      look up the value, returning the possibly empty weak reference to a value, or null if no value was found.
      Parameters:
      key - key to look up
      Returns:
      null if there is no entry, a weak reference if found

    • get

      public V get(K key)
      Get the value, creating if needed.
      Parameters:
      key - key.
      Returns:
      an instance.

    • create

      public V create(K key)
      Create a new instance under a key.

      The instance is created, added to the map and then the map value retrieved. This ensures that the reference returned is that in the map, even if there is more than one entry being created at the same time. If that race does occur, it will be logged the first time it happens for this specific map instance.

      HADOOP-18456 highlighted the risk of a concurrent GC resulting a null value being retrieved and so returned. To prevent this:

      1. A strong reference is retained to the newly created instance in a local variable.
      2. That variable is used after the resolution process, to ensure the JVM doesn't consider it "unreachable" and so eligible for GC.
      3. A check is made for the resolved reference being null, and if so, the put() is repeated
      Parameters:
      key - key
      Returns:
      the created value

    • put

      public V put(K key, V value)
      Put a value under the key. A null value can be put, though on a get() call a new entry is generated
      Parameters:
      key - key
      value - value
      Returns:
      any old non-null reference.

    • remove

      public V remove(K key)
      Remove any value under the key.
      Parameters:
      key - key
      Returns:
      any old non-null reference.

    • containsKey

      public boolean containsKey(K key)
      Does the map have a valid reference for this object? no-side effects: there's no attempt to notify or cleanup if the reference is null.
      Parameters:
      key - key to look up
      Returns:
      true if there is a valid reference.

    • resolve

      protected V resolve(WeakReference<V> r)
      Given a possibly null weak reference, resolve its value.
      Parameters:
      r - reference to resolve
      Returns:
      the value or null

    • prune

      public int prune()
      Prune all null weak references, calling the referenceLost callback for each one. non-atomic and non-blocking.
      Returns:
      the number of entries pruned.

    • getReferenceLostCount

      public final long getReferenceLostCount()
      Get count of references lost as detected during prune() or get() calls.
      Returns:
      count of references lost

    • getEntriesCreatedCount

      public final long getEntriesCreatedCount()
      Get count of entries created on demand.
      Returns:
      count of entries created