Package org.apache.hadoop.yarn.client.api.async

Class NMClientAsync

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.yarn.client.api.async.NMClientAsync
All Implemented Interfaces:
Closeable, AutoCloseable, Service
@Public @Stable public abstract class NMClientAsync extends AbstractService
NMClientAsync handles communication with all the NodeManagers and provides asynchronous updates on getting responses from them. It maintains a thread pool to communicate with individual NMs where a number of worker threads process requests to NMs by using NMClientImpl. The max size of the thread pool is configurable through YarnConfiguration.NM_CLIENT_ASYNC_THREAD_POOL_MAX_SIZE. It should be used in conjunction with a CallbackHandler. For example 
 
 class MyCallbackHandler extends NMClientAsync.AbstractCallbackHandler {
   public void onContainerStarted(ContainerId containerId,
       Map<String, ByteBuffer> allServiceResponse) {
     [post process after the container is started, process the response]
   }

   public void onContainerResourceIncreased(ContainerId containerId,
       Resource resource) {
     [post process after the container resource is increased]
   }

   public void onContainerStatusReceived(ContainerId containerId,
       ContainerStatus containerStatus) {
     [make use of the status of the container]
   }

   public void onContainerStopped(ContainerId containerId) {
     [post process after the container is stopped]
   }

   public void onStartContainerError(
       ContainerId containerId, Throwable t) {
     [handle the raised exception]
   }

   public void onGetContainerStatusError(
       ContainerId containerId, Throwable t) {
     [handle the raised exception]
   }

   public void onStopContainerError(
       ContainerId containerId, Throwable t) {
     [handle the raised exception]
   }
 }
The client's life-cycle should be managed like the following: 
 
 NMClientAsync asyncClient = 
     NMClientAsync.createNMClientAsync(new MyCallbackhandler());
 asyncClient.init(conf);
 asyncClient.start();
 asyncClient.startContainer(container, containerLaunchContext);
 [... wait for container being started]
 asyncClient.getContainerStatus(container.getId(), container.getNodeId(),
     container.getContainerToken());
 [... handle the status in the callback instance]
 asyncClient.stopContainer(container.getId(), container.getNodeId(),
     container.getContainerToken());
 [... wait for container being stopped]
 asyncClient.stop();

  • Field Details

    • client

      protected NMClient client

    • callbackHandler

      protected org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler

  • Constructor Details

    • NMClientAsync

      protected NMClientAsync(org.apache.hadoop.yarn.client.api.async.NMClientAsync.AbstractCallbackHandler callbackHandler)

    • NMClientAsync

      protected NMClientAsync(String name, org.apache.hadoop.yarn.client.api.async.NMClientAsync.AbstractCallbackHandler callbackHandler)

    • NMClientAsync

      protected NMClientAsync(String name, NMClient client, org.apache.hadoop.yarn.client.api.async.NMClientAsync.AbstractCallbackHandler callbackHandler)

    • NMClientAsync

      @Deprecated protected NMClientAsync(org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)
      Deprecated.
      Use NMClientAsync(AbstractCallbackHandler) instead.

    • NMClientAsync

      @Deprecated protected NMClientAsync(String name, org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)
      Deprecated.
      Use NMClientAsync(String, AbstractCallbackHandler) instead.

    • NMClientAsync

      @Private @VisibleForTesting @Deprecated protected NMClientAsync(String name, NMClient client, org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)
      Deprecated.

  • Method Details

    • createNMClientAsync

      public static NMClientAsync createNMClientAsync(org.apache.hadoop.yarn.client.api.async.NMClientAsync.AbstractCallbackHandler callbackHandler)

    • createNMClientAsync

      @Deprecated public static NMClientAsync createNMClientAsync(org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)
      Deprecated.
      Use createNMClientAsync(AbstractCallbackHandler) instead.

    • startContainerAsync

      public abstract void startContainerAsync(Container container, ContainerLaunchContext containerLaunchContext)

    • increaseContainerResourceAsync

      @Deprecated public abstract void increaseContainerResourceAsync(Container container)
      Deprecated.

    • updateContainerResourceAsync

      public abstract void updateContainerResourceAsync(Container container)

      Update the resources of a container.

      The ApplicationMaster or other applications that use the client must provide the details of the container, including the Id and the target resource encapsulated in the updated container token via Container.

      Parameters:
      container - the container with updated token.

    • reInitializeContainerAsync

      public abstract void reInitializeContainerAsync(ContainerId containerId, ContainerLaunchContext containerLaunchContex, boolean autoCommit)

      Re-Initialize the Container.

      Parameters:
      containerId - the Id of the container to Re-Initialize.
      containerLaunchContex - the updated ContainerLaunchContext.
      autoCommit - commit re-initialization automatically ?

    • restartContainerAsync

      public abstract void restartContainerAsync(ContainerId containerId)

      Restart the specified container.

      Parameters:
      containerId - the Id of the container to restart.

    • rollbackLastReInitializationAsync

      public abstract void rollbackLastReInitializationAsync(ContainerId containerId)

      Rollback last reInitialization of the specified container.

      Parameters:
      containerId - the Id of the container to restart.

    • commitLastReInitializationAsync

      public abstract void commitLastReInitializationAsync(ContainerId containerId)

      Commit last reInitialization of the specified container.

      Parameters:
      containerId - the Id of the container to commit reInitialize.

    • stopContainerAsync

      public abstract void stopContainerAsync(ContainerId containerId, NodeId nodeId)

    • getContainerStatusAsync

      public abstract void getContainerStatusAsync(ContainerId containerId, NodeId nodeId)

    • getClient

      public NMClient getClient()

    • setClient

      public void setClient(NMClient client)

    • getCallbackHandler

      public org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler getCallbackHandler()

    • setCallbackHandler

      public void setCallbackHandler(org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)