Package org.apache.hadoop.yarn.server.nodemanager

Class WindowsSecureContainerExecutor

java.lang.Object
org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor
org.apache.hadoop.yarn.server.nodemanager.WindowsSecureContainerExecutor
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable
public class WindowsSecureContainerExecutor extends DefaultContainerExecutor
Windows secure container executor (WSCE). This class offers a secure container executor on Windows, similar to the LinuxContainerExecutor. As the NM does not run on a high privileged context, this class delegates elevated operations to the helper hadoopwintuilsvc, implemented by the winutils.exe running as a service. JNI and LRPC is used to communicate with the privileged service.

  • Field Details

  • Constructor Details

  • Method Details

    • setConf

      public void setConf(org.apache.hadoop.conf.Configuration conf)
      Specified by:
      setConf in interface org.apache.hadoop.conf.Configurable
      Overrides:
      setConf in class ContainerExecutor

    • getRunCommand

      protected String[] getRunCommand(String command, String groupId, String userName, org.apache.hadoop.fs.Path pidFile, org.apache.hadoop.conf.Configuration conf)
      Description copied from class: ContainerExecutor
      Return a command line to execute the given command in the OS shell. On Windows, the {code}groupId{code} parameter can be used to launch and associate the given GID with a process group. On non-Windows hosts, the {code}groupId{code} parameter is ignored.
      Overrides:
      getRunCommand in class ContainerExecutor
      Parameters:
      command - the command to execute
      groupId - the job owner's GID
      userName - the job owner's username
      pidFile - the path to the container's PID file
      conf - the configuration
      Returns:
      the command line to execute

    • getLocalWrapperScriptBuilder

      protected DefaultContainerExecutor.LocalWrapperScriptBuilder getLocalWrapperScriptBuilder(String containerIdStr, org.apache.hadoop.fs.Path containerWorkDir)
      Description copied from class: DefaultContainerExecutor
      Create a DefaultContainerExecutor.LocalWrapperScriptBuilder for the given container ID and path that is appropriate to the current platform.
      Overrides:
      getLocalWrapperScriptBuilder in class DefaultContainerExecutor
      Parameters:
      containerIdStr - the container ID
      containerWorkDir - the container's working directory
      Returns:
      a new DefaultContainerExecutor.LocalWrapperScriptBuilder

    • copyFile

      protected void copyFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst, String owner) throws IOException
      Description copied from class: DefaultContainerExecutor
      Copy a file using the DefaultContainerExecutor.lfs FileContext.
      Overrides:
      copyFile in class DefaultContainerExecutor
      Parameters:
      src - the file to copy
      dst - where to copy the file
      owner - the owner of the new copy. Used only in secure Windows clusters
      Throws:
      IOException - when the copy fails
      See Also:

    • createDir

      protected void createDir(org.apache.hadoop.fs.Path dirPath, org.apache.hadoop.fs.permission.FsPermission perms, boolean createParent, String owner) throws IOException
      Description copied from class: DefaultContainerExecutor
      Use the DefaultContainerExecutor.lfs FileContext to create the target directory.
      Overrides:
      createDir in class DefaultContainerExecutor
      Parameters:
      dirPath - the target directory
      perms - the target permissions for the target directory
      createParent - whether the parent directories should also be created
      owner - the user as whom the target directory should be created. Used only on secure Windows hosts.
      Throws:
      IOException - if there's a failure performing a file operation
      See Also:

    • setScriptExecutable

      protected void setScriptExecutable(org.apache.hadoop.fs.Path script, String owner) throws IOException
      Description copied from class: DefaultContainerExecutor
      Make a file executable using the DefaultContainerExecutor.lfs FileContext.
      Overrides:
      setScriptExecutable in class DefaultContainerExecutor
      Parameters:
      script - the path to make executable
      owner - the new owner for the file. Used only in secure Windows clusters
      Throws:
      IOException - when the change mode operation fails
      See Also:

    • localizeClasspathJar

      public org.apache.hadoop.fs.Path localizeClasspathJar(org.apache.hadoop.fs.Path jarPath, org.apache.hadoop.fs.Path target, String owner) throws IOException
      Description copied from class: ContainerExecutor
      This function localizes the JAR file on-demand. On Windows the ContainerLaunch creates a temporary special JAR manifest of other JARs to workaround the CLASSPATH length. In a secure cluster this JAR must be localized so that the container has access to it. The default implementation returns the classpath passed to it, which is expected to have been created in the node manager's fprivate folder, which will not work with secure Windows clusters.
      Overrides:
      localizeClasspathJar in class ContainerExecutor
      Parameters:
      jarPath - the path to the JAR to localize
      target - the directory where the JAR file should be localized
      owner - the name of the user who should own the localized file
      Returns:
      the path to the localized JAR file
      Throws:
      IOException - if localization fails

    • startLocalizer

      public void startLocalizer(LocalizerStartContext ctx) throws IOException, InterruptedException
      Description copied from class: ContainerExecutor
      Prepare the environment for containers in this application to execute. 
       For $x in local.dirs
   create $x/$user/$appId
 Copy $nmLocal/appTokens -> $N/$user/$appId
 For $rsrc in private resources
   Copy $rsrc -> $N/$user/filecache/[idef]
 For $rsrc in job resources
   Copy $rsrc -> $N/$user/$appId/filecache/idef
      Overrides:
      startLocalizer in class DefaultContainerExecutor
      Parameters:
      ctx - LocalizerStartContext that encapsulates necessary information for starting a localizer.
      Throws:
      IOException - for most application init failures
      InterruptedException - if application init thread is halted by NM

    • buildCommandExecutor

      protected org.apache.hadoop.util.Shell.CommandExecutor buildCommandExecutor(String wrapperScriptPath, String containerIdStr, String userName, org.apache.hadoop.fs.Path pidFile, org.apache.hadoop.yarn.api.records.Resource resource, File wordDir, Map<String,String> environment, String[] numaCommands)
      Description copied from class: DefaultContainerExecutor
      Create a new Shell.ShellCommandExecutor using the parameters.
      Overrides:
      buildCommandExecutor in class DefaultContainerExecutor
      Parameters:
      wrapperScriptPath - the path to the script to execute
      containerIdStr - the container ID
      userName - the application owner's username
      pidFile - the path to the container's PID file
      resource - this parameter controls memory and CPU limits.
      wordDir - If not-null, specifies the directory which should be set as the current working directory for the command. If null, the current working directory is not modified.
      environment - the container environment
      numaCommands - list of prefix numa commands
      Returns:
      the new Shell.ShellCommandExecutor
      See Also:
      • Shell.ShellCommandExecutor

    • killContainer

      protected void killContainer(String pid, ContainerExecutor.Signal signal) throws IOException
      Description copied from class: DefaultContainerExecutor
      Send a specified signal to the specified pid
      Overrides:
      killContainer in class DefaultContainerExecutor
      Parameters:
      pid - the pid of the process [group] to signal.
      signal - signal to send
      Throws:
      IOException - if the command to kill the process fails