Package org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher

Class ContainerLaunch

java.lang.Object

org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch

All Implemented Interfaces:

Callable<Integer>

Direct Known Subclasses:

ContainerRelaunch, RecoveredContainerLaunch, RecoverPausedContainerLaunch

public class ContainerLaunch
extends Object
implements Callable<Integer>

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ContainerLaunch.ShellScriptBuilder

Field Summary

Fields

Modifier and Type	Field	Description
protected final Application	app
protected AtomicBoolean	completed
protected final Container	container
static final String	CONTAINER_PRE_LAUNCH_STDERR
static final String	CONTAINER_PRE_LAUNCH_STDOUT
static final String	CONTAINER_SCRIPT
protected AtomicBoolean	containerAlreadyLaunched
protected final LocalDirsHandlerService	dirsHandler
protected final org.apache.hadoop.yarn.event.Dispatcher	dispatcher
protected final ContainerExecutor	exec
static final String	FINAL_CONTAINER_TOKENS_FILE
static final String	KEYSTORE_FILE
protected org.apache.hadoop.fs.Path	pidFilePath
protected AtomicBoolean	shouldPauseContainer
static final String	SYSFS_DIR
static final String	TRUSTSTORE_FILE

Constructor Summary

Constructors

Constructor	Description
ContainerLaunch(Context context, org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.yarn.event.Dispatcher dispatcher, ContainerExecutor exec, Application app, Container container, LocalDirsHandlerService dirsHandler, ContainerManagerImpl containerManager)

Method Summary

Modifier and Type	Method	Description
void	addConfigsToEnv(Map<String,String> environment)	There are some configurations (such as "yarn.nodemanager.admin-env") whose values need to be added to the environment variables.
Integer	call()
protected void	cleanupContainerFiles(org.apache.hadoop.fs.Path containerWorkDir)	Clean up container's files for container relaunch or cleanup.
static String	expandEnvironment(String var, org.apache.hadoop.fs.Path containerLogDir)
protected List<String>	getApplicationLocalDirs(List<String> localDirs, String appIdStr)
protected List<String>	getContainerLocalDirs(List<String> localDirs)
protected List<String>	getContainerLogDirs(List<String> logDirs)
protected String	getContainerPrivateDir(String appIdStr, String containerIdStr)
protected org.apache.hadoop.fs.Path	getContainerWorkDir()
static String	getExitCodeFile(String pidFile)
protected Map<org.apache.hadoop.fs.Path,List<String>>	getLocalizedResources()
protected List<String>	getNMFilecacheDirs(List<String> localDirs)
protected String	getPidFileSubpath(String appIdStr, String containerIdStr)
static String	getRelativeContainerLogDir(String appIdStr, String containerIdStr)
protected List<String>	getUserFilecacheDirs(List<String> localDirs)
protected List<String>	getUserLocalDirs(List<String> localDirs)
protected void	handleContainerExitCode(int exitCode, org.apache.hadoop.fs.Path containerLogDir)
protected void	handleContainerExitWithFailure(org.apache.hadoop.yarn.api.records.ContainerId containerID, int ret, org.apache.hadoop.fs.Path containerLogDir, StringBuilder diagnosticInfo)	Tries to tail and fetch TAIL_SIZE_IN_BYTES of data from the error log.
protected int	launchContainer(ContainerStartContext ctx)
void	pauseContainer()	Pause the container.
protected int	prepareForLaunch(ContainerStartContext ctx)
protected int	relaunchContainer(ContainerStartContext ctx)
void	resumeContainer()	Resume the container.
void	sanitizeEnv(Map<String,String> environment, org.apache.hadoop.fs.Path pwd, List<org.apache.hadoop.fs.Path> appDirs, List<String> userLocalDirs, List<String> containerLogDirs, Map<org.apache.hadoop.fs.Path,List<String>> resources, org.apache.hadoop.fs.Path nmPrivateClasspathJarDir, Set<String> nmVars)
protected void	setContainerCompletedStatus(int exitCode)
void	signalContainer(org.apache.hadoop.yarn.api.records.SignalContainerCommand command)	Send a signal to the container.
static ContainerExecutor.Signal	translateCommandToSignal(org.apache.hadoop.yarn.api.records.SignalContainerCommand command)
protected boolean	validateContainerState()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

CONTAINER_PRE_LAUNCH_STDOUT

public static final String CONTAINER_PRE_LAUNCH_STDOUT

See Also:

• Constant Field Values

CONTAINER_PRE_LAUNCH_STDERR

public static final String CONTAINER_PRE_LAUNCH_STDERR

See Also:

• Constant Field Values

CONTAINER_SCRIPT

public static final String CONTAINER_SCRIPT

FINAL_CONTAINER_TOKENS_FILE

public static final String FINAL_CONTAINER_TOKENS_FILE

See Also:

• Constant Field Values

SYSFS_DIR

public static final String SYSFS_DIR

See Also:

• Constant Field Values

KEYSTORE_FILE

public static final String KEYSTORE_FILE

See Also:

• Constant Field Values

TRUSTSTORE_FILE

public static final String TRUSTSTORE_FILE

See Also:

• Constant Field Values

dispatcher

protected final org.apache.hadoop.yarn.event.Dispatcher dispatcher

exec

protected final ContainerExecutor exec

app

protected final Application app

container

protected final Container container

containerAlreadyLaunched

protected AtomicBoolean containerAlreadyLaunched

shouldPauseContainer

protected AtomicBoolean shouldPauseContainer

completed

protected AtomicBoolean completed

pidFilePath

protected org.apache.hadoop.fs.Path pidFilePath

dirsHandler

protected final LocalDirsHandlerService dirsHandler

Constructor Details

ContainerLaunch

public ContainerLaunch(Context context,
 org.apache.hadoop.conf.Configuration configuration,
 org.apache.hadoop.yarn.event.Dispatcher dispatcher,
 ContainerExecutor exec,
 Application app,
 Container container,
 LocalDirsHandlerService dirsHandler,
 ContainerManagerImpl containerManager)

Method Details

expandEnvironment

@VisibleForTesting
public static String expandEnvironment(String var,
 org.apache.hadoop.fs.Path containerLogDir)

call

public Integer call()

Specified by:

call in interface Callable<Integer>

validateContainerState

protected boolean validateContainerState()

getContainerLogDirs

protected List<String> getContainerLogDirs(List<String> logDirs)

getContainerLocalDirs

protected List<String> getContainerLocalDirs(List<String> localDirs)

getUserLocalDirs

protected List<String> getUserLocalDirs(List<String> localDirs)

getNMFilecacheDirs

protected List<String> getNMFilecacheDirs(List<String> localDirs)

getUserFilecacheDirs

protected List<String> getUserFilecacheDirs(List<String> localDirs)

getApplicationLocalDirs

protected List<String> getApplicationLocalDirs(List<String> localDirs,
 String appIdStr)

getLocalizedResources

protected Map<org.apache.hadoop.fs.Path,List<String>> getLocalizedResources()
                                                                     throws org.apache.hadoop.yarn.exceptions.YarnException

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

launchContainer

protected int launchContainer(ContainerStartContext ctx)
                       throws IOException,
org.apache.hadoop.yarn.exceptions.ConfigurationException

Throws:

IOException

org.apache.hadoop.yarn.exceptions.ConfigurationException

relaunchContainer

protected int relaunchContainer(ContainerStartContext ctx)
                         throws IOException,
org.apache.hadoop.yarn.exceptions.ConfigurationException

Throws:

IOException

org.apache.hadoop.yarn.exceptions.ConfigurationException

prepareForLaunch

protected int prepareForLaunch(ContainerStartContext ctx)
                        throws IOException

Throws:

IOException

setContainerCompletedStatus

protected void setContainerCompletedStatus(int exitCode)

handleContainerExitCode

protected void handleContainerExitCode(int exitCode,
 org.apache.hadoop.fs.Path containerLogDir)

handleContainerExitWithFailure

protected void handleContainerExitWithFailure(org.apache.hadoop.yarn.api.records.ContainerId containerID,
 int ret,
 org.apache.hadoop.fs.Path containerLogDir,
 StringBuilder diagnosticInfo)

Tries to tail and fetch TAIL_SIZE_IN_BYTES of data from the error log. ErrorLog filename is not fixed and depends upon app, hence file name pattern is used.

Parameters:

containerID -

ret -

containerLogDir -

diagnosticInfo -

getPidFileSubpath

protected String getPidFileSubpath(String appIdStr,
 String containerIdStr)

signalContainer

public void signalContainer(org.apache.hadoop.yarn.api.records.SignalContainerCommand command)
                     throws IOException

Send a signal to the container.

Throws:

IOException

translateCommandToSignal

@VisibleForTesting
public static ContainerExecutor.Signal translateCommandToSignal(org.apache.hadoop.yarn.api.records.SignalContainerCommand command)

pauseContainer

public void pauseContainer()
                    throws IOException

Pause the container. Cancels the launch if the container isn't launched yet. Otherwise asks the executor to pause the container.

Throws:

IOException - in case of errors.

resumeContainer

public void resumeContainer()
                     throws IOException

Resume the container. Cancels the launch if the container isn't launched yet. Otherwise asks the executor to pause the container.

Throws:

IOException - in case of error.

getRelativeContainerLogDir

public static String getRelativeContainerLogDir(String appIdStr,
 String containerIdStr)

getContainerPrivateDir

protected String getContainerPrivateDir(String appIdStr,
 String containerIdStr)

sanitizeEnv

public void sanitizeEnv(Map<String,String> environment,
 org.apache.hadoop.fs.Path pwd,
 List<org.apache.hadoop.fs.Path> appDirs,
 List<String> userLocalDirs,
 List<String> containerLogDirs,
 Map<org.apache.hadoop.fs.Path,List<String>> resources,
 org.apache.hadoop.fs.Path nmPrivateClasspathJarDir,
 Set<String> nmVars)
                 throws IOException

Throws:

IOException

addConfigsToEnv

public void addConfigsToEnv(Map<String,String> environment)

There are some configurations (such as "yarn.nodemanager.admin-env") whose values need to be added to the environment variables.

Parameters:

environment - The environment variables map to add the configuration values to.

getExitCodeFile

public static String getExitCodeFile(String pidFile)

getContainerWorkDir

protected org.apache.hadoop.fs.Path getContainerWorkDir()
                                                 throws IOException

Throws:

IOException

cleanupContainerFiles

protected void cleanupContainerFiles(org.apache.hadoop.fs.Path containerWorkDir)

Clean up container's files for container relaunch or cleanup.