Package org.apache.hadoop.yarn.service.provider

Class ProviderUtils

java.lang.Object

org.apache.hadoop.yarn.service.provider.ProviderUtils

All Implemented Interfaces:

YarnServiceConstants

public class ProviderUtils
extends Object
implements YarnServiceConstants

This is a factoring out of methods handy for providers. It's bonded to a log at construction time.

Field Summary

Fields

Modifier and Type	Field	Description
protected static final org.slf4j.Logger	log

Fields inherited from interface org.apache.hadoop.yarn.service.conf.YarnServiceConstants

APP_CONF_DIR, APP_LIB_DIR, APP_RESOURCES_DIR, APP_TYPE, CONTAINER_STATE_REPORT_AS_SERVICE_STATE, CONTENT, DEPENDENCY_DIR, DEPENDENCY_DIR_PERMISSIONS, DEPENDENCY_LOCALIZED_DIR_LINK, DEPENDENCY_TAR_GZ_FILE_EXT, DEPENDENCY_TAR_GZ_FILE_NAME, ERR_FILE, HADOOP_USER_NAME, KEYTAB_DIR, KEYTAB_LOCATION, OUT_FILE, PRINCIPAL, PROPERTY_LIB_DIR, RESOURCE_DIR, SERVICE_BASE_DIRECTORY, SERVICE_CORE_JAR, SERVICES_DIRECTORY, SERVICES_PUBLIC_DIRECTORY, STDERR_AM, STDOUT_AM, SUBMITTED_CONF_DIR, SYSPROP_LOG_DIR, SYSPROP_LOG4J_CONFIGURATION, TMP_DIR_PREFIX, UPGRADE_DIR, YARN_SERVICE_LOG4J_FILENAME

Constructor Summary

Constructors

Constructor	Description
ProviderUtils()

Method Summary

Modifier and Type	Method	Description
static void	addAllDependencyJars(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> providerResources, SliderFileSystem sliderFileSystem, org.apache.hadoop.fs.Path tempPath, String libDir, String libLocalSrcDir)	Loads all dependency jars from the default path.
static boolean	addProviderJar(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> providerResources, Class providerClass, String jarName, SliderFileSystem sliderFileSystem, org.apache.hadoop.fs.Path tempPath, String libdir, boolean miniClusterTestRun)	Add oneself to the classpath.
static void	createConfigFileAndAddLocalResource(AbstractLauncher launcher, SliderFileSystem fs, ContainerLaunchService.ComponentLaunchContext compLaunchContext, Map<String,String> tokensForSubstitution, ComponentInstance instance, org.apache.hadoop.yarn.service.ServiceContext context, ProviderService.ResolvedLaunchParams resolvedParams)
static void	handleStaticFilesForLocalization(AbstractLauncher launcher, SliderFileSystem fs, ContainerLaunchService.ComponentLaunchContext componentLaunchCtx, ProviderService.ResolvedLaunchParams resolvedParams)
static org.apache.hadoop.fs.Path	initCompInstanceDir(SliderFileSystem fs, ContainerLaunchService.ComponentLaunchContext compLaunchContext, ComponentInstance instance)
static org.apache.hadoop.fs.Path	initCompPublicResourceDir(SliderFileSystem fs, ContainerLaunchService.ComponentLaunchContext compLaunchContext, ComponentInstance instance)
static Map<String,String>	initCompTokensForSubstitute(ComponentInstance instance, org.apache.hadoop.yarn.api.records.Container container, ContainerLaunchService.ComponentLaunchContext componentLaunchContext)	Get initial component token map to be substituted into config values.
static String	replaceSpacesWithDelimiter(String content, String delimiter)
static void	substituteMapWithTokens(Map<String,String> configs, Map<String,String> tokenMap)
static String	substituteStrWithTokens(String content, Map<String,String> tokensForSubstitution)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

log

protected static final org.slf4j.Logger log

Constructor Details

ProviderUtils

public ProviderUtils()

Method Details

addProviderJar

public static boolean addProviderJar(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> providerResources,
 Class providerClass,
 String jarName,
 SliderFileSystem sliderFileSystem,
 org.apache.hadoop.fs.Path tempPath,
 String libdir,
 boolean miniClusterTestRun)
                              throws IOException,
SliderException

Add oneself to the classpath. This does not work on minicluster test runs where the JAR is not built up.

Parameters:

providerResources - map of provider resources to add these entries to

providerClass - provider to add

jarName - name of the jar to use

sliderFileSystem - target filesystem

tempPath - path in the cluster FS for temp files

libdir - relative directory to place resources

miniClusterTestRun - true if minicluster is being used

Returns:

true if the class was found in a JAR

Throws:

FileNotFoundException - if the JAR was not found and this is NOT a mini cluster test run

IOException - IO problems

SliderException - any Slider problem

addAllDependencyJars

public static void addAllDependencyJars(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> providerResources,
 SliderFileSystem sliderFileSystem,
 org.apache.hadoop.fs.Path tempPath,
 String libDir,
 String libLocalSrcDir)
                                 throws IOException,
SliderException

Loads all dependency jars from the default path.

Parameters:

providerResources - map of provider resources to add these entries to

sliderFileSystem - target filesystem

tempPath - path in the cluster FS for temp files

libDir - relative directory to place resources

libLocalSrcDir - explicitly supplied local libs dir

Throws:

IOException - trouble copying to HDFS

SliderException - trouble copying to HDFS

substituteStrWithTokens

public static String substituteStrWithTokens(String content,
 Map<String,String> tokensForSubstitution)

replaceSpacesWithDelimiter

public static String replaceSpacesWithDelimiter(String content,
 String delimiter)

substituteMapWithTokens

public static void substituteMapWithTokens(Map<String,String> configs,
 Map<String,String> tokenMap)

initCompInstanceDir

public static org.apache.hadoop.fs.Path initCompInstanceDir(SliderFileSystem fs,
 ContainerLaunchService.ComponentLaunchContext compLaunchContext,
 ComponentInstance instance)

initCompPublicResourceDir

public static org.apache.hadoop.fs.Path initCompPublicResourceDir(SliderFileSystem fs,
 ContainerLaunchService.ComponentLaunchContext compLaunchContext,
 ComponentInstance instance)

createConfigFileAndAddLocalResource

public static void createConfigFileAndAddLocalResource(AbstractLauncher launcher,
 SliderFileSystem fs,
 ContainerLaunchService.ComponentLaunchContext compLaunchContext,
 Map<String,String> tokensForSubstitution,
 ComponentInstance instance,
 org.apache.hadoop.yarn.service.ServiceContext context,
 ProviderService.ResolvedLaunchParams resolvedParams)
                                                throws IOException

Throws:

IOException

handleStaticFilesForLocalization

public static void handleStaticFilesForLocalization(AbstractLauncher launcher,
 SliderFileSystem fs,
 ContainerLaunchService.ComponentLaunchContext componentLaunchCtx,
 ProviderService.ResolvedLaunchParams resolvedParams)
                                             throws IOException

Throws:

IOException

initCompTokensForSubstitute

public static Map<String,String> initCompTokensForSubstitute(ComponentInstance instance,
 org.apache.hadoop.yarn.api.records.Container container,
 ContainerLaunchService.ComponentLaunchContext componentLaunchContext)

Get initial component token map to be substituted into config values.

Returns:

tokens to replace