Package org.apache.hadoop.mapreduce.v2.util

Class MRApps

java.lang.Object
org.apache.hadoop.yarn.util.Apps
org.apache.hadoop.mapreduce.v2.util.MRApps
@Private @Unstable public class MRApps extends org.apache.hadoop.yarn.util.Apps
Helper class for MR applications

  • Field Details

    • LOG

      public static final org.slf4j.Logger LOG

  • Constructor Details

    • MRApps

      public MRApps()

  • Method Details

    • toString

      public static String toString(JobId jid)

    • toJobID

      public static JobId toJobID(String jid)

    • toString

      public static String toString(TaskId tid)

    • toTaskID

      public static TaskId toTaskID(String tid)

    • toString

      public static String toString(TaskAttemptId taid)

    • toTaskAttemptID

      public static TaskAttemptId toTaskAttemptID(String taid)

    • taskSymbol

      public static String taskSymbol(TaskType type)

    • taskType

      public static TaskType taskType(String symbol)

    • taskAttemptState

      public static MRApps.TaskAttemptStateUI taskAttemptState(String attemptStateStr)

    • taskState

      public static MRApps.TaskStateUI taskState(String taskStateStr)

    • setClasspath

      public static void setClasspath(Map<String,String> environment, org.apache.hadoop.conf.Configuration conf) throws IOException
      Throws:
      IOException

    • setJobClassLoader

      public static void setJobClassLoader(org.apache.hadoop.conf.Configuration conf) throws IOException
      Creates and sets a ApplicationClassLoader on the given configuration and as the thread context classloader, if MRJobConfig.MAPREDUCE_JOB_CLASSLOADER is set to true, and the APP_CLASSPATH environment variable is set.
      Parameters:
      conf -
      Throws:
      IOException

    • createJobClassLoader

      public static ClassLoader createJobClassLoader(org.apache.hadoop.conf.Configuration conf) throws IOException
      Creates a ApplicationClassLoader if MRJobConfig.MAPREDUCE_JOB_CLASSLOADER is set to true, and the APP_CLASSPATH environment variable is set.
      Parameters:
      conf -
      Returns:
      the created job classloader, or null if the job classloader is not enabled or the APP_CLASSPATH environment variable is not set
      Throws:
      IOException

    • setClassLoader

      public static void setClassLoader(ClassLoader classLoader, org.apache.hadoop.conf.Configuration conf)
      Sets the provided classloader on the given configuration and as the thread context classloader if the classloader is not null.
      Parameters:
      classLoader -
      conf -

    • getStagingAreaDir

      public static org.apache.hadoop.fs.Path getStagingAreaDir(org.apache.hadoop.conf.Configuration conf, String user)

    • getJobFile

      public static String getJobFile(org.apache.hadoop.conf.Configuration conf, String user, org.apache.hadoop.mapreduce.JobID jobId)

    • getEndJobCommitSuccessFile

      public static org.apache.hadoop.fs.Path getEndJobCommitSuccessFile(org.apache.hadoop.conf.Configuration conf, String user, JobId jobId)

    • getEndJobCommitFailureFile

      public static org.apache.hadoop.fs.Path getEndJobCommitFailureFile(org.apache.hadoop.conf.Configuration conf, String user, JobId jobId)

    • getStartJobCommitFile

      public static org.apache.hadoop.fs.Path getStartJobCommitFile(org.apache.hadoop.conf.Configuration conf, String user, JobId jobId)

    • setupDistributedCache

      public static void setupDistributedCache(org.apache.hadoop.conf.Configuration conf, Map<String,org.apache.hadoop.yarn.api.records.LocalResource> localResources) throws IOException
      Throws:
      IOException

    • setupDistributedCacheLocal

      public static void setupDistributedCacheLocal(org.apache.hadoop.conf.Configuration conf) throws IOException
      Set up the DistributedCache related configs to make JobContextImpl.getLocalCacheFiles(Configuration) and JobContextImpl.getLocalCacheArchives(Configuration) working.
      Parameters:
      conf -
      Throws:
      IOException

    • getChildLogLevel

      public static String getChildLogLevel(org.apache.hadoop.conf.Configuration conf, boolean isMap)

    • addLog4jSystemProperties

      public static void addLog4jSystemProperties(org.apache.hadoop.mapred.Task task, List<String> vargs, org.apache.hadoop.conf.Configuration conf)
      Add the JVM system properties necessary to configure ContainerLogAppender or ContainerRollingLogAppender.
      Parameters:
      task - for map/reduce, or null for app master
      vargs - the argument list to append to
      conf - configuration of MR job

    • getSystemPropertiesToLog

      public static String getSystemPropertiesToLog(org.apache.hadoop.conf.Configuration conf)
      Return lines for system property keys and values per configuration.
      Returns:
      the formatted string for the system property lines or null if no properties are specified.

    • setEnvFromInputString

      public static void setEnvFromInputString(Map<String,String> env, String envString, org.apache.hadoop.conf.Configuration conf)

    • setEnvFromInputProperty

      public static void setEnvFromInputProperty(Map<String,String> env, String propName, String defaultPropValue, org.apache.hadoop.conf.Configuration conf)

    • addToEnvironment

      @Public @Unstable public static void addToEnvironment(Map<String,String> environment, String variable, String value, org.apache.hadoop.conf.Configuration conf)

    • crossPlatformifyMREnv

      public static String crossPlatformifyMREnv(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.ApplicationConstants.Environment env)