Package org.apache.hadoop.service.launcher

Class ServiceLauncher<S extends Service>

java.lang.Object
org.apache.hadoop.service.launcher.ServiceLauncher<S>
Type Parameters:
S - service class to cast the generated service to.
All Implemented Interfaces:
Thread.UncaughtExceptionHandler, LauncherArguments, LauncherExitCodes
public class ServiceLauncher<S extends Service> extends Object implements LauncherExitCodes, LauncherArguments, Thread.UncaughtExceptionHandler
A class to launch any YARN service by name. It's designed to be subclassed for custom entry points. Workflow:
  1. An instance of the class is created. It must be of the type Service
  2. If it implements LaunchableService.bindArgs(Configuration, List), it is given the binding args off the CLI after all general configuration arguments have been stripped.
  3. Its Service.init(Configuration) and Service.start() methods are called.
  4. If it implements it, LaunchableService.execute() is called and its return code used as the exit code.
  5. Otherwise: it waits for the service to stop, assuming that the Service.start() method spawns one or more thread to perform work
  6. If any exception is raised and provides an exit code, that is, it implements ExitCodeProvider, the return value of ExitCodeProvider.getExitCode(), becomes the exit code of the command.
Error and warning messages are logged to stderr.

  • Field Details

    • SHUTDOWN_PRIORITY

      protected static final int SHUTDOWN_PRIORITY
      Priority for the shutdown hook: 30.
      See Also:

    • NAME

      public static final String NAME
      The name of this class.
      See Also:

    • USAGE_NAME

      protected static final String USAGE_NAME
      See Also:

    • USAGE_SERVICE_ARGUMENTS

      protected static final String USAGE_SERVICE_ARGUMENTS
      See Also:

    • USAGE_MESSAGE

      public static final String USAGE_MESSAGE
      Usage message. Text: "Usage: ServiceLauncher [--conf <conf file>] [--hadoopconf <configuration classname>] service-classname <service arguments>"
      See Also:

    • DEFAULT_CONFIGS

      protected static final String[] DEFAULT_CONFIGS
      List of the standard configurations to create (and so load in properties). The values are Hadoop, HDFS and YARN configurations.

  • Constructor Details

    • ServiceLauncher

      public ServiceLauncher(String serviceClassName)
      Create an instance of the launcher.
      Parameters:
      serviceClassName - classname of the service

    • ServiceLauncher

      public ServiceLauncher(String serviceName, String serviceClassName)
      Create an instance of the launcher.
      Parameters:
      serviceName - name of service for text messages
      serviceClassName - classname of the service

  • Method Details

    • getService

      public final S getService()
      Get the service. Null until coreServiceLaunch(Configuration, Service, List, boolean, boolean) has completed.
      Returns:
      the service

    • setService

      protected void setService(S s)
      Setter is to give subclasses the ability to manipulate the service.
      Parameters:
      s - the new service

    • getConfiguration

      public final Configuration getConfiguration()
      Get the configuration constructed from the command line arguments.
      Returns:
      the configuration used to create the service

    • getServiceExitCode

      public final int getServiceExitCode()
      The exit code from a successful service execution.
      Returns:
      the exit code.

    • getServiceException

      public final ExitUtil.ExitException getServiceException()
      Get the exit exception used to end this service.
      Returns:
      an exception, which will be null until the service has exited (and System.exit has not been called)

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • launchServiceAndExit

      public void launchServiceAndExit(List<String> args)
      Launch the service and exit.
      1. Parse the command line.
      2. Build the service configuration from it.
      3. Start the service.
      4. If it is a LaunchableService: execute it
      5. Otherwise: wait for it to finish.
      6. Exit passing the status code to the exit(int, String) method.
      Parameters:
      args - arguments to the service. arg[0] is assumed to be the service classname.

    • bindCommandOptions

      protected void bindCommandOptions()
      Set the commandOptions field to the result of createOptions(); protected for subclasses and test access.

    • getUsageMessage

      protected String getUsageMessage()
      Get the usage message, ideally dynamically.
      Returns:
      the usage message

    • createOptions

      protected org.apache.commons.cli.Options createOptions()
      Override point: create an options instance to combine with the standard options set. Important. Synchronize uses of Option with Option.class
      Returns:
      the new options

    • createConfiguration

      protected Configuration createConfiguration()
      Override point: create the base configuration for the service. Subclasses can override to create HDFS/YARN configurations etc.
      Returns:
      the configuration to use as the service initializer.

    • getConfigurationsToCreate

      protected List<String> getConfigurationsToCreate()
      Override point: Get a list of configuration classes to create.
      Returns:
      the array of configs to attempt to create. If any are off the classpath, that is logged

    • loadConfigurationClasses

      @VisibleForTesting public int loadConfigurationClasses()
      Returns:
      This creates all the configurations defined by getConfigurationsToCreate() , ensuring that the resources have been pushed in. If one cannot be loaded it is logged and the operation continues except in the case that the class does load but it isn't actually a subclass of Configuration.
      Throws:
      ExitUtil.ExitException - if a loaded class is of the wrong type

    • launchService

      public ExitUtil.ExitException launchService(Configuration conf, List<String> processedArgs, boolean addShutdownHook, boolean execute)
      Launch a service catching all exceptions and downgrading them to exit codes after logging. Sets serviceException to this value.
      Parameters:
      conf - configuration to use
      processedArgs - command line after the launcher-specific arguments have been stripped out.
      addShutdownHook - should a shutdown hook be added to terminate this service on shutdown. Tests should set this to false.
      execute - execute/wait for the service to stop.
      Returns:
      an exit exception, which will have a status code of 0 if it worked

    • launchService

      public ExitUtil.ExitException launchService(Configuration conf, S instance, List<String> processedArgs, boolean addShutdownHook, boolean execute)
      Launch a service catching all exceptions and downgrading them to exit codes after logging. Sets serviceException to this value.
      Parameters:
      conf - configuration to use
      instance - optional instance of the service.
      processedArgs - command line after the launcher-specific arguments have been stripped out.
      addShutdownHook - should a shutdown hook be added to terminate this service on shutdown. Tests should set this to false.
      execute - execute/wait for the service to stop.
      Returns:
      an exit exception, which will have a status code of 0 if it worked

    • coreServiceLaunch

      protected int coreServiceLaunch(Configuration conf, S instance, List<String> processedArgs, boolean addShutdownHook, boolean execute) throws Exception
      Launch the service. All exceptions that occur are propagated upwards. If the method returns a status code, it means that it got as far starting the service, and if it implements LaunchableService, that the method LaunchableService.execute() has completed. After this method returns, the service can be retrieved returned by getService().
      Parameters:
      conf - configuration
      instance - optional instance of the service.
      processedArgs - arguments after the configuration parameters have been stripped out.
      addShutdownHook - should a shutdown hook be added to terminate this service on shutdown. Tests should set this to false.
      execute - execute/wait for the service to stop
      Returns:
      status code.
      Throws:
      ClassNotFoundException - classname not on the classpath
      IllegalAccessException - not allowed at the class
      InstantiationException - not allowed to instantiate it
      InterruptedException - thread interrupted
      ExitUtil.ExitException - any exception defining the status code.
      Exception - any other failure -if it implements ExitCodeProvider then it defines the exit code for any containing exception

    • instantiateService

      public Service instantiateService(Configuration conf)
      Parameters:
      conf - configuration to use
      Returns:
      Instantiate the service defined in serviceClassName. Sets the configuration field to the the value of conf, and the service field to the service created.

    • convertToExitException

      protected static ExitUtil.ExitException convertToExitException(Throwable thrown)
      Convert an exception to an ExitException. This process may just be a simple pass through, otherwise a new exception is created with an exit code, the text of the supplied exception, and the supplied exception as an inner cause.
      1. If is already the right type, pass it through.
      2. If it implements ExitCodeProvider.getExitCode(), the exit code is extracted and used in the new exception.
      3. Otherwise, the exit code LauncherExitCodes.EXIT_EXCEPTION_THROWN is used.
      Parameters:
      thrown - the exception thrown
      Returns:
      an ExitException with a status code

    • serviceCreationFailure

      protected ServiceLaunchException serviceCreationFailure(Exception exception)
      Generate an exception announcing a failure to create the service.
      Parameters:
      exception - inner exception.
      Returns:
      a new exception, with the exit code LauncherExitCodes.EXIT_SERVICE_CREATION_FAILURE

    • registerFailureHandling

      protected void registerFailureHandling()
      Override point: register this class as the handler for the control-C and SIGINT interrupts. Subclasses can extend this with extra operations, such as an exception handler: 
        Thread.setDefaultUncaughtExceptionHandler(
     new YarnUncaughtExceptionHandler());

    • uncaughtException

      public void uncaughtException(Thread thread, Throwable exception)
      Handler for uncaught exceptions: terminate the service.
      Specified by:
      uncaughtException in interface Thread.UncaughtExceptionHandler
      Parameters:
      thread - thread
      exception - exception

    • getServiceName

      public String getServiceName()
      Get the service name via Service.getName(). If the service is not instantiated, the classname is returned instead.
      Returns:
      the service name

    • warn

      protected void warn(String text)
      Print a warning message.

      This tries to log to the log's warn() operation. If the log at that level is disabled it logs to system error

      Parameters:
      text - warning text

    • error

      protected void error(String message, Throwable thrown)
      Report an error.

      This tries to log to LOG.error().

      If that log level is disabled disabled the message is logged to system error along with thrown.toString()

      Parameters:
      message - message for the user
      thrown - the exception thrown

    • exit

      protected void exit(int exitCode, String message)
      Exit the JVM. This is method can be overridden for testing, throwing an exception instead. Any subclassed method MUST raise an ExitException instance/subclass. The service launcher code assumes that after this method is invoked, no other code in the same method is called.
      Parameters:
      exitCode - code to exit
      message - input message.

    • exit

      protected void exit(ExitUtil.ExitException ee)
      Exit the JVM using an exception for the exit code and message, invoking ExitUtil.terminate(ExitUtil.ExitException). This is the standard way a launched service exits. An error code of 0 means success -nothing is printed. If ExitUtil.disableSystemExit() has been called, this method will throw the exception. The method may be subclassed for testing
      Parameters:
      ee - exit exception
      Throws:
      ExitUtil.ExitException - if ExitUtil exceptions are disabled

    • getClassLoader

      protected ClassLoader getClassLoader()
      Override point: get the classloader to use.
      Returns:
      the classloader for loading a service class.

    • extractCommandOptions

      public List<String> extractCommandOptions(Configuration conf, List<String> args)
      Extract the command options and apply them to the configuration, building an array of processed arguments to hand down to the service.
      Parameters:
      conf - configuration to update.
      args - main arguments. args[0]is assumed to be the service classname and is skipped.
      Returns:
      the remaining arguments
      Throws:
      ExitUtil.ExitException - if JVM exiting is disabled.

    • parseCommandArgs

      protected List<String> parseCommandArgs(Configuration conf, List<String> args)
      Parse the command arguments, extracting the service class as the last element of the list (after extracting all the rest). The field commandOptions field must already have been set.
      Parameters:
      conf - configuration to use
      args - command line argument list
      Returns:
      the remaining arguments
      Throws:
      ServiceLaunchException - if processing of arguments failed

    • createGenericOptionsParser

      protected GenericOptionsParser createGenericOptionsParser(Configuration conf, String[] argArray) throws IOException
      Override point: create a generic options parser or subclass thereof.
      Parameters:
      conf - Hadoop configuration
      argArray - array of arguments
      Returns:
      a generic options parser to parse the arguments
      Throws:
      IOException - on any failure

    • verifyConfigurationFilesExist

      protected void verifyConfigurationFilesExist(String[] filenames)
      Verify that all the specified filenames exist.
      Parameters:
      filenames - a list of files
      Throws:
      ServiceLaunchException - if a file is not found

    • startupShutdownMessage

      protected static String startupShutdownMessage(String classname, List<String> args)
      Parameters:
      classname - the class of the server
      args - arguments
      Returns:
      Build a log message for starting up and shutting down.

    • exitWithMessage

      protected static void exitWithMessage(int status, String message)
      Exit with a printed message.
      Parameters:
      status - status code
      message - message message to print before exiting
      Throws:
      ExitUtil.ExitException - if exceptions are disabled

    • exitWithUsageMessage

      protected static void exitWithUsageMessage()
      Exit with the usage exit code LauncherExitCodes.EXIT_USAGE and message USAGE_MESSAGE.
      Throws:
      ExitUtil.ExitException - if exceptions are disabled

    • main

      public static void main(String[] args)
      This is the JVM entry point for the service launcher. Converts the arguments to a list, then invokes serviceMain(List)
      Parameters:
      args - command line arguments.

    • serviceMain

      public static void serviceMain(String... args)
      Varargs version of the entry point for testing and other in-JVM use. Hands off to serviceMain(List)
      Parameters:
      args - command line arguments.

    • serviceMain

      public static void serviceMain(List<String> argsList)
      The real main function, which takes the arguments as a list. Argument 0 MUST be the service classname
      Parameters:
      argsList - the list of arguments