Package org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.deviceframework

Class DeviceResourceHandlerImpl

java.lang.Object

org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.deviceframework.DeviceResourceHandlerImpl

All Implemented Interfaces:

ResourceHandler

public class DeviceResourceHandlerImpl
extends Object
implements ResourceHandler

The Hooks into container lifecycle. Get device list from device plugin in bootstrap Assign devices for a container in preStart Restore statue in reacquireContainer Recycle devices from container in postComplete

Field Summary

Fields

Modifier and Type	Field	Description
static final String	ALLOWED_DEVICES_CLI_OPTION
static final String	CONTAINER_ID_CLI_OPTION
static final String	EXCLUDED_DEVICES_CLI_OPTION

Constructor Summary

Constructors

Constructor	Description
DeviceResourceHandlerImpl(String resName, DevicePluginAdapter devPluginAdapter, DeviceMappingManager devMappingManager, CGroupsHandler cgHandler, PrivilegedOperationExecutor operation, Context ctx)
DeviceResourceHandlerImpl(String resName, DevicePluginAdapter devPluginAdapter, DeviceMappingManager devMappingManager, CGroupsHandler cgHandler, PrivilegedOperationExecutor operation, Context ctx, ShellWrapper shell)

Method Summary

Modifier and Type	Method	Description
List<PrivilegedOperation>	bootstrap(org.apache.hadoop.conf.Configuration configuration)	Bootstrap resource susbsystem.
org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.deviceframework.DeviceResourceHandlerImpl.DeviceType	getDeviceType(Device device)
org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.deviceframework.DeviceResourceHandlerImpl.DeviceType	getDeviceTypeFromDeviceNumber(int major, int minor)	Get the device type used for cgroups value set.
List<PrivilegedOperation>	postComplete(org.apache.hadoop.yarn.api.records.ContainerId containerId)	Perform any tasks necessary after container completion.
List<PrivilegedOperation>	preStart(Container container)	Prepare a resource environment for container launch
List<PrivilegedOperation>	reacquireContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)	Require state for container that was already launched
List<PrivilegedOperation>	teardown()	Teardown environment for resource subsystem if requested.
String	toString()
List<PrivilegedOperation>	updateContainer(Container container)	Update state for container that was already launched

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

EXCLUDED_DEVICES_CLI_OPTION

public static final String EXCLUDED_DEVICES_CLI_OPTION

See Also:

• Constant Field Values

ALLOWED_DEVICES_CLI_OPTION

public static final String ALLOWED_DEVICES_CLI_OPTION

See Also:

• Constant Field Values

CONTAINER_ID_CLI_OPTION

public static final String CONTAINER_ID_CLI_OPTION

See Also:

• Constant Field Values

Constructor Details

DeviceResourceHandlerImpl

public DeviceResourceHandlerImpl(String resName,
 DevicePluginAdapter devPluginAdapter,
 DeviceMappingManager devMappingManager,
 CGroupsHandler cgHandler,
 PrivilegedOperationExecutor operation,
 Context ctx)

DeviceResourceHandlerImpl

@VisibleForTesting
public DeviceResourceHandlerImpl(String resName,
 DevicePluginAdapter devPluginAdapter,
 DeviceMappingManager devMappingManager,
 CGroupsHandler cgHandler,
 PrivilegedOperationExecutor operation,
 Context ctx,
 ShellWrapper shell)

Method Details

bootstrap

public List<PrivilegedOperation> bootstrap(org.apache.hadoop.conf.Configuration configuration)
                                    throws ResourceHandlerException

Description copied from interface: ResourceHandler

Bootstrap resource susbsystem.

Specified by:

bootstrap in interface ResourceHandler

Returns:

(possibly empty) list of operations that require elevated privileges

Throws:

ResourceHandlerException

preStart

public List<PrivilegedOperation> preStart(Container container)
                                   throws ResourceHandlerException

Description copied from interface: ResourceHandler

Prepare a resource environment for container launch

Specified by:

preStart in interface ResourceHandler

Parameters:

container - Container being launched

Returns:

(possibly empty) list of operations that require elevated privileges e.g a) create a custom cgroup b) add pid for container to tasks file for a cgroup.

Throws:

ResourceHandlerException

reacquireContainer

public List<PrivilegedOperation> reacquireContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
                                             throws ResourceHandlerException

Description copied from interface: ResourceHandler

Require state for container that was already launched

Specified by:

reacquireContainer in interface ResourceHandler

Parameters:

containerId - id of the container being reacquired.

Returns:

(possibly empty) list of operations that require elevated privileges

Throws:

ResourceHandlerException

updateContainer

public List<PrivilegedOperation> updateContainer(Container container)
                                          throws ResourceHandlerException

Description copied from interface: ResourceHandler

Update state for container that was already launched

Specified by:

updateContainer in interface ResourceHandler

Parameters:

container - the container being updated.

Returns:

(possibly empty) list of operations that require elevated privileges

Throws:

ResourceHandlerException

postComplete

public List<PrivilegedOperation> postComplete(org.apache.hadoop.yarn.api.records.ContainerId containerId)
                                       throws ResourceHandlerException

Description copied from interface: ResourceHandler

Perform any tasks necessary after container completion.

Specified by:

postComplete in interface ResourceHandler

Parameters:

containerId - of the container that was completed.

Returns:

(possibly empty) list of operations that require elevated privileges

Throws:

ResourceHandlerException

teardown

public List<PrivilegedOperation> teardown()
                                   throws ResourceHandlerException

Description copied from interface: ResourceHandler

Teardown environment for resource subsystem if requested. This method needs to be used with care since it could impact running containers.

Specified by:

teardown in interface ResourceHandler

Returns:

(possibly empty) list of operations that require elevated privileges

Throws:

ResourceHandlerException

toString

public String toString()

Overrides:

toString in class Object

getDeviceType

public org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.deviceframework.DeviceResourceHandlerImpl.DeviceType getDeviceType(Device device)

getDeviceTypeFromDeviceNumber

public org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.deviceframework.DeviceResourceHandlerImpl.DeviceType getDeviceTypeFromDeviceNumber(int major,
 int minor)

Get the device type used for cgroups value set. If sys file "/sys/dev/block/major:minor" exists, it's block device. Otherwise, it's char device. An exception is that Nvidia GPU doesn't create this sys file. so assume character device by default.