Package org.apache.hadoop.yarn.server.timeline

Class MemoryTimelineStore

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.yarn.server.timeline.MemoryTimelineStore

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.service.Service, TimelineReader, TimelineStore, TimelineWriter

@Private
@Unstable
public class MemoryTimelineStore
extends org.apache.hadoop.service.AbstractService

In-memory implementation of TimelineStore. This implementation is for test purpose only. If users improperly instantiate it, they may encounter reading and writing history data in different memory store. The methods are synchronized to avoid concurrent modification on the memory.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Nested classes/interfaces inherited from interface org.apache.hadoop.yarn.server.timeline.TimelineReader

TimelineReader.Field

Nested classes/interfaces inherited from interface org.apache.hadoop.yarn.server.timeline.TimelineStore

TimelineStore.SystemFilter

Field Summary

Fields

Modifier and Type	Field	Description
protected org.apache.hadoop.yarn.server.timeline.TimelineStoreMapAdapter<String,org.apache.hadoop.yarn.api.records.timeline.TimelineDomain>	domainById
protected org.apache.hadoop.yarn.server.timeline.TimelineStoreMapAdapter<String,Set<org.apache.hadoop.yarn.api.records.timeline.TimelineDomain>>	domainsByOwner
protected org.apache.hadoop.yarn.server.timeline.TimelineStoreMapAdapter<EntityIdentifier,org.apache.hadoop.yarn.api.records.timeline.TimelineEntity>	entities
protected org.apache.hadoop.yarn.server.timeline.TimelineStoreMapAdapter<EntityIdentifier,Long>	entityInsertTimes

Fields inherited from interface org.apache.hadoop.yarn.server.timeline.TimelineReader

DEFAULT_LIMIT

Constructor Summary

Constructors

Constructor	Description
MemoryTimelineStore()
MemoryTimelineStore(String name)

Method Summary

Modifier and Type	Method	Description
org.apache.hadoop.yarn.api.records.timeline.TimelineDomain	getDomain(String domainId)	This method retrieves the domain information for a given ID.
org.apache.hadoop.yarn.api.records.timeline.TimelineDomains	getDomains(String owner)	This method retrieves all the domains that belong to a given owner.
org.apache.hadoop.yarn.api.records.timeline.TimelineEntities	getEntities(String entityType, Long limit, Long windowStart, Long windowEnd, String fromId, Long fromTs, NameValuePair primaryFilter, Collection<NameValuePair> secondaryFilters, EnumSet<TimelineReader.Field> fields, TimelineDataManager.CheckAcl checkAcl)	This method retrieves a list of entity information, TimelineEntity, sorted by the starting timestamp for the entity, descending.
org.apache.hadoop.yarn.api.records.timeline.TimelineEntity	getEntity(String entityId, String entityType, EnumSet<TimelineReader.Field> fieldsToRetrieve)	This method retrieves the entity information for a given entity.
org.apache.hadoop.yarn.api.records.timeline.TimelineEvents	getEntityTimelines(String entityType, SortedSet<String> entityIds, Long limit, Long windowStart, Long windowEnd, Set<String> eventTypes)	This method retrieves the events for a list of entities all of the same entity type.
boolean	getServiceStopped()
void	put(org.apache.hadoop.yarn.api.records.timeline.TimelineDomain domain)	Store domain information to the timeline store.
org.apache.hadoop.yarn.api.records.timeline.TimelinePutResponse	put(org.apache.hadoop.yarn.api.records.timeline.TimelineEntities data)	Stores entity information to the timeline store.
protected void	serviceStop()

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceInit, serviceStart, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.service.Service

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, registerServiceListener, start, stop, unregisterServiceListener, waitForServiceToStop

Field Details

entities

protected org.apache.hadoop.yarn.server.timeline.TimelineStoreMapAdapter<EntityIdentifier,org.apache.hadoop.yarn.api.records.timeline.TimelineEntity> entities

entityInsertTimes

protected org.apache.hadoop.yarn.server.timeline.TimelineStoreMapAdapter<EntityIdentifier,Long> entityInsertTimes

domainById

protected org.apache.hadoop.yarn.server.timeline.TimelineStoreMapAdapter<String,org.apache.hadoop.yarn.api.records.timeline.TimelineDomain> domainById

domainsByOwner

protected org.apache.hadoop.yarn.server.timeline.TimelineStoreMapAdapter<String,Set<org.apache.hadoop.yarn.api.records.timeline.TimelineDomain>> domainsByOwner

Constructor Details

MemoryTimelineStore

public MemoryTimelineStore()

MemoryTimelineStore

public MemoryTimelineStore(String name)

Method Details

getServiceStopped

public boolean getServiceStopped()

serviceStop

protected void serviceStop()
                    throws Exception

Overrides:

serviceStop in class org.apache.hadoop.service.AbstractService

Throws:

Exception

getEntities

public org.apache.hadoop.yarn.api.records.timeline.TimelineEntities getEntities(String entityType,
 Long limit,
 Long windowStart,
 Long windowEnd,
 String fromId,
 Long fromTs,
 NameValuePair primaryFilter,
 Collection<NameValuePair> secondaryFilters,
 EnumSet<TimelineReader.Field> fields,
 TimelineDataManager.CheckAcl checkAcl)
                                                                         throws IOException

Description copied from interface: TimelineReader

This method retrieves a list of entity information, TimelineEntity, sorted by the starting timestamp for the entity, descending. The starting timestamp of an entity is a timestamp specified by the client. If it is not explicitly specified, it will be chosen by the store to be the earliest timestamp of the events received in the first put for the entity.

Specified by:

getEntities in interface TimelineReader

Parameters:

entityType - The type of entities to return (required).

limit - A limit on the number of entities to return. If null, defaults to TimelineReader.DEFAULT_LIMIT.

windowStart - The earliest start timestamp to retrieve (exclusive). If null, defaults to retrieving all entities until the limit is reached.

windowEnd - The latest start timestamp to retrieve (inclusive). If null, defaults to Long.MAX_VALUE

fromId - If fromId is not null, retrieve entities earlier than and including the specified ID. If no start time is found for the specified ID, an empty list of entities will be returned. The windowEnd parameter will take precedence if the start time of this entity falls later than windowEnd.

fromTs - If fromTs is not null, ignore entities that were inserted into the store after the given timestamp. The entity's insert timestamp used for this comparison is the store's system time when the first put for the entity was received (not the entity's start time).

primaryFilter - Retrieves only entities that have the specified primary filter. If null, retrieves all entities. This is an indexed retrieval, and no entities that do not match the filter are scanned.

secondaryFilters - Retrieves only entities that have exact matches for all the specified filters in their primary filters or other info. This is not an indexed retrieval, so all entities are scanned but only those matching the filters are returned.

fields - Specifies which fields of the entity object to retrieve (see TimelineReader.Field). If the set of fields contains TimelineReader.Field.LAST_EVENT_ONLY and not TimelineReader.Field.EVENTS, the most recent event for each entity is retrieved. If null, retrieves all fields.

Returns:

An TimelineEntities object.

Throws:

IOException

getEntity

public org.apache.hadoop.yarn.api.records.timeline.TimelineEntity getEntity(String entityId,
 String entityType,
 EnumSet<TimelineReader.Field> fieldsToRetrieve)

Description copied from interface: TimelineReader

This method retrieves the entity information for a given entity.

Specified by:

getEntity in interface TimelineReader

Parameters:

entityId - The entity whose information will be retrieved.

entityType - The type of the entity.

fieldsToRetrieve - Specifies which fields of the entity object to retrieve (see TimelineReader.Field). If the set of fields contains TimelineReader.Field.LAST_EVENT_ONLY and not TimelineReader.Field.EVENTS, the most recent event for each entity is retrieved. If null, retrieves all fields.

Returns:

An TimelineEntity object.

getEntityTimelines

public org.apache.hadoop.yarn.api.records.timeline.TimelineEvents getEntityTimelines(String entityType,
 SortedSet<String> entityIds,
 Long limit,
 Long windowStart,
 Long windowEnd,
 Set<String> eventTypes)

Description copied from interface: TimelineReader

This method retrieves the events for a list of entities all of the same entity type. The events for each entity are sorted in order of their timestamps, descending.

Specified by:

getEntityTimelines in interface TimelineReader

Parameters:

entityType - The type of entities to retrieve events for.

entityIds - The entity IDs to retrieve events for.

limit - A limit on the number of events to return for each entity. If null, defaults to TimelineReader.DEFAULT_LIMIT events per entity.

windowStart - If not null, retrieves only events later than the given time (exclusive)

windowEnd - If not null, retrieves only events earlier than the given time (inclusive)

eventTypes - Restricts the events returned to the given types. If null, events of all types will be returned.

Returns:

An TimelineEvents object.

getDomain

public org.apache.hadoop.yarn.api.records.timeline.TimelineDomain getDomain(String domainId)
                                                                     throws IOException

Description copied from interface: TimelineReader

This method retrieves the domain information for a given ID.

Specified by:

getDomain in interface TimelineReader

Returns:

a TimelineDomain object.

Throws:

IOException

getDomains

public org.apache.hadoop.yarn.api.records.timeline.TimelineDomains getDomains(String owner)
                                                                       throws IOException

Description copied from interface: TimelineReader

This method retrieves all the domains that belong to a given owner. The domains are sorted according to the created time firstly and the modified time secondly in descending order.

Specified by:

getDomains in interface TimelineReader

Parameters:

owner - the domain owner

Returns:

an TimelineDomains object.

Throws:

IOException

put

public org.apache.hadoop.yarn.api.records.timeline.TimelinePutResponse put(org.apache.hadoop.yarn.api.records.timeline.TimelineEntities data)

Description copied from interface: TimelineWriter

Stores entity information to the timeline store. Any errors occurring for individual put request objects will be reported in the response.

Specified by:

put in interface TimelineWriter

Parameters:

data - a TimelineEntities object.

Returns:

a TimelinePutResponse object.

put

public void put(org.apache.hadoop.yarn.api.records.timeline.TimelineDomain domain)
         throws IOException

Description copied from interface: TimelineWriter

Store domain information to the timeline store. If A domain of the same ID already exists in the timeline store, it will be COMPLETELY updated with the given domain.

Specified by:

put in interface TimelineWriter

Parameters:

domain - a TimelineDomain object

Throws:

IOException