Package org.apache.hadoop.yarn.server.timelineservice.documentstore

Class DocumentStoreUtils

java.lang.Object
org.apache.hadoop.yarn.server.timelineservice.documentstore.DocumentStoreUtils
public final class DocumentStoreUtils extends Object
This class consists of all the utils required for reading or writing documents for a DocumentStoreVendor.

  • Method Details

    • validateCosmosDBConf

      public static void validateCosmosDBConf(org.apache.hadoop.conf.Configuration conf) throws org.apache.hadoop.yarn.exceptions.YarnException
      Checks whether the cosmosdb conf are set properly in yarn-site.xml conf.
      Parameters:
      conf - related to yarn
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if required config properties are missing

    • getStoreVendor

      public static DocumentStoreVendor getStoreVendor(org.apache.hadoop.conf.Configuration conf)
      Retrieves DocumentStoreVendor configured.
      Parameters:
      conf - related to yarn
      Returns:
      Returns the DocumentStoreVendor that is configured, else uses DocumentStoreVendor.COSMOS_DB as default

    • fetchEvent

      public static org.apache.hadoop.yarn.api.records.timelineservice.TimelineEvent fetchEvent(org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity timelineEntity, String eventType)
      Retrieves a TimelineEvent from TimelineEntity.events.
      Parameters:
      timelineEntity - from which the set of events are examined.
      eventType - that has to be checked.
      Returns:
      TimelineEvent if found else null

    • isNullOrEmpty

      public static boolean isNullOrEmpty(String... values)
      Checks if the string is null or empty.
      Parameters:
      values - array of string to be checked
      Returns:
      false if any of the string is null or empty else true

    • createCosmosDBAsyncClient

      public static com.microsoft.azure.cosmosdb.rx.AsyncDocumentClient createCosmosDBAsyncClient(org.apache.hadoop.conf.Configuration conf)
      Creates CosmosDB Async Document Client.
      Parameters:
      conf - to retrieve cosmos db endpoint and key
      Returns:
      async document client for CosmosDB

    • getTopOfTheDayTimestamp

      public static long getTopOfTheDayTimestamp(long timeStamp)
      Returns the timestamp of the day's start (which is midnight 00:00:00 AM) for a given input timestamp.
      Parameters:
      timeStamp - Timestamp.
      Returns:
      timestamp of that day's beginning (midnight)

    • constructTimelineEntityDocId

      public static String constructTimelineEntityDocId(org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorContext collectorContext, String type)
      Creates a composite key for storing TimelineEntityDocument.
      Parameters:
      collectorContext - of the timeline writer
      type - of the entity
      Returns:
      composite key delimited with !

    • constructTimelineEntityDocId

      public static String constructTimelineEntityDocId(org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorContext collectorContext, String type, String id)
      Creates a composite key for storing TimelineEntityDocument.
      Parameters:
      collectorContext - of the timeline writer
      type - of the entity
      id - of the entity
      Returns:
      composite key delimited with !

    • constructFlowRunDocId

      public static String constructFlowRunDocId(org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorContext collectorContext)
      Creates a composite key for storing FlowRunDocument.
      Parameters:
      collectorContext - of the timeline writer
      Returns:
      composite key delimited with !

    • constructFlowActivityDocId

      public static String constructFlowActivityDocId(org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorContext collectorContext, long eventTimestamp)
      Creates a composite key for storing FlowActivityDocument.
      Parameters:
      collectorContext - of the timeline writer
      eventTimestamp - of the timeline entity
      Returns:
      composite key delimited with !

    • getCosmosDBDatabaseName

      public static String getCosmosDBDatabaseName(org.apache.hadoop.conf.Configuration conf)

    • createEntityToBeReturned

      public static org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity createEntityToBeReturned(TimelineEntityDocument timelineEntityDocument, org.apache.hadoop.yarn.server.timelineservice.reader.TimelineDataToRetrieve dataToRetrieve)
      Creates the final entity to be returned as the result.
      Parameters:
      timelineEntityDocument - which has all the information for the entity
      dataToRetrieve - specifies filters and fields to retrieve
      Returns:
      TimelineEntity as the result

    • createEntityToBeReturned

      public static org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity createEntityToBeReturned(TimelineEntityDocument timelineEntityDocument, org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterList confsToRetrieve, org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterList metricsToRetrieve)
      Creates the final entity to be returned as the result.
      Parameters:
      timelineEntityDocument - which has all the information for the entity
      confsToRetrieve - specifies config filters to be applied
      metricsToRetrieve - specifies metric filters to be applied
      Returns:
      TimelineEntity as the result

    • transformMetrics

      public static Set<org.apache.hadoop.yarn.api.records.timelineservice.TimelineMetric> transformMetrics(org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterList metricsToRetrieve, Map<String,Set<TimelineMetricSubDoc>> metrics)

    • applyConfigFilter

      public static Map<String,String> applyConfigFilter(org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterList configsToRetrieve, Map<String,String> configs)