Package org.apache.hadoop.hdfs.qjournal.protocol

Interface QJournalProtocol

All Known Implementing Classes:

JournalNodeRpcServer, QJournalProtocolTranslatorPB

@Private
public interface QJournalProtocol

Protocol used to communicate between QuorumJournalManager and each JournalNode. This is responsible for sending edits as well as coordinating recovery of the nodes.

Field Summary

Fields

Modifier and Type	Field	Description
static final long	versionID

Method Summary

Modifier and Type	Method	Description
void	acceptRecovery(RequestInfo reqInfo, QJournalProtocolProtos.SegmentStateProto stateToAccept, URL fromUrl)	Accept a proposed recovery for the given transaction ID.
Boolean	canRollBack(String journalId, String nameServiceid, StorageInfo storage, StorageInfo prevStorage, int targetLayoutVersion)
void	discardSegments(String journalId, String nameServiceId, long startTxId)	Discard journal segments whose first TxId is greater than or equal to the given txid.
void	doFinalize(String journalId, String nameServiceid)
void	doPreUpgrade(String journalId)
void	doRollback(String journalId, String nameServiceid)
void	doUpgrade(String journalId, StorageInfo sInfo)
void	finalizeLogSegment(RequestInfo reqInfo, long startTxId, long endTxId)	Finalize the given log segment on the JournalNode.
void	format(String journalId, String nameServiceId, NamespaceInfo nsInfo, boolean force)	Format the underlying storage for the given namespace.
QJournalProtocolProtos.GetEditLogManifestResponseProto	getEditLogManifest(String jid, String nameServiceId, long sinceTxId, boolean inProgressOk)
Long	getJournalCTime(String journalId, String nameServiceId)
QJournalProtocolProtos.GetJournaledEditsResponseProto	getJournaledEdits(String jid, String nameServiceId, long sinceTxId, int maxTxns)	Fetch edit logs present in the Journal's in-memory cache of edits (JournaledEditsCache).
QJournalProtocolProtos.GetJournalStateResponseProto	getJournalState(String journalId, String nameServiceId)	Get the current state of the journal, including the most recent epoch number and the HTTP port.
void	heartbeat(RequestInfo reqInfo)	Heartbeat.
boolean	isFormatted(String journalId, String nameServiceId)
void	journal(RequestInfo reqInfo, long segmentTxId, long firstTxnId, int numTxns, byte[] records)	Journal edit records.
QJournalProtocolProtos.NewEpochResponseProto	newEpoch(String journalId, String nameServiceId, NamespaceInfo nsInfo, long epoch)	Begin a new epoch.
QJournalProtocolProtos.PrepareRecoveryResponseProto	prepareRecovery(RequestInfo reqInfo, long segmentTxId)	Begin the recovery process for a given segment.
void	purgeLogsOlderThan(RequestInfo requestInfo, long minTxIdToKeep)
void	startLogSegment(RequestInfo reqInfo, long txid, int layoutVersion)	Start writing to a new log segment on the JournalNode.

Field Details

versionID

static final long versionID

See Also:

• Constant Field Values

Method Details

isFormatted

boolean isFormatted(String journalId,
 String nameServiceId)
             throws IOException

Returns:

true if the given journal has been formatted and contains valid data.

Throws:

IOException

getJournalState

QJournalProtocolProtos.GetJournalStateResponseProto getJournalState(String journalId,
 String nameServiceId)
                                                             throws IOException

Get the current state of the journal, including the most recent epoch number and the HTTP port.

Throws:

IOException

format

void format(String journalId,
 String nameServiceId,
 NamespaceInfo nsInfo,
 boolean force)
     throws IOException

Format the underlying storage for the given namespace.

Throws:

IOException

newEpoch

QJournalProtocolProtos.NewEpochResponseProto newEpoch(String journalId,
 String nameServiceId,
 NamespaceInfo nsInfo,
 long epoch)
                                               throws IOException

Begin a new epoch. See the HDFS-3077 design doc for details.

Throws:

IOException

journal

void journal(RequestInfo reqInfo,
 long segmentTxId,
 long firstTxnId,
 int numTxns,
 byte[] records)
      throws IOException

Journal edit records. This message is sent by the active name-node to the JournalNodes to write edits to their local logs.

Throws:

IOException

heartbeat

void heartbeat(RequestInfo reqInfo)
        throws IOException

Heartbeat. This is a no-op on the server, except that it verifies that the caller is in fact still the active writer, and provides up-to-date information on the most recently committed txid.

Throws:

IOException

startLogSegment

void startLogSegment(RequestInfo reqInfo,
 long txid,
 int layoutVersion)
              throws IOException

Start writing to a new log segment on the JournalNode. Before calling this, one should finalize the previous segment using finalizeLogSegment(RequestInfo, long, long).

Parameters:

txid - the first txid in the new log

layoutVersion - the LayoutVersion of the new log

Throws:

IOException

finalizeLogSegment

void finalizeLogSegment(RequestInfo reqInfo,
 long startTxId,
 long endTxId)
                 throws IOException

Finalize the given log segment on the JournalNode. The segment is expected to be in-progress and starting at the given startTxId.

Parameters:

startTxId - the starting transaction ID of the log

endTxId - the expected last transaction in the given log

Throws:

IOException - if no such segment exists

purgeLogsOlderThan

void purgeLogsOlderThan(RequestInfo requestInfo,
 long minTxIdToKeep)
                 throws IOException

Throws:

IOException

See Also:

• LogsPurgeable.purgeLogsOlderThan(long)

getEditLogManifest

QJournalProtocolProtos.GetEditLogManifestResponseProto getEditLogManifest(String jid,
 String nameServiceId,
 long sinceTxId,
 boolean inProgressOk)
                                                                   throws IOException

Parameters:

jid - the journal from which to enumerate edits

sinceTxId - the first transaction which the client cares about

inProgressOk - whether or not to check the in-progress edit log segment

Returns:

a list of edit log segments since the given transaction ID.

Throws:

IOException

getJournaledEdits

QJournalProtocolProtos.GetJournaledEditsResponseProto getJournaledEdits(String jid,
 String nameServiceId,
 long sinceTxId,
 int maxTxns)
                                                                 throws IOException

Fetch edit logs present in the Journal's in-memory cache of edits (JournaledEditsCache). To enable this cache, in-progress edit log tailing must be enabled via the "dfs.ha.tail-edits.in-progress" configuration key.

Parameters:

jid - The ID of the journal from which to fetch edits.

nameServiceId - The ID of the namespace for which to fetch edits.

sinceTxId - Fetch edits starting at this transaction ID

maxTxns - Request at most this many transactions to be returned

Returns:

Response containing serialized edits to be loaded

Throws:

IOException - If there was an issue encountered while fetching edits from the cache, including a cache miss (cache does not contain the requested edits). The caller should then attempt to fetch the edits via the streaming mechanism (starting with getEditLogManifest(String, String, long, boolean)).

See Also:

• JournaledEditsCache

prepareRecovery

QJournalProtocolProtos.PrepareRecoveryResponseProto prepareRecovery(RequestInfo reqInfo,
 long segmentTxId)
                                                             throws IOException

Begin the recovery process for a given segment. See the HDFS-3077 design document for details.

Throws:

IOException

acceptRecovery

void acceptRecovery(RequestInfo reqInfo,
 QJournalProtocolProtos.SegmentStateProto stateToAccept,
 URL fromUrl)
             throws IOException

Accept a proposed recovery for the given transaction ID.

Throws:

IOException

doPreUpgrade

void doPreUpgrade(String journalId)
           throws IOException

Throws:

IOException

doUpgrade

void doUpgrade(String journalId,
 StorageInfo sInfo)
        throws IOException

Throws:

IOException

doFinalize

void doFinalize(String journalId,
 String nameServiceid)
         throws IOException

Throws:

IOException

canRollBack

Boolean canRollBack(String journalId,
 String nameServiceid,
 StorageInfo storage,
 StorageInfo prevStorage,
 int targetLayoutVersion)
             throws IOException

Throws:

IOException

doRollback

void doRollback(String journalId,
 String nameServiceid)
         throws IOException

Throws:

IOException

discardSegments

void discardSegments(String journalId,
 String nameServiceId,
 long startTxId)
              throws IOException

Discard journal segments whose first TxId is greater than or equal to the given txid.

Throws:

IOException

getJournalCTime

Long getJournalCTime(String journalId,
 String nameServiceId)
              throws IOException

Throws:

IOException