Package org.apache.hadoop.hdfs.qjournal.server

Class JournalNodeRpcServer

java.lang.Object

org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer

All Implemented Interfaces:

InterQJournalProtocol, QJournalProtocol

@Private
@VisibleForTesting
public class JournalNodeRpcServer
extends Object
implements QJournalProtocol, InterQJournalProtocol

Field Summary

Fields inherited from interface org.apache.hadoop.hdfs.qjournal.protocol.InterQJournalProtocol

versionID

Fields inherited from interface org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocol

versionID

Method Summary

Modifier and Type	Method	Description
void	acceptRecovery(RequestInfo reqInfo, QJournalProtocolProtos.SegmentStateProto log, URL fromUrl)	Accept a proposed recovery for the given transaction ID.
Boolean	canRollBack(String journalId, String nameServiceId, StorageInfo storage, StorageInfo prevStorage, int targetLayoutVersion)
void	discardSegments(String journalId, String nameServiceId, long startTxId)	Discard journal segments whose first TxId is greater than or equal to the given txid.
void	doFinalize(String journalId, String nameServiceId)
void	doPreUpgrade(String journalId)
void	doRollback(String journalId, String nameServiceId)
void	doUpgrade(String journalId, StorageInfo sInfo)
void	finalizeLogSegment(RequestInfo reqInfo, long startTxId, long endTxId)	Finalize the given log segment on the JournalNode.
void	format(String journalId, String nameServiceId, NamespaceInfo nsInfo, boolean force)	Format the underlying storage for the given namespace.
InetSocketAddress	getAddress()
QJournalProtocolProtos.GetEditLogManifestResponseProto	getEditLogManifest(String jid, String nameServiceId, long sinceTxId, boolean inProgressOk)
QJournalProtocolProtos.GetEditLogManifestResponseProto	getEditLogManifestFromJournal(String jid, String nameServiceId, long sinceTxId, boolean inProgressOk)
protected int	getHandlerCount()
Long	getJournalCTime(String journalId, String nameServiceId)
QJournalProtocolProtos.GetJournaledEditsResponseProto	getJournaledEdits(String jid, String nameServiceId, long sinceTxId, int maxTxns)	Fetch edit logs present in the Journal's in-memory cache of edits (JournaledEditsCache).
QJournalProtocolProtos.GetJournalStateResponseProto	getJournalState(String journalId, String nameServiceId)	Get the current state of the journal, including the most recent epoch number and the HTTP port.
org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.StorageInfoProto	getStorageInfo(String jid, String nameServiceId)	Get the storage info for the specified journal.
void	heartbeat(RequestInfo reqInfo)	Heartbeat.
boolean	isFormatted(String journalId, String nameServiceId)
void	journal(RequestInfo reqInfo, long segmentTxId, long firstTxnId, int numTxns, byte[] records)	Journal edit records.
QJournalProtocolProtos.NewEpochResponseProto	newEpoch(String journalId, String nameServiceId, NamespaceInfo nsInfo, long epoch)	Begin a new epoch.
QJournalProtocolProtos.PrepareRecoveryResponseProto	prepareRecovery(RequestInfo reqInfo, long segmentTxId)	Begin the recovery process for a given segment.
void	purgeLogsOlderThan(RequestInfo reqInfo, long minTxIdToKeep)
void	startLogSegment(RequestInfo reqInfo, long txid, int layoutVersion)	Start writing to a new log segment on the JournalNode.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

getHandlerCount

@VisibleForTesting
protected int getHandlerCount()

getAddress

public InetSocketAddress getAddress()

isFormatted

public boolean isFormatted(String journalId,
 String nameServiceId)
                    throws IOException

Specified by:

isFormatted in interface QJournalProtocol

Returns:

true if the given journal has been formatted and contains valid data.

Throws:

IOException

getJournalState

public QJournalProtocolProtos.GetJournalStateResponseProto getJournalState(String journalId,
 String nameServiceId)
                                                                    throws IOException

Description copied from interface: QJournalProtocol

Get the current state of the journal, including the most recent epoch number and the HTTP port.

Specified by:

getJournalState in interface QJournalProtocol

Throws:

IOException

newEpoch

public QJournalProtocolProtos.NewEpochResponseProto newEpoch(String journalId,
 String nameServiceId,
 NamespaceInfo nsInfo,
 long epoch)
                                                      throws IOException

Description copied from interface: QJournalProtocol

Begin a new epoch. See the HDFS-3077 design doc for details.

Specified by:

newEpoch in interface QJournalProtocol

Throws:

IOException

format

public void format(String journalId,
 String nameServiceId,
 NamespaceInfo nsInfo,
 boolean force)
            throws IOException

Description copied from interface: QJournalProtocol

Format the underlying storage for the given namespace.

Specified by:

format in interface QJournalProtocol

Throws:

IOException

journal

public void journal(RequestInfo reqInfo,
 long segmentTxId,
 long firstTxnId,
 int numTxns,
 byte[] records)
             throws IOException

Description copied from interface: QJournalProtocol

Journal edit records. This message is sent by the active name-node to the JournalNodes to write edits to their local logs.

Specified by:

journal in interface QJournalProtocol

Throws:

IOException

heartbeat

public void heartbeat(RequestInfo reqInfo)
               throws IOException

Description copied from interface: QJournalProtocol

Heartbeat. This is a no-op on the server, except that it verifies that the caller is in fact still the active writer, and provides up-to-date information on the most recently committed txid.

Specified by:

heartbeat in interface QJournalProtocol

Throws:

IOException

startLogSegment

public void startLogSegment(RequestInfo reqInfo,
 long txid,
 int layoutVersion)
                     throws IOException

Description copied from interface: QJournalProtocol

Start writing to a new log segment on the JournalNode. Before calling this, one should finalize the previous segment using QJournalProtocol.finalizeLogSegment(RequestInfo, long, long).

Specified by:

startLogSegment in interface QJournalProtocol

txid - the first txid in the new log

layoutVersion - the LayoutVersion of the new log

Throws:

IOException

finalizeLogSegment

public void finalizeLogSegment(RequestInfo reqInfo,
 long startTxId,
 long endTxId)
                        throws IOException

Description copied from interface: QJournalProtocol

Finalize the given log segment on the JournalNode. The segment is expected to be in-progress and starting at the given startTxId.

Specified by:

finalizeLogSegment in interface QJournalProtocol

startTxId - the starting transaction ID of the log

endTxId - the expected last transaction in the given log

Throws:

IOException - if no such segment exists

purgeLogsOlderThan

public void purgeLogsOlderThan(RequestInfo reqInfo,
 long minTxIdToKeep)
                        throws IOException

Specified by:

purgeLogsOlderThan in interface QJournalProtocol

Throws:

IOException

See Also:

• LogsPurgeable.purgeLogsOlderThan(long)

getEditLogManifest

public QJournalProtocolProtos.GetEditLogManifestResponseProto getEditLogManifest(String jid,
 String nameServiceId,
 long sinceTxId,
 boolean inProgressOk)
                                                                          throws IOException

Specified by:

getEditLogManifest in interface QJournalProtocol

Parameters:

jid - the journal from which to enumerate edits

sinceTxId - the first transaction which the client cares about

inProgressOk - whether or not to check the in-progress edit log segment

Returns:

a list of edit log segments since the given transaction ID.

Throws:

IOException

getStorageInfo

public org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.StorageInfoProto getStorageInfo(String jid,
 String nameServiceId)
                                                                                       throws IOException

Description copied from interface: InterQJournalProtocol

Get the storage info for the specified journal.

Specified by:

getStorageInfo in interface InterQJournalProtocol

Parameters:

jid - the journal identifier

nameServiceId - the name service id

Returns:

the storage info object

Throws:

IOException

getJournaledEdits

public QJournalProtocolProtos.GetJournaledEditsResponseProto getJournaledEdits(String jid,
 String nameServiceId,
 long sinceTxId,
 int maxTxns)
                                                                        throws IOException

Description copied from interface: QJournalProtocol

Fetch edit logs present in the Journal's in-memory cache of edits (JournaledEditsCache). To enable this cache, in-progress edit log tailing must be enabled via the "dfs.ha.tail-edits.in-progress" configuration key.

Specified by:

getJournaledEdits in interface QJournalProtocol

Parameters:

jid - The ID of the journal from which to fetch edits.

nameServiceId - The ID of the namespace for which to fetch edits.

sinceTxId - Fetch edits starting at this transaction ID

maxTxns - Request at most this many transactions to be returned

Returns:

Response containing serialized edits to be loaded

Throws:

IOException - If there was an issue encountered while fetching edits from the cache, including a cache miss (cache does not contain the requested edits). The caller should then attempt to fetch the edits via the streaming mechanism (starting with QJournalProtocol.getEditLogManifest(String, String, long, boolean)).

See Also:

• JournaledEditsCache

prepareRecovery

public QJournalProtocolProtos.PrepareRecoveryResponseProto prepareRecovery(RequestInfo reqInfo,
 long segmentTxId)
                                                                    throws IOException

Description copied from interface: QJournalProtocol

Begin the recovery process for a given segment. See the HDFS-3077 design document for details.

Specified by:

prepareRecovery in interface QJournalProtocol

Throws:

IOException

acceptRecovery

public void acceptRecovery(RequestInfo reqInfo,
 QJournalProtocolProtos.SegmentStateProto log,
 URL fromUrl)
                    throws IOException

Description copied from interface: QJournalProtocol

Accept a proposed recovery for the given transaction ID.

Specified by:

acceptRecovery in interface QJournalProtocol

Throws:

IOException

doPreUpgrade

public void doPreUpgrade(String journalId)
                  throws IOException

Specified by:

doPreUpgrade in interface QJournalProtocol

Throws:

IOException

doUpgrade

public void doUpgrade(String journalId,
 StorageInfo sInfo)
               throws IOException

Specified by:

doUpgrade in interface QJournalProtocol

Throws:

IOException

doFinalize

public void doFinalize(String journalId,
 String nameServiceId)
                throws IOException

Specified by:

doFinalize in interface QJournalProtocol

Throws:

IOException

canRollBack

public Boolean canRollBack(String journalId,
 String nameServiceId,
 StorageInfo storage,
 StorageInfo prevStorage,
 int targetLayoutVersion)
                    throws IOException

Specified by:

canRollBack in interface QJournalProtocol

Throws:

IOException

doRollback

public void doRollback(String journalId,
 String nameServiceId)
                throws IOException

Specified by:

doRollback in interface QJournalProtocol

Throws:

IOException

discardSegments

public void discardSegments(String journalId,
 String nameServiceId,
 long startTxId)
                     throws IOException

Description copied from interface: QJournalProtocol

Discard journal segments whose first TxId is greater than or equal to the given txid.

Specified by:

discardSegments in interface QJournalProtocol

Throws:

IOException

getJournalCTime

public Long getJournalCTime(String journalId,
 String nameServiceId)
                     throws IOException

Specified by:

getJournalCTime in interface QJournalProtocol

Throws:

IOException

getEditLogManifestFromJournal

public QJournalProtocolProtos.GetEditLogManifestResponseProto getEditLogManifestFromJournal(String jid,
 String nameServiceId,
 long sinceTxId,
 boolean inProgressOk)
                                                                                     throws IOException

Specified by:

getEditLogManifestFromJournal in interface InterQJournalProtocol

Parameters:

jid - the journal from which to enumerate edits

sinceTxId - the first transaction which the client cares about

inProgressOk - whether or not to check the in-progress edit log segment

Returns:

a list of edit log segments since the given transaction ID.

Throws:

IOException