Package org.apache.zookeeper.server

Class ZKDatabase

java.lang.Object

org.apache.zookeeper.server.ZKDatabase

public class ZKDatabase
extends Object

This class maintains the in memory database of zookeeper server states that includes the sessions, datatree and the committed logs. It is booted up after reading the logs and snapshots from the disk.

Field Summary

Fields

Modifier and Type	Field	Description
static String	COMMIT_LOG_COUNT
int	commitLogCount
protected Queue<Leader.Proposal>	committedLog
protected DataTree	dataTree	make sure on a clear you take care of all these members.
static int	DEFAULT_COMMIT_LOG_COUNT
static double	DEFAULT_SNAPSHOT_SIZE_FACTOR
protected ReentrantReadWriteLock	logLock
protected long	maxCommittedLog
protected long	minCommittedLog
protected ConcurrentHashMap<Long,Integer>	sessionsWithTimeouts
protected FileTxnSnapLog	snapLog
static String	SNAPSHOT_SIZE_FACTOR	Default value is to use snapshot if txnlog size exceeds 1/3 the size of snapshot

Constructor Summary

Constructors

Constructor	Description
ZKDatabase(FileTxnSnapLog snapLog)	the filetxnsnaplog that this zk database maps to.

Method Summary

Modifier and Type	Method	Description
List<ACL>	aclForNode(DataNode n)
void	addCommittedProposal(Request request)	maintains a list of last committedLog or so committed requests.
void	addWatch(String basePath, Watcher watcher, int mode)	Add a watch
boolean	append(Request si)	append to the underlying transaction log
long	calculateTxnLogSizeLimit()
void	clear()	clear the zkdatabase.
void	close()	close this database.
void	commit()	commit to the underlying transaction log
boolean	compareDigest(TxnHeader header, Record txn, TxnDigest digest)
boolean	containsWatcher(String path, Watcher.WatcherType type, Watcher watcher)	Check whether the given watcher exists in datatree
DataTree	createDataTree()
void	deserializeSnapshot(InputArchive ia)	deserialize a snapshot from an input archive
void	deserializeSnapshot(InputArchive ia, CheckedInputStream is)	Deserialize a snapshot that contains FileHeader from an input archive.
void	dumpEphemerals(PrintWriter pwriter)	write a text dump of all the ephemerals in the datatree
long	fastForwardDataBase()	Fast forward the database adding transactions from the committed log into memory.
List<ACL>	getACL(String path, Stat stat)	get acl for a path
int	getAclSize()	get the acl size of the datatree
int	getAllChildrenNumber(String path)
List<String>	getChildren(String path, Stat stat, Watcher watcher)	get children list for this path
Collection<Leader.Proposal>	getCommittedLog()
byte[]	getData(String path, Stat stat, Watcher watcher)	get data and stat for a path
DataTree	getDataTree()	the datatree for this zkdatabase
long	getDataTreeLastProcessedZxid()	get the last processed zxid from a datatree
Map<Long,Set<String>>	getEphemerals()
Set<String>	getEphemerals(long sessionId)	the paths for ephemeral session id
ReentrantReadWriteLock	getLogLock()	Get the lock that controls the committedLog.
long	getmaxCommittedLog()	the committed log for this zk database
long	getminCommittedLog()	the minimum committed transaction log available in memory
DataNode	getNode(String path)	get the datanode for this path
int	getNodeCount()	the node count of the datatree
Iterator<Leader.Proposal>	getProposalsFromTxnLog(long startZxid, long sizeLimit)	Get proposals from txnlog.
long	getSessionCount()
Collection<Long>	getSessions()	return the sessions in the datatree
ConcurrentHashMap<Long,Integer>	getSessionWithTimeOuts()	get sessions with timeouts
int	getTxnCount()	Get the number of txn since last snapshot
long	getTxnSize()	Get the size of txn since last snapshot
void	initConfigInZKDatabase(QuorumVerifier qv)
boolean	isInitialized()	checks to see if the zk database has been initialized or not.
boolean	isSpecialPath(String path)	check if the path is special or not
boolean	isTxnLogSyncEnabled()
void	killSession(long sessionId, long zxid)	kill a given session in the datatree
long	loadDataBase()	load the database from the disk onto memory and also add the transactions to the committedlog in memory.
DataTree.ProcessTxnResult	processTxn(TxnHeader hdr, Record txn, TxnDigest digest)	the process txn on the data and perform digest comparision.
void	removeCnxn(ServerCnxn cnxn)	remove a cnxn from the datatree
boolean	removeWatch(String path, Watcher.WatcherType type, Watcher watcher)	Remove watch from the datatree
void	resetTxnCount()	Reset the number of txn since last rollLog
void	rollLog()	roll the underlying log
void	serializeSnapshot(OutputArchive oa)	serialize the snapshot
void	setlastProcessedZxid(long zxid)	the last processed zxid in the datatree
void	setSnapshotSizeFactor(double snapshotSizeFactor)	Use for unit testing, so we can turn this feature on/off
void	setWatches(long relativeZxid, List<String> dataWatches, List<String> existWatches, List<String> childWatches, List<String> persistentWatches, List<String> persistentRecursiveWatches, Watcher watcher)	set watches on the datatree
Stat	statNode(String path, ServerCnxn serverCnxn)	stat the path
boolean	truncateLog(long zxid)	Truncate the ZKDatabase to the specified zxid

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

dataTree

protected DataTree dataTree

make sure on a clear you take care of all these members.

sessionsWithTimeouts

protected ConcurrentHashMap<Long,Integer> sessionsWithTimeouts

snapLog

protected FileTxnSnapLog snapLog

minCommittedLog

protected long minCommittedLog

maxCommittedLog

protected long maxCommittedLog

SNAPSHOT_SIZE_FACTOR

public static final String SNAPSHOT_SIZE_FACTOR

Default value is to use snapshot if txnlog size exceeds 1/3 the size of snapshot

See Also:

Constant Field Values

DEFAULT_SNAPSHOT_SIZE_FACTOR

public static final double DEFAULT_SNAPSHOT_SIZE_FACTOR

See Also:

Constant Field Values

COMMIT_LOG_COUNT

public static final String COMMIT_LOG_COUNT

See Also:

Constant Field Values

DEFAULT_COMMIT_LOG_COUNT

public static final int DEFAULT_COMMIT_LOG_COUNT

See Also:

Constant Field Values

commitLogCount

public int commitLogCount

committedLog

protected Queue<Leader.Proposal> committedLog

logLock

protected ReentrantReadWriteLock logLock

Constructor Detail

ZKDatabase

public ZKDatabase(FileTxnSnapLog snapLog)

the filetxnsnaplog that this zk database maps to. There is a one to one relationship between a filetxnsnaplog and zkdatabase.

Parameters:

snapLog - the FileTxnSnapLog mapping this zkdatabase

Method Detail

isInitialized

public boolean isInitialized()

checks to see if the zk database has been initialized or not.

Returns:

true if zk database is initialized and false if not

clear

public void clear()

clear the zkdatabase. Note to developers - be careful to see that the clear method does clear out all the data structures in zkdatabase.

getDataTree

public DataTree getDataTree()

the datatree for this zkdatabase

Returns:

the datatree for this zkdatabase

getmaxCommittedLog

public long getmaxCommittedLog()

the committed log for this zk database

Returns:

the committed log for this zkdatabase

getminCommittedLog

public long getminCommittedLog()

the minimum committed transaction log available in memory

Returns:

the minimum committed transaction log available in memory

getLogLock

public ReentrantReadWriteLock getLogLock()

Get the lock that controls the committedLog. If you want to get the pointer to the committedLog, you need to use this lock to acquire a read lock before calling getCommittedLog()

Returns:

the lock that controls the committed log

getCommittedLog

public Collection<Leader.Proposal> getCommittedLog()

getDataTreeLastProcessedZxid

public long getDataTreeLastProcessedZxid()

get the last processed zxid from a datatree

Returns:

the last processed zxid of a datatree

getSessions

public Collection<Long> getSessions()

return the sessions in the datatree

Returns:

the data tree sessions

getSessionCount

public long getSessionCount()

Returns:

number of (global) sessions

getSessionWithTimeOuts

public ConcurrentHashMap<Long,Integer> getSessionWithTimeOuts()

get sessions with timeouts

Returns:

the hashmap of sessions with timeouts

loadDataBase

public long loadDataBase()
                  throws IOException

load the database from the disk onto memory and also add the transactions to the committedlog in memory.

Returns:

the last valid zxid on disk

Throws:

IOException

fastForwardDataBase

public long fastForwardDataBase()
                         throws IOException

Fast forward the database adding transactions from the committed log into memory.

Returns:

the last valid zxid.

Throws:

IOException

addCommittedProposal

public void addCommittedProposal(Request request)

maintains a list of last committedLog or so committed requests. This is used for fast follower synchronization.

Parameters:

request - committed request

isTxnLogSyncEnabled

public boolean isTxnLogSyncEnabled()

calculateTxnLogSizeLimit

public long calculateTxnLogSizeLimit()

getProposalsFromTxnLog

public Iterator<Leader.Proposal> getProposalsFromTxnLog(long startZxid,
                                                        long sizeLimit)

Get proposals from txnlog. Only packet part of proposal is populated.

Parameters:

startZxid - the starting zxid of the proposal

sizeLimit - maximum on-disk size of txnlog to fetch 0 is unlimited, negative value means disable.

Returns:

list of proposal (request part of each proposal is null)

aclForNode

public List<ACL> aclForNode(DataNode n)

removeCnxn

public void removeCnxn(ServerCnxn cnxn)

remove a cnxn from the datatree

Parameters:

cnxn - the cnxn to remove from the datatree

killSession

public void killSession(long sessionId,
                        long zxid)

kill a given session in the datatree

Parameters:

sessionId - the session id to be killed

zxid - the zxid of kill session transaction

dumpEphemerals

public void dumpEphemerals(PrintWriter pwriter)

write a text dump of all the ephemerals in the datatree

Parameters:

pwriter - the output to write to

getEphemerals

public Map<Long,Set<String>> getEphemerals()

getNodeCount

public int getNodeCount()

the node count of the datatree

Returns:

the node count of datatree

getEphemerals

public Set<String> getEphemerals(long sessionId)

the paths for ephemeral session id

Parameters:

sessionId - the session id for which paths match to

Returns:

the paths for a session id

setlastProcessedZxid

public void setlastProcessedZxid(long zxid)

the last processed zxid in the datatree

Parameters:

zxid - the last processed zxid in the datatree

processTxn

public DataTree.ProcessTxnResult processTxn(TxnHeader hdr,
                                            Record txn,
                                            TxnDigest digest)

the process txn on the data and perform digest comparision.

Parameters:

hdr - the txnheader for the txn

txn - the transaction that needs to be processed

digest - the expected digest. A null value would skip the check

Returns:

the result of processing the transaction on this datatree/zkdatabase

statNode

public Stat statNode(String path,
                     ServerCnxn serverCnxn)
              throws KeeperException.NoNodeException

stat the path

Parameters:

path - the path for which stat is to be done

serverCnxn - the servercnxn attached to this request

Returns:

the stat of this node

Throws:

KeeperException.NoNodeException

getNode

public DataNode getNode(String path)

get the datanode for this path

Parameters:

path - the path to lookup

Returns:

the datanode for getting the path

getData

public byte[] getData(String path,
                      Stat stat,
                      Watcher watcher)
               throws KeeperException.NoNodeException

get data and stat for a path

Parameters:

path - the path being queried

stat - the stat for this path

watcher - the watcher function

Throws:

KeeperException.NoNodeException

setWatches

public void setWatches(long relativeZxid,
                       List<String> dataWatches,
                       List<String> existWatches,
                       List<String> childWatches,
                       List<String> persistentWatches,
                       List<String> persistentRecursiveWatches,
                       Watcher watcher)

set watches on the datatree

Parameters:

relativeZxid - the relative zxid that client has seen

dataWatches - the data watches the client wants to reset

existWatches - the exists watches the client wants to reset

childWatches - the child watches the client wants to reset

persistentWatches - the persistent watches the client wants to reset

persistentRecursiveWatches - the persistent recursive watches the client wants to reset

watcher - the watcher function

addWatch

public void addWatch(String basePath,
                     Watcher watcher,
                     int mode)

Add a watch

Parameters:

basePath - watch base

watcher - the watcher

mode - a mode from ZooDefs.AddWatchModes

getACL

public List<ACL> getACL(String path,
                        Stat stat)
                 throws KeeperException.NoNodeException

get acl for a path

Parameters:

path - the path to query for acl

stat - the stat for the node

Returns:

the acl list for this path

Throws:

KeeperException.NoNodeException

getChildren

public List<String> getChildren(String path,
                                Stat stat,
                                Watcher watcher)
                         throws KeeperException.NoNodeException

get children list for this path

Parameters:

path - the path of the node

stat - the stat of the node

watcher - the watcher function for this path

Returns:

the list of children for this path

Throws:

KeeperException.NoNodeException

getAllChildrenNumber

public int getAllChildrenNumber(String path)
                         throws KeeperException.NoNodeException

Throws:

KeeperException.NoNodeException

isSpecialPath

public boolean isSpecialPath(String path)

check if the path is special or not

Parameters:

path - the input path

Returns:

true if path is special and false if not

getAclSize

public int getAclSize()

get the acl size of the datatree

Returns:

the acl size of the datatree

truncateLog

public boolean truncateLog(long zxid)
                    throws IOException

Truncate the ZKDatabase to the specified zxid

Parameters:

zxid - the zxid to truncate zk database to

Returns:

true if the truncate is successful and false if not

Throws:

IOException

deserializeSnapshot

public void deserializeSnapshot(InputArchive ia)
                         throws IOException

deserialize a snapshot from an input archive

Parameters:

ia - the input archive you want to deserialize from

Throws:

IOException

deserializeSnapshot

public void deserializeSnapshot(InputArchive ia,
                                CheckedInputStream is)
                         throws IOException

Deserialize a snapshot that contains FileHeader from an input archive. It is used by the admin restore command.

Parameters:

ia - the input archive to deserialize from

is - the CheckInputStream to check integrity

Throws:

IOException

serializeSnapshot

public void serializeSnapshot(OutputArchive oa)
                       throws IOException,
                              InterruptedException

serialize the snapshot

Parameters:

oa - the output archive to which the snapshot needs to be serialized

Throws:

IOException

InterruptedException

append

public boolean append(Request si)
               throws IOException

append to the underlying transaction log

Parameters:

si - the request to append

Returns:

true if the append was succesfull and false if not

Throws:

IOException

rollLog

public void rollLog()
             throws IOException

roll the underlying log

Throws:

IOException

commit

public void commit()
            throws IOException

commit to the underlying transaction log

Throws:

IOException

close

public void close()
           throws IOException

close this database. free the resources

Throws:

IOException

initConfigInZKDatabase

public void initConfigInZKDatabase(QuorumVerifier qv)

setSnapshotSizeFactor

public void setSnapshotSizeFactor(double snapshotSizeFactor)

Use for unit testing, so we can turn this feature on/off

Parameters:

snapshotSizeFactor - Set to minus value to turn this off.

containsWatcher

public boolean containsWatcher(String path,
                               Watcher.WatcherType type,
                               Watcher watcher)

Check whether the given watcher exists in datatree

Parameters:

path - node to check watcher existence

type - type of watcher

watcher - watcher function

removeWatch

public boolean removeWatch(String path,
                           Watcher.WatcherType type,
                           Watcher watcher)

Remove watch from the datatree

Parameters:

path - node to remove watches from

type - type of watcher to remove

watcher - watcher function to remove

createDataTree

public DataTree createDataTree()

resetTxnCount

public void resetTxnCount()

Reset the number of txn since last rollLog

getTxnCount

public int getTxnCount()

Get the number of txn since last snapshot

getTxnSize

public long getTxnSize()

Get the size of txn since last snapshot

compareDigest

public boolean compareDigest(TxnHeader header,
                             Record txn,
                             TxnDigest digest)