|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Objectjava.lang.Thread
org.tranche.commons.DebuggableThread
org.tranche.flatfile.HashSpanFixingThread
public class HashSpanFixingThread
"Heals" a server's data and meta-data based on the configured hash span.
| Nested Class Summary |
|---|
| Nested classes/interfaces inherited from class java.lang.Thread |
|---|
java.lang.Thread.State, java.lang.Thread.UncaughtExceptionHandler |
| Field Summary | |
|---|---|
static byte |
ACTIVITY_BALANCING
Static references to the activities this thread will undertake |
static byte |
ACTIVITY_DELETING
Static references to the activities this thread will undertake |
static byte |
ACTIVITY_DOWNLOADING
Static references to the activities this thread will undertake |
static byte |
ACTIVITY_HEALING
Static references to the activities this thread will undertake |
static byte |
ACTIVITY_NOTHING
Static references to the activities this thread will undertake |
protected long |
dataCopied
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread. |
protected long |
dataDeleted
Get the total number of data chunks deleted. |
protected long |
dataLocalChunkThrewException
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread. |
protected long |
dataNotRepaired
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread. |
protected long |
dataRepaired
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread. |
protected long |
dataSkipped
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread. |
static int |
DEFAULT_BATCH_SIZE_FOR_BLOCKS_TO_BALANCE
The number of DataBlocks to balance between DataDirectoryConfiguration objects before going on to next task. |
static int |
DEFAULT_BATCH_SIZE_FOR_CHUNKS_TO_DELETE
The number of chunks to check whether can be deleted, and to delete if can, before going on to next task. |
static int |
DEFAULT_BATCH_SIZE_FOR_CHUNKS_TO_DOWNLOAD
The number of chunks to download before going on to next task. |
static int |
DEFAULT_BATCH_SIZE_FOR_CHUNKS_TO_HEAL
The number of chunks to check to see whether needs replaced, and to replace if need be, before going on to next task. |
static long |
DEFAULT_MILLIS_PAUSE_BETWEEN_OPERATIONS
Default value for number of milliseconds to pause between operations in healing thread. |
protected long |
iterationCount
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread. |
protected long |
localDeleteCompleteIterations
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread. |
protected long |
localDeleteIteration
Used to do local deletion one cheap batch at a time. |
protected long |
localRepairCompleteIterations
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread. |
protected long |
localVerifyIteration
Used to do local verification one cheap batch at a time. |
protected long |
metaDataCopied
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread. |
protected long |
metaDataDeleted
Get the total number of meta data chunks deleted. |
protected long |
metaDataNotRepaired
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread. |
protected long |
metaDataRepaired
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread. |
protected long |
metaDataSkipped
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread. |
protected long |
metaLocalDataChunkThreadException
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread. |
protected java.lang.String |
serverBeingIndexed
The current server being checked by this thread to work on this server. |
| Fields inherited from class java.lang.Thread |
|---|
MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY |
| Constructor Summary | |
|---|---|
HashSpanFixingThread(FlatFileTrancheServer ffts)
|
|
| Method Summary | |
|---|---|
boolean |
addServerHostToUse(java.lang.String host)
Add a server to use. |
void |
clearServerHostsToUse()
Clear out all servers added to use. |
void |
close()
Close the thread down. |
int |
getBatchSizeForChunksToDelete(Configuration conf)
The number of chunks to inspect for potential deletion at a time. |
int |
getBatchSizeForChunksToDownload(Configuration conf)
The number of chunks to inspect for potential download at a time. |
int |
getBatchSizeForChunksToHeal(Configuration conf)
The number of chunks to inspect for repair work at a time. |
int |
getBatchSizeForDataBlocksToBalance(Configuration conf)
Get the number of DataBlock objects to balance between DataDirectoryConfiguration objects to check and balance before going on to the next task. |
java.lang.String |
getConnectedHost(BigHash h)
Get a connected verifyHost, preferring those that include in hash span, but defaulting to random server. |
java.lang.String |
getCurrentActivity()
Find out what the server is doing. |
long |
getCurrentPerformanceScore()
Current score for performance of the healing thread. |
java.lang.String |
getCurrentThreadState()
The state of the healing thread: |
long |
getDeleteDataIndex()
|
long |
getDeleteMetaDataIndex()
|
long |
getDownloadDataIndex()
|
long |
getDownloadMetaDataIndex()
|
java.lang.String |
getDownloadServer()
|
java.lang.String |
getEstimatedPercentageTimeSpentBalancing()
Builds a string suitable for human viewing with percentage of the time of this thread "balancing" |
java.lang.String |
getEstimatedPercentageTimeSpentDeleting()
Builds a string suitable for human viewing with percentage of the time of this thread doing "deleting" (looking for chunks to delete that don't belong on this server with sufficient replicates). |
java.lang.String |
getEstimatedPercentageTimeSpentDoingNothing()
Builds a string suitable for human viewing with percentage of the time of this thread doing "nothing" (loading or household cleaning). |
java.lang.String |
getEstimatedPercentageTimeSpentDownloading()
Builds a string suitable for human viewing with percentage of the time of this thread doing "download" (looking for new chunks to download that belong to this server's hash span). |
java.lang.String |
getEstimatedPercentageTimeSpentHealing()
Builds a string suitable for human viewing with percentage of the time of this thread doing "healing" (looking for corrupt chunks to fix). |
java.lang.String[] |
getHostsToReceiveChunk(BigHash h)
Returns all core servers that should receive chunk. |
java.lang.String[] |
getHostsToUse()
Get servers. |
long |
getPauseBetweenOperations(Configuration conf)
Get pause to relieve resources based on configuration options. |
long |
getRepairDataIndex()
|
long |
getRepairMetaDataIndex()
|
double |
getRequiredPercentageDifferenceToBalanceDataDirectories(Configuration conf)
The required difference in percentage between the data directories with least and most available space before transfering. |
double |
getRequiredPercentageForMostUsedDataDirectoryToBalance(Configuration conf)
The minimum percentage of its available space used by a data directory before it will be a candidate for balancing. |
int |
getRequiredReplicationsToDelete()
Deprecated. This may be used by testing, but should not be used in production. Instead, set the appropriate value in the server's Configuration object. |
int |
getRequiredRepsInHashSpanToDelete(Configuration conf)
This represents a parameter that is set in FlatFileTrancheServer's Configuration, and can be remotely changed. |
long |
getTimeSpentBalancing()
Get time, in milliseconds, spent balancing the data directories. |
long |
getTimeSpentDeleting()
Get time, in milliseconds, spent deleting inappropriate chunks on server. |
long |
getTimeSpentDoingNothing()
Get time, in milliseconds, spent in household cleaning or waiting for startup. |
long |
getTimeSpentDownloading()
Get time, in milliseconds, spent downloading new data that belongs on this server from other servers. |
long |
getTimeSpentHealing()
Get time, in milliseconds, spent healing local data chunks. |
static long |
getTimeToSleepIfNotAllowedToRun()
Time in milliseconds to sleep when not allowed to run before retrying. |
int |
getTotalRequiredRepsToDelete(Configuration conf)
This represents a parameter that is set in FlatFileTrancheServer's Configuration, and can be remotely changed. |
boolean |
isAllowedToBalance(Configuration conf)
This represents a parameter that is set in FlatFileTrancheServer's Configuration, and can be remotely changed. |
boolean |
isAllowedToDelete(Configuration conf)
This represents a parameter that is set in FlatFileTrancheServer's Configuration, and can be remotely changed. |
boolean |
isAllowedToRun(Configuration conf)
This represents a parameter that is set in FlatFileTrancheServer's Configuration, and can be remotely changed. |
boolean |
removeServerHostToUse(java.lang.String host)
If specified alternative servers to use, remove one. |
void |
run()
|
void |
setRequiredReplicationsToDelete(int num,
java.security.cert.X509Certificate cert,
java.security.PrivateKey key)
Deprecated. This may be used by testing, but should not be used in production. Instead, set the appropriate value in the server's Configuration object. |
void |
setTestingConcurrentReplication(boolean isTesting)
Deprecated. Doesn't do anything. Logic to handle data and meta data chunks removed in favor of cleaner logic |
static void |
setTimeToSleepIfNotAllowedToRun(long time)
Time in milliseconds to sleep when not allowed to run before retrying. |
void |
waitForIteration()
Blocks until next iteration. |
| Methods inherited from class org.tranche.commons.DebuggableThread |
|---|
debugErr, debugErr, debugOut, isDebug, setDebug |
| Methods inherited from class java.lang.Thread |
|---|
activeCount, checkAccess, countStackFrames, currentThread, destroy, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, stop, suspend, toString, yield |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
| Field Detail |
|---|
public static final byte ACTIVITY_NOTHING
Static references to the activities this thread will undertake
public static final byte ACTIVITY_HEALING
Static references to the activities this thread will undertake
public static final byte ACTIVITY_DELETING
Static references to the activities this thread will undertake
public static final byte ACTIVITY_DOWNLOADING
Static references to the activities this thread will undertake
public static final byte ACTIVITY_BALANCING
Static references to the activities this thread will undertake
public static final int DEFAULT_BATCH_SIZE_FOR_CHUNKS_TO_DOWNLOAD
The number of chunks to download before going on to next task.
Keep this low! If configuration should change, want to get new information ASAP.
public static final int DEFAULT_BATCH_SIZE_FOR_CHUNKS_TO_HEAL
The number of chunks to check to see whether needs replaced, and to replace if need be, before going on to next task.
Keep this low! If configuration should change, want to get new information ASAP.
public static final int DEFAULT_BATCH_SIZE_FOR_CHUNKS_TO_DELETE
The number of chunks to check whether can be deleted, and to delete if can, before going on to next task.
Keep this low! If configuration should change, want to get new information ASAP.
public static final int DEFAULT_BATCH_SIZE_FOR_BLOCKS_TO_BALANCE
The number of DataBlocks to balance between DataDirectoryConfiguration objects before going on to next task.
Keep this low! If configuration should change, want to get new information ASAP.
public static final long DEFAULT_MILLIS_PAUSE_BETWEEN_OPERATIONS
Default value for number of milliseconds to pause between operations in healing thread.
A low value allows thread to operate quickly, a high value slows it down to free up resources.
protected long dataCopied
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread.
protected long dataSkipped
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread.
protected long metaDataCopied
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread.
protected long metaDataSkipped
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread.
protected long dataRepaired
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread.
protected long metaDataRepaired
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread.
protected long dataNotRepaired
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread.
protected long metaDataNotRepaired
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread.
protected long dataLocalChunkThrewException
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread.
protected long metaLocalDataChunkThreadException
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread.
protected long localRepairCompleteIterations
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread.
protected long localDeleteCompleteIterations
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread.
protected long iterationCount
One of many status variables for the FlatFileTrancheServer to use to report progress of this thread.
protected long localVerifyIteration
Used to do local verification one cheap batch at a time.
protected long localDeleteIteration
Used to do local deletion one cheap batch at a time.
protected long dataDeleted
Get the total number of data chunks deleted.
protected long metaDataDeleted
Get the total number of meta data chunks deleted.
protected java.lang.String serverBeingIndexed
The current server being checked by this thread to work on this server.
| Constructor Detail |
|---|
public HashSpanFixingThread(FlatFileTrancheServer ffts)
ffts - | Method Detail |
|---|
public int getRequiredReplicationsToDelete()
Get the number of replications on other servers before chunk deleted on this server. A policy.
public void setRequiredReplicationsToDelete(int num,
java.security.cert.X509Certificate cert,
java.security.PrivateKey key)
throws java.lang.Exception
Set the number of required replications of chunk on other servers before deletion. Default is 3.
For example, if the required replications were 3, and 3+ servers have the chunk and this server has the chunk out of its hash span, the server will delete it.
However, if the required reps were 5, this server won't delete the chunk even though it is out of its hash span.
Note this operation blocks until the current iteration is complete.
num - cert - key -
java.lang.Exception
public void waitForIteration()
throws java.lang.InterruptedException
Blocks until next iteration. Mostly useful for tests or to determine whether affects have kicked in yet.
java.lang.InterruptedExceptionpublic void run()
run in interface java.lang.Runnablerun in class java.lang.Threadpublic java.lang.String[] getHostsToReceiveChunk(BigHash h)
Returns all core servers that should receive chunk.
Server should receive chunk if:
h -
public java.lang.String getConnectedHost(BigHash h)
Get a connected verifyHost, preferring those that include in hash span, but defaulting to random server. Uses following precedence:
h -
public java.lang.String[] getHostsToUse()
throws java.lang.Exception
Get servers. Might be core or test depending on environment.
java.lang.Exceptionpublic void setTestingConcurrentReplication(boolean isTesting)
Does nothing. Deprecated. Tests associated with 'concurrent' replication (injecting data and meta data chunks with same hash at same time) should be removed.
isTesting - boolean inputpublic boolean isAllowedToRun(Configuration conf)
This represents a parameter that is set in FlatFileTrancheServer's Configuration, and can be remotely changed. See param and return fields of this Javadoc for more information.
conf - The most recent Configuration from FlatFileTrancheServer.
public boolean isAllowedToDelete(Configuration conf)
This represents a parameter that is set in FlatFileTrancheServer's Configuration, and can be remotely changed. See param and return fields of this Javadoc for more information.
conf - The most recent Configuration from FlatFileTrancheServer.
public boolean isAllowedToBalance(Configuration conf)
This represents a parameter that is set in FlatFileTrancheServer's Configuration, and can be remotely changed. See param and return fields of this Javadoc for more information.
conf - The most recent Configuration from FlatFileTrancheServer.
public int getTotalRequiredRepsToDelete(Configuration conf)
This represents a parameter that is set in FlatFileTrancheServer's Configuration, and can be remotely changed. See param and return fields of this Javadoc for more information.
conf - The most recent Configuration from FlatFileTrancheServer.
public int getRequiredRepsInHashSpanToDelete(Configuration conf)
This represents a parameter that is set in FlatFileTrancheServer's Configuration, and can be remotely changed. See param and return fields of this Javadoc for more information.
conf - The most recent Configuration from FlatFileTrancheServer.
public double getRequiredPercentageDifferenceToBalanceDataDirectories(Configuration conf)
The required difference in percentage between the data directories with least and most available space before transfering.
This only matters if healing thread is set to auto balance.
Note that verifyHost the calculation is determined:
conf -
public double getRequiredPercentageForMostUsedDataDirectoryToBalance(Configuration conf)
The minimum percentage of its available space used by a data directory before it will be a candidate for balancing. This sets a threshold before data will be moved away toward other data blocks.
This only matters if healing thread is set to auto balance.
Note that this does not guarentee that data will be moved to other data directories. Three additional criteria must be met:
Note should always be a positive value less than 1.0.
conf -
public int getBatchSizeForChunksToDownload(Configuration conf)
The number of chunks to inspect for potential download at a time.
The following three items compete for single-thread's time:
By default, they have the same batch sizes. However, they might be adjusted to emphasize certain activities.
conf -
public int getBatchSizeForChunksToDelete(Configuration conf)
The number of chunks to inspect for potential deletion at a time.
The following three items compete for single-thread's time:
By default, they have the same batch sizes. However, they might be adjusted to emphasize certain activities.
conf -
public int getBatchSizeForChunksToHeal(Configuration conf)
The number of chunks to inspect for repair work at a time.
The following three items compete for single-thread's time:
By default, they have the same batch sizes. However, they might be adjusted to emphasize certain activities.
conf -
public int getBatchSizeForDataBlocksToBalance(Configuration conf)
Get the number of DataBlock objects to balance between DataDirectoryConfiguration objects to check and balance before going on to the next task.
conf -
public long getPauseBetweenOperations(Configuration conf)
throws java.lang.Exception
Get pause to relieve resources based on configuration options.
conf -
java.lang.Exceptionpublic boolean addServerHostToUse(java.lang.String host)
Add a server to use. If any servers are add, the healing thread will stop using core server and will use these instead.
To use core servers again, you will have to clear the added servers to use. See clearServersToUse().
verifyHost -
public boolean removeServerHostToUse(java.lang.String host)
If specified alternative servers to use, remove one.
verifyHost -
public void clearServerHostsToUse()
Clear out all servers added to use. This will cause the HashSpanFixingThread to use core servers again until more servers are added.
public java.lang.String getCurrentActivity()
Find out what the server is doing.
public long getTimeSpentDoingNothing()
Get time, in milliseconds, spent in household cleaning or waiting for startup.
public long getTimeSpentHealing()
Get time, in milliseconds, spent healing local data chunks.
public long getTimeSpentDeleting()
Get time, in milliseconds, spent deleting inappropriate chunks on server.
public long getTimeSpentDownloading()
Get time, in milliseconds, spent downloading new data that belongs on this server from other servers.
public long getTimeSpentBalancing()
Get time, in milliseconds, spent balancing the data directories.
public java.lang.String getEstimatedPercentageTimeSpentDoingNothing()
Builds a string suitable for human viewing with percentage of the time of this thread doing "nothing" (loading or household cleaning).
public java.lang.String getEstimatedPercentageTimeSpentHealing()
Builds a string suitable for human viewing with percentage of the time of this thread doing "healing" (looking for corrupt chunks to fix).
public java.lang.String getEstimatedPercentageTimeSpentDownloading()
Builds a string suitable for human viewing with percentage of the time of this thread doing "download" (looking for new chunks to download that belong to this server's hash span).
public java.lang.String getEstimatedPercentageTimeSpentDeleting()
Builds a string suitable for human viewing with percentage of the time of this thread doing "deleting" (looking for chunks to delete that don't belong on this server with sufficient replicates).
public java.lang.String getEstimatedPercentageTimeSpentBalancing()
Builds a string suitable for human viewing with percentage of the time of this thread "balancing"
public long getCurrentPerformanceScore()
Current score for performance of the healing thread.
There are several internal heuristics for determining the scores, but basically scores are incremented when activities are productive. In some contexts (e.g., low disk space), some activities are more highly weighted (e.g., deleting unnecessary space).
public java.lang.String getCurrentThreadState()
The state of the healing thread:
public long getDeleteDataIndex()
throws java.io.IOException
java.io.IOException
public long getDeleteMetaDataIndex()
throws java.io.IOException
java.io.IOException
public long getDownloadDataIndex()
throws java.io.IOException
java.io.IOException
public long getDownloadMetaDataIndex()
throws java.io.IOException
java.io.IOException
public long getRepairDataIndex()
throws java.io.IOException
java.io.IOException
public long getRepairMetaDataIndex()
throws java.io.IOException
java.io.IOException
public java.lang.String getDownloadServer()
throws java.io.IOException
java.io.IOExceptionpublic void close()
Close the thread down. Frees associated resources.
public static long getTimeToSleepIfNotAllowedToRun()
Time in milliseconds to sleep when not allowed to run before retrying.
public static void setTimeToSleepIfNotAllowedToRun(long time)
Time in milliseconds to sleep when not allowed to run before retrying.
time -
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||