com.timeindexing.io
Class AbstractFileIO

java.lang.Object
  extended bycom.timeindexing.io.AbstractIndexIO
      extended bycom.timeindexing.io.AbstractFileIO
All Implemented Interfaces:
IndexFileInteractor, IndexFileReader, IndexFileWriter, IndexInteractor, java.lang.Runnable
Direct Known Subclasses:
ExternalIndexIO, InlineIndexIO

public abstract class AbstractFileIO
extends AbstractIndexIO
implements IndexFileInteractor

Has code for indexes that are file-based.


Constructor Summary
AbstractFileIO()
           
 
Method Summary
 long addItem(ManagedIndexItem itemM)
          Add an item.
protected abstract  long alignForData()
          Align the index for an append of the Data This is done differently for each type of index.
protected  long alignForIndexItem()
          Align the index for an append of an IndexItem.
 boolean awaitWork()
           
protected abstract  long bufferedDataWrite(java.nio.ByteBuffer buffer)
          Write a buffer of data.
protected abstract  long bufferedIndexWrite(java.nio.ByteBuffer buffer)
          Write a buffer of index items.
protected  long bufferedWrite(java.nio.ByteBuffer buffer, java.nio.channels.FileChannel channel, ByteBufferRing ring)
          Write a buffer of data.
abstract  long calculateAppendPosition()
          Calculate the append position from the last item of the index.
 DataHolderObject convertDataReference(DataReference dataReference)
          Read some data, given a DataReference and return it as a DataHolderObject.
 void drainWriteQueue()
          This drains the write request queue by processing all the WriteRequests.
protected  long flushBuffer(java.nio.channels.FileChannel channel, java.nio.ByteBuffer flushBuffer, ByteBufferRing ring)
          Actually flush the buffer out.
 long getAppendPosition()
          Get the append position
abstract  ManagedIndexItem getItem(long position, boolean doLoadData)
          Get the item at index position Position.
 ManagedIndexItem getItem(Position position, boolean doLoadData)
          Get the item at index position Position.
 java.nio.channels.FileLock getWriteLock()
          Get a write-lock on this index.
 boolean gotoAppendPosition()
          Goto the append position
 boolean gotoFirstPosition()
          Goto the first position
 boolean isWriteLocked()
          Has the Index been write-locked.
 long loadIndex(LoadStyle loadStyle)
          Load the index
protected abstract  java.nio.ByteBuffer memoryMapData(long offset, long size)
          Memory map some data from a channel.
protected abstract  long processData(java.nio.ByteBuffer buffer)
          Processing of the data.
protected abstract  long processIndexItem(java.nio.ByteBuffer buffer)
          Processing of the idnex item.
 java.nio.ByteBuffer readData(DataReference ref)
          Read some data, given a DataReference.
 java.nio.ByteBuffer readData(long offset, long size)
          Read some data, given an offset and a size.
 java.nio.ByteBuffer readData(Offset offset, long size)
          Read some data, given an offset and a size.
protected abstract  long readDataIntoBuffer(java.nio.ByteBuffer buffer, long size)
          Actually read in the data.
 long readHeader(byte headerType)
          Read an index header from the header stream.
 java.nio.ByteBuffer readIndexItem(long offset)
          Read an IndexItem given an offset.
 java.nio.ByteBuffer readIndexItem(Offset offset, long size)
          Read an IndexItem given an offset.
 ManagedIndexItem readItem(long startOffset, boolean withData)
          Read the contents of the item It assumes the index file is alreayd open for writing.
 ManagedIndexItem readItem(Offset offset, boolean withData)
          Read the contents of the item It assumes the index file is alreayd open for writing.
protected  DataAbstraction readNormalData(long offset, long size, boolean withData)
          Read some data, from a specified offset for a number of bytes.
protected  IndexReferenceDataHolder readReferenceData(long offset, long size)
          Read a reference, from a specified offset for a number of bytes.
 boolean releaseWriteLock()
          Release a FileLock.
 void requestReadWork(Position position, boolean doLoadData)
          Add some work to the read queue.
 void requestWriteWork(java.nio.channels.FileChannel channel, java.nio.ByteBuffer flushBuffer, ByteBufferRing ring)
          Add some work to the write queue.
 void run()
          The Thread run method.
protected abstract  boolean seekToData(long position)
          Seek to a certain position in the data file.
protected  boolean seekToData(Offset offset)
          Seek to a certain position.
protected abstract  boolean seekToIndex(long position)
          Seek to a certain position.
protected  boolean seekToIndex(Offset offset)
          Seek to a certain position.
 boolean setAppendPosition()
          Set the append position from the indexChannelPosition.
protected  IndexFileInteractor setItemSize(int itemSize)
          Set the index item size.
 boolean skipData(long offset, long size)
          Skip over some data, given an offset and a size.
 boolean timeOut(long timeout)
          Wait for the timeout to go off.
 long writeFromWorkQueue()
          Write the contents of the ffirst ByteBuffer in the work queue to a FileChannel.
 long writeHeader(byte headerType)
          Write the contents of the header out It assumes the index file is alreayd open for writing.
 long writeItem(ManagedIndexItem itemM)
          Write the contents of the item It assumes the index file is alreayd open for writing.
 long writeNormal(ManagedIndexItem itemM)
          Write the contents of the item with normal data It assumes the index file is alreayd open for writing.
 long writeReference(ManagedIndexItem itemM)
          Write the contents of the item with a reference.
 
Methods inherited from class com.timeindexing.io.AbstractIndexIO
getIndex, getThread, initThread, startThread, stopThread
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface com.timeindexing.io.IndexInteractor
getIndex
 
Methods inherited from interface com.timeindexing.io.IndexFileReader
close, open, readMetaData
 
Methods inherited from interface com.timeindexing.io.IndexFileWriter
close, create, flush, open
 

Constructor Detail

AbstractFileIO

public AbstractFileIO()
Method Detail

readHeader

public long readHeader(byte headerType)
                throws java.io.IOException,
                       IndexOpenException
Read an index header from the header stream.

Specified by:
readHeader in interface IndexFileReader
Parameters:
headerType - the type of header, e.g FileType.INLINE_INDEX or FileType.EXTERNAL_INDEX
Throws:
java.io.IOException
IndexOpenException

writeHeader

public long writeHeader(byte headerType)
                 throws java.io.IOException
Write the contents of the header out It assumes the index file is alreayd open for writing.

Specified by:
writeHeader in interface IndexFileWriter
Parameters:
headerType - the type of header, e.g FileType.INLINE_INDEX or FileType.EXTERNAL_INDEX
Throws:
java.io.IOException

addItem

public long addItem(ManagedIndexItem itemM)
             throws java.io.IOException
Add an item.

Specified by:
addItem in interface IndexInteractor
Throws:
java.io.IOException

writeItem

public long writeItem(ManagedIndexItem itemM)
               throws java.io.IOException
Write the contents of the item It assumes the index file is alreayd open for writing.

Specified by:
writeItem in interface IndexFileWriter
Throws:
java.io.IOException

writeNormal

public long writeNormal(ManagedIndexItem itemM)
                 throws java.io.IOException
Write the contents of the item with normal data It assumes the index file is alreayd open for writing.

Throws:
java.io.IOException

writeReference

public long writeReference(ManagedIndexItem itemM)
                    throws java.io.IOException
Write the contents of the item with a reference. It assumes the index file is alreayd open for writing.

Throws:
java.io.IOException

alignForIndexItem

protected long alignForIndexItem()
                          throws java.io.IOException
Align the index for an append of an IndexItem.

Throws:
java.io.IOException

alignForData

protected abstract long alignForData()
                              throws java.io.IOException
Align the index for an append of the Data This is done differently for each type of index.

Throws:
java.io.IOException

processIndexItem

protected abstract long processIndexItem(java.nio.ByteBuffer buffer)
                                  throws java.io.IOException
Processing of the idnex item.

Throws:
java.io.IOException

processData

protected abstract long processData(java.nio.ByteBuffer buffer)
                             throws java.io.IOException
Processing of the data.

Throws:
java.io.IOException

bufferedIndexWrite

protected abstract long bufferedIndexWrite(java.nio.ByteBuffer buffer)
                                    throws java.io.IOException
Write a buffer of index items.

Throws:
java.io.IOException

bufferedDataWrite

protected abstract long bufferedDataWrite(java.nio.ByteBuffer buffer)
                                   throws java.io.IOException
Write a buffer of data.

Throws:
java.io.IOException

bufferedWrite

protected long bufferedWrite(java.nio.ByteBuffer buffer,
                             java.nio.channels.FileChannel channel,
                             ByteBufferRing ring)
                      throws java.io.IOException
Write a buffer of data. This flushes out large buffers a slice at a time.

Throws:
java.io.IOException

flushBuffer

protected long flushBuffer(java.nio.channels.FileChannel channel,
                           java.nio.ByteBuffer flushBuffer,
                           ByteBufferRing ring)
                    throws java.io.IOException
Actually flush the buffer out. Returns how man bytes were written.

Throws:
java.io.IOException

getItem

public ManagedIndexItem getItem(Position position,
                                boolean doLoadData)
                         throws java.io.IOException
Get the item at index position Position.

Specified by:
getItem in interface IndexInteractor
Parameters:
position - the position of the index item to get
doLoadData - read the data for this IndexItem if withData is true, the data needs to be read at a later time, otherwise
Throws:
java.io.IOException

getItem

public abstract ManagedIndexItem getItem(long position,
                                         boolean doLoadData)
                                  throws java.io.IOException
Get the item at index position Position.

Specified by:
getItem in interface IndexFileReader
Parameters:
position - the position of the index item to get
doLoadData - read the data for this IndexItem if withData is true, the data needs to be read at a later time, otherwise
Throws:
java.io.IOException

readItem

public ManagedIndexItem readItem(Offset offset,
                                 boolean withData)
                          throws java.io.IOException
Read the contents of the item It assumes the index file is alreayd open for writing.

Specified by:
readItem in interface IndexFileReader
Parameters:
offset - the byte offset in the file to start reading an item from
withData - read the data for this IndexItem if withData is true, the data needs to be read at a later time, otherwise
Throws:
java.io.IOException

readItem

public ManagedIndexItem readItem(long startOffset,
                                 boolean withData)
                          throws java.io.IOException
Read the contents of the item It assumes the index file is alreayd open for writing.

Specified by:
readItem in interface IndexFileReader
Parameters:
withData - read the data for this IndexItem if withData is true, the data needs to be read at a later time, otherwise
startOffset - the byte offset in the file to start reading an item from
Throws:
java.io.IOException

readNormalData

protected DataAbstraction readNormalData(long offset,
                                         long size,
                                         boolean withData)
                                  throws java.io.IOException
Read some data, from a specified offset for a number of bytes.

Throws:
java.io.IOException

readReferenceData

protected IndexReferenceDataHolder readReferenceData(long offset,
                                                     long size)
                                              throws java.io.IOException
Read a reference, from a specified offset for a number of bytes.

Throws:
java.io.IOException

readIndexItem

public java.nio.ByteBuffer readIndexItem(Offset offset,
                                         long size)
                                  throws java.io.IOException
Read an IndexItem given an offset.

Parameters:
offset - the byte offset in the file to start reading an item from
Throws:
java.io.IOException

readIndexItem

public java.nio.ByteBuffer readIndexItem(long offset)
                                  throws java.io.IOException
Read an IndexItem given an offset.

Parameters:
offset - the byte offset in the file to start reading an item from
Throws:
java.io.IOException

readData

public java.nio.ByteBuffer readData(Offset offset,
                                    long size)
                             throws java.io.IOException
Read some data, given an offset and a size.

Specified by:
readData in interface IndexFileReader
Parameters:
offset - the byte offset in the file to start reading an item from
size - the number of bytes to read
Throws:
java.io.IOException

readData

public java.nio.ByteBuffer readData(long offset,
                                    long size)
                             throws java.io.IOException
Read some data, given an offset and a size.

Specified by:
readData in interface IndexFileReader
Parameters:
offset - the byte offset in the file to start reading an item from
size - the number of bytes to read
Throws:
java.io.IOException

readDataIntoBuffer

protected abstract long readDataIntoBuffer(java.nio.ByteBuffer buffer,
                                           long size)
                                    throws java.io.IOException
Actually read in the data.

Throws:
java.io.IOException

readData

public java.nio.ByteBuffer readData(DataReference ref)
                             throws java.io.IOException
Read some data, given a DataReference.

Specified by:
readData in interface IndexFileReader
Throws:
java.io.IOException

memoryMapData

protected abstract java.nio.ByteBuffer memoryMapData(long offset,
                                                     long size)
                                              throws java.io.IOException
Memory map some data from a channel.

Throws:
java.io.IOException

seekToIndex

protected boolean seekToIndex(Offset offset)
                       throws java.io.IOException
Seek to a certain position.

Returns:
true if actually had to move the position, returns false if in correct place
Throws:
java.io.IOException

seekToIndex

protected abstract boolean seekToIndex(long position)
                                throws java.io.IOException
Seek to a certain position.

Returns:
true if actually had to move the position, returns false if in correct place
Throws:
java.io.IOException

seekToData

protected boolean seekToData(Offset offset)
                      throws java.io.IOException
Seek to a certain position.

Returns:
true if actually had to move the position, returns false if in correct place
Throws:
java.io.IOException

seekToData

protected abstract boolean seekToData(long position)
                               throws java.io.IOException
Seek to a certain position in the data file.

Returns:
true if actually had to move the position, returns false if in correct place
Throws:
java.io.IOException

skipData

public boolean skipData(long offset,
                        long size)
                 throws java.io.IOException
Skip over some data, given an offset and a size.

Parameters:
offset - the byte offset in the file to start reading an item from
size - the number of bytes to read
Throws:
java.io.IOException

convertDataReference

public DataHolderObject convertDataReference(DataReference dataReference)
Read some data, given a DataReference and return it as a DataHolderObject.

Specified by:
convertDataReference in interface IndexFileReader

loadIndex

public long loadIndex(LoadStyle loadStyle)
               throws java.io.IOException
Load the index

Specified by:
loadIndex in interface IndexFileReader
Throws:
java.io.IOException

getAppendPosition

public long getAppendPosition()
Get the append position

Specified by:
getAppendPosition in interface IndexFileWriter

gotoAppendPosition

public boolean gotoAppendPosition()
                           throws java.io.IOException
Goto the append position

Specified by:
gotoAppendPosition in interface IndexFileReader
Throws:
java.io.IOException

gotoFirstPosition

public boolean gotoFirstPosition()
                          throws java.io.IOException
Goto the first position

Specified by:
gotoFirstPosition in interface IndexFileReader
Throws:
java.io.IOException

setAppendPosition

public boolean setAppendPosition()
                          throws java.io.IOException
Set the append position from the indexChannelPosition.

Specified by:
setAppendPosition in interface IndexFileReader
Throws:
java.io.IOException

calculateAppendPosition

public abstract long calculateAppendPosition()
                                      throws java.io.IOException
Calculate the append position from the last item of the index.

Throws:
java.io.IOException

setItemSize

protected IndexFileInteractor setItemSize(int itemSize)
Set the index item size. The size is determined by the header I/O object at index create time. Return the old index item size in bytes.


getWriteLock

public java.nio.channels.FileLock getWriteLock()
Get a write-lock on this index.

Specified by:
getWriteLock in interface IndexFileWriter

releaseWriteLock

public boolean releaseWriteLock()
Release a FileLock.

Specified by:
releaseWriteLock in interface IndexFileWriter

isWriteLocked

public boolean isWriteLocked()
Has the Index been write-locked.

Specified by:
isWriteLocked in interface IndexFileWriter

requestWriteWork

public void requestWriteWork(java.nio.channels.FileChannel channel,
                             java.nio.ByteBuffer flushBuffer,
                             ByteBufferRing ring)
Add some work to the write queue.


requestReadWork

public void requestReadWork(Position position,
                            boolean doLoadData)
Add some work to the read queue.


writeFromWorkQueue

public long writeFromWorkQueue()
                        throws java.io.IOException
Write the contents of the ffirst ByteBuffer in the work queue to a FileChannel. It assumes the index file is alreayd open for writing.

Throws:
java.io.IOException

drainWriteQueue

public void drainWriteQueue()
                     throws java.io.IOException
This drains the write request queue by processing all the WriteRequests.

Throws:
java.io.IOException

timeOut

public boolean timeOut(long timeout)
Wait for the timeout to go off.

Returns:
true if the timeout happened, false if it dod not.

awaitWork

public boolean awaitWork()

run

public void run()
The Thread run method.

Specified by:
run in interface java.lang.Runnable