voldemort.store.readonly
Class ExternalSorter<V>

java.lang.Object
  voldemort.store.readonly.ExternalSorter<V>

Type Parameters:

V - The type of value being sorted

public class ExternalSorter<V>
extends java.lang.Object

Do an external sort on data coming from an input iterator and produce a new iterator over the values in sorted order

Field Summary

static org.apache.log4j.Logger

logger

Constructor Summary

ExternalSorter(Serializer<V> serializer, java.util.Comparator<V> comparator, int internalSortSize, int numThreads)
Create an external sorter using the given serializer and internal sort size.

ExternalSorter(Serializer<V> serializer, java.util.Comparator<V> comparator, int internalSortSize, java.lang.String tempDir, int bufferSize, int numThreads, boolean gzip)
Create an external sorter using the given serializer and internal sort size.

ExternalSorter(Serializer<V> serializer, int internalSortSize, int numThreads)
Create an external sorter using the given serializer and internal sort size.

Method Summary

java.lang.Iterable<V>

sorted(java.util.Iterator<V> input) Produce an iterator over the input values in sorted order.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

public static final org.apache.log4j.Logger logger

Constructor Detail

ExternalSorter

public ExternalSorter(Serializer<V> serializer,
                      int internalSortSize,
                      int numThreads)

Create an external sorter using the given serializer and internal sort size. Use natural ordering, system temp dir, and reasonable buffer size

Parameters:

serializer - The serializer used to write data to disk

internalSortSize - The number of objects in the internal sort buffer

ExternalSorter

public ExternalSorter(Serializer<V> serializer,
                      java.util.Comparator<V> comparator,
                      int internalSortSize,
                      int numThreads)

Create an external sorter using the given serializer and internal sort size. Use natural ordering, system temp dir, and reasonable buffer size

Parameters:

serializer - The serializer used to write data to disk

comparator - The comparator used to order the objects

internalSortSize - The number of objects to keep in the internal memory

ExternalSorter

public ExternalSorter(Serializer<V> serializer,
                      java.util.Comparator<V> comparator,
                      int internalSortSize,
                      java.lang.String tempDir,
                      int bufferSize,
                      int numThreads,
                      boolean gzip)

Create an external sorter using the given serializer and internal sort size. Use natural ordering, system temp dir, and reasonable buffer size

Parameters:

serializer - The serializer used to write data to disk

comparator - The comparator used to order the objects

internalSortSize - The number of objects to keep in the internal memory

tempDir - The temporary directory to which to write temporary data

bufferSize - The IO buffer size

Method Detail

sorted

public java.lang.Iterable<V> sorted(java.util.Iterator<V> input)

Produce an iterator over the input values in sorted order. Sorting will occur in the fixed space configured in the constructor, data will be dumped to disk as necessary.

Parameters:

input - An iterator over the input values

Returns:

An iterator over the values