@terascope/core-utils / collector / Collector
Class: Collector<T>
Defined in: packages/core-utils/src/collector.ts:11
An in-memory record collector, useful for batch data to a certain size or after a certain amount of time has passed.
NOTE: Records are store in an immutable array to be more memory efficient.
Type Parameters
| Type Parameter |
|---|
T |
Constructors
Constructor
new Collector<
T>(max):Collector<T>
Defined in: packages/core-utils/src/collector.ts:20
Parameters
| Parameter | Type |
|---|---|
max | { size: number; wait: number; } |
max.size | number |
max.wait | number |
Returns
Collector<T>
Properties
| Property | Modifier | Type | Description | Defined in |
|---|---|---|---|---|
size | readonly | number | the maximum wait time to collect the batch | packages/core-utils/src/collector.ts:13 |
wait | readonly | number | the maximum batch size of the batch | packages/core-utils/src/collector.ts:15 |
Accessors
length
Get Signature
get length():
number
Defined in: packages/core-utils/src/collector.ts:28
Get the current Queue Length
Returns
number
queue
Get Signature
get queue():
T[]
Defined in: packages/core-utils/src/collector.ts:35
Get the current queue
Returns
T[]
Methods
add()
add(
_records):void
Defined in: packages/core-utils/src/collector.ts:42
Add a record, or records, to the in-memory queue.
Parameters
| Parameter | Type |
|---|---|
_records | T | T[] |
Returns
void
flushAll()
flushAll():
T[]
Defined in: packages/core-utils/src/collector.ts:75
Flush all of the records in the queue.
NOTE: This can potentially return more records than specified than the max size.
Returns
T[]
getBatch()
getBatch():
T[] |null
Defined in: packages/core-utils/src/collector.ts:57
Get the batch of data if it is full or has exceeded the time threshold.
Returns
T[] | null
null if the batch isn't ready