@terascope/utils / collector / Collector

Class: Collector<T>

An in-memory record collector, useful for batch data to a certain size or after a certain amount of time has passed.

NOTE: Records are store in an immutable array to be more memory efficient.

Type parameters

Type parameter
T

Constructors

new Collector()

new Collector<T>(max): Collector<T>

Parameters

Parameter	Type
max	object
max.size	number
max.wait	number

Returns

Collector<T>

Source

packages/utils/src/collector.ts:20

Properties

size

readonly size: number

the maximum wait time to collect the batch

Source

packages/utils/src/collector.ts:13

---

wait

readonly wait: number

the maximum batch size of the batch

Source

packages/utils/src/collector.ts:15

Accessors

length

get length(): number

Get the current Queue Length

Returns

number

Source

packages/utils/src/collector.ts:28

---

queue

get queue(): T[]

Get the current queue

Returns

T[]

Source

packages/utils/src/collector.ts:35

Methods

add()

add(_records): void

Add a record, or records, to the in-memory queue.

Parameters

Parameter	Type
_records	T | T[]

Returns

void

Source

packages/utils/src/collector.ts:42

---

flushAll()

flushAll(): T[]

Flush all of the records in the queue.

NOTE: This can potentially return more records than specified than the max size.

Returns

T[]

Source

packages/utils/src/collector.ts:75

---

getBatch()

getBatch(): null | T[]

Get the batch of data if it is full or has exceeded the time threshold.

Returns

null | T[]

null if the batch isn't ready

the a batch of records, less or equal to the max size

Source

packages/utils/src/collector.ts:57