@terascope/job-components / index / Collector

Class: Collector<T>

An in-memory record collector, useful for batch data to a certain size or after a certain amount of time has passed.

NOTE: Records are store in an immutable array to be more memory efficient.

Type parameters

Type parameter
T

Constructors

new Collector()

new Collector<T>(max): Collector<T>

Parameters

Parameter	Type
max	object
max.size	number
max.wait	number

Returns

Collector<T>

Source

packages/utils/dist/src/collector.d.ts:16

Properties

size

readonly size: number

the maximum wait time to collect the batch

Source

packages/utils/dist/src/collector.d.ts:11

---

wait

readonly wait: number

the maximum batch size of the batch

Source

packages/utils/dist/src/collector.d.ts:13

Accessors

length

get length(): number

Get the current Queue Length

Returns

number

Source

packages/utils/dist/src/collector.d.ts:23

---

queue

get queue(): T[]

Get the current queue

Returns

T[]

Source

packages/utils/dist/src/collector.d.ts:27

Methods

add()

add(_records): void

Add a record, or records, to the in-memory queue.

Parameters

Parameter	Type
_records	T | T[]

Returns

void

Source

packages/utils/dist/src/collector.d.ts:31

---

flushAll()

flushAll(): T[]

Flush all of the records in the queue.

NOTE: This can potentially return more records than specified than the max size.

Returns

T[]

Source

packages/utils/dist/src/collector.d.ts:45

---

getBatch()

getBatch(): null | T[]

Get the batch of data if it is full or has exceeded the time threshold.

Returns

null | T[]

null if the batch isn't ready

the a batch of records, less or equal to the max size

Source

packages/utils/dist/src/collector.d.ts:38