@terascope/job-components / index / Collector

Class: Collector<T>

Defined in: packages/utils/dist/src/collector.d.ts:9

An in-memory record collector, useful for batch data to a certain size or after a certain amount of time has passed.

NOTE: Records are store in an immutable array to be more memory efficient.

Type Parameters

Type Parameter
T

Constructors

new Collector()

new Collector<T>(max): Collector<T>

Defined in: packages/utils/dist/src/collector.d.ts:16

Parameters

Parameter	Type
max	{ size: number; wait: number; }
max.size	number
max.wait	number

Returns

Collector<T>

Properties

Property	Modifier	Type	Description	Defined in
size	readonly	number	the maximum wait time to collect the batch	packages/utils/dist/src/collector.d.ts:11
wait	readonly	number	the maximum batch size of the batch	packages/utils/dist/src/collector.d.ts:13

Accessors

length

Get Signature

get length(): number

Defined in: packages/utils/dist/src/collector.d.ts:23

Get the current Queue Length

Returns

number

---

queue

Get Signature

get queue(): T[]

Defined in: packages/utils/dist/src/collector.d.ts:27

Get the current queue

Returns

T[]

Methods

add()

add(_records): void

Defined in: packages/utils/dist/src/collector.d.ts:31

Add a record, or records, to the in-memory queue.

Parameters

Parameter	Type
_records	T | T[]

Returns

void

---

flushAll()

flushAll(): T[]

Defined in: packages/utils/dist/src/collector.d.ts:45

Flush all of the records in the queue.

NOTE: This can potentially return more records than specified than the max size.

Returns

T[]

---

getBatch()

getBatch(): null | T[]

Defined in: packages/utils/dist/src/collector.d.ts:38

Get the batch of data if it is full or has exceeded the time threshold.

Returns

null | T[]

null if the batch isn't ready

the a batch of records, less or equal to the max size