Job Components: `Builtin Collect Processor Default`
builtin/collect/processor.default
A variation of "Processor" that deals with a batch of data at a time.
Hierarchy
-
↳
default
Table of contents
Constructors
Properties
Methods
Constructors
constructor
• new default(context
, opConfig
, executionConfig
): default
Parameters
Name | Type |
---|---|
context | WorkerContext |
opConfig | CollectConfig |
executionConfig | ExecutionConfig |
Returns
Overrides
Defined in
packages/job-components/src/builtin/collect/processor.ts:9
Properties
collector
• collector: Collector
<DataEntity
<Record
<string
, any
>, _DataEntityMetadata
<Record
<string
, any
>>>>
Defined in
packages/job-components/src/builtin/collect/processor.ts:7
context
• Readonly
context: Readonly
<WorkerContext
>
Inherited from
Defined in
packages/job-components/src/operations/core/core.ts:10
deadLetterAction
• deadLetterAction: string
Inherited from
Defined in
packages/job-components/src/operations/core/operation-core.ts:28
events
• Readonly
events: EventEmitter
Inherited from
Defined in
packages/job-components/src/operations/core/core.ts:13
executionConfig
• Readonly
executionConfig: Readonly
<ExecutionConfig
>
Inherited from
Defined in
packages/job-components/src/operations/core/core.ts:11
logger
• Readonly
logger: Logger
Inherited from
Defined in
packages/job-components/src/operations/core/core.ts:12
opConfig
• Readonly
opConfig: Readonly
<OpConfig
& CollectConfig
>
Inherited from
Defined in
packages/job-components/src/operations/core/operation-core.ts:27
Methods
createAPI
▸ createAPI<A
>(name
, ...params
): Promise
<A
>
Create an API and add it to the operation lifecycle
Type parameters
Name | Type |
---|---|
A | extends OpAPI = OpAPI |
Parameters
Name | Type |
---|---|
name | string |
...params | any [] |
Returns
Promise
<A
>
Inherited from
Defined in
packages/job-components/src/operations/core/operation-core.ts:51
getAPI
▸ getAPI<A
>(name
): A
Get a reference to an existing API
Type parameters
Name | Type |
---|---|
A | extends OpAPI = OpAPI |
Parameters
Name | Type |
---|---|
name | string |
Returns
A
Inherited from
Defined in
packages/job-components/src/operations/core/operation-core.ts:58
handle
▸ handle(input
): Promise
<DataEntity
<Record
<string
, any
>, _DataEntityMetadata
<Record
<string
, any
>>>[]>
A generic method called by the Teraslice framework to a give a "Processor" the ability to handle the input and output of operation
Parameters
Name | Type | Description |
---|---|---|
input | DataEntity <Record <string , any >, _DataEntityMetadata <Record <string , any >>>[] | an array of DataEntities |
Returns
Promise
<DataEntity
<Record
<string
, any
>, _DataEntityMetadata
<Record
<string
, any
>>>[]>
an array of DataEntities
Inherited from
Defined in
packages/job-components/src/operations/batch-processor.ts:15
initialize
▸ initialize(): Promise
<void
>
Called during execution initialization, when this is called perform any async setup.
Returns
Promise
<void
>
Inherited from
Defined in
packages/job-components/src/operations/core/operation-core.ts:40
onBatch
▸ onBatch(batch
): Promise
<DataEntity
<Record
<string
, any
>, _DataEntityMetadata
<Record
<string
, any
>>>[]>
A method called by BatchProcessor#handle
Parameters
Name | Type |
---|---|
batch | DataEntity <Record <string , any >, _DataEntityMetadata <Record <string , any >>>[] |
Returns
Promise
<DataEntity
<Record
<string
, any
>, _DataEntityMetadata
<Record
<string
, any
>>>[]>
an array of DataEntities
Overrides
Defined in
packages/job-components/src/builtin/collect/processor.ts:14
rejectRecord
▸ rejectRecord(input
, err
): null
Reject a record using the dead letter action
Based on OpConfig._dead_letter_action the transformation can be handled any of the following ways:
- "throw": throw the original error
- "log": log the error and the data
- "none": skip the error entirely OR a string to specify the api to use as the dead letter queue
Parameters
Name | Type |
---|---|
input | unknown |
err | Error |
Returns
null
null
Inherited from
Defined in
packages/job-components/src/operations/core/operation-core.ts:96
shutdown
▸ shutdown(): Promise
<void
>
Called during execution shutdown, when this is cleanup any open connections or destroy any in-memory state.
Returns
Promise
<void
>
Overrides
Defined in
packages/job-components/src/builtin/collect/processor.ts:20
tryRecord
▸ tryRecord<I
, R
>(fn
): (input
: I
) => null
| R
Try catch a transformation on a record and place any failed records in a dead letter queue
See #rejectRecord for handling
Type parameters
Name |
---|
I |
R |
Parameters
Name | Type | Description |
---|---|---|
fn | (input : I ) => R | a function to transform the data with |
Returns
fn
a curried a function that will be called with the data and handle the dead letter action
▸ (input
): null
| R
Try catch a transformation on a record and place any failed records in a dead letter queue
See #rejectRecord for handling
Parameters
Name | Type |
---|---|
input | I |
Returns
null
| R
a curried a function that will be called with the data and handle the dead letter action
Inherited from
Defined in
packages/job-components/src/operations/core/operation-core.ts:71