@terascope/job-components / operations/batch-processor / default
Abstract Class: default<T>
Defined in: packages/job-components/src/operations/batch-processor.ts:8
A variation of "Processor" that deals with a batch of data at a time.
Extends
default<T>
Extended by
Type Parameters
| Type Parameter | Default type |
|---|---|
T | OpConfig |
Constructors
Constructor
new default<
T>(context,opConfig,executionConfig):BatchProcessor<T>
Defined in: packages/job-components/src/operations/core/operation-core.ts:26
Parameters
| Parameter | Type |
|---|---|
context | Context |
opConfig | OpConfig & T |
executionConfig | ExecutionConfig |
Returns
BatchProcessor<T>
Inherited from
Properties
| Property | Modifier | Type | Inherited from | Defined in |
|---|---|---|---|---|
context | readonly | Readonly<T> | default.context | packages/job-components/src/operations/core/core.ts:10 |
deadLetterAction | public | string | default.deadLetterAction | packages/job-components/src/operations/core/operation-core.ts:24 |
events | readonly | EventEmitter | default.events | packages/job-components/src/operations/core/core.ts:13 |
executionConfig | readonly | Readonly<ExecutionConfig> | default.executionConfig | packages/job-components/src/operations/core/core.ts:11 |
logger | readonly | Logger | default.logger | packages/job-components/src/operations/core/core.ts:12 |
opConfig | readonly | Readonly<OpConfig & T> | default.opConfig | packages/job-components/src/operations/core/operation-core.ts:23 |
Methods
createAPI()
createAPI<
A>(name, ...params):Promise<A>
Defined in: packages/job-components/src/operations/core/operation-core.ts:47
Create an API and add it to the operation lifecycle
Type Parameters
| Type Parameter | Default type |
|---|---|
A extends OpAPI | OpAPI |
Parameters
| Parameter | Type |
|---|---|
name | string |
...params | any[] |
Returns
Promise<A>
Inherited from
getAPI()
getAPI<
A>(name):A
Defined in: packages/job-components/src/operations/core/operation-core.ts:54
Get a reference to an existing API
Type Parameters
| Type Parameter | Default type |
|---|---|
A extends OpAPI | OpAPI |
Parameters
| Parameter | Type |
|---|---|
name | string |
Returns
A
Inherited from
handle()
handle(
input):Promise<DataEntity<Record<string,any>,_DataEntityMetadata<Record<string,any>>>[]>
Defined in: packages/job-components/src/operations/batch-processor.ts:15
A generic method called by the Teraslice framework to a give a "Processor" the ability to handle the input and output of operation
Parameters
| Parameter | Type | Description |
|---|---|---|
input | DataEntity<Record<string, any>, _DataEntityMetadata<Record<string, any>>>[] | an array of DataEntities |
Returns
Promise<DataEntity<Record<string, any>, _DataEntityMetadata<Record<string, any>>>[]>
an array of DataEntities
Overrides
initialize()
initialize():
Promise<void>
Defined in: packages/job-components/src/operations/core/operation-core.ts:36
Called during execution initialization, when this is called perform any async setup.
Returns
Promise<void>
Inherited from
onBatch()
abstractonBatch(batch):Promise<DataEntity<Record<string,any>,_DataEntityMetadata<Record<string,any>>>[]>
Defined in: packages/job-components/src/operations/batch-processor.ts:13
A method called by BatchProcessor#handle
Parameters
| Parameter | Type |
|---|---|
batch | DataEntity<Record<string, any>, _DataEntityMetadata<Record<string, any>>>[] |
Returns
Promise<DataEntity<Record<string, any>, _DataEntityMetadata<Record<string, any>>>[]>
an array of DataEntities
rejectRecord()
rejectRecord(
input,err):null
Defined in: packages/job-components/src/operations/core/operation-core.ts:92
Reject a record using the dead letter action
Based on OpConfig._dead_letter_action the transformation can be handled any of the following ways:
- "throw": throw the original error
- "log": log the error and the data
- "none": skip the error entirely OR a string to specify the api to use as the dead letter queue
Parameters
| Parameter | Type |
|---|---|
input | unknown |
err | Error |
Returns
null
null
Inherited from
shutdown()
shutdown():
Promise<void>
Defined in: packages/job-components/src/operations/core/operation-core.ts:40
Called during execution shutdown, when this is cleanup any open connections or destroy any in-memory state.
Returns
Promise<void>
Inherited from
tryRecord()
tryRecord<
I,R>(fn): (input) =>null|R
Defined in: packages/job-components/src/operations/core/operation-core.ts:67
Try catch a transformation on a record and place any failed records in a dead letter queue
See #rejectRecord for handling
Type Parameters
| Type Parameter |
|---|
I |
R |
Parameters
| Parameter | Type | Description |
|---|---|---|
fn | (input) => R | a function to transform the data with |
Returns
a curried a function that will be called with the data and handle the dead letter action
(
input):null|R
Parameters
| Parameter | Type |
|---|---|
input | I |
Returns
null | R