@terascope/job-components / operations/core/fetcher-core / default

Class: abstract default<T>

A base class for supporting "Fetcher" that run on a "Worker". The "Fetcher" is a part of the "Reader" component of a job. See [[OperationCore]]

Extends

• default<T>

Extended by

• default

Type parameters

Type parameter	Value
T	OpConfig

Constructors

new default()

new default<T>(context, opConfig, executionConfig): default<T>

Parameters

Parameter	Type
context	Context
opConfig	OpConfig & T
executionConfig	ExecutionConfig

Returns

default<T>

Inherited from

default.constructor

Source

packages/job-components/src/operations/core/operation-core.ts:26

Properties

context

readonly context: Readonly<Context>

Inherited from

default.context

Source

packages/job-components/src/operations/core/core.ts:10

---

deadLetterAction

deadLetterAction: string

Inherited from

default.deadLetterAction

Source

packages/job-components/src/operations/core/operation-core.ts:24

---

events

readonly events: EventEmitter<DefaultEventMap>

Inherited from

default.events

Source

packages/job-components/src/operations/core/core.ts:13

---

executionConfig

readonly executionConfig: Readonly<ExecutionConfig>

Inherited from

default.executionConfig

Source

packages/job-components/src/operations/core/core.ts:11

---

logger

readonly logger: Logger

Inherited from

default.logger

Source

packages/job-components/src/operations/core/core.ts:12

---

opConfig

readonly opConfig: Readonly<OpConfig & T>

Inherited from

default.opConfig

Source

packages/job-components/src/operations/core/operation-core.ts:23

Methods

createAPI()

createAPI<A>(name, ...params): Promise<A>

Create an API and add it to the operation lifecycle

Type parameters

Type parameter	Value
A extends OpAPI	OpAPI

Parameters

Parameter	Type
name	string
...params	any[]

Returns

Promise<A>

Inherited from

default.createAPI

Source

packages/job-components/src/operations/core/operation-core.ts:47

---

getAPI()

getAPI<A>(name): A

Get a reference to an existing API

Type parameters

Type parameter	Value
A extends OpAPI	OpAPI

Parameters

Parameter	Type
name	string

Returns

A

Inherited from

default.getAPI

Source

packages/job-components/src/operations/core/operation-core.ts:54

---

handle()

abstract handle(sliceRequest?): Promise<DataEntity<Record<string, any>, _DataEntityMetadata<Record<string, any>>>[]>

A generic method called by the Teraslice framework to a give a "Fetcher" the ability to handle the fetch operation

Parameters

Parameter	Type
sliceRequest?	unknown

Returns

Promise<DataEntity<Record<string, any>, _DataEntityMetadata<Record<string, any>>>[]>

Source

packages/job-components/src/operations/core/fetcher-core.ts:16

---

initialize()

initialize(): Promise<void>

Called during execution initialization, when this is called perform any async setup.

Returns

Promise<void>

Inherited from

default.initialize

Source

packages/job-components/src/operations/core/operation-core.ts:36

---

rejectRecord()

rejectRecord(input, err): null

Reject a record using the dead letter action

Based on OpConfig._dead_letter_action the transformation can be handled any of the following ways:

• "throw": throw the original error
• "log": log the error and the data
• "none": skip the error entirely OR a string to specify the api to use as the dead letter queue

Parameters

Parameter	Type
input	unknown
err	Error

Returns

null

null

Inherited from

default.rejectRecord

Source

packages/job-components/src/operations/core/operation-core.ts:92

---

shutdown()

shutdown(): Promise<void>

Called during execution shutdown, when this is cleanup any open connections or destroy any in-memory state.

Returns

Promise<void>

Inherited from

default.shutdown

Source

packages/job-components/src/operations/core/operation-core.ts:40

---

tryRecord()

tryRecord<I, R>(fn): (input) => null | R

Try catch a transformation on a record and place any failed records in a dead letter queue

See #rejectRecord for handling

Type parameters

Type parameter
I
R

Parameters

Parameter	Type	Description
fn	(input) => R	a function to transform the data with

Returns

Function

a curried a function that will be called with the data and handle the dead letter action

Parameters

Parameter	Type
input	I

Returns

null | R

Inherited from

default.tryRecord

Source

packages/job-components/src/operations/core/operation-core.ts:67