@terascope/job-components / builtin/test-reader/fetcher / default

Class: default

Defined in: packages/job-components/src/builtin/test-reader/fetcher.ts:9

The simpliest varient of "Fetcher"

Extends

• default<TestReaderConfig>

Constructors

Constructor

new default(context, opConfig, executionConfig): TestFetcher

Defined in: packages/job-components/src/operations/core/operation-core.ts:26

Parameters

Parameter	Type
context	Context
opConfig	OpConfig & TestReaderConfig
executionConfig	ExecutionConfig

Returns

TestFetcher

Inherited from

default.constructor

Properties

Property	Modifier	Type	Default value	Inherited from	Defined in
cachedData	public	null | Buffer<ArrayBufferLike>	null	-	packages/job-components/src/builtin/test-reader/fetcher.ts:10
context	readonly	Readonly<T>	undefined	default.context	packages/job-components/src/operations/core/core.ts:10
deadLetterAction	public	string	undefined	default.deadLetterAction	packages/job-components/src/operations/core/operation-core.ts:24
events	readonly	EventEmitter	undefined	default.events	packages/job-components/src/operations/core/core.ts:13
executionConfig	readonly	Readonly<ExecutionConfig>	undefined	default.executionConfig	packages/job-components/src/operations/core/core.ts:11
lastFilePath	public	string	''	-	packages/job-components/src/builtin/test-reader/fetcher.ts:11
logger	readonly	Logger	undefined	default.logger	packages/job-components/src/operations/core/core.ts:12
opConfig	readonly	Readonly<OpConfig & T>	undefined	default.opConfig	packages/job-components/src/operations/core/operation-core.ts:23

Methods

createAPI()

createAPI<A>(name, ...params): Promise<A>

Defined in: packages/job-components/src/operations/core/operation-core.ts:47

Create an API and add it to the operation lifecycle

Type Parameters

Type Parameter	Default type
A extends OpAPI	OpAPI

Parameters

Parameter	Type
name	string
...params	any[]

Returns

Promise<A>

Inherited from

default.createAPI

---

fetch()

fetch(slice?): Promise<DataEntity<Record<string, any>, _DataEntityMetadata<Record<string, any>>>[]>

Defined in: packages/job-components/src/builtin/test-reader/fetcher.ts:17

A method called by Fetcher#handle

Parameters

Parameter	Type
slice?	SliceRequest[]

Returns

Promise<DataEntity<Record<string, any>, _DataEntityMetadata<Record<string, any>>>[]>

a DataEntity compatible array

Overrides

default.fetch

---

getAPI()

getAPI<A>(name): A

Defined in: packages/job-components/src/operations/core/operation-core.ts:54

Get a reference to an existing API

Type Parameters

Type Parameter	Default type
A extends OpAPI	OpAPI

Parameters

Parameter	Type
name	string

Returns

A

Inherited from

default.getAPI

---

handle()

handle(sliceRequest?): Promise<DataEntity<Record<string, any>, _DataEntityMetadata<Record<string, any>>>[]>

Defined in: packages/job-components/src/operations/fetcher.ts:16

A generic method called by the Teraslice framework to a give a "Fetcher" the ability to handle the fetch operation

Parameters

Parameter	Type
sliceRequest?	unknown

Returns

Promise<DataEntity<Record<string, any>, _DataEntityMetadata<Record<string, any>>>[]>

Inherited from

default.handle

---

initialize()

initialize(): Promise<void>

Defined in: packages/job-components/src/builtin/test-reader/fetcher.ts:13

Called during execution initialization, when this is called perform any async setup.

Returns

Promise<void>

Overrides

default.initialize

---

rejectRecord()

rejectRecord(input, err): null

Defined in: packages/job-components/src/operations/core/operation-core.ts:92

Reject a record using the dead letter action

Based on OpConfig._dead_letter_action the transformation can be handled any of the following ways:

• "throw": throw the original error
• "log": log the error and the data
• "none": skip the error entirely OR a string to specify the api to use as the dead letter queue

Parameters

Parameter	Type
input	unknown
err	Error

Returns

null

null

Inherited from

default.rejectRecord

---

shutdown()

shutdown(): Promise<void>

Defined in: packages/job-components/src/operations/core/operation-core.ts:40

Called during execution shutdown, when this is cleanup any open connections or destroy any in-memory state.

Returns

Promise<void>

Inherited from

default.shutdown

---

tryRecord()

tryRecord<I, R>(fn): (input) => null | R

Defined in: packages/job-components/src/operations/core/operation-core.ts:67

Try catch a transformation on a record and place any failed records in a dead letter queue

See #rejectRecord for handling

Type Parameters

Type Parameter
I
R

Parameters

Parameter	Type	Description
fn	(input) => R	a function to transform the data with

Returns

a curried a function that will be called with the data and handle the dead letter action

(input): null | R

Parameters

Parameter	Type
input	I

Returns

null | R

Inherited from

default.tryRecord