@terascope/file-asset-apis / s3/s3-sender / S3Sender

Class: S3Sender

Defined in: s3/s3-sender.ts:24

Extends

• ChunkedFileSender

Implements

• RouteSenderAPI

Constructors

Constructor

new S3Sender(client, config, logger): S3Sender

Defined in: s3/s3-sender.ts:27

Parameters

Parameter	Type
client	S3Client
config	ChunkedFileSenderConfig
logger	Logger

Returns

S3Sender

Overrides

ChunkedFileSender.constructor

Properties

Property	Modifier	Type	Default value	Inherited from	Defined in
client	public	S3Client	undefined	-	s3/s3-sender.ts:25
config	readonly	ChunkedFileSenderConfig	undefined	ChunkedFileSender.config	base/chunked-file-sender.ts:56
formatter	protected	Formatter	undefined	ChunkedFileSender.formatter	base/chunked-file-sender.ts:53
logger	readonly	Logger	undefined	ChunkedFileSender.logger	base/chunked-file-sender.ts:57
pathList	readonly	Map<string, boolean>	undefined	ChunkedFileSender.pathList	base/chunked-file-sender.ts:54
sliceCount	protected	number	-1	ChunkedFileSender.sliceCount	base/chunked-file-sender.ts:51
type	readonly	FileSenderType	undefined	ChunkedFileSender.type	base/chunked-file-sender.ts:55

Accessors

concurrency

Get Signature

get concurrency(): number

Defined in: base/chunked-file-sender.ts:103

Returns

number

Inherited from

ChunkedFileSender.concurrency

---

filePerSlice

Get Signature

get filePerSlice(): boolean

Defined in: base/chunked-file-sender.ts:111

Returns

boolean

Inherited from

ChunkedFileSender.filePerSlice

---

format

Get Signature

get format(): Format

Defined in: base/chunked-file-sender.ts:115

Returns

Format

Inherited from

ChunkedFileSender.format

---

id

Get Signature

get id(): string

Defined in: base/chunked-file-sender.ts:91

Returns

string

Inherited from

ChunkedFileSender.id

---

isRouter

Get Signature

get isRouter(): boolean

Defined in: base/chunked-file-sender.ts:99

Returns

boolean

Inherited from

ChunkedFileSender.isRouter

---

lineDelimiter

Get Signature

get lineDelimiter(): string

Defined in: base/chunked-file-sender.ts:107

Returns

string

Inherited from

ChunkedFileSender.lineDelimiter

---

nameOptions

Get Signature

get nameOptions(): NameOptions

Defined in: base/chunked-file-sender.ts:119

Returns

NameOptions

Inherited from

ChunkedFileSender.nameOptions

---

path

Get Signature

get path(): string

Defined in: base/chunked-file-sender.ts:95

Returns

string

Inherited from

ChunkedFileSender.path

Methods

createFileDestinationName()

createFileDestinationName(filePath): Promise<string>

Defined in: base/chunked-file-sender.ts:154

Ensures the root destination file path exists and returns the full destination file path.

Parameters

Parameter	Type
filePath	string

Returns

Promise<string>

Note

this API might change since it should not have knowledge of the different sender types

Inherited from

ChunkedFileSender.createFileDestinationName

---

ensureBucket()

ensureBucket(): Promise<void>

Defined in: s3/s3-sender.ts:123

Used to verify that the bucket exists, will attempt to create one if it does not exist

Returns

Promise<void>

---

incrementCount()

protected incrementCount(): void

Defined in: base/chunked-file-sender.ts:236

Increases the internal sliceCount used to create filenames, this is already executed in the "send" method, do not use unless you are implementing your own send logic

Returns

void

Inherited from

ChunkedFileSender.incrementCount

---

joinPath()

protected joinPath(route?): string

Defined in: base/chunked-file-sender.ts:177

Method to help create proper file paths, mainly used in the abstract "verify" method

Parameters

Parameter	Type
route?	string

Returns

string

Inherited from

ChunkedFileSender.joinPath

---

prepareDispatch()

prepareDispatch(data): Promise<SendBatchConfig[]>

Defined in: base/chunked-file-sender.ts:192

Batches records in a slice into groups based on the "path" config or by the DataEntity metadata 'standard:route' override if dynamic routing is being used, this is called in the "send" method

Parameters

Parameter	Type
data	SendRecords

Returns

Promise<SendBatchConfig[]>

Inherited from

ChunkedFileSender.prepareDispatch

---

send()

send(records): Promise<number>

Defined in: base/chunked-file-sender.ts:251

Write data to file, uses parent "sendToDestination" method to determine location

Parameters

Parameter	Type
records	SendRecords

Returns

Promise<number>

Example

s3Sender.send([{ some: 'data' }]) => Promise<void>
  s3Sender.send([DataEntity.make({ some: 'data' })]) => Promise<void>

Implementation of

RouteSenderAPI.send

Inherited from

ChunkedFileSender.send

---

sendToDestination()

protected sendToDestination(__namedParameters): Promise<void>

Defined in: s3/s3-sender.ts:37

This is a low level API, it is not meant to be used externally, please use the "send" method instead

Parameters

Parameter	Type
__namedParameters	SendBatchConfig

Returns

Promise<void>

Overrides

ChunkedFileSender.sendToDestination

---

simpleSend()

simpleSend(records): Promise<void>

Defined in: base/chunked-file-sender.ts:283

Write data to file, uses parent "sendToDestination" method to determine location. Use this to avoid having to bucket the data into different paths. Normally you don't want this but it can be used in specific case.

Parameters

Parameter	Type
records	SendRecords

Returns

Promise<void>

Example

s3Sender.simpleSend([{ some: 'data' }]) => Promise<void>
  s3Sender.simpleSend([DataEntity.make({ some: 'data' })]) => Promise<void>

Inherited from

ChunkedFileSender.simpleSend

---

verify()

verify(_route): Promise<void>

Defined in: s3/s3-sender.ts:151

This is currently a noop, may change in future and work to ensure dynamic bucket exists

Parameters

Parameter	Type
_route	string

Returns

Promise<void>

Implementation of

RouteSenderAPI.verify

Overrides

ChunkedFileSender.verify