@terascope/job-components / index / DataEntity

Class: DataEntity<T, M>

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:9

A wrapper for data that can hold additional metadata properties. A DataEntity should be essentially transparent to use within operations.

NOTE: Use DataEntity.make, DataEntity.fromBuffer and DataEntity.makeArray in production for potential performance gains

Type Parameters

Type Parameter	Default type
T	Record<string, any>
M	_DataEntityMetadata<Record<string, any>>

Indexable

[prop: string]: any

Constructors

new DataEntity()

new DataEntity<T, M>(data, metadata?): DataEntity<T, M>

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:65

Parameters

Parameter	Type
data	undefined | null | T
metadata?	M

Returns

DataEntity<T, M>

Methods

getCreateTime()

getCreateTime(): Date

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:89

Get the time at which this entity was created.

Returns

Date

---

getEventTime()

getEventTime(): undefined | false | Date

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:124

Get time associated from a specific field on source data or message

If none is found, undefined will be returned. If an invalid date is found, false will be returned.

Returns

undefined | false | Date

---

getIngestTime()

getIngestTime(): undefined | false | Date

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:96

Get the time at which the data was ingested into the source data

If none is found, undefined will be returned. If an invalid date is found, false will be returned.

Returns

undefined | false | Date

---

getKey()

getKey(): string | number

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:80

Get the unique document _key from the metadata. If no _key is found, an error will be thrown

Returns

string | number

---

getMetadata()

Call Signature

getMetadata(key?): _DataEntityMetadata<M>

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:70

Get the metadata for the DataEntity. If a field is specified, it will get that property of the metadata

Parameters

Parameter	Type
key?	undefined

Returns

_DataEntityMetadata<M>

Call Signature

getMetadata<K>(key): EntityMetadataValue<M, K>

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:71

Get the metadata for the DataEntity. If a field is specified, it will get that property of the metadata

Type Parameters

Type Parameter
K extends string | number | symbol

Parameters

Parameter	Type
key	K

Returns

EntityMetadataValue<M, K>

---

getProcessTime()

getProcessTime(): undefined | false | Date

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:110

Get the time at which the data was consumed by the reader.

If none is found, undefined will be returned. If an invalid date is found, false will be returned.

Returns

undefined | false | Date

---

getRawData()

getRawData(): Buffer

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:136

Get the raw data, usually used for encoding type raw. If there is no data, an error will be thrown

Returns

Buffer

---

setEventTime()

setEventTime(val?): void

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:131

Set time associated from a specific field on source data or message

If the value is empty it will set the time to now. If an invalid date is given, an error will be thrown.

Parameters

Parameter	Type
val?	string | number | Date

Returns

void

---

setIngestTime()

setIngestTime(val?): void

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:103

Set the time at which the data was ingested into the source data

If the value is empty it will set the time to now. If an invalid date is given, an error will be thrown.

Parameters

Parameter	Type
val?	string | number | Date

Returns

void

---

setKey()

setKey(key): void

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:85

Set the unique document _key from the metadata. If no _key is found, an error will be thrown

Parameters

Parameter	Type
key	string | number

Returns

void

---

setMetadata()

setMetadata<K, V>(field, value): void

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:75

Given a field and value set the metadata on the record

Type Parameters

Type Parameter
K extends string | number | symbol
V

Parameters

Parameter	Type
field	K
value	V

Returns

void

---

setProcessTime()

setProcessTime(val?): void

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:117

Set the time at which the data was consumed by the reader

If the value is empty it will set the time to now. If an invalid date is given, an error will be thrown.

Parameters

Parameter	Type
val?	string | number | Date

Returns

void

---

setRawData()

setRawData(buf): void

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:141

Set the raw data, usually used for encoding type raw If given null, it will unset the data

Parameters

Parameter	Type
buf	null | string | Buffer<ArrayBufferLike>

Returns

void

---

toBuffer()

toBuffer(opConfig?): Buffer

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:148

Convert the DataEntity to an encoded buffer

Parameters

Parameter	Type	Description
opConfig?	EncodingConfig	The operation config used to get the encoding type of the buffer,

Returns

Buffer

Default

json

---

[hasInstance]()

static [hasInstance](instance): boolean

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:62

Parameters

Parameter	Type
instance	unknown

Returns

boolean

---

fork()

static fork<T>(input, withData?): T

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:33

Create a new instance of a DataEntity.

If the second param withData is set to true both the data and metadata will be forked, if set to false only the metadata will be copied. Defaults to true

Type Parameters

Type Parameter	Default type
T extends DataEntity<any, any>	DataEntity<Record<string, any>, _DataEntityMetadata<Record<string, any>>>

Parameters

Parameter	Type
input	T
withData?	boolean

Returns

T

---

fromBuffer()

static fromBuffer<T, M>(input, opConfig?, metadata?): DataEntity<T, M>

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:43

A utility for converting a Buffer to a DataEntity, using the DataEntity encoding.

Type Parameters

Type Parameter	Default type
T	Record<string, any>
M	Record<string, any>

Parameters

Parameter	Type	Description
input	string | Buffer<ArrayBufferLike>	A Buffer to parse to JSON
opConfig?	EncodingConfig	The operation config used to get the encoding type of the Buffer, defaults to "json"
metadata?	M	Optionally add any metadata

Returns

DataEntity<T, M>

---

getMetadata()

static getMetadata<V>(input, field?): undefined | V

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:61

Safely get the metadata from a DataEntity. If the input is object it will get the property from the object

DEPRECATED: Since this isn't recommended to be used, and will be removed in future releases.

Type Parameters

Type Parameter	Default type
V	any

Parameters

Parameter	Type
input	unknown
field?	string

Returns

undefined | V

Deprecated

---

isDataEntity()

static isDataEntity<T, M>(input): input is DataEntity<T, M>

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:47

Verify that an input is the DataEntity

Type Parameters

Type Parameter	Default type
T	Record<string, any>
M	Record<string, any>

Parameters

Parameter	Type
input	unknown

Returns

input is DataEntity<T, M>

---

isDataEntityArray()

static isDataEntityArray<T, M>(input): input is DataEntity<T, M>[]

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:51

Verify that an input is an Array of DataEntities,

Type Parameters

Type Parameter	Default type
T	Record<string, any>
M	Record<string, any>

Parameters

Parameter	Type
input	unknown

Returns

input is DataEntity<T, M>[]

---

make()

Call Signature

static make<T, M>(input, metadata?): T

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:17

A utility for safely converting an object a DataEntity. If the input is a DataEntity it will return it and have no side-effect. If you want a create new DataEntity from an existing DataEntity either use new DataEntity or shallow clone the input before passing it to DataEntity.make.

Type Parameters

Type Parameter	Default type
T extends DataEntity<any, any>	-
M	Record<string, any>

Parameters

Parameter	Type
input	T
metadata?	M

Returns

T

Call Signature

static make<T, M>(input, metadata?): DataEntity<T, M>

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:18

A utility for safely converting an object a DataEntity. If the input is a DataEntity it will return it and have no side-effect. If you want a create new DataEntity from an existing DataEntity either use new DataEntity or shallow clone the input before passing it to DataEntity.make.

Type Parameters

Type Parameter	Default type
T	Record<string, any>
M	Record<string, any>

Parameters

Parameter	Type
input	Record<string, any>
metadata?	M

Returns

DataEntity<T, M>

---

makeArray()

static makeArray<T, M>(input): DataEntity<T, M>[]

Defined in: packages/utils/dist/src/entities/data-entity.d.ts:24

A utility for safely converting an input of an object, or an array of objects, to an array of DataEntities. This will detect if passed an already converted input and return it.

Type Parameters

Type Parameter	Default type
T	Record<string, any>
M	Record<string, any>

Parameters

Parameter	Type
input	DataArrayInput

Returns

DataEntity<T, M>[]