@terascope/data-mate / teal-frame/append
teal-frame/append
Functions
appendInTransaction()
appendInTransaction(
instance,config,table,source,options):Promise<number>
Defined in: data-mate/src/teal-frame/append.ts:142
Runs one TealFrame.append as an atomic transaction on its own connection.
The separate connection is required for correctness. A DuckDB transaction belongs to a
connection, and the shared connection is used by the whole process. If two appends shared it,
the second BEGIN would throw, its ROLLBACK would discard the first append's rows, and every
later statement would fail with Current transaction is aborted.
With one connection per append, concurrent appends to the same table are safe: DuckDB's MVCC lets appends proceed without write conflicts.
Parameters
| Parameter | Type |
|---|---|
instance | DuckDBInstance |
config | FrameConfig |
table | string |
source | AppendSource |
options | AppendOptions |
Returns
Promise<number>
appendParquet()
appendParquet(
connection,table,paths):Promise<number>
Defined in: data-mate/src/teal-frame/append.ts:47
Appends Parquet files with INSERT ... BY NAME.
Columns are matched by name because a positional INSERT ... SELECT * fails when a file's
column order differs from the table's, and separately produced files may not agree on order.
No coercion is needed: Parquet is typed, and the values were coerced by fromRecords when
the file was written.
Parameters
| Parameter | Type |
|---|---|
connection | DuckDBConnection |
table | string |
paths | string | readonly string[] |
Returns
Promise<number>
appendRecords()
appendRecords(
connection,config,table,records,options):Promise<number>
Defined in: data-mate/src/teal-frame/append.ts:64
Coerces records with coerceToType and appends them through a typed DuckDB appender.
strict (the default) throws on the first bad value, as DataFrame's builder does.
lenient stores null instead and collects each failing field with a count and an example.
Parameters
| Parameter | Type |
|---|---|
connection | DuckDBConnection |
config | FrameConfig |
table | string |
records | readonly Record<string, unknown>[] |
options | AppendOptions |
Returns
Promise<number>
countQuietly()
countQuietly(
connection,table):Promise<number|undefined>
Defined in: data-mate/src/teal-frame/append.ts:24
Counts a table's rows without throwing. Used while reporting a failure, where a second error would hide the original one.
Parameters
| Parameter | Type |
|---|---|
connection | DuckDBConnection |
table | string |
Returns
Promise<number | undefined>