Scripts

A collection of terascope monorepo scripts

Installation

# Using pnpm
pnpm add -g @terascope/scripts
# Using npm
npm install --global @terascope/scripts

Test Runner

Many tests require the use of various database's for testing. Built into the test framework we provide the means to test against supported databases including opensearch, kafka, minio(S3) and rabbitmq.

To enable these databases into the tests include the appropriate environmental variables to the test scripts in the package.json

Example:

"test": "TEST_OPENSEARCH='true' ts-scripts test . --",
"test:opensearch2": "TEST_OPENSEARCH='true' OPENSEARCH_VERSION='2.15.0' ts-scripts test . --",

In the above example, we specify TEST_OPENSEARCH which alerts the test harness to setup and wait for an opensearch instance to be available before any tests are run. It defaults to v2.15.0 which can be overridden by the OPENSEARCH_VERSION variable

All defaults and variables that are used can be found at /packages/scripts/src/helpers/config.ts file. These values are exported and a few of them are set in the test process environment for client setup.

Multiple databases can be setup at the same time by using the correct environmental variables

"test": "TEST_OPENSEARCH='true' TEST_KAFKA='true' ts-scripts test --suite e2e --",

In the above scenario, both opensearch and kafka are setup before the tests are run

List of environmental variables to setup a database:

• "TEST_OPENSEARCH"
• "TEST_KAFKA"
• "TEST_MINIO"
• "TEST_RESTRAINED_OPENSEARCH" (this contains bulk queue limits to test api bulk overflows)
• "TEST_RABBITMQ"
• "TEST_TERASLICE"

Asset E2E Testing with Teraslice

This section applies to asset repos that need to validate themselves against a running Teraslice instance. This is not for testing Teraslice itself.

Setting TEST_TERASLICE=true enables e2e testing of assets against a live Teraslice instance. When enabled, the test harness will:

1. Start an OpenSearch instance (required by Teraslice)
2. Fetch the latest Teraslice image from GHCR for the current Node.js major version (or use a pinned image via TERASLICE_IMAGE)
3. Start a Teraslice container configured to connect to the OpenSearch and any other services in the test suite
4. Wait for the Teraslice API to become healthy before running tests

The TERASLICE_HOST environment variable is set automatically and made available to tests for connecting to the Teraslice API.

Example package.json test script for an asset:

"test:e2e": "earl assets build --overwrite && ASSET_ZIP_PATH=$(./scripts/asset-zip-path.sh) TEST_TERASLICE=true ts-scripts test -- --testPathPatterns=test/e2e"

./scripts/asset_zip_path.sh provides the path to the zipped asset within ./build that matches the asset version and node version of the terminal running the script.

Environment Variables

Variable	Description	Default
TEST_TERASLICE	Start a Teraslice instance for testing	false
TERASLICE_IMAGE	Pin a specific Teraslice Docker image (e.g. ghcr.io/terascope/teraslice:v3.4.2-nodev24.14.0). If not set, the latest release is fetched from GitHub.	null
TERASLICE_PORT	Port to expose Teraslice on the host	45678
ASSET_ZIP_PATH	Path to the built asset .zip file to upload during tests	null
TERASLICE_DOCKER_VOLUME_PATHS	Comma-separated list of absolute host paths to local packages to inject into the Teraslice container. See Dev Package Injection below.	null

Dev Package Injection

By default, e2e tests run against a pre-built Teraslice release image. This means bugs introduced in a local connector or library dependency may not be caught — the pre-built image ships its own copy of those packages.

TERASLICE_DOCKER_VOLUME_PATHS solves this by injecting local packages directly into the Teraslice container at test time. When this variable is set, the test harness:

1. Builds a thin wrapper image on top of the base Teraslice image
2. Mounts each listed host path into the container under /mnt/dev/<package-basename>
3. Rewrites all matching package.json dependency entries inside the container to file: references pointing at the mounts
4. Runs pnpm install to resolve the rewritten dependencies before starting Teraslice

This ensures the running Teraslice instance uses exactly the local code under test, not a cached registry version.

Example — testing a local Kafka connector against Teraslice:

# In an asset repo (e.g. kafka-assets), point to the connector package directory
TERASLICE_DOCKER_VOLUME_PATHS=/path/to/kafka-assets/packages/terafoundation_kafka_connector \
TEST_TERASLICE=true \
ts-scripts test -- --testPathPatterns=test/e2e

Or in package.json:

"test:e2e": "TERASLICE_DOCKER_VOLUME_PATHS=$(pwd)/packages/terafoundation_kafka_connector TEST_TERASLICE=true ts-scripts test -- --testPathPatterns=test/e2e"

Multiple packages can be injected at once using a comma-separated list:

TERASLICE_DOCKER_VOLUME_PATHS=/path/to/pkg-a,/path/to/pkg-b TEST_TERASLICE=true ts-scripts test -- --testPathPatterns=test/e2e

Each path must point to a directory containing a package.json with a name field. The name is used to find matching dependency entries across all package.json files inside the container.