Class: TransformersJsAdapter
Defined in: src/batteries/llm/transformers_js/adapter.ts:342
Dual-environment executor adapter for transformers.js text generation.
Remarks
Construct with at least { model }; wire new TransformersJsAdapter(opts).executor() into a DispatchRunner as the executorCallback. The pipeline is resolved lazily on first dispatch (or eagerly via TransformersJsAdapter.preload); pass pipeline to inject a pre-built one.
Constructors
Constructor
new TransformersJsAdapter(options: unknown): TransformersJsAdapter;Defined in: src/batteries/llm/transformers_js/adapter.ts:370
Parameters
| Parameter | Type | Description |
|---|---|---|
options | unknown | Raw adapter options, validated against transformersJsOptionsSchema. |
Returns
TransformersJsAdapter
Throws
@nhtio/adk/batteries!E_INVALID_TRANSFORMERS_JS_OPTIONS when options are invalid.
Properties
| Property | Modifier | Type | Description | Defined in |
|---|---|---|---|---|
STASH_KEY | readonly | "transformersJs" | The ctx.stash key under which per-dispatch option overrides are read. | src/batteries/llm/transformers_js/adapter.ts:344 |
Methods
dispose()
dispose(): Promise<void>;Defined in: src/batteries/llm/transformers_js/adapter.ts:414
Release the loaded model's underlying ONNX sessions + GPU/wasm buffers, then drop all cached references (so the next dispatch re-resolves a fresh pipeline).
Returns
Promise<void>
Remarks
reset() only nulls the JS references — it does NOT free the native ONNX Runtime sessions or the WebGPU/wasm device memory they hold. Those leak until GC, and in a browser session that loads many models back-to-back (e.g. a full matrix run) the accumulated sessions exhaust the heap, surfacing as Can't create a session … Failed to load external data file … memory copy. transformers.js exposes PreTrainedModel.dispose() ("disposes of all the ONNX sessions created during inference") and Pipeline.dispose() — this awaits them so the memory is actually reclaimed between loads. Settles any in-flight load first, swallows per-handle disposal errors (a half-loaded model must not throw out of teardown), and finishes with reset(). Idempotent and safe to call when nothing is loaded.
executor()
executor(overrides?: Partial<TransformersJsAdapterOptions>): DispatchExecutorFn;Defined in: src/batteries/llm/transformers_js/adapter.ts:686
Produce the bound DispatchExecutorFn the DispatchRunner invokes.
Parameters
| Parameter | Type | Description |
|---|---|---|
overrides? | Partial<TransformersJsAdapterOptions> | Option overrides layered above the constructor baseline (below ctx.stash). |
Returns
isAvailable()
isAvailable(): boolean;Defined in: src/batteries/llm/transformers_js/adapter.ts:376
Instance availability probe (honours the isAvailable option override).
Returns
boolean
preload()
preload(overrides?: Partial<TransformersJsAdapterOptions>): Promise<TextGenerationPipeline>;Defined in: src/batteries/llm/transformers_js/adapter.ts:385
Eagerly resolve (load) the pipeline before the first dispatch.
Parameters
| Parameter | Type | Description |
|---|---|---|
overrides? | Partial<TransformersJsAdapterOptions> | Optional option overrides applied for this load. |
Returns
Promise<TextGenerationPipeline>
recycle()
recycle(overrides?: Partial<TransformersJsAdapterOptions>): Promise<void>;Defined in: src/batteries/llm/transformers_js/adapter.ts:452
Free the WebGPU buffer cache by releasing the model's ONNX sessions, then reload the same model.
Parameters
| Parameter | Type | Description |
|---|---|---|
overrides? | Partial<TransformersJsAdapterOptions> | Optional option overrides applied to the reload (same as preload). |
Returns
Promise<void>
Remarks
The consumer-facing lever for the ONNX Runtime Web WebGPU buffer-freelist high-water-mark (see @nhtio/adk/batteries!probeGpuBudget and the battery's GPU-budget notes). ORT-web parks freed activation buffers in per-size buckets sized to the largest tensor shape the model has run; the pool is flushed ONLY when every session of the model is released (ORT clears the cache at sessionCount === 0; microsoft/onnxruntime#22490). There is no public flag to flush it mid-life, so the supported way to reclaim that retained working-set without permanently unloading the model is to dispose the sessions and load again.
This is exactly dispose() followed by preload() — surfaced as a named method because "recycle to free the GPU buffer cache" is a distinct, intentional operation (e.g. an application offering a "free GPU memory" action after a @nhtio/adk/batteries!E_LLM_GPU_OUT_OF_MEMORY), not a teardown. It is NOT invoked automatically by the battery — the ADK surfaces the lever and leaves the decision to the consumer. Re-incurs the cold-load cost (download is cached; the WebGPU graph/shader compile is not). Idempotent.
reset()
reset(): void;Defined in: src/batteries/llm/transformers_js/adapter.ts:393
Drop the cached pipeline/engine and any in-flight load so the next dispatch re-resolves it.
Returns
void
isAvailable()
static isAvailable(): boolean;Defined in: src/batteries/llm/transformers_js/adapter.ts:362
Whether this battery is available. transformers.js is environment-neutral (Node + browser), so this is true whenever the runtime can import the peer — there is no WebGPU requirement. Static form returns true; the instance form honours an injected isAvailable override.
Returns
boolean