Class: LiteRtLmAdapter

Defined in: src/batteries/llm/litert_lm/adapter.ts:296

Cross-environment executor adapter for LiteRT-LM.

Remarks

Construct with at least { model }; wire new LiteRtLmAdapter(opts).executor() into a DispatchRunner as the executorCallback. The engine is resolved lazily on first dispatch (or eagerly via LiteRtLmAdapter.preload); pass engine to inject a pre-built one (e.g. in tests).

Constructors

Constructor

new LiteRtLmAdapter(options: unknown): LiteRtLmAdapter;

Defined in: src/batteries/llm/litert_lm/adapter.ts:319

Parameters

Parameter	Type	Description
options	unknown	Raw adapter options, validated against liteRtLmOptionsSchema.

Returns

LiteRtLmAdapter

Throws

@nhtio/adk/batteries!E_INVALID_LITERT_LM_OPTIONS when options are invalid.

Properties

Property	Modifier	Type	Description	Defined in
STASH_KEY	readonly	"liteRtLm"	The ctx.stash key under which per-dispatch option overrides are read.	src/batteries/llm/litert_lm/adapter.ts:298

Methods

dispose()

dispose(): Promise<void>;

Defined in: src/batteries/llm/litert_lm/adapter.ts:356

Release the loaded engine's native resources (Engine.delete()), then drop the cached reference.

Returns

Promise<void>

Remarks

reset() only nulls the JS reference; the LiteRT engine holds a WebGPU device + compiled graph that stay alive until GC. Loading many .litertlm engines back-to-back in one browser session accumulates those until the GPU heap is exhausted. @litert-lm/core's Engine exposes delete() — this settles any in-flight load, awaits delete(), swallows a teardown error (teardown must not throw), and finishes with reset(). Idempotent and safe when nothing is loaded.

---

executor()

executor(overrides?: Partial<LiteRtLmAdapterOptions>): DispatchExecutorFn;

Defined in: src/batteries/llm/litert_lm/adapter.ts:549

Produce the bound DispatchExecutorFn the DispatchRunner invokes.

Parameters

Parameter	Type	Description
overrides?	Partial<LiteRtLmAdapterOptions>	Option overrides layered above the constructor baseline (below ctx.stash).

Returns

DispatchExecutorFn

---

isAvailable()

isAvailable(): boolean;

Defined in: src/batteries/llm/litert_lm/adapter.ts:325

Instance WebGPU-availability probe (honours the isWebGPUAvailable option override).

Returns

boolean

---

preload()

preload(overrides?: Partial<LiteRtLmAdapterOptions>): Promise<Engine>;

Defined in: src/batteries/llm/litert_lm/adapter.ts:335

Eagerly resolve (load) the engine before the first dispatch.

Parameters

Parameter	Type	Description
overrides?	Partial<LiteRtLmAdapterOptions>	Optional option overrides applied for this load.

Returns

Promise<Engine>

The resolved LiteRtLmEngine.

---

recycle()

recycle(overrides?: Partial<LiteRtLmAdapterOptions>): Promise<void>;

Defined in: src/batteries/llm/litert_lm/adapter.ts:384

Free the WebGPU buffer cache by deleting the engine, then reload the same model.

Parameters

Parameter	Type	Description
overrides?	Partial<LiteRtLmAdapterOptions>	Optional option overrides applied to the reload (same as preload).

Returns

Promise<void>

Remarks

The consumer-facing lever for the ONNX Runtime Web WebGPU buffer-freelist high-water-mark (see @nhtio/adk/batteries!probeGpuBudget). The pool is flushed only when the engine's sessions are released; there is no public flag to flush it mid-life, so the supported way to reclaim the retained working-set without permanently unloading is to delete the engine and load again. This is exactly dispose() then preload(), surfaced as a named, intentional operation (e.g. an application offering a "free GPU memory" action after a @nhtio/adk/batteries!E_LLM_GPU_OUT_OF_MEMORY). NOT invoked automatically — the ADK surfaces the lever and leaves the decision to the consumer. Re-incurs the cold-load cost. Idempotent.

---

reset()

reset(): void;

Defined in: src/batteries/llm/litert_lm/adapter.ts:341

Drop the cached engine and any in-flight load so the next dispatch re-resolves it.

Returns

void

---

isAvailable()

static isAvailable(): boolean;

Defined in: src/batteries/llm/litert_lm/adapter.ts:307

Returns true when the current runtime exposes WebGPU (navigator.gpu).

Returns

boolean