Six ways a cheap AI call breaks
Free and low-cost models come and go, and gateways have bad days. Every tool that calls them ends up rewriting the same defensive code. Lodestone does it once, for all your tools.
The model disappears
A hard-coded model id stops working without warning when the provider withdraws it.
One model eats the clock
A single slow model uses the whole serverless time limit and the person sees an unreadable gateway error.
"429" means four things
Daily limit, per-minute limit, busy upstream, or no credit. Each needs a different message to the person.
explain() turns each into advice a person can act on.The provider itself is down
One gateway's outage takes every model behind it, and every tool that depends on it.
The same dead model, every time
Free models fail in bursts, and without memory every request pays to find out again.
"Free" is not free if it hangs
A paid fallback with no limit is risky on a public tool, and free models that never answer cost your users time.
Pick a bad day and see what Lodestone does
A call runs through up to three phases until one answers: OpenRouter's free models, a low-cost paid model, then your fallback providers. These are the real rules; the timings and model names are illustrative, except where a scenario says it was measured.
What one bad day looked like
On 28 October 2026 the free OpenRouter models behind a real tool stopped answering, and scoring a job posting ran its whole 24 s budget and failed. These are the measurements that changed how Lodestone's defaults work. One day, one posting and one CV, so read them as an example, not a benchmark.
1 usable answer in 15 attempts
Three attempts each on five free models. Two were rate-limited every time (429), two held the connection open for 45 s and never answered, and the router once chose a safety classifier that replied "User Safety: safe". The one good answer took 25 s.
About 3 s, for under a tenth of a cent
Reading a posting: claude-haiku-5.5 took 3.0–3.2 s in all three runs for $0.0004. Scoring took about 5 s. Gemini was 5–6 s; Mistral Small was erratic (7–9 s, and one rate limit).
Time until the whole answer
My first speed table timed only until the response headers arrived, which made every model look much faster. OpenRouter sends headers early and the body follows seconds later. Lodestone's own tests and tools now time the full body.
| Scoring a CV against a posting | Time | Cost | Note |
|---|---|---|---|
claude-haiku-5.5, reasoning left on | 10 s | $0.0012 | Spent about 1,000 tokens thinking, even when asked for "low" |
claude-haiku-5.5, reasoning off | 5 s | $0.0006 | Same answers on both runs, so half the time and half the cost |
gemini-3.8-flash | 5–6 s | $0.003 | Refuses to run with reasoning off ("mandatory for this endpoint") |
qwen3.7-flash | n/a | n/a | Spent all 3,500 tokens thinking and returned nothing |
So reasoning is set per model (openrouter({ reasoning })), a paid-first mode exists (paidFirst), and the free models become the backup. The result for that tool: reading a posting in 2.6–3.1 s and scoring in 4.5–4.9 s, instead of failing after 24 s.
Other providers, each with its own key
Any service that speaks OpenAI's chat API works through openaiCompatible(), and presets exist for the ones Lodestone has been checked against. Here is exactly how far each check went.
| Preset | Service | What was checked |
|---|---|---|
requesty() | Requesty | checked model list read live (807 chat models, with price, lab and data policy) chat call not made: no key |
llmapi() | LLM API | checked model list read live (152 chat of 393 models; image, video, speech and embedding ignored) chat call not made: no key |
groq(), cerebras() | Groq, Cerebras | partly endpoint answers (needs a key); free-tier terms not checked untested with a key |
openai(), mistral(), together() | the providers themselves | partly endpoint answers (needs a key) untested with a key |
ollama() | a local server | partly the standard local endpoint untested here |
(built in) openrouter | OpenRouter | checked used in production for the default path catalogue, calls, cost and expiry dates all seen live |
anthropic(), google() | direct | partly request shape tested against the documented format not called live |
"Model list read" means Lodestone's reader was run against the service's real public list and parsed it correctly. No chat call was made to a service without a key, so those paths are tested against fakes of the documented request, not against the service itself. Requesty and LLM API have no free models; they add redundancy on the paid path. Free redundancy comes from providers with free tiers, such as Groq and Cerebras, whose terms Lodestone has not checked.
Discovery that never makes a request slower
A model list that nobody edits goes stale; one that fills itself can grow until it is slow, or admit something you would not choose. Lodestone does the first and bounds the second.
Outdated models drop out
A daily checkModels() saves each provider's catalogue. A model it no longer lists is dropped. One with an expiry date within a week is left out (OpenRouter publishes the date; 14 models carried one the day this was written). One failing right now is skipped by the breaker.
A pool of 8, three tried
The ranked pool lives in the background. Models that are down are filtered out, and a request tries at most three. When one fails, the next best takes its place for later requests, so adding models adds choice, not delay.
Trust is checked, not assumed
A discovered model must come from a trusted lab. You can block labs or models, refuse any a provider says is trained on or keeps prompts too long, and mark a single call private.
Illustrative: a pool of eight. The two that are down are skipped; the next three live ones are tried.
What the trust policy decides
| Model found by discovery | Default policy | Strict: no training, prompts kept ≤ 30 days |
|---|---|---|
google/gemma-4-31b-it:freeOpenRouter, publishes nothing on data use | used | refused: provider does not say what happens to prompts |
shady-lab/free-llm:freeOpenRouter, a lab nobody has heard of | refused: lab not on the trusted list | refused: provider does not say what happens to prompts |
poolside/laguna-s-2.1:freeOpenRouter, retiring on 31 Oct 2026 | refused: retiring within a week | refused: retiring within a week |
bedrock/claude-haiku-5-5Requesty: not trained on, prompts kept 0 days | used | used |
openai/gpt-6.1-terra:flexRequesty: marked as trained on | used | refused: trained on |
gemini-3.8-flashLLM API, family "gemini-flash", publishes nothing on data use | used | refused: provider does not say what happens to prompts |
Verdicts are computed by the library's own vet() as this page is built, on rows shaped like real lists. On 28 Oct 2026, of Requesty's 807 chat models, 205 came from a lab not on the default list and 9 were marked as trained on. LLM API publishes nothing about what happens to prompts, so under a strict policy none of its models can be discovered; models you name yourself still work.
What this is not. There is no vulnerability database for hosted AI models, so Lodestone does not claim to scan one. It checks what providers actually publish, and enforces your rules on it. Those facts are self-reported by providers. SECURITY.md lists exactly what is and is not checked.
Eleven kinds of failure, each with its own advice
complete() and json() throw a LodestoneError with one of these kind values. Tap one to read the sentence explain(err, { product: 'MyApp', settingsHint: 'in Settings' }) gives a person.
The sentences are the library's own: this page calls explain() for each kind when it is built, so they cannot drift from the code.
Scorecards: let measurements choose the order
Two small JSON formats let an evaluator and Lodestone feed each other without sharing code. Naija Eval, which measures translation and speech quality for Hausa, Igbo and Yoruba, is the first to use them. Its scores are translation quality, so they help tasks in those languages, not every task.
Evaluator (Naija Eval)
Measures answer quality, such as chrF++ for English→Yoruba, per model.
Lodestone (in your tool)
Tries the best-measured models first for tasks you name, lists the models it can call, and records its own success rates and speeds.
- A scorecard only changes the order models are tried in. Nothing is added or dropped.
- Scores from fewer than 30 items, or over a year old, are ignored.
- Unmeasured models keep their slot, so a good unmeasured model is never pushed down.
- An explicit
routeis never reordered. - If the scorecard fetch fails or takes over 1.5 s, the order is left alone.
- Live stats keep the last 200 attempts per model and never see prompts or answers.
What a tool writes
// set up once, and share the store const ai = createLodestone({ openrouter: { apiKey: OPENROUTER_KEY, allowPaid: true, paidFirst: true, paidModels: ['anthropic/claude-haiku-5.5'], reasoning: (m) => m.startsWith('anthropic/') ? { enabled: false } : { effort: 'low', exclude: true } }, fallbacks: [ { provider: requesty(), apiKey: REQUESTY_KEY, models: ['bedrock/claude-haiku-5-5'], discover: { fill: { maxPromptUsdPerMTok: 1, maxCompletionUsdPerMTok: 5 } } }, { provider: groq(), apiKey: GROQ_KEY, models: ['llama-3.3-70b-versatile'], free: true }, ], store, health: createHealth({ store }), spend: { allow: underDailyLimit, record: addToLedger }, trust: { requireNoTraining: true }, }); // each call names a task, not a model try { const { data } = await ai.json({ prompt, validate: isJob, signal: req.signal, private: true }); // a CV: do not keep or train on it } catch (err) { return explain(err, { product: 'MyApp' }); }
- Tiers, not model ids. A task maps to
fast,strongorvision(picked automatically when a message has an image). Or pass a fixedrouteto name provider, model and key yourself. - Customer keys stay separate. A caller's own key replaces yours and unlocks the paid model, but your fallback providers and your spend ledger are never touched by that call.
- Daily upkeep.
checkModels()refreshes every provider's catalogue and reports what has gone or is retiring.cacheKeyandpeek()reuse answers for repeatable tasks. - Cancelling. Pass a
signal; when the person leaves, the call and every model after it stop. - Private by design.
onAttemptreports provider, model, outcome and timing only. Prompts and answers are never logged.
What it does not do yet
Streaming
complete() and json() wait for the whole answer. Streaming needs a rule for a model that fails halfway; the likely one is to fail over only before the first token.
Tool calling, embeddings, speech
Lodestone is for one-shot text and JSON calls today. It does not let a model call functions, and it has no tier for embeddings or speech.
Other runtimes
It uses only fetch, AbortSignal and URL, with no Node-specific APIs, so it should run on Workers, Deno and edge runtimes. It has only been tested on Node 20 and 22.
Where each piece lives in src/
Generated from the files in the repository when this page is built. +tests marks a file with its own test file; the others are covered by a shared one or by the call-path tests.
candidates.tsmodel-candidates/1: the models in use, for an evaluator
catalog.ts+testsReads a provider's catalogue; picks and ranks models; drops withdrawn ones
discover.ts+testsKeeps a provider's list alive and short: prune, then fill within a price limit
errors.tsLodestoneError, ProviderError and the failure kinds
explain.tsTurns an error into a sentence a person can read
health.ts+testsThe circuit breaker: skips models that keep failing, shared across instances
index.tsPublic exports
json.ts+testsStrips code fences and parses; json() uses it to skip bad answers
lodestone.ts+testsThe core: createLodestone, the phases, the time budget, complete() and json()
models.tsDefault free model lists per tier
rateLimit.ts+testsTells the four kinds of 429 apart
scorecard.ts+testsReads model-scorecard/1 and orders models by it
stats.ts+testsLive success rate and speed per model, as a scorecard
store.tsA key-value store: memory by default, yours to share between functions
trust.ts+testsThe trust policy: trusted labs, block list, privacy and expiry checks
types.tsShared types
providers/Adapters: anthropic, google, mock, openaiCompatible, openrouter
examples/A Netlify function, the daily check and a multi-provider setup, all type-checked in CI
.github/workflowsTests, builds and publishes to npm with provenance on a version tag
docs/scorecards.mdThe two JSON formats in full