Inference Portal

HTTP API

Base URL: https://inference.gille.ai/v1
Authentication: Authorization: Bearer <your-key>

List models your key can use:

curl https://inference.gille.ai/v1/models \
  -H "Authorization: Bearer $HS_KEY"

Public usage stats (no key needed):

curl https://inference.gille.ai/portal/stats

Returns content-blind grand totals only — {total_tokens, total_requests, since} — summed across everything the box has ever served (survives restarts). No per-user, per-key, or per-model detail is exposed. This powers the “Served so far” card above.

Chat completions (streaming — recommended):

curl https://inference.gille.ai/v1/chat/completions \
  -H "Authorization: Bearer $HS_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vibethinker-3b",
    "messages": [{"role": "user", "content": "Solve this carefully: 17 * 19"}],
    "temperature": 1.0,
    "top_p": 0.95,
    "top_k": 0,
    "min_p": 0,
    "max_tokens": 32768,
    "stream": true
  }'

Always use stream: true for longer completions — non-streaming requests are cut off after ~100 seconds by the network layer. Streaming keeps the connection alive and tokens arrive as they are generated.

OpenAI Python SDK:

from openai import OpenAI

client = OpenAI(
    base_url="https://inference.gille.ai/v1",
    api_key="hs_guest_...",
)
stream = client.chat.completions.create(
    model="vibethinker-3b",
    messages=[{"role": "user", "content": "Solve this carefully: 17 * 19"}],
    temperature=1.0,
    top_p=0.95,
    max_tokens=32768,
    extra_body={"top_k": 0, "min_p": 0},
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

The server enforces a per-key rate limit and a lifetime credit budget. Both are set when your invite is created and shown on your dashboard above. Ordinary models cap max_tokens at 12,288. Explicit vibethinker-3b requests may use up to 32,768; omitting the field keeps the safer 12,288 default. Standard OpenAI clients understand temperature and top_p. Send llama.cpp extensions top_k and min_p as extra request-body fields, as above.

Speech-to-text (transcription):

curl https://inference.gille.ai/v1/audio/transcriptions \
  -H "Authorization: Bearer $HS_KEY" \
  -F [email protected] \
  -F model=whisper-1

Send a multipart file (any common audio format). Optional: language (default auto-detect), response_format (json → {text}, verbose_json → full object with timings, text → plain body), and temperature (0–1). Unlike chat, this is billed per second of audio, not per token — up to 30 min / 32 MB per request. Any OpenAI Whisper client works unchanged.

Errors — what they mean and what to do:

HTTP	Code	Meaning	What to do
401	invalid_api_key	Missing, wrong, or revoked key	Check your key or ask the owner
402	credits_exhausted	Your credit budget is used up	Ask the owner for more credits
403	model_not_allowed	Your key is restricted to specific models and the requested model is not in its allow-list	Use a model from GET /v1/models
429	rate_limit_exceeded	You hit your RPM, TPM, or daily budget	Retry after the Retry-After seconds
503	server_busy	Box is busy — owner’s work has priority	Normal — retry after the Retry-After seconds
502	upstream_unavailable	Model backend is not reachable	Retry shortly (no Retry-After is sent)
504	upstream_timeout	Model took too long to respond	Retry after the Retry-After seconds
Mid-stream failure: if a streaming response is cut short, a terminal error frame is sent so your client knows the stream was truncated — not cleanly finished. Retry the request.

All errors use the standard OpenAI error envelope {"error": {"message": "...", "type": "...", "code": "..."}} so any OpenAI SDK’s error handling works without changes.

MCP (Claude Code)

Add the box to Claude Code as an MCP server in one command. Once added, Claude can offload self-contained sub-tasks — code generation, drafting, classification, summarization — to the local models as tools, keeping data on the box and saving frontier tokens.

claude mcp add --transport http local-llm \
  https://inference.gille.ai/mcp \
  --header "Authorization: Bearer hs_..."

The same key, credit budget, and model allow-list apply as for the HTTP API. Two tools are exposed:

Tool	What it does
list_models	Returns the models your key may use (same as GET /v1/models), each with a one-line strength hint to help Claude pick the right one.
ask	Runs a completion: {model, prompt, system?, max_tokens?, temperature?, top_p?, top_k?, min_p?} → returns the text. A model outside your allow-list, exhausted credits, a quota hit, or a busy box come back as a tool error with a clear message.

CLI (hs)

A zero-dependency command-line wrapper. Requires Node 18+. Served directly from this gateway at /hs — no GitHub account needed.

# one-liner install
curl -fsSL https://inference.gille.ai/hs -o ~/bin/hs && chmod +x ~/bin/hs
# or via npm: npm install -g gille-inference  (or: npx gille-inference ask "...")

Commands:

Command	What it does
hs redeem inv_…	Trade an invite code for a key and save credentials to ~/.config/hs/config.json
hs login --url <base> --key <key>	Save an existing key manually
hs models	List the models your key may use
hs ask [-m <model>] [--system <s>] <prompt>	Ask a question — streams tokens as they arrive
hs usage	Show your tier, allowed models, and credit usage (calls GET /portal/me)
hs whoami	Show stored base URL and masked key from ~/.config/hs/config.json

Under the hood hs ask calls POST /v1/chat/completions with streaming, and hs models calls GET /v1/models — so you can swap it for any OpenAI-compatible SDK at any time.