LLM Chat

Connect to any OpenAI-compatible API — OpenAI, Groq, Together AI, Mistral, Ollama, LM Studio, or any local/remote provider. Configure model parameters and view response metrics. Your API key stays in your browser.

Selects default URL and model. You can override below.

Settings are saved as cookies on your local machine. Nothing is sent to any server other than the API endpoint above.

0.50
1.00
0.0
0.0

Not all providers support every parameter. Unsupported fields are typically ignored.

The system prompt is prepended to the conversation history on each request. It is not stored in the chat.

Send a message to see response metrics.

Code

const chat = new quikchat('#chat', handleLLMRequest);

async function handleLLMRequest(chatInstance, userInput) {
  chatInstance.messageAddNew(userInput, 'user', 'right');

  const response = await fetch(baseUrl, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': 'Bearer ' + apiKey
    },
    body: JSON.stringify({
      model: model,
      messages: [
        { role: 'system', content: systemPrompt },
        ...chatInstance.historyGet()
      ],
      temperature: 0.5,
      max_tokens: 2048,
      stream: true,
      stream_options: { include_usage: true }
    })
  });

  const reader = response.body.getReader();
  let start = true, id;

  // Read SSE stream token-by-token
  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    // parse chunks, extract content, append to message
    if (start) {
      id = chatInstance.messageAddNew(content, 'bot', 'left');
      start = false;
    } else {
      chatInstance.messageAppendContent(id, content);
    }
    // final chunk may contain usage: { prompt_tokens, completion_tokens }
  }
}