UniLM.jl

A unified Julia interface for large language models.

What is UniLM.jl?

UniLM.jl provides a Julian, type-safe interface to LLM providers with first-class native backends — OpenAI (Chat Completions + Responses), Anthropic (Messages), and Google Gemini (generateContent + agentic Interactions) — plus any OpenAI-compatible provider (Azure, DeepSeek, Mistral, Ollama, vLLM, LM Studio). It covers Chat Completions & Responses, a cross-provider agentic respond verb, Image Generation/Edits, Embeddings, Files/Vector Stores, Conversations, Audio, Batch, Moderations, Fine-tuning, Webhooks, Realtime, and MCP (client & server) — with built-in token/cost accounting.

Key Features

• 🗣️ Chat Completions — stateful conversations with automatic history management
• 🔮 Responses API & Agentic Verb — OpenAI's Responses API plus a cross-provider respond verb that also drives Google's Gemini Interactions
• 🖼️ Image Generation & Edits — create and edit images with gpt-image-2
• 🔧 Tool/Function Calling — first-class function tools in both APIs, with an automated tool_loop
• 🔌 MCP (Model Context Protocol) — connect to MCP servers or build your own, with seamless tool-loop integration
• 📊 Embeddings — text embedding generation
• 💰 Cost & Token Accounting — per-call estimated_cost, per-Chat cumulative_cost, and a built-in multi-provider pricing table
• 🌊 Streaming — real-time token streaming with do-block syntax
• 📐 Structured Output — JSON Schema–constrained generation
• ☁️ Multi-Backend — native OpenAI/Anthropic/Gemini plus Azure, DeepSeek, Ollama, Mistral, vLLM, LM Studio, and any OpenAI-compatible provider
• ✅ Type Safety & Capability Introspection — invalid states are unrepresentable and unsupported requests fail fast via provider_capabilities; tested with JET.jl and Aqua.jl

Chat Completions vs Responses (OpenAI)

For OpenAI, use either conversational API — Chat Completions (Chat; also the path for the native Anthropic/Gemini and OpenAI-compatible backends) or the newer Responses (respond; basis for the cross-provider agentic verb):

Installation

UniLM requires Julia 1.12+ and is registered in Julia's General registry:

using Pkg
Pkg.add("UniLM")

Or in the Pkg REPL:

pkg> add UniLM

For the latest unreleased changes, install from GitHub instead:

Pkg.add(url="https://github.com/algunion/UniLM.jl")

Quick Example

Building requests — these construct objects locally without calling the API:

using UniLM
using JSON

# Chat Completions request
chat = Chat(model="gpt-5.2")
push!(chat, Message(Val(:system), "You are a Julia expert."))
push!(chat, Message(Val(:user), "Explain multiple dispatch in one sentence."))
println("Chat has ", length(chat), " messages, model: ", chat.model)
println("Request body preview:")
println(JSON.json(chat))

Chat has 2 messages, model: gpt-5.2
Request body preview:
{"messages":[{"content":"You are a Julia expert.","role":"system"},{"content":"Explain multiple dispatch in one sentence.","role":"user"}],"model":"gpt-5.2"}

# Responses API request
r = Respond(input="What makes Julia special?")
println("Respond model: ", r.model)
println(JSON.json(r))

Respond model: gpt-5.6-sol
{"input":"What makes Julia special?","model":"gpt-5.6-sol"}

# Image Generation request. `model` is a sentinel that resolves at serialization
# time, so the field itself stays "" — read the body to see what gets sent.
ig = ImageGeneration(prompt="A watercolor Julia logo", quality="high")
println("Image model field: ", repr(ig.model))
println(JSON.json(ig))

Image model field: ""
{"model":"gpt-image-2","prompt":"A watercolor Julia logo","quality":"high"}

With a valid API key, actual API calls return structured results:

Responses API (recommended for new code):

result = respond("Explain Julia's multiple dispatch in 2-3 sentences.", model="gpt-5.4-mini")
if result isa ResponseSuccess
    println(output_text(result))
else
    println("Request failed — ", output_text(result))
end

Request failed — Error: KeyError: key "OPENAI_API_KEY" not found

Chat Completions:

chat = Chat(model="gpt-5.4-mini")
push!(chat, Message(Val(:system), "You are a concise Julia programming tutor."))
push!(chat, Message(Val(:user), "What is multiple dispatch? Answer in 2-3 sentences."))
result = chatrequest!(chat)
if result isa LLMSuccess
    println(result.message.content)
else
    println("Request failed — see result for details")
end

Request failed — see result for details

Image Generation:

result = generate_image(
    "A watercolor painting of a friendly robot reading a Julia programming book",
    size="1024x1024", quality="medium"
)
println("Success: ", result isa ImageSuccess)
if result isa ImageSuccess
    save_image(image_data(result)[1], joinpath(@__DIR__, "assets", "generated_robot.png"))
    println("Saved to assets/generated_robot.png")
else
    println("Image generation failed — see result for details")
end

Next Steps

• Getting Started — setup and first requests
• Chat Completions Guide — deep dive into Chat and chatrequest!
• Responses API Guide — the newer Responses API
• Image Generation Guide — create images from text prompts
• MCP Guide — connect to MCP servers or build your own
• Timeouts & Retries — RequestConfig, typed timeout failures, retry and concurrency contracts
• API Reference — full type and function reference

Platform APIs

Beyond chat and generation, UniLM wraps the full OpenAI platform surface (OpenAI-only) — each has an API-reference page:

• Storage & retrieval — Files, Vector Stores, Conversations, Uploads, Containers
• Generation & media — Audio, Videos, Images
• Jobs & ops — Batch, Fine-tuning, Moderations, Webhooks, Realtime
• Cross-cutting — Cost Tracking, Provider Capabilities, Retrieval & File Search