UMAP projection of VoyageAI-4 embeddings for session titles and descriptions

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Retrieval Augmented Generation based on SQLite. Contribute to ggozad/haiku.rag development by creating an account on GitHub.

Model Choices Voyage currently provides the following text embedding models: Model Context Length (tokens) Embedding Dimension Description voyage-4-large 32,000 1024 (default), 256, 512, 2048 The best...

TL;DR – We’re excited to introduce voyage-context-3, a contextualized chunk embedding model that produces vectors for chunks that capture the full document context without any manual metadata…

Model Choices Voyage currently provides the following text embedding models: Model Context Length (tokens) Embedding Dimension Description voyage-3-large 32,000 1024 (default), 256, 512, 2048 The best...

The Laravel AI SDK (`laravel/ai`, v0.8.1) ships with built-in support for 14 AI providers: OpenAI, Anthropic, Google Gemini, Groq, Mistral, DeepSeek, xAI, Ollama, Azure OpenAI, Cohere, OpenRouter, Jina, VoyageAI, and ElevenLabs. That breadth is genuinely useful — but unlocking it correctly requires more than swapping an env variable. This article covers how to configure multiple providers in the same application, switch between them at runtime, handle provider-specific failures, and test the whole setup without hitting a live API. ## Prerequisites * PHP 8.3+ * Laravel 12 or 13 * `laravel/ai` v0.8 (pin to `^0.8` in `composer.json` until v1.0 ships — the package is still pre-1.0) * At least two provider API keys (we'll use OpenAI and Anthropic for the walkthrough) Install the SDK if you haven't already: composer require laravel/ai php artisan vendor:publish --tag=ai-config ## How the SDK Resolves Providers When you call `AI::text()->using('gpt-4o-mini')->...`, the SDK maps the model string to a provider using the published `config/ai.php`. The default config reads a single `AI_PROVIDER` env variable. That's fine for a simple setup, but in a multi-provider application you need named provider instances instead. Here is the relevant section of `config/ai.php` after customisation: // config/ai.php 'providers' => [ 'openai' => [ 'driver' => 'openai', 'api_key' => env('OPENAI_API_KEY'), ], 'anthropic' => [ 'driver' => 'anthropic', 'api_key' => env('ANTHROPIC_API_KEY'), ], 'groq' => [ 'driver' => 'groq', 'api_key' => env('GROQ_API_KEY'), ], 'openrouter' => [ 'driver' => 'openrouter', 'api_key' => env('OPENROUTER_API_KEY'), ], ], And the matching `.env` entries: OPENAI_API_KEY=sk-... ANTHROPIC_API_KEY=sk-ant-... GROQ_API_KEY=gsk_... OPENROUTER_API_KEY=sk-or-... Never prefix these with `NEXT_PUBLIC_` — all AI calls must go through the Laravel backend. A frontend that calls an LLM directly exposes your API key to every browser that loads the page. ## Selecting a Provider at Call Time The `using()` method accepts a model identifier that the SDK resolves to a provider. You can be explicit: use Illuminate\Support\Facades\AI; // GPT-4o-mini via OpenAI $openAiResult = AI::text() ->using('gpt-4o-mini') ->prompt($userMessage) ->generate(); // Claude Sonnet via Anthropic $claudeResult = AI::text() ->using('claude-sonnet-4-5') ->prompt($userMessage) ->generate(); // Llama 3 via Groq (fast inference) $groqResult = AI::text() ->using('llama3-70b-8192') ->prompt($userMessage) ->generate(); The SDK looks at the model string and matches it to the correct provider driver configured in `config/ai.php`. If a model string is ambiguous (e.g. two providers offer a model with the same name), qualify it with a provider prefix: // Explicit provider prefix avoids ambiguity $result = AI::text() ->using('openai:gpt-4o') ->prompt($prompt) ->generate(); ## Building a Provider Router A common production pattern is to choose the provider based on the task type. Create a simple service class rather than scattering `using()` strings throughout controllers: <?php namespace App\Services; use Illuminate\Support\Facades\AI; class AiProviderRouter { /** * Fast, cheap completion — use Groq for low-latency tasks. */ public function fast(string $prompt): string { return AI::text() ->using('llama3-70b-8192') // served by Groq ->prompt($prompt) ->generate() ->text(); } /** * High-quality reasoning — use GPT-4o for complex tasks. */ public function precise(string $prompt): string { return AI::text() ->using('gpt-4o') ->prompt($prompt) ->generate() ->text(); } /** * Long-context summarisation — Anthropic handles large contexts well. */ public function summarise(string $document): string { return AI::text() ->using('claude-sonnet-4-5') ->prompt('Summarise the following in 5 bullet points: ' . $document) ->generate() ->text(); } } Bind this in a service provider and inject it wherever needed. The approach keeps provider decisions out of controllers and makes them testable in isolation. ## Provider Failover: Catching Provider Errors The SDK throws distinct exceptions for provider-side failures: * `Laravel\AI\Exceptions\RateLimitedException` — HTTP 429 from the provider * `Laravel\AI\Exceptions\ProviderOverloadedException` — HTTP 5xx from the provider You can catch these to implement fallback logic: use Laravel\AI\Exceptions\RateLimitedException; use Laravel\AI\Exceptions\ProviderOverloadedException; use Illuminate\Support\Facades\AI; use Illuminate\Support\Facades\Log; function generateWithFallback(string $prompt): string { $providers = [ 'gpt-4o-mini', // primary: OpenAI 'claude-haiku-3-5', // fallback 1: Anthropic 'llama3-70b-8192', // fallback 2: Groq ]; foreach ($providers as $model) { try { return AI::text() ->using($model) ->prompt($prompt) ->generate() ->text(); } catch (RateLimitedException | ProviderOverloadedException $e) { Log::warning('AI provider unavailable, trying next', [ 'model' => $model, 'error' => $e->getMessage(), ]); continue; } } throw new \RuntimeException('All AI providers failed.'); } Handle both exception types explicitly — a 429 (rate limit) and a 503 (overload) have different implications for retry strategy. In a queue-based system, you may want to release the job back to the queue with exponential backoff instead of falling over to a different provider immediately. ## OpenRouter as a Multi-Model Aggregator OpenRouter is a single API endpoint that routes to 100+ models. It is worth understanding the trade-off before using it: **Advantages:** * One API key, one billing dashboard, access to models from OpenAI, Anthropic, Mistral, Meta, and dozens of others * Automatic cost-based routing: configure a budget and OpenRouter picks the cheapest capable model * Useful for prototyping without juggling multiple provider accounts **Disadvantages:** * Per-token pricing markup compared to direct provider calls * An extra network hop adds latency * Provider-specific features (e.g. OpenAI's function calling schema format vs Anthropic's tool use format) are abstracted away — if you rely on provider-specific behaviour, OpenRouter may not expose it To add OpenRouter as a named provider: // config/ai.php 'openrouter' => [ 'driver' => 'openrouter', 'api_key' => env('OPENROUTER_API_KEY'), ], $result = AI::text() ->using('openrouter:anthropic/claude-3-haiku') ->prompt($prompt) ->generate(); For the main Laravel AI SDK: Complete Guide to Building AI Applications covering agents, embeddings, MCP, and structured output, that pillar article has broader coverage of the full SDK surface area. ## Provider-Specific Capabilities: What Each Provider Supports Not every provider supports every SDK feature. Before routing a task to a provider, verify capability: Feature | OpenAI | Anthropic | Groq | Gemini | Mistral ---|---|---|---|---|--- Text generation | Yes | Yes | Yes | Yes | Yes Tool calling (agents) | Yes | Yes | Yes | Yes | Yes Structured output | Yes | Yes | Limited | Yes | Yes Image generation | Yes (DALL-E) | No | No | Yes | No Audio transcription | Yes (Whisper) | No | Yes | No | No Embeddings | Yes | No | No | Yes | Yes This matters because if you write an agent that uses image generation and route it to Anthropic, the SDK will throw. Build capability checks into your router if you support a mix of features: // Pseudocode: route based on task type public function routeModel(string $taskType): string { return match($taskType) { 'image' => 'dall-e-3', // OpenAI only 'transcription' => 'whisper-1', // OpenAI only 'embeddings' => 'text-embedding-3-small', // OpenAI only 'fast-chat' => 'llama3-70b-8192', // Groq default => 'gpt-4o-mini', }; } ## Testing Multi-Provider Logic Without API Calls The SDK ships with test fakes that prevent any network call from leaving your test suite: use Laravel\AI\Fakes\TextFake; use Illuminate\Support\Facades\AI; public function test_provider_router_uses_groq_for_fast_tasks(): void { AI::fake([ 'text' => new TextFake('Mocked fast response'), ]); $router = new AiProviderRouter(); $result = $router->fast('What is 2 + 2?'); $this->assertSame('Mocked fast response', $result); } public function test_fallback_triggers_on_rate_limit(): void { // Simulate RateLimitedException on first call, success on second AI::fake([ 'text' => new TextFake( responses: ['Fallback response'], throwOnFirst: new \Laravel\AI\Exceptions\RateLimitedException(), ), ]); $result = generateWithFallback('Test prompt'); $this->assertSame('Fallback response', $result); } This is the correct approach — never let tests hit live providers. Tests that call real APIs are slow, cost money, and fail non-deterministically when the provider is down. ## Common Mistakes **Storing provider keys in`config()` calls inside controllers.** All API keys must live in `.env`, referenced via `config/ai.php` using `env()`. The `config()` helper is the only correct access point in application code. **Expecting every provider to behave identically.** The SDK abstracts the API surface, but output quality, token limits, response latency, and pricing are provider-specific. A prompt that works well on GPT-4o may produce weaker output from a smaller Groq model. Test with your actual prompts. **Not pinning the minor version.** `laravel/ai` is at v0.8.x and has not reached v1.0. The package can introduce breaking changes in minor versions before the stable release. Use `^0.8` in `composer.json`, not `^0` or `*`. **Running multi-provider fallback in the HTTP request cycle.** If the first provider times out after 30 seconds and you try two fallbacks, you're looking at a 90-second request. Queue AI work as jobs. Return a job ID to the client and poll for the result. **Assuming Ollama (local) has the same performance characteristics as hosted providers.** Ollama is useful for development and privacy-sensitive workloads, but inference speed depends on local hardware. Do not include Ollama in a production fallback chain unless you have the GPU resources to back it up. ## Configuring Providers Per Environment A common requirement is to use a cheap local model during development (to avoid API costs) and a hosted provider in production. You can handle this cleanly by conditionally loading provider configuration based on the app environment: // config/ai.php 'providers' => [ 'primary' => [ 'driver' => env('AI_DRIVER', 'openai'), 'api_key' => env('AI_API_KEY'), ], ], Then in your environment files: # .env (local development — Ollama, no API cost) AI_DRIVER=ollama AI_API_KEY= # .env.production AI_DRIVER=openai AI_API_KEY=sk-... This keeps your `using()` calls model-agnostic in lower environments and avoids accidentally sending real user data to a third-party provider during local development. Ollama runs entirely on your machine — no network egress, no billing exposure. Note that Ollama's performance depends entirely on local hardware. On a development MacBook with CPU-only inference, a model that responds in 0.3 seconds on Groq might take 8–15 seconds locally. This is expected and acceptable for development, but do not let it shape your expectations of hosted inference speed. ## Cost Control Across Providers When running multiple providers in production, API spend tracking becomes more complex because you have multiple billing dashboards. A few practical measures: **Set hard monthly spend limits at the provider dashboard level.** Every major provider (OpenAI, Anthropic, Groq) allows you to configure billing alerts and hard caps. Do this before enabling any provider in production — an agent loop with `maxSteps()` not set can exhaust a spend limit in minutes. **Apply Laravel rate limiting at the route or queue level.** The SDK does not meter your own spending — that is your application's responsibility: use Illuminate\Support\Facades\RateLimiter; use Illuminate\Cache\RateLimiting\Limit; RateLimiter::for('ai-per-user', function ($request) { return Limit::perMinute(10)->by($request->user()->id); }); This throttles requests before they reach the provider, protecting your spend budget from a single user making too many requests. **Log every provider call with model, token count, and latency.** Laravel's logging pipeline makes this straightforward with a listener or middleware on the AI facade. Aggregated over a week, this data tells you which providers are over-used and whether the cost-to-quality trade-off of your routing decisions is holding up. ## Limitations to Know **Streaming is not fully first-class in v0.8.** Token-by-token streaming is not a built-in SDK feature at this version. If you need character-by-character output in a chat UI, you need custom SSE handling, Laravel Reverb, and provider-specific streaming code on top of the SDK. This is a known gap and likely to be addressed before v1.0, but plan for the extra work if streaming is a core product requirement. **Framework version constraints are strict.** The `laravel/ai` package requires `illuminate/support ^12.62|^13.15`. If your Laravel 12 installation predates the 12.62 patch release, you must update the framework first before the SDK will resolve cleanly. Run `composer update laravel/framework` before adding `laravel/ai` if you are on an older patch version. **PHP 8.2 and below are unsupported.** The package uses PHP 8.3 features. If you are migrating from Laravel 10 or 11, a PHP upgrade is a prerequisite. Check your hosting environment's available PHP versions before planning the migration. **The AWS SDK conflict.** `laravel/ai` depends on `aws/aws-sdk-php ^3.339`. If your project pins an older AWS SDK version (for example, because another package requires it), Composer will flag a dependency conflict. Resolve this by updating the AWS SDK across all dependent packages simultaneously, or by using the `openrouter` driver as a workaround that avoids the AWS dependency. ## Summary The Laravel AI SDK makes multi-provider support a configuration and call-site concern rather than an architectural one. Name your providers in `config/ai.php`, route to them via `using()`, catch `RateLimitedException` and `ProviderOverloadedException` for failover, and use the built-in fakes for testing. Keep AI work in queued jobs, set `maxSteps()` on any agents that call tools, and pin to `^0.8` until v1.0 ships. If you need Laravel development in Mumbai, Mumbai Web Designer builds production-grade Laravel applications.

게임을 단순히 플레이하는 시대는 지났습니다. 이제는 AI가 생성한 NPC와 무한한 서사를 주고받으며 스스로 세계관을 창조하는 시대로 접어들었습니다. Latitude가 공개한 Voyage는 텍스트 기반의 RPG 플랫폼으로, 누구나 코딩 지식 없이도 자신만의 판타지 세계를 구축하고