Island AI

Island AI

Island AI is a collection of low-level utilities and high-level tools for handling structured data streams from Large Language Models (LLMs). The packages range from basic JSON streaming parsers to complete LLM clients, giving you the flexibility to build custom solutions or use pre-built integrations.

Core Packages

1. schema-stream (moved) ⇨

schema-stream is now maintained in its standalone repository, including its source, documentation, tests, and releases. The npm package name is unchanged.

2. zod-stream ⇨

Extends schema-stream with OpenAI integration and Zod-specific features.

Key Features:

  • OpenAI completion streaming
  • Multiple response modes (TOOLS, FUNCTIONS, JSON, etc.)
  • Schema validation during streaming
import { OAIStream } from "zod-stream";
import { withResponseModel } from "zod-stream";
import { z } from "zod";
 
// Define extraction schema
const ExtractionSchema = z.object({
  users: z.array(z.object({
    name: z.string(),
    handle: z.string(),
    twitter: z.string()
  })).min(3),
  location: z.string(),
  budget: z.number()
});
 
// Configure OpenAI params with schema
const params = withResponseModel({
  response_model: { 
    schema: ExtractionSchema, 
    name: "Extract" 
  },
  params: {
    messages: [{ role: "user", content: textBlock }],
    model: "gpt-4"
  },
  mode: "TOOLS"
});
 
// Stream completions
const stream = OAIStream({ 
  res: await oai.chat.completions.create({
    ...params,
    stream: true
  })
});
 
// Process results
const client = new ZodStream();
const extractionStream = await client.create({
  completionPromise: () => stream,
  response_model: { 
    schema: ExtractionSchema, 
    name: "Extract" 
  }
});
 
for await (const data of extractionStream) {
  console.log('Progressive update:', data);
}

3. stream-hooks ⇨

React hooks for consuming streaming JSON data with Zod schema validation.

Key Features:

  • Ready-to-use React hooks
  • Automatic schema validation
  • Progress tracking
  • Error handling
import { useJsonStream } from "stream-hooks";
 
function DataViewer() {
  const { loading, startStream, data, error } = useJsonStream({
    schema: ExtractionSchema,
    onReceive: (update) => {
      console.log('Progressive update:', update);
    },
  });
 
  return (
    <div>
      {loading && <div>Loading...</div>}
      {error && <div>Error: {error.message}</div>}
      {data && (
        <pre>{JSON.stringify(data, null, 2)}</pre>
      )}
      <button onClick={() => startStream({
        url: "/api/extract",
        method: "POST",
        body: { text: "..." }
      })}>
        Start Extraction
      </button>
    </div>
  );
}

4. evalz ⇨

Structured evaluation tools for assessing LLM outputs across multiple dimensions. Built with TypeScript and the OpenAI Responses API, it enables both automated evaluation and human-in-the-loop assessment workflows.

Key Features:

  • 🎯 Model-Graded Evaluation: Leverage LLMs to assess response quality
  • 📊 Accuracy Measurement: Compare outputs using semantic and lexical similarity
  • 🔍 Context Validation: Evaluate responses against source materials
  • ⚖️ Composite Assessment: Combine multiple evaluation types with custom weights
// Combine different evaluator types
const compositeEval = createWeightedEvaluator({
  evaluators: {
    entities: createContextEvaluator({ type: "entities-recall" }),
    accuracy: createAccuracyEvaluator({
      weights: { 
        factual: 0.9,   // High weight on exact matches
        semantic: 0.1    // Low weight on similar terms
      }
    }),
    quality: createEvaluator({
      client: oai,
      model: "gpt-4-turbo",
      evaluationDescription: "Rate quality"
    })
  },
  weights: {
    entities: 0.3,
    accuracy: 0.4,
    quality: 0.3
  }
});
 
// Must provide all required fields for each evaluator type
await compositeEval({
  data: [{
    prompt: "Summarize the earnings call",
    completion: "CEO Jane Smith announced 15% growth",
    expectedCompletion: "The CEO reported strong growth",
    groundTruth: "CEO discussed Q3 performance",
    contexts: [
      "CEO Jane Smith presented Q3 results",
      "Company saw 15% growth in Q3 2023"
    ]
  }]
});

5. llm-polyglot ⇨

A universal LLM client that extends the OpenAI SDK to provide consistent interfaces across different providers that may not follow the OpenAI API specification.

Key Features:

  • OpenAI-compatible interface for non-OpenAI providers
  • Support for major providers:
    • OpenAI (direct SDK proxy)
    • Anthropic (Claude models)
    • Google (Gemini models)
    • Together
    • Microsoft/Azure
    • Anyscale
  • Streaming support across providers
  • Function/tool calling compatibility
  • Context caching for Gemini
  • Structured output support

Basic OpenAI-Style Usage

import { createLLMClient } from "llm-polyglot";
 
// Create provider-specific client
const anthropicClient = createLLMClient({
  provider: "anthropic"
});
 
// Use consistent OpenAI-style API
const completion = await anthropicClient.chat.completions.create({
  model: "claude-3-opus-20240229",
  max_tokens: 1000,
  messages: [{ role: "user", content: "Extract data..." }]
});

Next

Concepts

On this page