Models API

Model generation and streaming APIs

Models API

The Models API provides functions for interacting with generative AI models.

generate()

Generate calls a generative model based on the provided prompt and configuration.

Signature:

generate<O extends z.ZodTypeAny = z.ZodTypeAny, CustomOptions extends z.ZodTypeAny = typeof GenerationCommonConfigSchema>(
  registry: Registry,
  options: GenerateOptions<O, CustomOptions> | PromiseLike<GenerateOptions<O, CustomOptions>>
): Promise<GenerateResponse<z.infer<O>>>

Parameters

options · GenerateOptions · required

Generation options

Show properties

model · ModelArgument

The model to use (name string or ModelAction). If not provided, uses the default model.

prompt · string | Part[]

User prompt - can be text or multipart content

system · string | Part[]

System instructions for the model

messages · MessageData[]

Conversation history for multi-turn interactions

tools · ToolAction[]

Tools the model can call. Model will automatically invoke tools unless returnToolRequests is true.

returnToolRequests · boolean · default: "false"

If true, return tool requests without executing them

config · GenerationCommonConfig

Model configuration including:

• temperature: Number (0-2) - Sampling temperature
• maxOutputTokens: Maximum tokens to generate
• topK: Top-K sampling parameter
• topP: Top-P (nucleus) sampling parameter
• stopSequences: Array of sequences that stop generation

output · object

Output schema specification

• schema: Zod schema for structured output
• format: Output format (e.g., ‘json’, ‘text’)

docs · Document[]

Documents for retrieval-augmented generation (RAG)

Returns

response · GenerateResponse<O>

Generation response object

Show properties

text · string

The generated text content

output() · () => z.infer<O>

Parses and returns structured output according to the schema

messages · MessageData[]

Complete conversation including user prompts, model responses, and tool calls

usage · GenerationUsage

Token usage statistics:

• inputTokens: Number of input tokens
• outputTokens: Number of generated tokens
• totalTokens: Total tokens used

finishReason · string

Why generation stopped (e.g., ‘stop’, ‘length’, ‘safety’)

finishMessage · string

Additional details about finish reason

Example

import { generate } from '@genkit-ai/ai';
import { z } from 'zod';

// Simple text generation
const response = await generate(registry, {
  model: 'googleai/gemini-2.0-flash-exp',
  prompt: 'Write a haiku about code',
});
console.log(response.text);

// Structured output
const response = await generate(registry, {
  model: 'googleai/gemini-2.0-flash-exp',
  prompt: 'List 3 programming languages',
  output: {
    schema: z.object({
      languages: z.array(z.string()),
    }),
  },
});
const { languages } = response.output();

// With tools
const response = await generate(registry, {
  model: 'googleai/gemini-2.0-flash-exp',
  prompt: 'What is the weather in Paris?',
  tools: [weatherTool],
});

generateStream()

Streaming version of generate() that yields response chunks as they are generated.

Signature:

generateStream<O extends z.ZodTypeAny = z.ZodTypeAny, CustomOptions extends z.ZodTypeAny = typeof GenerationCommonConfigSchema>(
  registry: Registry,
  options: GenerateStreamOptions<O, CustomOptions> | PromiseLike<GenerateStreamOptions<O, CustomOptions>>
): GenerateStreamResponse<z.infer<O>>

Parameters

Accepts the same GenerateOptions as generate(), plus:

onChunk · StreamingCallback

Callback invoked for each chunk:

(chunk: GenerateResponseChunk) => void | Promise<void>

Returns

response · GenerateStreamResponse

Streaming response object

Show properties

response · Promise<GenerateResponse>

Promise that resolves to the final complete response

stream · Channel<GenerateResponseChunk>

Async iterable channel of response chunks. Each chunk contains:

• text: Incremental text
• content: Array of content parts
• usage: Cumulative token usage

Example

import { generateStream } from '@genkit-ai/ai';

const { response, stream } = generateStream(registry, {
  model: 'googleai/gemini-2.0-flash-exp',
  prompt: 'Write a long story',
});

// Process chunks as they arrive
for await (const chunk of stream) {
  process.stdout.write(chunk.text);
}

// Get final response
const final = await response;
console.log('\nUsage:', final.usage);

modelRef()

Creates a reference to a model by name.

Signature:

modelRef<ConfigSchema extends z.ZodTypeAny = z.ZodTypeAny>(options: {
  name: string;
  config?: z.infer<ConfigSchema>;
  info?: ModelInfo;
}): ModelReference<ConfigSchema>

Parameters

options · object · required

Show properties

name · string · required

Model name (e.g., ‘googleai/gemini-2.0-flash-exp’)

config · object

Default configuration for this model reference

info · ModelInfo

Model metadata (capabilities, version, etc.)

Returns

reference · ModelReference

A reference object that can be passed to generate() or other model functions

Example

import { modelRef } from '@genkit-ai/ai/model';

const flash = modelRef({
  name: 'googleai/gemini-2.0-flash-exp',
  config: {
    temperature: 0.7,
  },
});

const response = await generate(registry, {
  model: flash,
  prompt: 'Hello!',
});

Types

GenerateRequest

The raw request object sent to a model.

interface GenerateRequest<CustomOptions extends z.ZodTypeAny = z.ZodTypeAny> {
  messages: MessageData[];
  config?: z.infer<CustomOptions>;
  tools?: ToolDefinition[];
  output?: {
    format?: string;
    schema?: JSONSchema;
  };
  candidates?: number;
}

GenerateResponse

The response from a model.

interface GenerateResponse<O = any> {
  message?: MessageData;
  text: string;
  output: () => O;
  messages: MessageData[];
  usage?: GenerationUsage;
  finishReason?: string;
  finishMessage?: string;
  request: GenerateRequest;
}

Part

Content part within a message.

type Part = TextPart | MediaPart | DataPart | ToolRequestPart | ToolResponsePart | CustomPart;

interface TextPart {
  text: string;
}

interface MediaPart {
  media: {
    url: string;
    contentType?: string;
  };
}

interface DataPart {
  data: any;
}

interface ToolRequestPart {
  toolRequest: {
    name: string;
    ref?: string;
    input: any;
  };
}

interface ToolResponsePart {
  toolResponse: {
    name: string;
    ref?: string;
    output: any;
  };
}

MessageData

A message in the conversation.

interface MessageData {
  role: 'user' | 'model' | 'system' | 'tool';
  content: Part[];
  metadata?: Record<string, any>;
}

See Also

• Genkit API
• Tools API
• Prompts API