Troubleshooting - Kimi API Platform

Why does the model repeatedly call the same tool when using `tool_calls`?

When using tool calls (tool_calls), the model may issue multiple consecutive tool calls based on the context.If you find that the model repeatedly calls the same tool, each call uses exactly the same function.name and function.arguments, and the tool result does not provide new useful information, you can treat it as a repeated tool call.When handling this issue, we recommend checking the message layout first:

When the Kimi API returns finish_reason=tool_calls, make sure the returned choice.message has been added to the messages list as is.
Make sure each tool_call has a corresponding message with role=tool.
Make sure the tool_call_id in the role=tool message exactly matches the corresponding tool_call.id.
If you use streaming output with stream=True, make sure the streamed tool_calls chunks have been assembled correctly, especially the function.arguments field.

If the message layout is correct but the model still repeatedly calls the same tool with the same arguments, you can add repeated-call detection on the client side and append a reminder to the system prompt in the next request.When the same tool and the same arguments are repeated 3 consecutive times, you can append:

<system-reminder>
You are repeating the exact same tool call with identical parameters. Please carefully analyze the previous result. If the task is not yet complete, try a different method or parameters instead of repeating the same call.
</system-reminder>

When the repeated call reaches 5 consecutive times, you can append a stronger reminder that includes the tool name, repeat count, and arguments:

<system-reminder>
You have repeatedly called the same tool with identical parameters many times.
Repeated tool call detected:
- tool: {tool_name}
- repeated_times: {repeat_count}
- arguments: {tool_arguments}
The previous repeated calls did not make progress. Do not call this exact same tool with the exact same arguments again.
Carefully inspect the latest tool result and choose a different next action, different parameters, or finish the task if enough evidence has been gathered.
</system-reminder>

If the same tool and the same arguments are repeated 8 consecutive times, we recommend appending the stronger reminder again.Note: <system-reminder> is only an example prompt, not a special field of the Kimi API. You can merge it into the next role=system message, or write it into the system prompt based on your own message management logic. To avoid false positives, we recommend triggering this reminder only when the same tool, the same arguments, consecutive repetition, and no new progress from the tool result are all true.

Why are API results different from Kimi Assistant results?

The API and Kimi Assistant use the same model. If you notice differences in output, try adjusting the system prompt. Kimi Assistant also provides tools such as a calculator, while the API does not include these tools by default. You need to assemble them yourself when using the API.

Previously, the Kimi API only provided model interaction and did not include additional content search or web page browsing capabilities, commonly known as web search.Now, the Kimi API provides web search. See our guide:Use Web Search with the Kimi APIIf you want to implement web search yourself through the Kimi API, you can also refer to our tool_calls guide:Use the Kimi API for Tool Calls (tool_calls)If you want help from the open-source community, you can refer to these open-source projects:

If you want services from professional vendors, these options are available:

The content returned by the Kimi API is incomplete or truncated

If the content returned by the Kimi API is incomplete, truncated, or shorter than expected, first check the value of choice.finish_reason in the response body. If the value is length, it means the number of tokens generated by the current model exceeded the max_completion_tokens parameter in the request. In this case, the Kimi API only returns up to max_completion_tokens tokens, and any extra content is discarded.When you encounter finish_reason=length, and you want the Kimi model to continue from the previous response, you can use Partial Mode. For details, see:Use Partial Mode with the Kimi APITo avoid finish_reason=length, we recommend increasing max_completion_tokens appropriately. A best practice is to use the estimate-token-count API to calculate the number of tokens in the input, then subtract that number from the maximum context window supported by the selected model. For example, moonshot-v1-32k supports up to 32k tokens, while kimi-k2.6, kimi-k2.5, kimi-k2-0905-preview, and kimi-k2-turbo-preview support up to 256k tokens. The remaining value can be used as the upper bound for max_completion_tokens in the current request.

What is the output length of the Kimi model?

For moonshot-v1-8k, the maximum output length is 8*1024 - prompt_tokens.
For moonshot-v1-32k, the maximum output length is 32*1024 - prompt_tokens.
For moonshot-v1-128k, the maximum output length is 128*1024 - prompt_tokens.
For kimi-k2.6, kimi-k2.5, kimi-k2-0905-preview, and kimi-k2-turbo-preview, the maximum output length is 256*1024 - prompt_tokens.

How many Chinese characters does the Kimi model support?

moonshot-v1-8k supports approximately 15,000 Chinese characters.
moonshot-v1-32k supports approximately 60,000 Chinese characters.
moonshot-v1-128k supports approximately 200,000 Chinese characters.
kimi-k2.6, kimi-k2.5, kimi-k2-0905-preview, and kimi-k2-turbo-preview support approximately 400,000 Chinese characters.

Note: These are estimates. Actual results may vary.

File extraction is inaccurate, or images cannot be recognized

We provide file upload and parsing services for various file formats. For text files, we extract the text content. For image files, we use OCR to recognize text in the image. For PDF documents, if the PDF only contains images, we use OCR to extract text from those images; otherwise, we only extract the text content.Note: For images, we only use OCR to extract text. If your image does not contain any text, parsing will fail.For the complete list of supported file formats, see:Files API

When using the `files` API, I want to reference file content with `file_id`

We currently do not support referencing file content as context through a file file_id.

Error: `content_filter: The request was rejected because it was considered high risk`

The input to the Kimi API, or the output from the Kimi model, contains unsafe or sensitive content. Note: Content generated by the Kimi model may also contain unsafe or sensitive content, which can trigger a content_filter error.

The TPM or RPM limit in the error message does not match my account tier

If you encounter a rate_limit_reached_error while using the Kimi API, for example:

rate_limit_reached_error: Your account {uid}<{ak-id}> request reached TPM rate limit, current:{current_tpm}, limit:{max_tpm}

but the TPM or RPM limit in the error message does not match the TPM or RPM shown in the dashboard, first check whether you are using the correct api_key for the current account. In most cases, this mismatch is caused by using the wrong api_key, such as accidentally using a key provided by another user or mixing up keys across multiple accounts.

Error: `model_not_found`

Make sure your SDK is configured with base_url=https://api.moonshot.ai/v1. The model_not_found error is usually caused by using the OpenAI SDK without setting base_url, which sends the request to OpenAI servers instead. OpenAI then returns the model_not_found error.

The Kimi model makes numerical calculation errors

Due to the uncertainty of model generation, the Kimi model may make calculation errors of varying severity when performing numerical computations. We recommend using tool calls (tool_calls) to provide calculator functionality to the Kimi model. For more information, see our tool calling guide:Use the Kimi API for Tool Calls (tool_calls)

The Kimi model cannot answer today's date

The Kimi model cannot access highly time-sensitive information such as the current date. However, you can provide this information in the system prompt. For example:

python
node.js

import os
from datetime import datetime
from openai import OpenAI

client = OpenAI(
    api_key=os.environ['MOONSHOT_API_KEY'],
    base_url="https://api.moonshot.ai/v1",
)

# Generate the current date with datetime and add it to the system prompt.
system_prompt = f"""
You are Kimi, and today's date is {datetime.now().strftime('%d.%m.%Y %H:%M:%S')}
"""

completion = client.chat.completions.create(
    model="moonshot-v1-128k",
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": "What is today's date?"},
    ],
    temperature=0.3,
)

print(completion.choices[0].message.content)  # Output: Today's date is July 31, 2024.

const OpenAI = require('openai')

client = new OpenAI({
    apiKey: process.env.MOONSHOT_API_KEY,
    baseURL: "https://api.moonshot.ai/v1",
})

// Generate the current date with Date and add it to the system prompt.
system_prompt = `You are Kimi, and today's date is ${new Date().toString()}`

async function main() {
    completion = await client.chat.completions.create({
        model: "moonshot-v1-128k",
        messages: [
            {role: "system", content: system_prompt},
            {role: "user", content: "What is today's date?"},
        ],
        temperature: 0.3,
    })

    console.log(completion.choices[0].message.content)  // Output: Today's date is July 31, 2024.
}

main()

How do I handle errors without using an SDK?

In some cases, you may need to integrate with the Kimi API directly instead of using the OpenAI SDK. When integrating directly, decide the next processing step based on the status returned by the API. In general, HTTP status code 200 indicates success, while 4xx and 5xx status codes indicate failure. We return error information in JSON format. See the following code snippets for example handling logic:

python
node.js

import os
import httpx

header = {
    "Authorization": f"Bearer {os.environ['MOONSHOT_API_KEY']}",
}

messages = [
    {"role": "system", "content": "You are Kimi"},
    {"role": "user", "content": "Hello."},
]

r = httpx.post("https://api.moonshot.ai/v1/chat/completions",
               headers=header,
               json={
                   "model": "moonshot-v1-128k",  # A correct model enters the status_code == 200 branch below.
                   # "model": "moonshot-v1-129k",  # An incorrect model name enters the else branch below.
                   "messages": messages,
                   "temperature": 0.3,
               })

if r.status_code == 200:  # Normal handling for a request with a correct model.
    completion = r.json()
    print(completion["choices"][0]["message"]["content"])
else:  # Error handling for a request with an incorrect model name.
    # For demonstration, we only print the error here.
    # In production logic, you may want to log, interrupt the request, retry, and so on.
    error = r.json()
    print(f"error: status={r.status_code}, type='{error['error']['type']}', message='{error['error']['message']}'")

const axios = require('axios');

header = {
    "Authorization": `Bearer ${process.env.MOONSHOT_API_KEY}`,
}

messages = [
    {"role": "system", "content": "You are Kimi"},
    {"role": "user", "content": "Hello."},
]

async function main() {
    r = await axios.post("https://api.moonshot.ai/v1/chat/completions",
        {
            "model": "moonshot-v1-128k",  // A correct model enters the success branch below.
            //"model": "moonshot-v1-129k",  // An incorrect model name enters the error branch.
            "messages": messages,
            "temperature": 0.3,
        },
        {
            headers: header,
            validateStatus: function (status) {
                return status == 200; // Resolve only when the status code is 200.
            }
        },
     ).catch(function (error) {
        console.log(`error: ${error.message}`)
     })

    if (r) {  // Normal handling for a request with a correct model.
        console.log(r.data.choices[0].message.content)
    }
}

main()

Our error messages follow this format:

{
	"error": {
		"type": "error_type",
		"message": "error_message"
	}
}

For the complete error reference, see:Error Reference

Why do some requests respond quickly while others respond slowly when the prompts are similar?

If some requests with similar prompts respond quickly, for example in 3 seconds, while others respond slowly, for example in 20 seconds, this is usually because the Kimi model generated different numbers of tokens. In general, the number of generated tokens is proportional to the total response time of the Kimi API. The more tokens generated, the longer the full response takes.Note that the number of generated tokens only affects the response time for the complete request, meaning the time until the final token is generated. You can set stream=True and observe the time to first token, abbreviated as TTFT. In normal cases, when prompt lengths are similar, TTFT should not vary significantly.

I set `max_completion_tokens=2000` to make Kimi output 2,000 characters, but the output is shorter

Note: max_tokens is deprecated. Please use max_completion_tokens instead. The two fields have the same meaning.

The max_completion_tokens parameter means: when calling /v1/chat/completions, it specifies the maximum number of tokens the model is allowed to generate. Once the number of generated tokens exceeds max_completion_tokens, the model stops generating the next token.max_completion_tokens is used to:

Help the caller determine which model to use. For example, when prompt_tokens + max_completion_tokens <= 8 * 1024, you can choose moonshot-v1-8k.
Prevent the Kimi model from generating too much unexpected content in unusual cases, which could cause extra cost. For example, the model might repeatedly output whitespace.

max_completion_tokens does not tell the Kimi model how many tokens to output. In other words, max_completion_tokens is not used as part of the prompt input to the Kimi model. If you want the model to output a specific number of characters, use these general approaches:

For outputs under 1,000 characters:
1. Clearly specify the desired character count in the prompt.
2. Manually or programmatically check whether the output length meets expectations. If not, tell the model in a second round that the output is too long or too short, then ask it to generate a new version.
For outputs over 1,000 characters or much longer:
1. Split the expected output by structure or chapter, create a template, and use placeholders to mark where the model should fill in content.
2. Ask the Kimi model to fill each placeholder one by one, then assemble the complete long-form text.

I only made one request in a minute, but triggered `Your account reached max request`

The OpenAI SDK usually includes a retry mechanism:

Certain errors are automatically retried 2 times by default, with a short exponential backoff. Connection errors (for example, due to a network connectivity problem), 408 Request Timeout, 409 Conflict, 429 Rate Limit, and >=500 Internal errors are all retried by default.

This retry mechanism retries 2 times by default when an error occurs, for a total of 3 requests. In unstable network conditions or other cases that may cause request errors, using the OpenAI SDK can expand one request into 2 or 3 requests. All of these requests count toward your RPM quota.Note: For users on a tier0 account using the OpenAI SDK, one failed request may consume the entire RPM quota because of the default retry mechanism.

I used `base64` encoding for my text content to make transmission easier

Please do not do this. Encoding files with base64 can cause massive token consumption. If your file type is supported by our /v1/files API, upload the file through the files API and extract its content there.For binary files or files in other encoded formats, the Kimi model currently cannot parse the content. Do not add this content to the context.

Why can't I use a key from another Kimi platform on platform.kimi.ai?

Kimi Open Platform provides separate regional platforms. For users outside mainland China, use platform.kimi.ai. Accounts and keys from different regional platforms are independent and cannot be mixed.If you use the wrong key for a platform, you will receive a 401 invalid_authentication_error. If you receive a 401 error, first check whether you are using a key from the wrong platform.

International open platform base_url: https://api.moonshot.ai/v1