Vision in Text Models

This article describes a specific capability of text models: vision, which enables image-to-text and video-to-text conversion. With vision support, models can interpret visual content and return structured or natural-language responses based on what they see.

Common use cases include describing images, analyzing screenshots, extracting text, understanding charts and documents, identifying objects, summarizing scenes, and processing video frames or clips.

The sections below explain how to work with image and video inputs, along with request examples and supported models.

🏝️ Image analysis

Supported Model List

import requests
import json   # for getting a structured output with indentation

response = requests.post(
    url = "https://api.aimlapi.com/v1/chat/completions",
    headers = {
        # Insert your AIML API Key instead of <YOUR_AIMLAPI_KEY>:  
        "Authorization": "Bearer <YOUR_AIMLAPI_KEY>",
        "Content-Type": "application/json"
    },

    json = {
        "model": "alibaba/qwen3.5-omni-flash",
        "messages": [
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "Describe the content of this image."
                    },
                    {
                        "type": "image_url",
                        "image_url": {
                            "url": "https://raw.githubusercontent.com/aimlapi/api-docs/main/reference-files/handwriting.jpg"
                        }
                    }
                ]
            }
        ]
    }
)

data = response.json()
print(json.dumps(data, indent=2, ensure_ascii=False))

Response

{
  "choices": [
    {
      "message": {
        "content": "This image shows a piece of lined notebook paper with handwritten text in black ink. The handwriting is neat, flowing, and appears to be written with a fountain pen — as noted in the text itself.\n\nThe content reads:\n\n> This is a handwriting test to see how it looks on lined paper. For the past two weeks I have been trying to improve my writing along with learning how to write with fountain pens. If you have any suggestions, tips or free resources I would love to check it out. Hope everyone is having a good day. :)\n\nAt the end, there’s a simple smiley face drawn with two dots for eyes and a curved line for a mouth.\n\nThe writer is sharing their progress in improving their handwriting while also learning to use fountain pens, and they’re politely asking for advice or recommendations from others. The tone is friendly and open-ended, inviting interaction.\n\nOverall, it’s a casual, personal note — likely shared online (perhaps on social media or a forum) as part of a “handwriting challenge” or community engagement around calligraphy or penmanship.",
        "reasoning_content": "",
        "role": "assistant"
      },
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null
    }
  ],
  "object": "chat.completion",
  "usage": {
    "prompt_tokens": 1253,
    "completion_tokens": 222,
    "total_tokens": 1475,
    "prompt_tokens_details": {
      "image_tokens": 1234,
      "text_tokens": 19
    },
    "completion_tokens_details": {
      "text_tokens": 222
    }
  },
  "created": 1777062253,
  "system_fingerprint": null,
  "model": "qwen3.5-omni-flash",
  "id": "chatcmpl-f7bde975-e2d2-9609-b890-bb1eb983f853",
  "meta": {
    "usage": {
      "credits_used": 2574,
      "usd_spent": 0.001287
    }
  }
}

🎦 Video analysis

Supported Model List

import requests
import json   # for getting a structured output with indentation

response = requests.post(
    url = "https://api.aimlapi.com/v1/chat/completions",
    headers = {
        # Insert your AIML API Key instead of <YOUR_AIMLAPI_KEY>:  
        "Authorization": "Bearer <YOUR_AIMLAPI_KEY>",
        "Content-Type": "application/json"
    },

    json = {
        "model": "alibaba/qwen3.5-omni-flash",
        "messages": [
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "Describe this scene:"
                    },
                    {
                        "type": "video_url",
                        "video_url": {
                            "url": "https://raw.githubusercontent.com/aimlapi/api-docs/main/reference-files/aimlapi.mp4"
                        }
                    }
                ]
            }
        ]
    }
)

data = response.json()
print(json.dumps(data, indent=2, ensure_ascii=False))

Response

{
  "choices": [
    {
      "message": {
        "content": "This image shows a piece of lined notebook paper with handwritten text in black ink. The handwriting is neat, flowing, and appears to be written with a fountain pen — as noted in the text itself.\n\nThe content reads:\n\n> This is a handwriting test to see how it looks on lined paper. For the past two weeks I have been trying to improve my writing along with learning how to write with fountain pens. If you have any suggestions, tips or free resources I would love to check it out. Hope everyone is having a good day. :)\n\nAt the end, there’s a simple smiley face drawn with two dots for eyes and a curved line for a mouth.\n\nThe writer is sharing their progress in improving their handwriting while also learning to use fountain pens, and they’re politely asking for advice or recommendations from others. The tone is friendly and open-ended, inviting interaction.\n\nOverall, it’s a casual, personal note — likely shared online (perhaps on social media or a forum) as part of a “handwriting challenge” or community engagement around calligraphy or penmanship.",
        "reasoning_content": "",
        "role": "assistant"
      },
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null
    }
  ],
  "object": "chat.completion",
  "usage": {
    "prompt_tokens": 1253,
    "completion_tokens": 222,
    "total_tokens": 1475,
    "prompt_tokens_details": {
      "image_tokens": 1234,
      "text_tokens": 19
    },
    "completion_tokens_details": {
      "text_tokens": 222
    }
  },
  "created": 1777062253,
  "system_fingerprint": null,
  "model": "qwen3.5-omni-flash",
  "id": "chatcmpl-f7bde975-e2d2-9609-b890-bb1eb983f853",
  "meta": {
    "usage": {
      "credits_used": 2574,
      "usd_spent": 0.001287
    }
  }
}

PreviousFunction Calling

NextWeb Search

Last updated 4 months ago

Was this helpful?