QVQ model usage - Alibaba Cloud Model Studio - Alibaba Cloud Documentation Center

The QVQ model has powerful visual reasoning capability. It first outputs the thinking process, then the response content. Currently, QVQ only supports streaming output.

Supported models

QVQ is a visual reasoning model that supports visual input and chain-of-thought output. It shows stronger capabilities in mathematics, programming, visual analysis, creation, and general tasks.

Name	Version	Context window	Maximum input	Maximum CoT	Maximum response	Input price	Output price	Free quota (Note)
		(Tokens)				(Million tokens)
qvq-max Same performance as qvq-max-2025-03-25	Stable	131,072	106,496 Up to 16,384 per image	16,384	8,192	Time-limited free trial After the free quota runs out, you cannot access this model. Please stay tuned for updates.		1 million tokens each Valid for 180 days after activation
qvq-max-latest Always same performance as the latest snapshot	Latest
qvq-max-2025-03-25 Also qvq-max-0325	Snapshot

Calculate image and video tokens

Image

Each 28×28 pixels correspond to one token, and an image requires at least 4 tokens. You can estimate the tokens for an image using the following code:

Python

import math
# Use the following command to install the Pillow library: pip install Pillow
from PIL import Image

def token_calculate(image_path):
    # Open the specified PNG image file
    image = Image.open(image_path)

    # Get the original dimensions of the image
    height = image.height
    width = image.width
    
    # Adjust the height to a multiple of 28
    h_bar = round(height / 28) * 28
    # Adjust the width to a multiple of 28
    w_bar = round(width / 28) * 28
    
    # Minimum token limit for an image: 4 tokens
    min_pixels = 28 * 28 * 4
    # Maximum token limit for an image: 1280 tokens
    max_pixels = 1280 * 28 * 28
        
    # Scale the image to ensure the total number of pixels is within the range [min_pixels, max_pixels]
    if h_bar * w_bar > max_pixels:
        # Calculate the scaling factor beta so that the scaled image's total pixels do not exceed max_pixels
        beta = math.sqrt((height * width) / max_pixels)
        # Recalculate the adjusted height, ensuring it's a multiple of 28
        h_bar = math.floor(height / beta / 28) * 28
        # Recalculate the adjusted width, ensuring it's a multiple of 28
        w_bar = math.floor(width / beta / 28) * 28
    elif h_bar * w_bar < min_pixels:
        # Calculate the scaling factor beta so that the scaled image's total pixels are not less than min_pixels
        beta = math.sqrt(min_pixels / (height * width))
        # Recalculate the adjusted height, ensuring it's a multiple of 28
        h_bar = math.ceil(height * beta / 28) * 28
        # Recalculate the adjusted width, ensuring it's a multiple of 28
        w_bar = math.ceil(width * beta / 28) * 28
    return h_bar, w_bar

# Replace test.png with the path to your local image
h_bar, w_bar = token_calculate("test.png")
print(f"Scaled image dimensions: height {h_bar}, width {w_bar}")

# Calculate the number of tokens for the image: total pixels divided by 28 * 28
token = int((h_bar * w_bar) / (28 * 28))

# The system automatically adds <|vision_bos|> and <|vision_eos|> visual markers (1 token each)
print(f"The image token count is {token + 2}")

Node.js

// Use the following command to install sharp: npm install sharp
import sharp from 'sharp';
import fs from 'fs';

async function tokenCalculate(imagePath) {
    // Open the specified PNG image file
    const image = sharp(imagePath);
    const metadata = await image.metadata();

    // Get the original dimensions of the image
    const height = metadata.height;
    const width = metadata.width;

    // Adjust the height to a multiple of 28
    let hBar = Math.round(height / 28) * 28;
    // Adjust the width to a multiple of 28
    let wBar = Math.round(width / 28) * 28;

    // Minimum token limit for an image: 4 tokens
    const minPixels = 28 * 28 * 4;
    // Maximum token limit for an image: 1280 tokens
    const maxPixels = 1280 * 28 * 28;

    // Scale the image to ensure the total number of pixels is within the range [min_pixels, max_pixels]
    if (hBar * wBar > maxPixels) {
        // Calculate the scaling factor beta so that the scaled image's total pixels do not exceed max_pixels
        const beta = Math.sqrt((height * width) / maxPixels);
        // Recalculate the adjusted height, ensuring it's a multiple of 28
        hBar = Math.floor(height / beta / 28) * 28;
        // Recalculate the adjusted width, ensuring it's a multiple of 28
        wBar = Math.floor(width / beta / 28) * 28;
    } else if (hBar * wBar < minPixels) {
        // Calculate the scaling factor beta so that the scaled image's total pixels are not less than min_pixels
        const beta = Math.sqrt(minPixels / (height * width));
        // Recalculate the adjusted height, ensuring it's a multiple of 28
        hBar = Math.ceil(height * beta / 28) * 28;
        // Recalculate the adjusted width, ensuring it's a multiple of 28
        wBar = Math.ceil(width * beta / 28) * 28;
    }

    return { hBar, wBar };
}

// Replace test.png with the path to your local image
const imagePath = 'test.png';
tokenCalculate(imagePath).then(({ hBar, wBar }) => {
    console.log(`Scaled image dimensions: height ${hBar}, width ${wBar}`);

    // Calculate the number of tokens for the image: total pixels divided by 28 * 28
    const token = Math.floor((hBar * wBar) / (28 * 28));

    // The system automatically adds <|vision_bos|> and <|vision_eos|> visual markers (1 token each)
    console.log(`The total image token count is ${token + 2}`);
}).catch(err => {
    console.error('Error processing image:', err);
});

Java

import javax.imageio.ImageIO;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;

public class Main {

    // Custom class to store the resized dimensions
    public static class ResizedSize {
        public final int height;
        public final int width;

        public ResizedSize(int height, int width) {
            this.height = height;
            this.width = width;
        }
    }

    public static ResizedSize smartResize(String imagePath) throws IOException {
        // 1. Load the image
        BufferedImage image = ImageIO.read(new File(imagePath));
        if (image == null) {
            throw new IOException("Cannot load image file: " + imagePath);
        }

        int originalHeight = image.getHeight();
        int originalWidth = image.getWidth();

        final int minPixels = 28 * 28 * 4;
        final int maxPixels = 1280 * 28 * 28;
        // 2. Initial adjustment to multiples of 28
        int hBar = (int) (Math.round(originalHeight / 28.0) * 28);
        int wBar = (int) (Math.round(originalWidth / 28.0) * 28);
        int currentPixels = hBar * wBar;

        // 3. Adjust dimensions based on conditions
        if (currentPixels > maxPixels) {
            // Current pixels exceed the maximum, need to reduce
            double beta = Math.sqrt(
                    (originalHeight * (double) originalWidth) / maxPixels
            );
            double scaledHeight = originalHeight / beta;
            double scaledWidth = originalWidth / beta;

            hBar = (int) (Math.floor(scaledHeight / 28) * 28);
            wBar = (int) (Math.floor(scaledWidth / 28) * 28);
        } else if (currentPixels < minPixels) {
            // Current pixels below the minimum, need to enlarge
            double beta = Math.sqrt(
                    (double) minPixels / (originalHeight * originalWidth)
            );
            double scaledHeight = originalHeight * beta;
            double scaledWidth = originalWidth * beta;

            hBar = (int) (Math.ceil(scaledHeight / 28) * 28);
            wBar = (int) (Math.ceil(scaledWidth / 28) * 28);
        }

        return new ResizedSize(hBar, wBar);
    }

    public static void main(String[] args) {
        try {
            ResizedSize size = smartResize(
                    // Replace xxx/test.png with your image path
                    "xxx/test.png"
            );

            System.out.printf("Scaled image dimensions: height %d, width %d%n", size.height, size.width);

            // Calculate tokens (total pixels / 28×28 + 2)
            int token = (size.height * size.width) / (28 * 28) + 2;
            System.out.printf("Total image tokens: %d%n", token);

        } catch (IOException e) {
            System.err.println("Error: " + e.getMessage());
            e.printStackTrace();
        }
    }
}

Video

You can estimate the tokens for a video using the following code:

Python

# Install with before use: pip install opencv-python
import math
import os
import logging
import cv2

logger = logging.getLogger(__name__)

FRAME_FACTOR = 2
IMAGE_FACTOR = 28
# Aspect ratio of video frames
MAX_RATIO = 200

# Minimum token limit for video frames
VIDEO_MIN_PIXELS = 128 * 28 * 28
# Maximum token limit for video frames
VIDEO_MAX_PIXELS = 768 * 28 * 28

# If the user does not pass the FPS parameter, use the default value
FPS = 2.0
# Minimum number of frames to extract
FPS_MIN_FRAMES = 4
# Maximum number of frames to extract. When using the qwen2.5-vl model, set FPS_MAX_FRAMES to 512; for other models, set it to 80
FPS_MAX_FRAMES = 512

# Maximum pixel value for video input,
# When using the qwen2.5-vl model, set VIDEO_TOTAL_PIXELS to 65536 * 28 * 28; for other models, set it to 24576 * 28 * 28
VIDEO_TOTAL_PIXELS = int(float(os.environ.get('VIDEO_MAX_PIXELS', 65536 * 28 * 28)))

def round_by_factor(number: int, factor: int) -> int:
    """Return the integer closest to 'number' that is divisible by 'factor'."""
    return round(number / factor) * factor

def ceil_by_factor(number: int, factor: int) -> int:
    """Return the smallest integer greater than or equal to 'number' that is divisible by 'factor'."""
    return math.ceil(number / factor) * factor

def floor_by_factor(number: int, factor: int) -> int:
    """Return the largest integer less than or equal to 'number' that is divisible by 'factor'."""
    return math.floor(number / factor) * factor

def smart_nframes(ele,total_frames,video_fps):
    """Calculate the number of video frames to extract.

    Args:
        ele (dict): Dictionary containing video configuration
            - fps: fps is used to control the number of input frames extracted by the model.
        total_frames (int): Total frames of the original video.
        video_fps (int | float): Original frame rate of the video

    Raises:
        nframes should be within [FRAME_FACTOR, total_frames], otherwise throws an error

    Returns:
        The number of video frames for model input.
    """
    assert not ("fps" in ele and "nframes" in ele), "Only accept either `fps` or `nframes`"
    fps = ele.get("fps", FPS)
    min_frames = ceil_by_factor(ele.get("min_frames", FPS_MIN_FRAMES), FRAME_FACTOR)
    max_frames = floor_by_factor(ele.get("max_frames", min(FPS_MAX_FRAMES, total_frames)), FRAME_FACTOR)
    duration = total_frames / video_fps if video_fps != 0 else 0
    if duration-int(duration) > (1/fps):
        total_frames = math.ceil(duration * video_fps)
    else:
        total_frames = math.ceil(int(duration) * video_fps)
    nframes = total_frames / video_fps * fps
    if nframes > total_frames:
        logger.warning(f"smart_nframes: nframes[{nframes}] > total_frames[{total_frames}]")
    nframes = int(min(min(max(nframes, min_frames), max_frames), total_frames))
    if not (FRAME_FACTOR <= nframes and nframes <= total_frames):
        raise ValueError(f"nframes should be in interval [{FRAME_FACTOR}, {total_frames}], but got {nframes}.")

    return nframes

def get_video(video_path):
    # Get video information
    cap = cv2.VideoCapture(video_path)

    frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    # Get video height
    frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
    total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))

    video_fps = cap.get(cv2.CAP_PROP_FPS)
    return frame_height, frame_width, total_frames, video_fps

def smart_resize(ele, path, factor=IMAGE_FACTOR):
    # Get the original video's width and height
    height, width, total_frames, video_fps = get_video(path)
    # Minimum token limit for video frames
    min_pixels = VIDEO_MIN_PIXELS
    total_pixels = VIDEO_TOTAL_PIXELS
    # Number of extracted video frames
    nframes = smart_nframes(ele, total_frames, video_fps)
    max_pixels = max(min(VIDEO_MAX_PIXELS, total_pixels / nframes * FRAME_FACTOR), int(min_pixels * 1.05))

    # The aspect ratio of the video should not exceed 200:1 or 1:200
    if max(height, width) / min(height, width) > MAX_RATIO:
        raise ValueError(
            f"absolute aspect ratio must be smaller than {MAX_RATIO}, got {max(height, width) / min(height, width)}"
        )

    h_bar = max(factor, round_by_factor(height, factor))
    w_bar = max(factor, round_by_factor(width, factor))
    if h_bar * w_bar > max_pixels:
        beta = math.sqrt((height * width) / max_pixels)
        h_bar = floor_by_factor(height / beta, factor)
        w_bar = floor_by_factor(width / beta, factor)
    elif h_bar * w_bar < min_pixels:
        beta = math.sqrt(min_pixels / (height * width))
        h_bar = ceil_by_factor(height * beta, factor)
        w_bar = ceil_by_factor(width * beta, factor)
    return h_bar, w_bar
    

def token_calculate(video_path, fps):
    # Pass video path and fps extraction parameter
    messages = [{"content": [{"video": video_path, "fps": fps}]}]
    vision_infos = extract_vision_info(messages)[0]

    resized_height, resized_width = smart_resize(vision_infos, video_path)

    height, width, total_frames, video_fps = get_video(video_path)
    num_frames = smart_nframes(vision_infos, total_frames, video_fps)
    print(f"Original video size: {height}*{width}, size for model input: {resized_height}*{resized_width}, total frames of video: {total_frames}, when fps is {fps}, total frames extracted: {num_frames}", end=", ")
    video_token = int(math.ceil(num_frames / 2) * resized_height / 28 * resized_width / 28)
    video_token += 2  # The system will automatically add <|vision_bos|> and <|vision_eos|> visual markers (each counts as 1 Token)
    return video_token

def extract_vision_info(conversations):
    vision_infos = []
    if isinstance(conversations[0], dict):
        conversations = [conversations]
    for conversation in conversations:
        for message in conversation:
            if isinstance(message["content"], list):
                for ele in message["content"]:
                    if (
                        "image" in ele
                        or "image_url" in ele
                        or "video" in ele
                        or ele.get("type", "") in ("image", "image_url", "video")
                    ):
                        vision_infos.append(ele)
    return vision_infos


video_token = token_calculate("xxx/test.mp4", 1)
print("Video tokens:", video_token)

Node.js

// Install before use: npm i node-ffprobe
import ffprobeInstaller from '@ffprobe-installer/ffprobe';
import ffprobe from 'node-ffprobe';
import probe from "node-ffprobe";
// Set ffprobe path (global configuration)
ffprobe.FFPROBE_PATH = ffprobeInstaller.path;

// Get video information
async function getVideoInfo(videoPath) {
  try {
    const probeData = await probe(videoPath);
    const videoStream = probeData.streams.find(
      stream => stream.codec_type === 'video'
    );

    if (!videoStream) {
      throw new Error('No video stream found in the video');
    }

    const width = videoStream.width;
    const height = videoStream.height;
    const totalFrames = videoStream.nb_frames;
    const [numerator, denominator] = videoStream.avg_frame_rate.split('/');
    const frameRate = parseFloat(numerator / denominator);

    return {
      width,
      height,
      totalFrames,
      frameRate
    };
  } catch (error) {
    console.error('Failed to get video information:', error);
    throw error;
  }
}

// Configuration parameters
const FRAME_FACTOR = 2; 
const IMAGE_FACTOR = 28;
const MAX_RATIO = 200;
// Minimum token limit for video frames
const VIDEO_MIN_PIXELS = 128 * 28 * 28; 
// Maximum token limit for video frames
const VIDEO_MAX_PIXELS = 768 * 28 * 28; 
const FPS = 2.0; // If the user does not pass the FPS parameter, use the default value
// Minimum number of frames to extract
const FPS_MIN_FRAMES = 4;
// Maximum number of frames to extract. When using the qwen2.5-vl model, set FPS_MAX_FRAMES to 512; for other models, set it to 80
const FPS_MAX_FRAMES = 512; 
// Maximum pixel value for video input,
// When using the qwen2.5-vl model, set VIDEO_TOTAL_PIXELS to 65536 * 28 * 28; for other models, set it to 24576 * 28 * 28
const VIDEO_TOTAL_PIXELS = parseInt(process.env.VIDEO_MAX_PIXELS) || 65536 * 28 * 28;

// Math utility functions
function roundByFactor(number, factor) {
    return Math.round(number / factor) * factor;
}

function ceilByFactor(number, factor) {
    return Math.ceil(number / factor) * factor;
}

function floorByFactor(number, factor) {
    return Math.floor(number / factor) * factor;
}

// Calculate the number of frames to extract
function smartNFrames(ele, totalFrames, frameRate) {
    const fps = ele.fps || FPS;
    const minFrames = ceilByFactor(ele.min_frames || FPS_MIN_FRAMES, FRAME_FACTOR);
    const maxFrames = floorByFactor(
        ele.max_frames || Math.min(FPS_MAX_FRAMES, totalFrames),
        FRAME_FACTOR
    );
    const duration = frameRate !== 0 ? parseFloat(totalFrames / frameRate) : 0;

    let totalFramesAdjusted = duration % 1 > (1 / fps)
        ? Math.ceil(duration * frameRate)
        : Math.ceil(Math.floor(parseInt(duration)) * frameRate);

    const nframes = (totalFramesAdjusted / frameRate) * fps;
    const finalNFrames = parseInt(Math.min(
        Math.max(nframes, minFrames),
        Math.min(maxFrames, totalFramesAdjusted)
    ));

    if (finalNFrames < FRAME_FACTOR || finalNFrames > totalFramesAdjusted) {
        throw new Error(
            `nframes should be between ${FRAME_FACTOR} and ${totalFramesAdjusted}, got ${finalNFrames}`
        );
    }
    return finalNFrames;
}

// Smartly adjust resolution
async function smartResize(ele, videoPath) {
    const { height, width, totalFrames, frameRate } = await getVideoInfo(videoPath);
    const minPixels = VIDEO_MIN_PIXELS;
    const nframes = smartNFrames(ele, totalFrames, frameRate);
    const maxPixels = Math.max(
        Math.min(VIDEO_MAX_PIXELS, VIDEO_TOTAL_PIXELS / nframes * FRAME_FACTOR),
        Math.floor(minPixels * 1.05)
    );

    // Check aspect ratio
    const ratio = Math.max(height, width) / Math.min(height, width);
    if (ratio > MAX_RATIO) {
        throw new Error(`Aspect ratio ${ratio} exceeds ${MAX_RATIO}`);
    }

    let hBar = Math.max(IMAGE_FACTOR, roundByFactor(height, IMAGE_FACTOR));
    let wBar = Math.max(IMAGE_FACTOR, roundByFactor(width, IMAGE_FACTOR));

    if (hBar * wBar > maxPixels) {
        const beta = Math.sqrt((height * width) / maxPixels);
        hBar = floorByFactor(height / beta, IMAGE_FACTOR);
        wBar = floorByFactor(width / beta, IMAGE_FACTOR);
    } else if (hBar * wBar < minPixels) {
        const beta = Math.sqrt(minPixels / (height * width));
        hBar = ceilByFactor(height * beta, IMAGE_FACTOR);
        wBar = ceilByFactor(width * beta, IMAGE_FACTOR);
    }

    return { hBar, wBar };
}

// Calculate token count
async function tokenCalculate(videoPath, fps) {
    const messages = [{ content: [{ video: videoPath, fps }] }];
    const visionInfos = extractVisionInfo(messages);

    const { hBar, wBar } = await smartResize(visionInfos[0], videoPath);
    const { height, width, totalFrames, frameRate } = await getVideoInfo(videoPath);
    const numFrames = smartNFrames(visionInfos[0], totalFrames, frameRate);

    console.log(
        `Original video size: ${height}*${width}, size for model input: ${hBar}*${wBar}, total frames of video: ${totalFrames}, when fps is ${fps}, total frames extracted: ${numFrames}`
    );

    const videoToken = Math.ceil(numFrames / 2) * Math.floor(hBar / 28) * Math.floor(wBar / 28) + 2;
    return videoToken;
}

// Extract vision information
function extractVisionInfo(conversations) {
    const visionInfos = [];
    if (!Array.isArray(conversations)) {
        conversations = [conversations];
    }
    conversations.forEach(conversation => {
        if (!Array.isArray(conversation)) {
            conversation = [conversation];
        }
        conversation.forEach(message => {
            if (Array.isArray(message.content)) {
                message.content.forEach(ele => {
                    if (ele.image || ele.image_url || ele.video || ['image', 'image_url', 'video'].includes(ele.type)) {
                        visionInfos.push(ele);
                    }
                });
            }
        });
    });
    return visionInfos;
}

// Example usage
(async () => {
    try {
        const videoPath = "xxx/test.mp4"; // Replace with your local path
        const videoToken = await tokenCalculate(videoPath, 1);
        console.log('Video tokens:', videoToken);
    } catch (error) {
        console.error('Error:', error.message);
    }
})();

For concurrent rate limiting, see Rate limits.

Get started

Prerequisites: You must have obtained an API key and configured it as an environment variable. To use the SDKs, you must install OpenAI or DashScope SDK. The DashScope SDK for Java must be version 2.19.0 or later.

Important

Due to the long reasoning time, currently QVQ only supports streaming output.
Thinking cannot be disabled for QVQ.
QVQ does not support System Message.
For the DashScope method;
- incremental_output defaults to true and cannot be false.
- result_format defaults to "message", and cannot be "text".

The following sample codes use image URL for understanding.

Check the limitations on input images section. To use local images, see Using local files.