DeepSeek V4 Flash Vision Exp for E2E Testing: Midscene 1.12 Benchmarks & Integration

Feature Overview

Midscene 1.12 adds official support for DeepSeek V4 Flash Vision Exp, a multimodal model from DeepSeek. The article evaluates its performance for end-to-end (E2E) testing across speed, accuracy, and cost.

UI Location Speed Leads

In Midscene's standard location mode, DeepSeek V4 Flash Vision Exp shows a clear speed advantage. The P50 (median) latency is 0.6 seconds, meaning half of location requests complete within 0.6s. Compared to the fastest P50 of other models, it leads by 0.8–1 second; against mainstream flagship models, the gap widens to 1.5–2 seconds.

Competitive Location Accuracy

On clear-target, simple-structure pages, DeepSeek's accuracy approaches some flagship models. However, on dense or complex interfaces with many elements, it tends to mis-select or offset elements, lagging behind latest flagship versions. Enabling Midscene's deepLocate optimization raises accuracy to near-flagship levels while keeping P50 around 1.4s, retaining a performance edge.

Low Token Cost

Per official pricing, uncached input costs 3 CNY per million tokens at peak, 1.5 CNY off-peak, with each image counted as up to 384 tokens. At the upper bound, 10,000 images ≈ 3.84M tokens. Even at peak rates, 10k images cost under 12 CNY; off-peak or with caching, costs drop further, offering significant optimization for E2E test budgets.

Applicable Scenarios & Selection Advice

DeepSeek's standout advantage is response speed. It performs well on straightforward pages but still trails flagships on dense, small-target, or spatially complex UIs. While the low cost (≈12 CNY per 10k images) is attractive, many domestic models now fall within affordable ranges for E2E testing, so price alone shouldn't drive selection. Real E2E tasks also require complex page-state understanding and step planning; as DeepSeek's first official multimodal API, its capability in these areas needs further validation. The authors recommend developers test multiple Midscene-supported models against their specific business scenarios, comparing location effectiveness, response speed, call cost, and task planning to choose the best fit.

Case Demonstrations

Case 1: Link Game (连连看)

A demo video shows Midscene + DeepSeek playing a Link game entirely via visual location, unedited and unaccelerated, with operation speed approaching human level. Full report:

https://lf3-static.bytednsdoc.com/obj/eden-cn/luljzkpt/ljhwZthlaukjlkulzlp/showcases/deepseek/deepseek-link.html

Case 2: GitHub Signup

Another video demonstrates autonomous form filling for GitHub registration (select China, pass validations, stop before final submit), completing in ~50 seconds unedited. Task instruction:

Fill out the GitHub sign-up form, select China as the country, and pass all form validations, but do not click the final “Create account” or submit button.

Full report:

https://lf3-static.bytednsdoc.com/obj/eden-cn/luljzkpt/ljhwZthlaukjlkulzlp/showcases/deepseek/deepseek-github-signup.html

Integration Guide

Upgrade to the latest Midscene version.

Set the following environment variables:

MIDSCENE_MODEL_BASE_URL="https://api.deepseek.com";
MIDSCENE_MODEL_API_KEY="......";
MIDSCENE_MODEL_NAME="deepseek-v4-flash-vision-exp";
MIDSCENE_MODEL_FAMILY="deepseek";

Complete guide:

https://midscenejs.com/zh/model-common-config.html#deepseek

Getting the New Version

Official site: https://midscenejs.com GitHub repo: https://github.com/web-infra-dev/midscene Changelog:

https://midscenejs.com/zh/changelog.html