Developer Diary: Building a Handy Web‑to‑Markdown Tool with Jina AI

Problem: LLM knowledge staleness

Large language models can only answer based on the data they were trained on and cannot retrieve the latest web pages, news or documentation. Enterprise agents that need current product information, stock prices or technical specs therefore require a way to fetch real‑time content.

Solution: JinaReaderTool

JinaReaderTool is a lightweight Java implementation that forwards a request to the free Jina AI web‑content extraction service. The service is invoked with a URL of the form https://r.jina.ai/http://{targetUrl} (or the HTTPS variant) and returns the page as structured Markdown, automatically removing ads, navigation and other irrelevant elements.

Key features

Free to use – no registration or API key required.

Clean output – extracts the main text and filters out advertisements and navigation.

Markdown format – produces structured text that LLMs can consume directly.

Supports both HTTP and HTTPS URLs.

Implementation details

The tool implements the AgentTool interface. It validates the url parameter, builds a Jina‑compatible request URL that tolerates missing or mixed protocols, and performs an asynchronous HTTP request using Feat’s HTTP client. Errors are captured with exceptionally so the agent can decide how to handle failures. The code stays compatible with JDK 8 and uses CompletableFuture for async processing without any JDK 9+ features.

public class JinaReaderTool implements AgentTool {
    private static final String NAME = "jina_web_reader";
    private static final String DESCRIPTION = "使用 Jina AI 读取任意网页的内容并返回结构化文本";

    @Override
    public CompletableFuture<String> execute(JSONObject parameters) {
        String url = parameters.getString("url");
        if (FeatUtils.isBlank(url)) {
            return CompletableFuture.completedFuture("错误：必须提供 'url' 参数");
        }
        String jinaUrl = buildJinaUrl(url);
        return Feat.httpClient(jinaUrl, httpOptions -> httpOptions.debug(false))
            .get().submit()
            .thenApply(response -> formatResult(url, response.body()))
            .exceptionally(t -> "执行失败: " + t.getMessage());
    }

    private String buildJinaUrl(String targetUrl) {
        if (targetUrl.startsWith("http://")) {
            return "https://r.jina.ai/http://" + targetUrl.substring(7);
        } else if (targetUrl.startsWith("https://")) {
            return "https://r.jina.ai/https://" + targetUrl.substring(8);
        }
        return "https://r.jina.ai/https://" + targetUrl;
    }
    // getName(), getDescription(), getParametersSchema() omitted for brevity
}

Registering the tool with a Feat agent

FeatAgent agent = FeatAI.agent(opts -> {
    opts.chatOptions().model(ChatModelVendor.GiteeAI.Qwen2_5_72B_Instruct);
    opts.tool(new JinaReaderTool());
    opts.maxIterations(10);
});
String result = agent.execute("访问 https://smartboot.tech 并总结主要内容").get();

When a user query contains a URL, the agent automatically selects the jina_web_reader tool, calls the Jina API, receives the Markdown content, and uses it to answer the question without any manual intervention.

Comparison with Feat's built‑in WebPageReaderTool

Dependency : JinaReaderTool calls the external Jina AI service; WebPageReaderTool performs a local HTTP request and HTML parsing.

HTML parsing : Handled in the cloud by Jina AI for JinaReaderTool; performed locally for WebPageReaderTool.

Network requirement : JinaReaderTool needs outbound access to r.jina.ai; WebPageReaderTool accesses the target site directly.

Special handling : JinaReaderTool has no site‑specific adapters; WebPageReaderTool includes custom parsers for Baidu, Bing and OSChina.

Result format : JinaReaderTool returns plain Markdown; WebPageReaderTool returns Markdown plus site metadata.

Choose JinaReaderTool for simplicity and zero HTML‑parsing code, or WebPageReaderTool when fine‑grained control, internal‑network operation, or extra metadata are required.

Typical use cases

News summarization – e.g., "Read this news article and summarize the key points: https://example.com/news".

Product research – e.g., "Analyze competitor pricing: https://competitor.com/pricing".

Technical documentation – e.g., "Extract installation steps from https://docs.example.com".

Content monitoring – periodically fetch pages to track price or content changes.

Limitations

Jina AI imposes request‑rate limits.

Some websites block crawling.

Pages rendered with JavaScript may not be fully captured.

Conclusion

With fewer than a hundred lines of Java, JinaReaderTool demonstrates how a compact, well‑encapsulated component can give Feat AI agents the ability to break through the static knowledge boundary of LLMs and fetch fresh internet information in real time.