Quick Guide to Using ModelScope Library for Multi‑Modal AI Model Inference

Overview

The ModelScope Python library offers a complete set of tools for loading official models, performing inference, fine‑tuning, data preprocessing, post‑processing, and evaluation across five AI domains (vision, NLP, speech, multimodal, and scientific). With a few lines of code, users can run inference, training, and evaluation, and extend the library for custom development.

Supported Deep Learning Frameworks

ModelScope currently supports PyTorch and TensorFlow. Future updates will add more frameworks. All official models can be invoked through the library’s pipeline API.

Model Inference Pipeline

An inference pipeline typically consists of three stages: data pre‑processing, model forward pass, and data post‑processing. The core function pipeline() enables one‑line inference for a wide range of tasks.

Pipeline Basics

Environment preparation

Key parameters

Basic pipeline usage

Specifying custom pre‑processor and model

Examples for different scenarios

Chinese Word Segmentation Example

Load the word‑segmentation task and run inference on a single sentence:

from modelscope.pipelines import pipeline
word_segmentation = pipeline('word-segmentation')
input_str = '开源技术小栈作者是Tinywan，你知道不？'
print(word_segmentation(input_str))

PHP equivalent (using PyCore):

<?php
$operator = PyCore::import('operator');
$builtins = PyCore::import('builtins');
$pipeline = PyCore::import('modelscope.pipelines')->pipeline;
$word_segmentation = $pipeline('word-segmentation');
$input_str = "开源技术小栈作者是Tinywan，你知道不？";
PyCore::print($word_segmentation($input_str));
?>

Online conversion tool: https://www.swoole.com/py2php/

Batch Inference

The pipeline can process a list of inputs, returning a list of results. Batch size can be controlled via the batch_size argument.

inputs = ['今天天气不错，适合出去游玩', '这本书很好，建议你看看']
print(word_segmentation(inputs, batch_size=2))

Dataset Input

Use MsDataset to wrap a list of sentences and feed it directly to the pipeline:

from modelscope.msdatasets import MsDataset
from modelscope.pipelines import pipeline

dataset = MsDataset.load(['今天天气不错，适合出去游玩', '这本书很好，建议你看看'], target='sentence')
word_segmentation = pipeline('word-segmentation')
for out in word_segmentation(dataset):
    print(out)

Custom Pre‑processor and Model

Instantiate a model and optionally a pre‑processor, then pass them to pipeline() for full control:

from modelscope.models import Model
from modelscope.pipelines import pipeline
from modelscope.preprocessors import Preprocessor

model = Model.from_pretrained('damo/nlp_structbert_word-segmentation_chinese-base')
preprocessor = Preprocessor.from_pretrained(model.model_dir)
word_segmentation = pipeline('word-segmentation', model=model, preprocessor=preprocessor)
print(word_segmentation(['开源技术小栈作者是Tinywan，你知道不？', 'webman这个框架不错，建议你看看']))

Image Matting Pipeline

Install OpenCV before running image‑related pipelines: pip install opencv-python Run the portrait-matting pipeline on an online image and save the result:

import cv2
from modelscope.pipelines import pipeline

portrait_matting = pipeline('portrait-matting')
result = portrait_matting('https://modelscope.oss-cn-beijing.aliyuncs.com/test/images/image_matting.png')
cv2.imwrite('result.png', result['output_img'])

PHP version (using PyCore):

<?php
$operator = PyCore::import('operator');
$builtins = PyCore::import('builtins');
$cv2 = PyCore::import('cv2');
$pipeline = PyCore::import('modelscope.pipelines')->pipeline;
$portrait_matting = $pipeline('portrait-matting');
$result = $portrait_matting('https://modelscope.oss-cn-beijing.aliyuncs.com/test/images/image_matting.png');
$cv2->imwrite('tinywan_result.png', $result->__getitem__('output_img'));
?>

Load a local image file with $portrait_matting('./tinywan.png') .