How to Upload, Test, and Deploy MiniLM on Modelers.cn: A Step‑by‑Step Guide

About Modelers.cn

Modelers.cn is a neutral, non‑profit AI community that provides hosting, display and collaborative services for AI tools, models and data, creating an open platform for developers and enthusiasts.

What is MiniLM?

MiniLM is a compression method for Transformer‑based pretrained models that reduces model size via knowledge distillation while preserving or improving accuracy, making it suitable for resource‑constrained environments such as mobile or embedded devices. It can be applied to various NLP tasks including text classification, named‑entity recognition, question answering and language generation.

Why Test a Model?

Testing ensures model stability and usability across different environments, helps other users configure the model correctly and reduces errors.

Uploading and Testing MiniLM on Modelers.cn

When uploading, include an

example

folder with test scripts and configuration files. After upload, run the built‑in usability test; if it fails, consult the logs.

Running a Simple Test with openMind

<code>from openmind import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("Congruent/MiniLM-L12-H384-uncased")
tokenizer = AutoTokenizer.from_pretrained("Congruent/MiniLM-L12-H384-uncased")
inputs = tokenizer("Hello world!", return_tensors="pt")
outputs = model(**inputs)
print(outputs)
</code>

The result is shown in the following screenshot.

Fine‑tuning MiniLM for Classification

Using the Iris dataset, a logistic regression classifier is trained on features extracted by MiniLM. The accuracy is printed at the end.

<code>import torch
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from transformers import AutoModel, AutoTokenizer, pipeline
from datasets import load_dataset

model = AutoModel.from_pretrained("Congruent/MiniLM-L12-H384-uncased")
tokenizer = AutoTokenizer.from_pretrained("Congruent/MiniLM-L12-H384-uncased")
dataset = load_dataset('imdb')
# ... (feature extraction and training code) ...
print(f'Accuracy: {accuracy:.4f}')
</code>

Simple Question‑Answering with MiniLM

Encode questions and candidate answers, compute cosine similarity, and return the most similar answer.

<code>import torch
from openmind import AutoModel, AutoTokenizer
from sklearn.metrics.pairwise import cosine_similarity
import numpy as np

model_name = "Congruent/MiniLM-L12-H384-uncased"
model = AutoModel.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

qa_pairs = [
    {"question": "What is the capital of France?", "answer": "Paris"},
    {"question": "Who is the CEO of Apple?", "answer": "Tim Cook"},
    {"question": "How many legs does a spider have?", "answer": "Eight"}
]

# functions encode_question_and_answers, find_best_answer ...

for question in ["What is the capital of France?", "Who leads Apple?", "How many legs do spiders have?"]:
    best_answer = find_best_answer(question, qa_pairs, tokenizer, model)
    print(f"Question: {question}\nAnswer: {best_answer}\n")
</code>

Running the script yields the expected answers.

Experience Space

An experience space was created on Modelers.cn to interactively test the QA system. Screenshots of the interface are shown below.