HarmonyOS 7.0 Text-to-Image Search: Building a Storage Memory Box Retrieval Loop
The article demonstrates building a "storage memory box" app using HarmonyOS 7.0's text-to-image search (Core Vision Kit) to retrieve photos of stored items and their logged locations, covering data modeling, implementation flow, candidate resolution, UI state management, and validation methodology.
Problem Context
Seasonal storage often leads to forgotten box locations. Even with photos, searching by capture date is inefficient. The author builds a "Storage Memory Box" app using HarmonyOS 7.0's text-to-image search (Core Vision Kit) to let users describe an item (e.g., "red knitted scarf") and retrieve the relevant photo plus its logged storage location.
Capability Boundaries
Core Vision Kit's textSearchImage searches app-sandbox images by descriptive text, returning imagePath and similarity. It does not know storage box numbers; room, box ID, and registration time must be maintained by the app. The UI therefore shows "registered location" and "registered time" — historical clues, not real-time positions. The article targets HarmonyOS 7.0, API/SDK 26.0.0, noting concurrency limits and no emulator support; real-device testing is required.
Mechanism Design
Three Data Types, Three Responsibilities
Original pixels, retrieval index (image features), and location records are distinct. Existence of an original photo doesn't guarantee indexing; successful indexing doesn't guarantee complete location data. The app must interpret candidates from the API before display.
Data Model (ArkTS)
interface RoomRecord { roomId: string; name: string; }interface BoxRecord { boxId: string; roomId: string; label: string; }interface PhotoRecord { photoId: string; path: string; boxId: string; registeredAt: number; }Example: bedroom R01, box B02 ("top closet storage box"), photo P01 linked to B02. Stable IDs avoid rename issues; capture time is stored separately from registration time.
Identity Not Guessed from Filenames
Search returns paths; the app maps the exact indexed path to a PhotoRecord, then follows boxId → roomId. Filenames can collide across directories; item identity (e.g., item-01) is separate from photo identity ( P01). New photos of the same item get new photo IDs.
Candidate Card as Conditional Promise
When UI shows a photo and "Bedroom·Top Closet·B02", the app must verify the linkage before display. Missing photo record → "photo record missing"; unreadable file → "original unavailable"; missing box/room → "registration info missing". Similarity scores are kept in debug logs, not shown as probabilities.
Core Implementation
Repeatable Call Sequence
Fixed set of 10–20 photos (including target P01 and distractors). Each photo gets a metadata record, then indexing. Library version, actual paths, and results are logged for reproducibility.
Official entry: import { textSearchImage } from '@kit.CoreVisionKit'. Basic order: init → per-photo insertImage → search → release. Pseudocode for a single prepare-and-search task:
prepareAndSearch(config, photos, query):
initialized = false
stage = "初始化"
try:
await vision.initializeChecked()
initialized = true
stage = "入库"
for photo in photos:
await files.requireReadable(photo.path)
await vision.insertChecked(photo.path, config.scope)
log.recordInserted(photo.photoId)
stage = "查询"
hits = await vision.searchChecked(query, config.scope, config.topKey)
log.saveRawHits(hits)
return hits
catch error:
log.recordFailure(stage, error)
throw error
finally:
if initialized:
await vision.releaseAndRecordError()Any insertion failure aborts the whole batch; completed insertions are not rolled back. Release errors are logged separately. Interactive pages can defer release until session end, but must await current task completion; release is not a cancellation mechanism nor does it delete originals/index.
Converting Results to Displayable Information
Candidate resolution is isolated in a function that answers "can this candidate be verified by the user now?" Pseudocode:
resolveCandidate(hit):
photo = photos.findByExactPath(hit.imagePath)
if photo is missing:
return unavailable("照片记录缺失")
if not await files.canRead(photo.path):
return unavailable("原图不可用", photo.photoId)
box = boxes.find(photo.boxId)
room = box exists ? rooms.find(box.roomId) : missing
if box is missing or room is missing:
return imageOnly(photo.path, "登记信息缺失")
return candidate(
photoId = photo.photoId,
imagePath = photo.path,
location = room.name + "·" + box.label + "·" + box.boxId,
registeredAt = photo.registeredAt
)For P01 this resolves to R01 and B02. Detail view re-checks file readability because state may change between list generation and user tap. Raw API results are saved before transformation to distinguish retrieval failures from data-linkage failures.
Page State Follows Task State
Distinct states for library preparation (progress shown, search disabled) and querying (duplicate submissions blocked). Outcomes: candidate list, empty results ("no usable candidates found"), or failure (retry prompt). Unreadable images are reported as file issues, not lumped into "not found".
Validation Methodology
Verify Both Retrieval and Answer
Test conditions fixed: device, OS, API/SDK, region, date, library version, indexed count, query Q01 ("red knitted scarf"), request params. Relevance list frozen before run.
Two observations: (1) whether P01 appears in results and at what rank; (2) whether displayed candidate opens correctly and shows matching photo ID and location. Both recorded separately.
Evidence to save for each check object:
Raw candidates — Paths, original order, similarity — Answers: What did the interface return?
Files — Read results, open records — Answers: Can user verify the photo?
Location linkage — Photo ID, box ID, room, registration time — Answers: Does displayed info belong to that photo?
UI — Displayed items, unavailable items with reasons — Answers: How does the app interpret results?
Report candidate openable count vs. total checked, location-match count vs. verifiable records, and list unmatched entries. Counting only successful opens hides missing items.
Actively Check Failure Paths
Missing target, irrelevant similar images — record "not observed" without fabricating errors. File unreadability and missing metadata are actively induced: temporarily move files or remove box links in isolated records, then verify app shows correct unavailable state. Restore baseline after test. If the API doesn't return a moved file's path, inject that path into the resolver to test error handling — this proves error handling works, not that the index returns stale paths.
Filename Baseline Comparison
Same query Q01 against filenames (e.g., P01.jpg lacks "red knitted scarf") shows filename search cannot use visual appearance. Text-to-image search provides a different entry point. The comparison explains what information each method uses, not to declare one universally superior.
Conclusion
Version 0.1 establishes the core chain: image features enable query → candidate path links to photo record → photo record links to manually registered location. Along this chain, basic calls, candidate interpretation, and validation steps are individually verifiable. Stable baseline: same app, same scarf ( item-01), same photo P01, same query Q01, same location B02. Next article varies photo capture angle (e.g., only a corner visible in a group shot) to study indexing and capture strategies, assigning new photo IDs while retaining P01.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
