Industrial Agent UX: Designing for Gloves, Noise, and Offline Reality

Follow a Task End-to-End Before Choosing Voice or Glasses

Design starts by observing how maintenance personnel receive a task, locate equipment, review records, contact specialists, and submit results — under actual site conditions, across shifts and roles. Office-based chat simulations cannot replace this. HSE human-factors guidance stresses involving people who know the work and accounting for task, equipment, and environment effects on performance.

For a quality anomaly, the interaction can be split into five steps: confirm equipment and batch, review anomaly and evidence, acknowledge ownership, record inspection facts, submit disposition. Each step first answers "what does the person need to know, what does the system need to receive" before selecting input/output methods. Acknowledging a task may need only a single clear action, but "notification received" must be distinguished from "accepting responsibility for subsequent actions". Chat and task cards can coexist when each handles appropriate work.

The concept of micro-interactions (NN/G) applies: a concrete trigger paired with targeted feedback. Pressing "Confirm Acceptance" shows "Submitting"; on server confirmation the task displays the assignee. If the request times out, show "Acceptance status pending verification" — not a premature success tick. "Micro" means the handled problem is specific, not that buttons are small or necessary confirmations are omitted.

First Screen: Identify the Object, Clarify the Current Task

When a maintainer opens a task, they must immediately confirm: which equipment, which location, which batch, which time window, and what they are now responsible for. A task card presents object ID, anomaly summary, current owner, confirmed status, and pending items, with links to raw inspection records and candidate explanations — organized by task, not a compressed report.

Critical constraints must not be hidden in collapsible sections. For example, "current batch not yet verified" or "referenced case does not cover current tool combination" should appear alongside recommendations if they affect the next judgment. Candidate causes must be explicitly labeled as unverified; prominent fault icons must not turn speculation into confirmed conclusions.

Scanning a QR code reduces manual entry of equipment numbers, but a scanned label does not guarantee correct association. The UI must still show equipment name, location, and key identifiers for human verification. If labels are damaged, duplicated, or mismapped, provide error feedback and a manual lookup entry — never silently pick a similar object.

Information needs differ by role: frontline staff need pending items and operating conditions; specialists need trends, raw records, versions; managers need impact scope and open tasks. Layered display is acceptable, but evidence required for accountability must not be hidden.

Two illustrative sketches contrast the approaches (images are mechanistic illustrations, not validated designs):

Figure A — Next action buried in a long chat: The layout forces reading a lengthy report before replying, yet it remains unclear how to accept the task.

Figure B — Confirm object first, then accept task: After object verification, the task shows a pending-acceptance state. On acceptance, the UI moves to recording inspections, reporting material shortages (with reason codes), and handling holds and handovers per site process. The interface provides actions appropriate to each task state, using text to clarify actions, color only as aid. Touch target sizes and spacing must still be validated with actual gloves.

Choose Interaction Methods Together with Their Failure Paths

The following comparison table (engineering synthesis, not a terminal ranking) maps roles/tasks to candidate interactions, required verifications, and fallback behaviors:

Maintainer confirms object: Scan code / existing terminal query. Verify name, location, ID match. On label anomaly: manual lookup and log conflict.

Shift personnel accept task: Structured task card with explicit button. Verify acceptance scope, personal identity, current task status. On unclear submission state: query first, no repeated clicks.

Specialist verifies evidence: Evidence pages paired with chat for follow-up questions. Verify source, time window, applicability conditions. On missing or conflicting data: enter evidence-supplementation flow.

Field personnel add observations: Short options, text, approved voice or photo. Verify object, negation words, numeric values and units, record nature. On transcription failure: switch to available entry, keep incomplete status.

Shift handover and feedback: Open-items checklist, result forms. Verify who takes over, what is done, what awaits review. On system outage: use existing handover channels, reconcile and backfill after recovery.

Glove suitability for touchscreens must be tested with the actual gloves, screens, postures, and environment — button size, spacing, click feedback, and adjacent-action consequences all matter. A designer's accurate taps at a desk do not prove a maintainer holding a rugged device with safety gloves can do the same. If a routine acceptance forces removal of mandated gloves, adjust the entry point or terminal, don't blame user unfamiliarity.

Fixed terminals or physical buttons may suit few well-defined actions, but must clearly indicate current object and function — avoid one button changing meaning with hidden pages. Mobile devices travel with the task but face battery, network, cleanliness, and site-use limits. Wearables or AR must be evaluated for donning, PPE compatibility, field of view, comfort, and maintenance. If the overlay only shows a model's suspected location, it must not be presented as a confirmed fault point. If existing terminals already complete the task, adding a new device and its maintenance burden may not be justified.

Voice Can Record, Not Replace Execution Authorization

Voice suits some hands-busy situations, but background noise, multiple speakers, accents, equipment numbers, and domain abbreviations must be validated locally. A smooth demo in a quiet meeting room does not prove field suitability.

In the example, "no looseness found" mis-transcribed as "looseness found" would flip subsequent diagnosis. The system must let users verify critical transcription results — especially object references, negations, numbers, and units. When reliable verification is impossible, retain as a pending-confirmation record. Confirming transcription accuracy only means the report was captured faithfully; whether it suffices as diagnostic evidence depends on observation scope and supporting evidence.

Distinguish raw human observation from agent summarization. A worker saying "sound differs from normal" must not auto-become "bearing anomaly" in the summary. Corrections to transcriptions should preserve necessary amendment records and accountability.

If users must stop work to read confirmations, provide appropriate confirmation moments instead of demanding simultaneous operation and screen-watching. When site rules prohibit recording, photography, or wearing certain devices, the design must offer alternative entry points — PPE requirements cannot be waived to accommodate an interaction mode. Reducing duplicate entry has value, but critical identity, object, and authorization confirmations must remain. "Zero-click" must not turn a background conversation into an execution permit.

When Offline, Don't Label "Saved Locally" as "Submitted"

The word "Done" on screen easily conflates distinct states: record saved locally, task accepted, inspection executed, or quality issue cleared after review. These states must be explicitly separated.

For example, after clicking "Submit Inspection Record", feedback should distinguish "Local draft saved", "Server received", and "Awaiting review". Without a server receipt, indicate local save success but never claim submission complete. The system must provide status query and prevent duplicate processing of the same request — not just warn users not to click repeatedly.

Offline capability must be scoped: allowed to view cached documents with version and sync timestamp, store observation drafts, but never display stale cache as current equipment status. Online-required actions (acceptance, authorization, state changes) must be clearly marked unconfirmable offline and guide users to existing site channels.

On reconnection, verify whether someone else already accepted or completed the task before processing pending syncs — don't blindly replay all offline clicks. Offline observations retain actual observation time and backfill time; conflicts are referred to the responsible person, not automatically overwritten by newer records. Key state changes need persistent visible results, not fleeting toasts.

Shift handover follows the same principle. Auto-generated summaries reduce clerical work, but both outgoing and incoming parties must still verify open items. The UI must preserve whether responsibility transfer completed, not substitute "message read" for acceptance confirmation.

More Alerts Do Not Mean Clearer Awareness

Repeated pushes for the same anomaly and low-priority interruptions can drown the items that truly need response. Alert design must state why this role is notified, what response is required, and when escalation triggers.

Industrial interfaces should not measure value by dwell time. After necessary review and confirmation, attention should return to the field; subsequent reminders need concrete task reasons. HSE alarm management guidance emphasizes alarms must be useful, relevant, have explicit response requirements, and consider personnel capability and response time — this principle of matching information to task is borrowed, not equating agent messages with safety alarms.

Agent may merge duplicate notifications for the same task per confirmed rules, but new risks, scope changes, and escalation notices must not be suppressed just because the task ID matches, nor should they override existing safety alarms. Reducing noise must not make unhandled tasks disappear; open-item lists and escalation paths must persist.

Critical cues must not rely solely on color or sound; clear text, shape, or status indicators are required. Actual font size, contrast, display position, and alert method must be validated against viewing distance, lighting, and PPE conditions. HSE control-room design guidance covers display, control, and environmental coordination, but control-room experience does not directly prove which voice or wearable solution suits mobile maintenance.

Acceptance Tests Task Completion, Not Just Smooth UI

Before testing, field roles and designers agree on tasks, conditions, and unacceptable errors. Representative personnel — including new and experienced staff, different shifts, and actual task owners — perform controlled tests. Developers alone demonstrating is insufficient.

Tests start with playback and isolated environments, then progress to site-approved scope. Use approved gloves and terminals; simulate network loss, recognition failure, task takeover by others — without creating hazardous noise, equipment faults, or removing protections.

Record wrong-object selections, missed constraints, mis-confirmations, task completion times, and ability to continue after failure. Faster average speed that increases batch misidentification is not improvement; counting only successful tasks hides abandonment and phone-call fallback costs. When comparing two interfaces, keep task difficulty, training, and environment comparable to avoid conflating proficiency differences with interaction gains.

Acceptance must trace downstream: do field-submitted records link to the correct task? Can specialists understand them? Is responsibility cleanly transferred? If easier entry creates more ambiguity and rework, adjust fields and verification methods.

Interaction designers own the UI, system engineers own state and receipts, domain experts confirm terminology and evidence meaning, safety personnel assess usage conditions. Low-risk local improvements can run small-scale A/B; changes affecting critical confirmations require corresponding re-validation. Final choice of voice, task cards, or existing terminals must be supported by these task results.

Summary

Good field interaction lets people, under permitted working conditions, identify the right object, see the evidence, complete necessary confirmations, and pass accurate facts to the next station. It may include chat, or it may need only a task card and clear feedback.

References

[1] HSE: Design. Used to illustrate requirements for user involvement and task, equipment, and environment human-factors design. https://www.hse.gov.uk/humanfactors/topics/design.htm [2] HSE: Alarm management. Used to clarify principles of explicit response, information relevance, and personnel capability; not to assign safety-alarm functions to agent messages. https://www.hse.gov.uk/humanfactors/topics/alarm-management.htm [3] HSE: Control room design. Used for coordination of displays, controls, and environmental conditions; not to copy control-room parameters or extrapolate to mobile maintenance terminals. https://www.hse.gov.uk/comah/sragtech/techmeascontrol.htm [4] Nielsen Norman Group: Microinteractions in User Experience. Used to explain the trigger-feedback relationship of micro-interactions; not to prove task-card effectiveness in industrial settings. https://www.nngroup.com/articles/microinteractions/