Build a Real‑Time Webcam Posture Monitor with TensorFlow.js

Background

Remote work increases the risk of poor posture, irregular meals, insufficient hydration, and low activity. The solution treats personal health as a software requirement and implements a lightweight web application that runs entirely in the browser.

Posture Monitoring

1. Video Capture

Access the built‑in webcam (or an external camera) with the HTTPS‑only API navigator.mediaDevices.getUserMedia. The call returns a Promise that resolves to a MediaStream which can be attached to a <video> element.

2. Pose Estimation

Load TensorFlow.js PoseNet model ( posenet.load()) with default parameters (architecture, outputStride, inputResolution, multiplier, quantBytes). After the model is ready, call estimateSinglePose(videoFrame, {flipHorizontal:true}) to obtain a pose object.

{
  "score": 0.36588028040440645,
  "keypoints": [
    {"score":0.998099148273468,"part":"nose","position":{"x":318.63,"y":371.86}},
    {"score":0.996922492980957,"part":"leftEye","position":{"x":260.77,"y":307.91}}
    // ... other keypoints
  ]
}

The score field (0‑1) indicates confidence; values below 0.1 can be discarded as “no person detected”.

3. Posture Classification

Compute geometric relationships between keypoints to detect six common bad‑posture patterns. Example calculations:

Slouching : distance from nose to bottom of the video frame is below a calibrated threshold.

Chin on hand : absolute difference between the Y‑coordinates of the two eyes exceeds a small tolerance, indicating head tilt.

Cross‑eye (head turned) : difference between left eye‑left ear X‑distance and right eye‑right ear X‑distance exceeds a threshold.

Looking up : vertical gap between nose Y and eye Y is smaller than a lower bound.

Looking down : vertical gap between nose Y and ear Y is smaller than a lower bound.

Too close to screen : inter‑eye X‑distance is larger than a calibrated “near” value.

Thresholds are obtained empirically: move the head to the extremes, record the numeric values, and choose margins that separate “good” from “bad” postures.

4. Alerting Logic

Maintain a timestamp for each detected bad‑posture condition. When the same condition persists for more than three seconds, trigger an alert:

Play a short audio cue using new Audio('alert.mp3').play().

Optionally use a Google‑Translate TTS file that announces the specific posture.

To reduce false positives (e.g., stepping away, drinking water), ignore detections with pose.score < 0.1. The detection loop can run with setInterval(()=>{…}, 500) for a balance between responsiveness and CPU usage, or with requestAnimationFrame for smoother visual feedback at the cost of higher load.

Meal Suggestion

Define four ingredient categories (staple, protein, vegetables, snacks). Implement a chooseMeal() function that randomly selects one item from each category, then shuffles the selections to produce a meal plan. Add de‑duplication logic so that two consecutive calls never return the identical combination. Expose the function through a UI button that updates the displayed suggestion.

Water and Exercise Reminders

Configure four daily reminder times (e.g., 09:00, 12:00, 15:00, 18:00). Use a setInterval that runs every minute, compares new Date() to the target times, and when a match occurs, plays the same audio mechanism used for posture alerts. Separate timers can be created for water‑drink prompts and stretch‑break prompts.

Implementation Sketch

// Initialize webcam
const video = document.querySelector('#video');
navigator.mediaDevices.getUserMedia({video:true})
  .then(stream=>video.srcObject=stream);

// Load PoseNet
let net;
posenet.load().then(model=>{net=model;});

// Main loop (every 500 ms)
setInterval(async()=>{
  if(!net) return;
  const pose = await net.estimateSinglePose(video, {flipHorizontal:true});
  if(pose.score<0.1) return; // no person
  const posture = classify(pose.keypoints);
  handlePosture(posture);
}, 500);

function classify(kps){ /* compute distances, return string or null */ }
function handlePosture(p){ /* start timer, play audio after 3 s */ }

// Meal suggestion
function chooseMeal(){ /* random pick + dedup */ }

// Reminder timers
const reminders=[{type:'water',hour:9,minute:0},{type:'stretch',hour:10,minute:30}];
setInterval(()=>{
  const now=new Date();
  reminders.forEach(r=>{
    if(now.getHours()===r.hour && now.getMinutes()===r.minute){
      new Audio(`${r.type}.mp3`).play();
    }
  });
},60000);

References

TensorFlow.js PoseNet repository: https://github.com/tensorflow/tfjs-models/tree/master/posenet

MDN navigator.mediaDevices.getUserMedia documentation: https://developer.mozilla.org/zh-CN/docs/Web/API/MediaDevices/getUserMedia

Sample demo of similar alarm logic: https://www.doverr.com/alarm/index.html