Detecting Safety Hazards at Bulk Terminals

A practical guide to using computer vision for workplace safety—from choosing cameras and training models to detecting hazards and turning alerts into action.

Detecting Safety Hazards at Bulk Terminals

A practical architecture for PPE screening, footwear assessment and vehicle warnings—covering optics, datasets, annotation, model training and the workflow after an alert.

The problem begins before the neural network

A worker walks beside a conveyor while a loader approaches a crossing. A fixed camera records the scene. The helmet occupies a recognisable patch of pixels; the shoes merge into the dark floor. A dust plume briefly hides the worker. The video connection remains healthy throughout.

What should the system conclude?

An ordinary object detector returns boxes and scores. The terminal needs something more demanding: a defensible statement about a particular person, in a particular zone, at a particular time, with enough remaining time for someone to respond. When the evidence disappears, the system must also say that its ability to assess the situation has disappeared.

That distinction defines the implementation. We are engineering an evidence-processing and response system whose inputs happen to include video.

In his article Building an AI Consistency Engine for Financial Reports, Pranav Pathekar separates extraction, contextualised facts, deterministic validation and evidence-backed review. The same architectural principle applies here: learned perception supplies observations; explicit rules interpret those observations; a workflow assigns and verifies action. A plausible model answer is not an operational fact.

For a terminal, four difficulties interact:

  • Observability: dust, shadows, glare, tiny targets and occlusion can make the requested judgement impossible from the available image.
  • Association: a helmet, vest or shoe must be associated with the correct person. Detecting equipment somewhere in a frame is insufficient.
  • Time and geometry: one ambiguous frame differs from a sustained event; two overlapping image boxes do not establish physical collision risk.
  • Intervention: an alert can be generated correctly yet arrive late, reach nobody, or be acknowledged without the hazard being addressed.

The engineering objective is to preserve these distinctions from annotation through the operator interface.

Define the decision before choosing the model

Start with an operational design domain: the locations, lighting, visibility, camera geometry, target sizes and operating conditions in which each use case will be assessed. Record what happens outside that domain. An excluded condition is a coverage gap with an operational fallback, not permission to silently display a green status.

Use case Observable question Necessary evidence Appropriate output
Helmet screening Is the visible head wearing a helmet? Assessable head region and person association Helmet present, visible absence, unknown
Vest screening Is a vest visibly worn on this torso? Assessable torso and worn-versus-carried distinction Visible match, mismatch candidate, unknown
Footwear screening What footwear appearance is visible on each foot? Adequate original detail and per-foot association Appearance category, mismatch candidate, unknown
Restricted-zone intrusion Has a person entered the configured region? Valid calibration, tracked ground contact and boundary uncertainty Entry candidate, confirmed episode, uncertain location
Vehicle warning Are trajectories and separation creating a conflict? Fresh relative position/motion, vehicle geometry and sensor health Directional warning, risk state, unavailable state

“Certified safety shoe” is not an honest image-only class. Toe protection, penetration resistance, electrical properties, authenticity and suitability are not reliably visible. A trainer-style shoe can be certified; a boot-shaped shoe may not be. Procurement records, product markings and inspection supply a different kind of evidence.

If the local rule prohibits open footwear, a clearly visible sandal can support a mismatch finding. A dark closed shoe, by itself, cannot establish either certification or non-compliance. This prevents the system from learning a convenient but false definition of safety.

Build three sensing paths around a shared evidence platform

Use a controlled pedestrian checkpoint for difficult PPE observations; fixed cameras for zone and area monitoring; vehicle-local sensing for warnings that depend on short response times. Their evidence can share a platform without sharing the same timing assumptions.

Three sensing paths feed a shared evidence platform while vehicle warnings remain local.
Three sensing paths feed a shared evidence platform while vehicle warnings remain local.

The cloud or central server may support investigation, reporting and model management. An urgent vehicle warning must not depend on an internet round trip or a supervisor opening a dashboard.

Conceptual bulk terminal with separate PPE checkpoints, area surveillance and vehicle-warning locations.
Different safety decisions need different observation points. Conceptual layout; not a site engineering drawing. AI-generated concept illustration.

Solve the optical problem with a measurable pixel budget

At a particular target plane, a first approximation is:

target_width_pixels ≈ image_width_pixels × projected_target_width_metres / scene_width_metres

Suppose a 2,560-pixel image covers 20 metres at the worker’s position. A shoe feature projected as 0.10 metres is approximately 13 pixels wide. Resize the complete frame to 640 pixels and that feature becomes approximately three pixels wide. No choice of network architecture repairs that information loss.

This estimate ignores perspective, blur and optical resolution; the actual image must be measured. It is not a minimum-pixel standard. Establish minimum usable size experimentally for each task using held-out footage.

The practical pipeline keeps access to the decoded high-resolution frame. A first-stage detector finds a person. The second stage crops the relevant body region from the original frame and only then resizes it for inference. Cropping a three-pixel shoe out of the already reduced detector image merely enlarges three pixels.

At commissioning, collect target-size measurements at near, middle and far positions. Include oblique views, walking motion and actual night exposure. Measure pixels on the feature the classifier needs, not merely person height.

For footwear, create a sheltered, accessible viewpoint along an established pedestrian route. Compare one oblique lower-body camera with two complementary views. Keep the geometry from creating a new trip hazard or pedestrian bottleneck. Use controlled lighting where justified and physically protect a low-mounted camera from dirt and impact.

Evaluate exposure, shutter speed, compression and denoising together. A bright image with motion-smeared shoes is not useful evidence. A sharp still image does not establish performance on moving workers. Generative enhancement and super-resolution must not invent the detail used to decide compliance.

Comparison of distant footwear in a wide berth image and a closer controlled pedestrian checkpoint view.
Original image detail determines what footwear screening can establish. Appearance alone does not establish protective certification. AI-generated concept illustration.

Treat visibility as an independent subsystem

A stream can decode successfully while the camera window is opaque. Health monitoring therefore needs separate layers:

Layer Checks Failure response
Transport Stream connectivity, packet/decoder errors Reconnect and report outage
Freshness Source timestamps, arrival timestamps, sequence progress, repeated content Reject stale observations; diagnose frozen feed
Image usability Blur, clipped highlights, obstruction, contrast, condensation Mark affected tasks degraded or unavailable
Geometry Landmark alignment, camera movement, changed zoom Invalidate spatial rules pending calibration
Inference Queue age, GPU memory, throughput, model health Shed stale work and report degraded service

Start with interpretable features: gradient energy for sharpness, saturation fraction, local contrast, edge density, and reference-region alignment. Then train a small image/region quality classifier against human-labelled usability if those rules are inadequate. A low Laplacian variance is not a universal dust detector: an empty uniform scene can also have low variance.

Make quality task-specific. The scene may still support person detection while footwear is unassessable. Maintain separate masks and states for head, torso, feet and zone geometry. A single global “camera healthy” Boolean discards that distinction.

Use persistence and hysteresis to avoid flickering health states, but independently reject stale inputs immediately once their permitted age is exceeded. Detect a frozen feed using timestamps and content jointly; a genuinely static scene is not necessarily frozen.

Dust on the exterior window needs a maintenance action. Airborne dust needs a visibility-dependent operating response. IP protection does not clean glass. Salt corrosion is usually a physical inspection and component-reliability issue; do not imply that a video-quality model certifies bracket or connector condition.

Illustrations of airborne dust, a dirty optical window, salt corrosion, vibration and glare.
A connected camera can still provide unusable evidence. Each failure needs its own monitoring and maintenance response. AI-generated concept illustration.

Build a dataset around failure conditions

The dataset is a collection of independent operating situations, not a bag of frames. Ten thousand adjacent images of the same worker are mostly repeated evidence.

Use three data layers:

  1. General pretraining: a permitted pretrained model supplies generic person and object features. COCO includes detection and person-keypoint tasks, but does not supply the terminal’s compliance definition.
  2. Relevant external examples: use PPE datasets only after provenance, annotation semantics and licence review.
  3. Permissioned site data: capture the real optics, backgrounds, equipment, clothing, materials and operating conditions that determine acceptance.

SH17 contains 8,099 images, 75,994 instances and 17 categories, including foot and shoes. Its repository lists CC BY-NC-SA 4.0. Treat it as a research candidate requiring rights clearance before commercial use; its shoe class does not establish certification. Ultralytics Construction-PPE documents 1,416 images and 11 classes, including boots and missing-equipment labels, under AGPL-3.0. Review the actual data, model and software terms separately; attribution alone is not a blanket production clearance.

The illustrations in this article are explanatory assets. They are not ground truth and must not enter the acceptance test set.

Capture matrix

Dimension Capture variations Why it matters
Viewpoint Checkpoint, berth, crossing; near/mid/far; front/side/rear Perspective and target size
Light Day, twilight, actual night lighting, backlight, headlights Appearance and exposure changes
Visibility Clear, airborne dust, deposited dust, rain/condensation Abstention and health detection
Material/background Relevant cargo, wet/dry ground, stockpiles, vessel backgrounds Prevent background shortcuts
People Different workers, heights, clothes, gait and body orientation Prevent person-specific learning
PPE Approved models, styles, wear and dirt; visually similar alternatives Prevent colour/shape shortcuts
Occlusion Trouser cuffs, other people, rails, machinery, carried objects Association and missing evidence
Vehicles Type, load, speed, reversing, turning, parked state Motion and footprint differences

Use EHS-approved controlled capture for missing PPE outside hazardous operations. Do not stage real near-collisions. Use isolated test areas, suitable targets, protected procedures and simulation/replay to explore risk cases, followed by controlled physical validation.

As a first pilot annotation budget, plan roughly 10,000–20,000 deliberately selected full frames and 15,000–30,000 footwear crops from hundreds of independent passages. Also budget continuous event sequences and several hundred camera-hours of representative negative footage for alert evaluation. These are workload estimates, not sufficient-data guarantees. Expand according to learning curves and unresolved condition gaps.

Preserve continuous sequences around changes: appearance, occlusion, crossing, warning, clearance and re-entry. Sparse sampling can build detector labels; it cannot by itself establish warning latency or temporal event behaviour.

Annotate observations, relationships and uncertainty

Keep a rich master annotation format. A detector export is a derived training view, not the authoritative dataset. CVAT’s track/shape representations support attributes and explicit occluded/outside properties; relationships and event labels may additionally live in a linked schema.

Master annotation schema

Entity Geometry Required attributes
Person Bounding box and track Local track ID, truncation, occlusion, assessability
Head region Box; optional visible keypoints Parent person, visibility, helmet observation
Torso region Box Parent person, visibility, vest observation
Footwear item One box per visible shoe/foot Parent person, anatomical side if known, appearance, occlusion, usable detail
Helmet/vest Item box Worn, carried or uncertain; parent person if defensible
Vehicle Box or footprint-related geometry Type, track, movement state, visible extent
Operational zone Versioned polygon or surveyed region Rule, calibration version, valid time range
Event Time interval Participants, zone, category, evidence status, review outcome

For detector geometry, use a written convention for tight boxes around the visible extent; record truncation separately. Do not mix visible-only boxes with guessed full hidden extent. If amodal geometry is needed, store it as a distinct annotation type. Define the convention with examples before distributing labelling work.

For footwear, store appearance such as open_footwear, closed_low_cut, boot_like, bare_foot or unknown. Store assessment separately: assessable, occluded, too_small, blurred, out_of_frame or other_unknown. An appearance category is not a compliance verdict.

Keep independently verified product facts in a different table: product ID, inspection source, verification date and scope. Do not automatically transfer “this worker was issued certified shoes” into the visual label for every image of that worker.

Annotation rules that prevent expensive errors

  • If feet are behind a barrier, label them unassessable. Do not invent a no_safety_shoe box.
  • If only one foot is assessable, retain the other as unknown. Do not infer a matching pair.
  • If two workers overlap, assign an item to a person only when the relationship is defensible. Permit an unassigned item.
  • A helmet in a hand is not a worn helmet. A vest draped over a rail is not a compliant torso.
  • A tracked object leaving the image is outside the view; an object hidden within the view is occluded. These are different states.
  • Do not guess anatomical left/right from screen position. Allow unknown side.
  • If a public dataset uses no_boots, inspect its meaning before mapping it to the site’s ontology.

Example: a valid uncertainty record

{
  "schema_version": "1.0",
  "frame_id": "checkpoint03_session14_frame0284",
  "person_track_id": "camera03_session14_person07",
  "image_size": [2560, 1440],
  "coordinate_space": "original_image_pixels",
  "footwear": [
    {
      "bbox_xyxy": [1120, 1010, 1196, 1054],
      "side": "unknown",
      "appearance": "closed_low_cut",
      "assessment": "assessable",
      "protective_certification": "not_established_by_image"
    },
    {
      "bbox_xyxy": null,
      "side": "unknown",
      "appearance": "unknown",
      "assessment": "occluded"
    }
  ],
  "visual_screening_result": "cannot_assess_complete_pair",
  "reviewed": true
}

This record is more useful than a forced safe/unsafe answer because it identifies the next action: obtain another view or perform a manual check.

Annotation quality control

Have two annotators independently label an initial calibration batch of roughly 200–300 varied examples. An EHS representative resolves policy semantics; an ML lead resolves geometry conventions. Keep disagreements as a guide to unclear instructions.

During production, double-label a stratified sample—an initial 10–20% is a planning choice—and review all difficult footwear/occlusion cases. Measure class disagreement, box overlap, identity switches and parent-association errors. Maintain adjudicated examples and a versioned instruction manual. Automated checks should catch orphan PPE items, impossible coordinates, duplicate IDs and inconsistent unknown states.

Split the dataset before extracting training frames

Group by capture session and passage, and enforce that neighbouring frames, derived crops and augmented variants remain in the same split. Where identity or particular shoe pairs would permit memorisation, group them too.

A useful starting allocation is 70% training, 15% validation and 15% locked test by independent groups, adjusted for event rarity and condition coverage. The percentage is less important than independence.

Use two distinct tests: a later-time holdout on intended cameras to assess same-site operation, and held-out viewpoints or zones if claiming transfer to unseen locations. Report these separately. Do not insist on unseen-camera evaluation for every claim, or pretend same-camera validation proves generalisation.

Preserve a representative operational test set for real false-alert burden. Maintain a separate enriched challenge set for rare conditions. Neither should be substituted for the other. Use temporal proximity and perceptual similarity checks to detect leakage.

Implement the perception pipeline as testable components

Stage A: decode, timestamp and preserve coordinates

Receive the stream, decode on appropriate hardware, and preserve source presentation timestamps plus monotonic arrival times. Record original dimensions, crop offsets, scaling and letterbox padding. Without these transforms, boxes from a resized crop can be incorrectly mapped to the original scene or assigned to the wrong person.

Bound queues. For current hazard assessment, old frames are often worse than dropped frames: a fast model working through a backlog still issues late warnings. Evidence recording and current inference may have different buffering policies.

Stage B: detect people and vehicles

Start with a small or medium pretrained detector from a supported framework. A YOLO-family detector is a practical baseline; compare a larger detector or another architecture only when a measured failure justifies the additional compute. Pin code, weights and licences. The current Ultralytics documentation describes configurable training and export workflows.

The detector predicts class scores and boxes. It does not know whether a missing shoe box means absent footwear, an occluded foot or a very small target. That interpretation belongs downstream.

Compare full-frame inference at multiple resolutions with original-frame crops or overlapping tiles. Tiling can recover detail but increases compute and introduces duplicate boxes at tile boundaries. Maintain coordinate-aware merging and evaluate duplicate handling in crowds.

Stage C: track identities within a camera session

A tracking-by-detection baseline can use a Kalman motion model and assignment between predictions and detections. ByteTrack is a relevant implementation approach: it associates lower-scoring detections as well as high-scoring ones to reduce track fragmentation. This does not make every low-scoring detection trustworthy PPE evidence.

Use motion/overlap gating; consider appearance embeddings if crossing people cause unacceptable identity switches. Measure performance with similar uniforms and occlusion. During ambiguous identity transitions, suspend transfer of PPE state. A predicted track through dust is a location hypothesis, not a fresh observation of the person’s shoes.

Use camera/session-scoped IDs by default. Cross-camera identity is a separate capability with accuracy and privacy implications; it is not required to run local event episodes.

Stage D: locate relevant body regions

Use person geometry for an initial head/torso/lower-body region proposal. Add pose estimation where it measurably improves localisation, especially bent bodies and unusual viewpoints. Ankle keypoints are proposals, not proof that a foot is visible. Keypoint models can confidently estimate hidden joints.

A dedicated footwear detector within the lower-body crop can refine the region. Evaluate the full cascade as well as each component with ground-truth crops. Good classification on perfectly labelled crops can conceal a weak runtime crop generator.

Stage E: classify only assessable evidence

Train a lightweight footwear appearance classifier on original-image crops. A compact convolutional backbone is a sensible starting point. Classify usability separately or use a masked multi-task head so unusable samples do not teach fabricated appearance labels.

Associate items to people using spatial compatibility, body-region location and track continuity. Resolve crowded scenes with gated assignment rather than assigning every overlapping item to everyone. Permit unmatched items and ambiguous ownership.

Return a structured observation containing timestamp, person track, region, appearance, quality state, score and model version. A score becomes a calibrated probability only after calibration is tested; a raw 0.9 detector score does not mean 90% certainty of compliance.

The vision pipeline preserves uncertainty before creating or updating an event episode.
The vision pipeline preserves uncertainty before creating or updating an event episode.

Train through controlled experiments

Begin with a detector baseline and a footwear crop baseline. Do not build an elaborate multimodal network before understanding their failure distributions.

For a YOLO-style detector, localisation, classification and, where used by the selected architecture, distribution-based box losses train complementary outputs. For the footwear classifier, cross-entropy is a baseline; use class weighting or focal loss only when imbalance experiments justify it. Mask the appearance loss on unassessable examples, while retaining those examples for quality training.

The following is an experiment specification, not universal optimal hyperparameters:

experiment: terminal_ppe_baseline_01
data:
  manifest: terminal_dataset_v1
  split_unit: independent_capture_group
  test_access: final_evaluation_only
detector:
  initialisation: approved_pretrained_checkpoint
  input_size_candidates: [960, 1280]
  epoch_budget: 100
  early_stopping_patience: 20
  batch_size: measured_for_available_memory
footwear:
  crop_source: original_decoded_frame
  input_size_candidates: [224, 320]
  unknown_appearance_loss: masked
  quality_head: separately_evaluated
evaluation:
  seeds: [17, 29, 43]
  report_by: [viewpoint, light, dust, target_size, occlusion]

Pilot these settings on a subset before committing the training budget. Select learning rate and weight decay on validation data using the pretrained backbone’s supported training recipe. Track wall-clock cost and performance variation; do not select a model because of one favourable random seed.

Use realistic brightness, exposure, blur, compression, scale and partial-occlusion augmentations. Test colour changes carefully if reflective clothing colour contributes to recognition. Excessive downscaling or mosaic augmentation can turn the very shoes we need to learn into uninterpretable dots. Synthetic haze does not reproduce every optical effect of real dust and cannot replace real dusty validation.

Mine hard negatives: parked equipment, shadows, sacks resembling people, helmets being carried, reflective machine panels, railings hiding feet and ordinary footwear that looks protective. Audit them before training. Preserve real operational prevalence in evaluation even if training oversamples rare cases.

Maintain an experiment table comparing baseline, higher resolution, native crops, improved illumination, different viewpoint and two-view fusion. A camera relocation may outperform a more complex model. That is a successful engineering result.

After training, fit any confidence calibration on a separate calibration subset or properly separated validation data. Select thresholds against missed-event cost, false alerts and assessment coverage. Treat out-of-distribution scores as supporting indicators, not reliable detectors of every unfamiliar scene.

Convert observations into events without inventing certainty

For PPE, use a bounded recent time window containing eligible observations. An observation is eligible only if the relevant region is assessable, its association is defensible, and it is fresh.

An illustrative rule might require at least four mismatch observations over one second, a minimum eligible observation count, and a minimum weighted mismatch fraction. The actual parameters come from validation and the required response time. Adjacent frames are correlated; four matching frames do not provide four independent proofs.

For a calibrated per-observation mismatch score p and quality weight q, a smoothing statistic is:

S = sum(q_i * p_i) / sum(q_i)

Use a recent bounded window and minimum evidence requirements. S is a smoothed score, not automatically a probability that the event is true. Validate the complete rule.

Unknown observations neither prove compliance nor clear an active event. If valid evidence ages out, mark the current assessment unknown; preserve the event’s unresolved workflow state. Require fresh sufficient evidence to record visible clearance. A supervisor can separately resolve an event through a documented physical check.

Keep perception state and case-management state separate. A person can leave the view while an incident remains open. A supervisor can acknowledge while the hazard remains visible.

The response workflow tracks ownership, acknowledgement, escalation and verified closure.
The response workflow tracks ownership, acknowledgement, escalation and verified closure.

Deduplicate by rule, camera/session, person track, zone and episode. Merge repeat observations into an existing episode. Rearm after a validated clearance or new entry according to policy. Do not let a long global cooldown hide a different person’s event. Track fragmentation requires cautious episode reconciliation; spatial proximity alone can merge two workers incorrectly.

Engineer zone detection in physical space

For a fixed camera viewing approximately planar ground, undistort the image and estimate an image-to-ground homography using surveyed correspondences. OpenCV documents the underlying planar mapping. Use more than the minimum correspondences, distribute them across the working area, and measure independent residual errors.

Map a defensible ground-contact estimate, such as visible foot contact, into the ground plane. A bounding-box bottom centre is a rough fallback and becomes unreliable under occlusion. Do not map a person’s head or an elevated truck feature as if it lay on the ground.

Determine zone membership using a polygon test with uncertainty near the boundary. Define entry/exit hysteresis and direction where relevant. Camera movement, a changed zoom or a moving crane-mounted camera invalidates a fixed mapping unless pose is independently updated. Sloped ramps and multiple ground planes need separate mappings or a 3D approach.

An intrusion rule is not a collision predictor. It can report that a worker entered a prohibited region without knowing whether a vehicle will intersect their path.

Extend to vehicle warnings with motion and uncertainty

Begin with a machine-specific hazard assessment: vehicle types, blind areas, reversing behaviour, turning envelopes, speeds, payloads, surface conditions and warning response. A loader’s bucket and articulation matter; a point at the centre of its camera box does not represent its swept volume.

A candidate local stack combines cameras for semantic context with radar for appropriate range and relative-motion measurements. Radar supplies its own limitations: multipath, clutter, angular resolution and sensor-specific detection behaviour. Metallic terminal infrastructure can complicate association. Dust robustness must be tested, not assumed from the sensor name.

Calibrate camera intrinsics, radar-to-camera extrinsics and time offsets. Transform detections into a common vehicle coordinate system. Gate associations by time, position, direction and uncertainty, then use an appropriate assignment method. Many radars primarily measure radial velocity; do not treat a radial return as a complete two-dimensional pedestrian velocity without estimation and evidence.

Maintain object position/velocity with a Kalman or extended Kalman filter where its motion assumptions fit. Include ego-motion from approved vehicle interfaces or suitable independent sensing. Track covariance and age. When a person becomes visually occluded, an existing radar track may still support an obstacle warning; lack of a camera classification must not automatically erase a radar hazard.

For a simplified constant-relative-velocity case, let r be relative position and u relative velocity:

t_closest = clamp(-(r · u) / (u · u), 0, prediction_horizon)

d_closest = norm(r + u * t_closest)

Handle near-zero relative speed explicitly. This calculation estimates closest approach of points, not time to physical contact. Compare predicted footprints or occupied regions over time, include dimensions and uncertainty, and account for turning. Constant velocity is only a short-horizon approximation.

A warning region should reflect system delay and stopping behaviour. For straight-line stationary-obstacle reasoning, an illustrative bound is:

distance = speed * total_prebraking_delay + speed² / (2 * effective_deceleration) + margin

At 15 km/h, a 1.5-second combined sensing/processing/warning/driver delay, 1.5 m/s² effective deceleration and a 2-metre margin give about 14 metres. This is an educational calculation, not a terminal setting. Real evaluation must include load, grade, traction, brake dynamics, turning, moving pedestrians and uncertainty.

Measure capture-to-warning age under load, including queues and warning-device actuation. Use source timing or external instrumentation when software timestamps exclude camera buffering. A 25 ms inference benchmark is not a 25 ms warning system.

ISO 21815-1 addresses collision warning/avoidance for specified machinery categories; applicability depends on the machine and function. Automatic braking or motion inhibition adds a safety-control function requiring separate OEM integration, assurance and validation. The proposed vision model does not become a certified safety controller because its dashboard displays a warning.

Conceptual loader camera and radar arrangement with a local in-cab crossing warning.
A vehicle warning depends on fresh motion evidence and a timely driver interface. Warning is not automatic collision avoidance. AI-generated concept illustration.

Make exceptions first-class outputs

Observation System result Response
Foot hidden behind a rail Footwear unknown Another view or manual checkpoint check
Closed shoe visible, product identity unknown Appearance available; certification unverified Use approved verification process if required
Helmet carried, head visibly bare Worn-helmet mismatch candidate Confirm temporally and route event
Two workers cross and identities become ambiguous Association uncertain Stop carrying PPE state across uncertain identity
Dust obscures the relevant zone Monitoring degraded/unavailable Notify operations and invoke agreed fallback
Stream connected but frozen Stale evidence rejected Technical incident and recovery verification
Camera shifts after maintenance Zone calibration invalid Recalibrate before re-enabling spatial decisions
Event delivered but no acknowledgement Response overdue Escalate to designated backup
Radar track lacks visual classification Unclassified obstacle evidence Apply validated obstacle-warning policy

An operator-facing case should say what is known, what is not known and what action is required. “Footwear cannot be assessed because both feet are occluded” is actionable. “AI confidence 63%” is not an adequate explanation.

The event record should preserve an immutable ID, timestamps, zone/calibration version, participant references, rule version, model versions, observation quality, evidence locations, delivery attempts, owner, acknowledgement, actions and closure reason. Corrections should append provenance rather than silently overwrite history.

Keep snapshots where that is the agreed scope. Pre/post-event clips require separately agreed retention, access, privacy and storage arrangements. A snapshot can show a foot but cannot by itself demonstrate a motion-based warning decision; where clips are unavailable, retain permitted time-series metadata and explicitly state the evidence limitation.

Deploy for bounded latency and recoverable failure

A practical implementation can use hardware video decoding, a GStreamer/DeepStream-style pipeline, an optimised inference runtime, a local event service, durable local buffering and a central evidence store. NVIDIA provides pipeline and frame-latency instrumentation; measure the deployment rather than importing a vendor example’s latency figure.

Size compute from the workload: camera count × actual sampling rate × detector cost, plus person-dependent crop inference, tracking, decoding, recording and peak density. PPE crop processing load grows when more people enter, even if camera FPS remains constant. Benchmark at expected maximum concurrent load and thermally steady conditions.

Evaluate FP16 first if supported. Introduce INT8 only after representative quantisation calibration and exported-model regression testing, especially for small footwear and low-light crops. Freeze preprocessing, postprocessing and coordinate transforms alongside the model. Export success is not equivalent to equivalent predictions.

Use separate priority paths for immediate vehicle warnings and slower review/reporting. Preserve local warning operation through backhaul loss where the architecture supports it. Buffer event records durably and upload later with stable event IDs and idempotent handling. Do not replay old buffered events as fresh urgent warnings.

Protect field equipment through appropriate enclosures, thermal design, power conditioning, surge/bonding design, corrosion-compatible assemblies and maintainable optics. Select exact hardware against site environmental and hazardous-area assessments. A GPU specification cannot compensate for a dirty optical window or an unserviceable mounting point.

Maintain segmented networks, limited remote access, credential management, controlled updates and signed/versioned deployment packages where supported. A release should bind model, thresholds, rule definitions and camera calibration versions. Canary a new release on limited cameras, compare in shadow mode and retain a tested rollback path.

Accept the whole system, including what it cannot assess

Frame mAP is a diagnostic metric. It is not the acceptance statement.

Metric Definition/use
Event recall Ground-truth eligible events detected within the required time window
False alerts False episodes per camera-hour or vehicle operating-hour
Assessment coverage Assessable eligible passages or monitored time divided by all eligible passages/time; declare denominator
Association error PPE assigned to the wrong person or state transferred across a wrong identity
Warning latency End-to-end age at actual warning, including tail percentiles and stale-drop counts
Degraded-state detection Delay and accuracy in recognising unavailable monitoring
Response performance Delivery, acknowledgement, action and verified closure intervals

Report performance separately for footwear, helmet, vest and zones, and by dust, light, target size, occlusion and camera type. For footwear, a useful view is error rate versus assessment coverage: an apparently accurate classifier may simply abstain on most difficult passages.

Count events and independent passages as well as frames. Use confidence intervals suitable for the sampling structure; repeated frames from one passage are not independent trials. Investigate misses through the cascade: absent person detection, broken track, wrong crop, bad association, appearance error, rule suppression, delivery failure or late warning.

For example, 98% correct decisions among the 40% of passages deemed assessable does not demonstrate dependable screening for all arrivals. It demonstrates a need to redesign the view or provide a staffed fallback for the remaining passages.

Commission with operationally representative negative footage and controlled positive scenarios. Exercise power loss, frozen streams, dust, camera movement, overload, full storage, disconnected warning devices and unacknowledged cases. Recovery must include restored calibration and evidence freshness, not simply a restarted process.

Improve through reviewed evidence, not automatic self-training

Collect candidate learning examples from false alerts, missed events, uncertain assessments, new PPE, new viewpoints and changing materials. Also sample ordinary accepted and non-alerted footage: an alert-only review loop cannot discover invisible misses.

Prioritise uncertain and diverse samples, cap repeated examples from one camera, and have humans adjudicate before adding them to training. A dismissed operator alert is not automatically a negative label; the dismissal may reflect workload or misunderstanding. Log the review reason.

Keep the existing test set locked. Add a new independently held-out test cohort when the operating domain changes. Track dataset, annotation, model, threshold and camera-configuration versions. Retrain when evidence identifies a failure pattern, not because a calendar says that a new model must be better.

An implementation sequence with concrete exit evidence

Stage Deliverable Exit evidence
Decision and survey Use-case matrix, operating domain, camera/zone survey EHS/operations agree observable claims and fallback
Optical pilot Measured views at one checkpoint and one representative area Adequate original detail under relevant conditions
Dataset and labels Versioned corpus, annotation manual, independent splits Audited labels and coverage gaps documented
Perception baseline Detector, tracker, crops, quality and appearance models Component and complete-cascade results
Event integration Rules, deduplication, ownership, evidence and timeouts Replay proves episode and exception behaviour
Shadow deployment Non-operational alert observation and review Real false-alert load, misses and coverage measured
Operational pilot Limited deployment with agreed fallback Acceptance tests and response drills passed
Vehicle extension Machine-specific sensing and warning design Separate controlled validation and applicable assurance

The technical success condition is precise: every output must preserve the chain from physical observation to model evidence, explicit rule, accountable response and verified disposition. When that chain breaks, the system must expose the break. That is what turns a detector into operational safety support.

This article describes a proposed engineering approach. Numerical examples and training budgets are illustrative; they are not reported deployment results. Site-specific performance, equipment suitability and applicable requirements must be validated for the installation.