Purpose and operating boundary
Evaluate known-class coverage, novel-item handling and physical route association separately rather than reporting one flattering color accuracy number.
This tutorial defines an engineering evaluation, not a ready-to-run product certification. The current Deyi Vision public PC example covers an EAN-13 workflow on confirmed sample images. Native OCR and detection work in the engineering code must be checked against the installed runtime, bound models and job before use. A browser template does not prove browser execution: the planned OpenCV.js backend needs separate verification. Cloud recognition requires explicit approval of the image scope, provider, model and cost. A review archive is not an executable release package. Camera, firmware and driver compatibility require their own check. All worksheet values below are hypothetical; they are not Deyi Vision measurements or customer results.
Prepare the evidence
Freeze classes, unknown policy, masks and any SKU mapping with an independent specimen set. Begin with output simulation and supervised routing; keep unsupported execution paths inhibited. Define which timestamp starts the deadline and which acknowledgement proves completion. Save queueing and communication delays beside image processing so that a fast algorithm does not conceal a late line result. Use monotonic timing for intervals and a separately documented clock mapping when devices use different clocks. Record tail delays, missing results and reconnect behavior, not only the average. A new recipe must not reinterpret results already in flight; either finish under the recorded revision or apply the agreed inhibit procedure. Keep the output sequence, product position and confirmation together. The acceptance record should distinguish correct classification from successful communication and from confirmed physical routing. Color sorting and SKU recognition answer different questions. A color region may match a red reference while the item is the wrong size, label or package variant. Define whether the job sorts a physical color, verifies an expected SKU or requires both independent checks. Use color images with known channel order and keep exposure, white balance and light spectrum in the recipe. Test acceptable material lots, glossy surfaces, shadows and foreign products rather than only a clean reference swatch. Low saturation makes hue unreliable, and clipped highlights remove information that an algorithm cannot restore. Unknown, unobservable and mixed-color items need an explicit route. If semantic SKU recognition uses a model, validate that model separately from the color threshold and require approval before any cloud transfer. Write the line contract before connecting a verdict to an actuator. Define acquisition identity, recipe revision, result state, deadline, acknowledgement and the disposition of missing or late data. Exercise the handshake with outputs inhibited, then use a supervised line trial under the plant acceptance procedure. Measure from the same defined event on each run; processing time alone omits queueing, transfer and output delay. A repeated product identifier is not a unique frame identifier. Test stop, restart, reconnect and recipe change with outstanding work. Preserve the full sequence so that the part being rejected can be traced to the evidence that justified that action.
Procedure
- Build a confusion record with an explicit unknown column and retain all missing, Review and error outcomes.
- Evaluate color agreement independently of semantic SKU identity; identical colors do not establish identical products.
- Send event ID, class or unknown reason and revision, then verify acknowledgement and the supervised destination.
- Inject ambiguous color, unseen SKU, missing frame and late response; confirm the declared unknown or inhibited route without substituting a nearby known class.
Interpret the trial
The hypothetical set has 20 red, 20 blue and 10 novel items. Known-class routing covers 37 of 40, or 92.5%; it is not an overall accuracy claim. For novel items, 2 of 10 are wrongly assigned known classes, or 20%. Keeping those errors visible reveals a failure hidden by reporting only the successful known routes.
Recording worksheet
| Record | Value / observation | Interpretation |
|---|---|---|
| True red | 20 | 18 red / 0 blue / 2 unknown |
| True blue | 20 | 0 red / 19 blue / 1 unknown |
| Novel item | 10 | 1 red / 1 blue / 8 unknown |
known_coverage = (18 + 19) / (20 + 20) = 37 / 40 = 92.5%
novel_wrong_known = (1 + 1) / 10 = 2 / 10 = 20%
N = 20 + 20 + 10 = 50Two engineering views
Two engineering views · 1
The diagram shows the order of checks. Each arrow requires recorded evidence; it does not imply an implemented automatic connection. Acquisition → Verdict → Acknowledgement → Physical route.
- Acquisition
- Verdict
- Acknowledgement
- Physical route
Two engineering views · 2
The diagram shows the order of checks. Each arrow requires recorded evidence; it does not imply an implemented automatic connection. Normal → Missing → Late → Inhibited.
- Normal
- Missing
- Late
- Inhibited
Failure diagnosis
- A dashboard omitting unknowns inflates apparent performance.
- A correct class on the wrong event or physical bin still fails acceptance, even if the image classification was correct.
Acceptance and handover
Accept each agreed limit for known classes, novel rejection, complete accounting and actual route handling. The worksheet illustrates how to calculate the metrics; only independent real samples and witnessed line evidence can establish the deployed result.
Keep the source images, the expected decision, the exact revision and the complete result record together. Freeze the evaluation set before selecting the final parameters. If those answers are later used to tune the workflow, mark that set as development data and obtain a new independent acceptance set. Keep Review, NoResult, execution error and missing acquisition visible in the denominator; do not turn them into passes by filtering rows. Save a reproducible failure alongside the successful sample. The handover should identify who may change the recipe, how a prior approved revision is restored and which device output remains inhibited until acceptance. A software result alone does not establish that a moving physical part reached the correct destination.
Sources and technical references
These references explain the methods and limits used in this tutorial. Example checks are guidance; production acceptance requires evidence from your own samples.
- Deyi Vision machine vision software: evaluation scopeDeyi Vision
Public EAN-13 PC evaluation scope and separate hardware/production acceptance.
Checked - Build a PLC–vision handshake with explicit statesDeyi Vision
Associate pending work, decisions, acknowledgements, timeout and restart recovery with the same acquisition/event identity.
Checked - Load-test latency from trigger to PLC decisionDeyi Vision
Measure end-to-end timing and retain late, lost and execution-error outcomes; hypothetical budgets do not establish measured speed.
Checked - Inspection threshold confusion matrixDeyi Vision
Use explicit false-positive/false-negative denominators and independent evaluation; do not hide Review, errors or missing acquisition.
Checked
Component and selection references
- machine vision software
- Automated Visual Inspection
- Visual Inspection System Design Checklist
- Load-test latency from trigger to PLC decision
- Accept and integrate visible assembly verification
- Build an AI-assisted visible assembly workflow
- EAN-13 traceability: line acceptance
- EAN-13 traceability: decode and data rules
- Stable camera acquisition: buffers, timeouts and reconnection
- Accept and integrate a visible fill-level workflow
- AI-assisted workflow for a visible level proxy
- Link inspection records, images and part identifiers
- OCR labels: acceptance and PLC contract
- OCR labels: native workflow and exact match
- Accept and integrate open-package completeness checks
- Build an AI-assisted open-package completeness workflow
- Seal and bead appearance: line acceptance
- Seal and bead appearance: workflow rules
- SKU color sorting: typed color and SKU rules
- Surface defects: acceptance and line output
- Surface defects: bounded AI evaluation
- Version and roll back the complete vision recipe

