Picture a plant manager at a mid-size Ontario dairy preparing to put AI-guided vision inspection and rejection on a cup-sealing line. This manager is a fictional composite, but the choice is familiar: two integrators have convincing demonstrations, and neither demonstration shows what happens during a film change, after sanitation, or when the network drops. The manager wants fewer defective cups leaving the plant without sending good product to rework. Just as important, the system must remain safe, supportable and under the dairy’s control after the supplier’s technicians go home.
That last concern becomes more concrete when the manager reads about Harlan Bakeries in Indianapolis. Harlan bought a $3.6 million, three-line system to wrap and box cream-cheese-filled bagel bars. Although the equipment was intended to handle 1,050 bars a minute, Harlan alleged it could not meet a specified minimum of 700 even with all three lines running. Faulty sensors and vacuum valves figured in the dispute, while an expert pointed to thinner packaging film that reportedly had not been disclosed. Repairs went on for nearly a year. These were allegations reported in a lawsuit, not findings established by the court. Still, they give the manager a useful starting question: which operating conditions and responsibilities will the acceptance test actually cover?
Before the demo
That question sends the manager to QA before any supplier showroom.
1. Validate against your own hazards and SKUs
First, validate the proposed AI against the plant’s hazard analysis, not against a supplier’s general accuracy percentage. Seal integrity, incorrect labels, allergen-related errors and visible defects do not carry identical consequences. QA therefore defines defect-by-defect missed-detection and false-reject limits for every SKU, including rare critical faults, uncertain cases, seasonal ingredients and start-up after cleaning. An A3 specification guide reports that only one in ten customers arrives with a real vision specification, while CFIA validation guidance calls for evidence that a control works under its intended conditions. Interviews with ML practitioners reinforce why offline test results alone cannot establish live performance. The dairy will define those conditions before inviting a demonstration.
2. Make one partner own the whole integration
Once QA knows what must be detected, the controls team asks what happens to each cup afterward. Second, give one integrator end-to-end responsibility for the camera, model, PLC, reject device and plant information systems. A correct classification on a screen means little if a cylinder fires against the wrong cup. In a firsthand manufacturing-forum account, conveyor timing logic could reject a good part between two bad ones. A separate bottle co-packer question described the tracking challenge posed by different containers between camera and reject cylinder; it was a design question, not a proven failure. Those accounts lead the dairy to demand signed interface definitions, PLC acknowledgments, reject confirmation and tests at variable conveyor speeds.
3. Prove edge inference works with the cloud cut off
With the physical reject path mapped, the manager follows the decision upstream to the computer making it. Third, run production-critical inference on edge equipment inside the plant and prove it works offline. The trial will physically disconnect the internet and vendor cloud, then measure worst-case camera-to-actuator latency at maximum line speed. Missing frames, full storage, an overloaded processor and lost connectivity need defined safe responses; a remote licence heartbeat cannot be allowed to halt critical inspection. The December 2021 AWS outage disrupted connected household devices, outside food manufacturing but a clear example of physical equipment becoming dependent on remote services. The dairy can permit optional cloud analytics while retaining local control of production.
4. Specify washdown for the whole assembly
That offline test would be incomplete if the installation fails its first cleaning cycle. Fourth, specify hygienic and environmental performance for the entire installed assembly, including lights, optics, connectors, cables, enclosures and edge computer, rather than accepting a camera’s rating in isolation. Condensation, background colour, film reflectivity and lens contamination can alter the image; food-image research describes the sensitivity of image-based inspection to acquisition conditions. The manager schedules tests immediately after sanitation and checks maintenance access against CFIA equipment and sanitation inspection guidance. Now the hardware test reflects the line people actually operate, which makes the contractual questions harder to evade.
Reader offer · ends November 11Where is the hidden cost in your factory?Watch it with 3 months of Premier free →At the contract table
Because the cameras will see real production, the first contract question concerns information leaving the building.
5. Keep plant data under your control
Fifth, keep plant data under plant control and prohibit secondary training without explicit permission. The integrator must map outbound images, employee-identifying footage, recipes, line settings, labels, metadata, backups and annotation access, then demonstrate that unauthorized transfer is blocked. No routine collection in the vendor’s cloud is permitted. Canada’s Tim Hortons privacy investigation concerned an app, not plant AI, but found excessive collection and overly permissive third-party terms. A Reuters investigation into Tesla camera recordings shows the related risk of human access to vision data outside food. The manager requests a Canadian-residency option and auditable exceptions, while recognizing that PIPEDA does not categorically ban cross-border processing.
6. Own the images, labels and model weights
Controlling data flow, however, does not guarantee ownership of what the system learns. Sixth, secure ownership or perpetual operating rights to plant-specific images, annotations, defect definitions, trained weights, thresholds and deployment files. The contract must identify any supplier-owned base model separately and allow the dairy or a replacement engineer to maintain and retrain the commissioned system. In a manufacturing engineer’s firsthand account, plant personnel could not access installed PLC diagnostics or logic; it illustrates practical lock-in, not a legal ruling about ownership. The manager therefore requires a third party to export and restore the accepted model on a clean plant-controlled machine. A folder of files without usable licences and runtime dependencies will not pass.
7. Approve and roll back every model update
Once that restore works, the next concern is that somebody could silently replace the running model. Seventh, require plant approval for every change to models, thresholds, preprocessing, calibration and AI-directed behaviour. Each release needs a versioned dataset, configuration, runtime, QA sign-off where appropriate and an immediate rollback test. The July 2024 CrowdStrike failure affected automotive manufacturing; the incident was a software update, not an AI-model release, but it demonstrates why externally managed changes can interrupt production. The manager asks to watch a model update, compare its decisions against the previous version and restore that version with its matching configuration before accepting automatic patching arrangements.
8. Lock down OT security and vendor remote access
That change control also requires a secure route for legitimate supplier support. Eighth, treat edge computers and supplier remote access as part of the plant’s operational-technology security program. The dairy’s OT owner must approve network zones, asset and software inventories, multifactor authentication, time-limited sessions, recorded access, patch responsibility and recovery images. JBS’s 2021 ransomware disruption shows that food processing can suffer serious cyber interruptions, although reporting does not identify an AI integrator as the entry point. Outside food, a US advisory on internet-exposed PLCs with default passwords explains why unmanaged vendor tunnels are unacceptable. The manager expects an architecture reviewed against NIST OT security guidance, not a promise that the vendor’s network is secure.
9. Assess robot safety when AI changes motion
If a supplier proposes AI-guided picking or robotic rejection, cybersecurity is only part of the exposure. Ninth, commission an application-level robot risk assessment and require safety reapproval when AI changes motion. Safety-rated protection must work independently of a model’s prediction; grippers, conveyors, cleaning access, jams and foreseeable human interventions belong in the assessment. A fatal 2023 incident at a South Korean produce distribution centre involved a worker crushed by a box-handling robot. Reporting did not establish whether it used adaptive AI, but the physical-integration danger is directly relevant. The manager assigns responsibility under ISO 10218-2:2025 before any robot joins the dairy line.
10. Price the system over five to seven years
Having defined the technical boundaries, the manager asks what keeping them in place will cost. Tenth, demand a five- to seven-year operating-cost model with protected growth pricing. It must include edge-computer replacement, licences, cameras, storage, retraining, annotation, specialist support, security patches and the cost of adding SKUs or leaving a platform. An AI practitioner’s self-reported cloud-GPU bill exceeded $400 monthly despite mostly idle capacity; that is not a verified invoice or a food-plant example, but it shows why compute assumptions need scrutiny. Each partner prices the dairy’s base case, increased throughput and more frequent retraining on the same worksheet.
11. Test the exit plan before you sign
The longer the cost horizon, the more serious supplier survival becomes. Eleventh, test an exit plan before relying on the vendor’s licence server or expertise. After DoorDash’s decision to close Chowbotics in 2022, reporting indicated that deployed salad-making robots would be retired. That happened in foodservice rather than a dairy plant, while the announced retirement of a major cloud IoT service offers a separate connected-device analogue. The manager requires perpetual rights to run the last accepted version, practical source or deployment escrow where feasible, spare-hardware and support plans, and a simulated vendor shutdown. Those rights mean little until the plant can use them during acceptance.
At commissioning
Signed promises still have to be proven on the real line, so commissioning is where the manager makes the partner show its work.
12. Witness whole-line factory and site acceptance tests
Because contract promises need a witnessed proof, twelfth, make factory and site acceptance tests cover the complete system and every agreed product condition. The manager has read about the Carando Gourmet Frozen Foods dispute: during a May 2017 factory acceptance test attended by the owners, equipment was unfinished and a motor failed; later unwitnessed tests were contested, and an October test covered only one shell size. These are disputed case details, not a general verdict on integrators. For the dairy, they justify plant witnesses and deliberate trials with adjacent good and bad cups, missed triggers, restarts, full reject bins, MES outages, different films and the post-sanitation first run. The acceptance document records what passed, what failed and who must fix it.
13. Keep decision records QA can reconstruct
Those records must remain useful after the witnesses leave. Thirteenth, retain reconstructable AI decisions and assign independent food-safety verification. QA should be able to retrieve a lot’s input image, SKU, model and threshold versions, override, PLC reject command and physical reject confirmation without the vendor cloud. CFIA record-keeping guidance addresses electronic records, retrieval, access and modification history, while its verification guidance calls for verification by someone other than the person monitoring the control. Applicable preventive-control records must meet retention requirements, including the two-year period described in CFIA guidance. Vision only examines visible features, so separate controls such as metal detection and X-ray remain necessary where the hazard plan requires them.
Turn demo accuracy into daily losses
Once QA can trace a rejection to the actual cup, the manager turns a supplier’s impressive percentage into an everyday workload. In an illustrative calculation, the line runs 100 cups a minute for 15 hours: 90,000 cups daily. Partner A shows 99.5% accuracy on curated samples, which does not establish its production false-reject rate. Partner B proposes collecting three weeks of real images across six SKUs, including changeovers and the first hour after sanitation, with a witnessed false-reject target below 0.5% per SKU. At an illustrative 0.5% rate, the dairy would pull 450 good cups for recheck each day; if a film change pushed that rate to 2%, it would pull 1,800. That is why the manager evaluates missed defects and rejected good product separately, rather than treating one accuracy percentage as proof.
After go-live
Go-live is not the finish line, because the plant keeps changing after the partner’s engineers leave.
14. Detect drift and contract the retraining
Because even a thorough trial cannot reproduce every future film or ingredient, fourteenth, make drift detection and retraining a paid, timed service with plant approval. QA samples ground truth by SKU, shift and defect class, while the integrator watches false rejects, escapes, image quality and confidence. A 2023 study using CNC milling data, outside food, found model performance deteriorated under changing conditions. A separate firsthand predictive-maintenance report described roughly 40% false alerts at month three and fewer than 10% at month eighteen; it was not vision inspection and was not independently verified. Together these examples persuade the manager to name a responsible ML engineer, set response deadlines, define retraining triggers and revalidate each accepted release.
15. Transfer recovery skills to your own staff
That expert still cannot be the only person able to recover the line. Fifteenth, require hands-on skills transfer to operators, QA, maintenance, controls staff and a designated plant AI owner. People must be able to clean and realign optics, distinguish uncertain predictions from actual faults, label examples with expert review, restore an edge computer and roll back a model. Research on human review of low-confidence images supports using targeted expert checks rather than indiscriminately labelling everything, while industrial MLOps research describes persistent data, integration and operational challenges. The manager wants a named specialist, approval of replacements, backup coverage and a final drill where plant staff retrieve an audit record and restore service without supplier hands on the keyboard.
Score the two proposals side by side
With all fifteen commitments defined, the manager can compare the two proposals without letting a polished demonstration outweigh an operational risk. The following illustrative scorecard uses weights adding to 100 and evidence scores from 0 to 3: 0 = no evidence; 1 = proposed; 2 = partly demonstrated; 3 = demonstrated and written into the contract. Each weighted score equals weight multiplied by rating, so the maximum is 300.
| What to do | Why it matters, with a linked source | Weight (example) | Partner A | Partner B |
|---|---|---|---|---|
| Validate every SKU and critical hazard against real production | CFIA requires evidence that controls are effective | 10 | 1 | 3 |
| Assign responsibility for PLC, tracking, reject and MES integration | Practitioner reports reject-timing faults | 9 | 2 | 3 |
| Prove edge inference and safe failures with cloud disconnected | An AWS outage disrupted connected devices | 9 | 0 | 3 |
| Validate washdown, lighting and the entire assembly | Image conditions affect food-vision performance | 5 | 2 | 3 |
| Block unauthorized data export and secondary training | Canadian investigation found overly permissive data terms | 8 | 0 | 3 |
| Secure model and label rights; prove export and restore | Manufacturing personnel describe locked diagnostics | 8 | 1 | 3 |
| Approve, version and roll back every production AI release | Software changes disrupted automotive production | 8 | 0 | 3 |
| Control OT access, segmentation, patches and recovery | JBS ransomware disrupted food processing | 8 | 1 | 3 |
| Assess robot application safety and AI motion changes | Produce-handling robot fatally injured a worker | 6 | 1 | 2 |
| Price full lifecycle, growth and retraining costs | AI compute-user reported paying for idle capacity | 5 | 2 | 2 |
| Demonstrate independence in a simulated vendor shutdown | Chowbotics closure affected deployed foodservice robots | 6 | 0 | 3 |
| Witness whole-line factory and site acceptance tests | Carando dispute exposed contested and narrow testing | 4 | 2 | 3 |
| Preserve lot-level decision records and independent QA checks | CFIA addresses electronic preventive-control records | 5 | 1 | 3 |
| Monitor drift, contract retraining and revalidation | CNC study found degradation as conditions drifted | 5 | 1 | 2 |
| Prove staff recovery skills and named specialist support | Industrial MLOps review describes lifecycle challenges | 4 | 1 | 3 |
| Total weighted points (out of 300) | Weights total 100 | 100 | 92 | 284 |
With the scores calculated, Partner A finishes at 92/300 and Partner B at 284/300. These are illustrative evaluations, not scores of real bidders. A has some conventional integration and equipment capability, but little evidence of offline independence, data control, governed updates or supplier exit. B earns the recommendation provided its demonstrated commitments and three-week acceptance program become enforceable contract terms; the partially demonstrated safety, cost and drift items still need closing before purchase.
What the manager does this week
That leaves the Ontario manager with a concrete job this week rather than another vendor meeting about AI potential. QA will assemble good, defective and uncertain cup images from all six SKUs, controls will map camera-to-reject timing, and maintenance will document what changes after cleaning. The same hazard-based trial and fifteen contract demands go to both shortlisted integrators. When the next film change reaches the sealer, the dairy will need more than an accurate camera: it will need evidence, access and a partner accountable for the whole working line.
