An AI-Assisted Approach to Building a Vision Inspection Pipeline for Real-Time Production

A food-manufacturing client came to us with a familiar set of constraints: three cameras looking down at a moving conveyor; they needed to inspect every product for quality in real time; and their project timeline favored a fast, structured build. The production line wasn’t available for data collection, and the legacy system we were replacing had no way to export live footage. So, we started with a specification and a few sample images.

This post describes how we went from that starting point to a deployed real-time inspection pipeline, and how AI assisted along the way. The inspection itself is classical, deterministic computer vision: color segmentation, morphology, and geometric measurement. And while AI didn’t run the inspection, it accelerated how quickly we designed, communicated, built, and tested the system that does.

What the Client Needed

Despite the project constraints, the requirement was straightforward: as products travel down the belt under three adjacent cameras, identify each one, measure its quality against the client’s spec, and return a pass/fail result fast enough to keep up with the line. Every product had to be analyzed in the time between frame captures, and the counts had to be accurate even though the cameras overlapped and the belt was always moving. The problem is clear, but turning it into a pipeline is where the work begins.

Brainstorming the Approach

Before writing any code, we had to decide what the pipeline should do and how to structure it. This is one of the most important parts of a system like this, and it’s where AI was useful as a thinking partner. We used it to sketch candidate pipeline structures, talk through trade-offs, and review ideas before committing to any of them.

The high-level version of the problem, “look at the belt and decide pass or fail”, involves a number of decisions: how to combine three views into one, how to separate product from belt, how to avoid counting the same product twice, and which quality metrics to compute. Brainstorming with AI helped us surface those decisions early rather than discover them mid-build.

Defining the Pipeline

Out of that brainstorming came a concrete sequence of stages, each simple to describe but with its own speed-versus-accuracy trade-offs:

1. Capture a frame from each camera.
2. Crop and stitch the frames into a single wide view of the belt.
3. Detect products by separating them from the belt background.
4. Deduplicate detections across overlapping cameras and across time.
5. Analyze each new product’s quality metrics.
6. Evaluate those metrics against pass/fail thresholds.
7. Report results to the operator interface.

On this system, deduplication has to work in two directions: across cameras, where overlapping fields of view mean one product can appear twice, and across time, where the same product reappears in consecutive frames as the belt moves. Resolving both correctly keeps counts accurate and analysis efficient, because then you only run the full analysis on newly visible products and track them as such.

The analysis stage depends on choosing fast, well-behaved proxy metrics for human-defined quality terms. Shape complaints become contour measurements, such as how far a product’s outline deviates from its convex hull or the ratio of the axes of a fitted ellipse. Surface complaints become pixel-share calculations over the segmented product mask, such as the fraction of pixels falling outside the acceptable color range or the fraction belonging to interior gaps. Each returns one number per product, cheap enough to compute inside the per-frame budget and stable enough to hold a threshold against.

Confirming the Design with the Client

A pipeline that the engineering team understands isn’t enough; the client needs to recognize their process in it and point out anything we missed. The clearest way to have that conversation was a visual, so we used AI to turn the pipeline into a Mermaid diagram we could share with the client and our own team.

The diagram served its purpose: it let us walk the client through the data flow, confirm that we fully understood the process, and identify bottlenecks and edge cases while the design was still easy to change. Reaching that alignment before writing production code was critical and helped us avoid building the wrong thing.

Meeting the Real-Time Requirement

Accuracy isn’t enough if the pipeline is too slow; every product has to be analyzed before the next frames arrive. Rather than guess at what was slow, we used AI to write profiling that timed each stage of the pipeline and reported where the per-frame budget went. That pointed us at the actual bottlenecks, and from there we redesigned the expensive hot-path metrics into more efficient formulations that produced nearly identical results at lower cost.

From Notebook to Production, with Engineers in Control

The analysis logic is the most sensitive part of the system, because a quiet mistake there directly changes what gets passed or rejected; therefore, we prototyped and validated those components in notebooks against synthetic data first, reviewed them by hand, and only then moved the validated logic into the production application.

For example, if we wanted to create a metric around acceptable product discoloration, we could start by collecting a curated set of examples to test against and play around with different metric implementations to see which ones passed and failed the samples according to our expectations. Some choices of metrics will pass or fail too often, some may not be easily tunable, some may be inflexible, and some may never be computationally efficient enough to fit within our budget.

This is where the earlier distinction matters. The deployed inspection is deterministic computer vision: fast, explainable, and free of any trained model that could drift or need retraining. AI helped us design, communicate, build, and test that pipeline more quickly, but the engineering judgment, particularly around the logic that determines pass or fail, stayed with our team.

Key Takeaways

Real-time inspection pipelines are a sequence of deliberate stages, each with its own speed and accuracy trade-offs. AI is most effective, in this context, for brainstorming architecture and scaffolding a build, not owning final design decisions. It can also greatly improve communication with the client by aligning them early and then serving as a reference for a build later on. Tools like synthetic data or generated images can unblock testing and critical points and turn profiling into an optimized, measured process.

In all of these cases, AI can meaningfully shorten the path from a client’s requirements to a deployed inspection pipeline. The value comes from using it at each step to brainstorm, communicate, build, and test, while preserving the engineer’s ability to discern whether a product passes or fails.