In the world of electronics assembly, parts are really small, quantities are really large, and equipment is really expensive. The process of putting all the components onto circuit boards thousands of times per hour with really high accuracy is challenging, but tools are getting easier and cheaper every day.
At my job (Quietyme), we’re doing our own circuit board assembly. We had to purchase some equipment, like a pick and place, which for $5000 automates the process of taking the small components off of reels and putting the components on the circuit board. We also have a stencil screenprinter and a reflow oven (toaster oven). Total investment so far has been under $6000.
The most recent tool to add to our arsenal is an Automated Optical Inspector. The purpose of the AOI is to examine the circuit board to find defects in the assembly process. Sometimes a component doesn’t solder on correctly, or there is a bridge where there shouldn’t be. These need to be found and fixed before they are programmed and used. We had been doing it manually with just visual inspection, but that was tedious and not very accurate, especially after the 100th board.
I had been reading into OpenCV and decided I’d give it a shot. The idea was to take a regular webcam, point it at the circuit board, do some image analysis to figure out if there were parts that didn’t match a “good” board, and highlight those so that a full inspection by a human wouldn’t be necessary. After a little bit of playing, I realized it could work. I designed a holder and laser cut it out of acrylic, then glued it together. It was pretty slick. With the threaded rod I could raise the camera up until the PCB occupied the entire field of view, so no pixels were wasted. I wanted consistent even bright light, so I added a bunch of white LED strips. I immediately noticed a problem with hot spots, so I added a layer of acrylic with a diffuser (a piece of paper glued to it). Finally, I glued the bottom of the enclosure to the base. The PCB was designed for the enclosure, so it was a perfect holder that would consistently keep it in the same place.
The camera is a 720p Microsoft LifeCam that I had laying around. In the next version I’ll upgrade to a better camera. The LED strips are just 12V white LEDs.
Now the software. I used the OpenCV CV2 python library, and ran it both on my main dev computer (Linux Mint), and my test laptop (Win XP (shut up)). The older computer was definitely slower, and setting up OpenCV in windows is a pain, but I made it work.
The first step is to get the video stream from the camera and display it. Fairly simple. Then I experimented with diffing against a known good board. The idea was that by subtracting the video from a good image, only the differences would be visible. This is good in theory, but it didn’t work at all in practice:
This is way too busy to think about and try to analyze to find poorly placed components. It wasn’t going to work.
The next idea was to mask out anything we didn’t care about. There are unpopulated components on this board, and we don’t care about the silkscreen or the outside edges. There’s no point showing them to the user. So I created a black and white mask:
This mask is displayed over the top of the video stream, so that the operator only has to look at the unmasked parts to evaluate the board. Like so (note that this is an intentionally sabotaged PCB which has bridges, tombstones, and misaligned components that I put in to test the pattern matching):
Great! Now the work is already a lot simpler. From here I wrote a quick routine that would zoom in on the Zigbee chip to digitally and show a 4x zoomed chip to look for bridges. So no pattern matching yet and we’ve already significantly improved the inspection process and reduced eye strain.
The next step was adding in the pattern matching. The idea is to use the OpenCV template matching feature to identify components that are correctly soldered and use those as a template, then when the video stream sees those templates matched it’ll draw a black box over the component indicating that it doesn’t need to be examined. So the above image after pattern matching should look like this:
Bam! Now we’ve gone from having to inspect a full board to just having to check over a few components! In reality it ends up not matching perfectly so we usually see a few extra components that are just fine, but it’s better than having the threshold too low and accidentally blocking out bad joints.
So how does the setup process work? By drawing boxes! In the python application, just drag a box around the component you want to use as a good template, and it will create an image and store it and use it as a template from then on. I started by first looking over the entire video stream for each template, with the idea that a capacitor in one place might look like a capacitor somewhere else. This was a dumb idea and slowed down the system to unusable. Then I got the idea to save the image coordinates as the filename, and only look within a certain bit of slop within that area for that template. That got me back up to near real time processing of the feed. It also allowed me to have multiple good templates for a single component to bump up the likelihood of matching without sacrificing the threshold of matching. So it ends up working pretty well. Here are some examples of component templates:
So to summarize, I’ve put a few hours of time (<20), a webcam, and some hardware into this, and I’ve got a functional AOI to add to our assembly line. It’s really only useful for small circuit boards, and a single camera that doesn’t move definitely has limitations, but it’s really a testament to the awesomeness of OpenCV for making this so easy.
It should be noted that a cheap AOI can be had for $20k, and the ones used in some factories sell for over $100k. Mine is by no means competitive with theirs, and couldn’t find nearly as many types of faults on as large a board. They move the camera around and look at solder quality using colored lights and blah blah blah. But for the price of a webcam, some acrylic, an LED strip, and a few hours of time, this is a pretty big step.
Want the code? Here ya go: AOI Python code