Deborah Schmidt, Johannes Fritzsche
How do we create markers that are visible to cameras, but not to our eyes?
Tracking markers (e.g. QR Codes) are a common tool to create mixed reality applications. They look ugly, they can’t be too small for successful detection. In our case, we wanted to use them to recognize book pages and project onto them, but the projection on top of the blunt black pattern would not give the content justice.
A common technique to recognize something visually outside of the humanly receptive spectrum is the usage of Near-Infrared (NIR) cameras. There are high quality / high price solutions to print the markers with the same special ink used on paper money to create invisible watermarks. We experimented a lot to find a cheaper DIY solution and here it is!
Build a Near-Infrared receptive camera
First, we need a Near-Infrared receptive camera. You can specifically buy one or you can also modify a cheap webcam by removing the IR filter and adding a visible light filter (e.g. dia film). Here is a guide.
Material and color experiments
Experiment with the camera. What do you see? How do different colors look like on the material you want to use?
-
- Johannes is inspecting how color is printed onto the paper.
-
- This is what the webcam we altered to be NIR sensitive displays. My hoodie is black.
Since we wanted to print markers, we used a color printer with black cartridge as well as color cartridges and took standard printer test pages (this one from Ubuntu was perfect) to get a feeling for what the IR cam can detect. We realized that what we printed with the black ink was also visible in NIR, but stayed invisible to the camera if printed using the color cartridges.
This marker was printed in a middle tone grey with the black ink and inserted a second time into the printer to print a colored motive on top of it. You can hardly see the marker and utilize any fitting motive to cover it.
For our scenario we needed a plane, light, non-distracting surface. It also had to be something that we could mass print. Therefore we printed the markers in the lightest grey that was still detectable using the black ink, and printed the remaining white quads (the negative space of the marker) in a similar grey by mixing the color inks in a similar amount.
Marker image creation
In Inkscape there is a simple way to achieve that: Assign the color using the CMYK color picker and make sure that C, M and Y are zero. The printer will use the black ink. Add a small amount to Cyan, Magenta or Yellow and the printer will solely use the color cartridges, but create a similar grey tone. This way, we did not have to insert the page twice into the printer. Of course, if this works depends on the driver and the system – experiment.
Result
We were able to print a 100 page book with markers of the size of the full page using our home printer. We were able to project onto the markers without anyone noticing them (they were still recognizable though).
Have a look at the final project video to see the result: