Color Binoculars landed on the App Store today, promising to infuse the real world with color for anyone with the three most common forms of color blindness. This isn’t the first app designed to help color-blind folks see a broader spectrum of colors, but it comes from two Microsoft software engineers (one of whom is color blind), and its straightforward filter method is simple to use.
The app uses the iPhone camera to adjust colors in a way that makes them easier to distinguish for color-blind people. The enhanced image shows up on the iPhone screen, allowing users to pick out flowers, choose matching outfits or take in the beauty of fall, for example.
Tom Overton and Tingting Zhu started working on Color Binoculars during Microsoft’s 2015 Hackathon and they finished it in the company’s Garage program, which helps experimental apps go public. Overton is color blind, so he was both a developer and the app’s main tester.
Tom Overton and Tingting Zhu (Image credit: Scott Eklund / Red Box Pictures)
“It’s an app that helps color blind people distinguish color combinations that they would normally have trouble telling apart,” Overton tells the Microsoft blog. “For example, since I have difficulty distinguishing between red and green, our app makes reds brighter and greens darker so that the difference is more obvious. It replaces difficult color combinations, like red and green, with more easily distinguishable combinations, like pink and green.”
Microsoft’s Word Flow keyboard for iPhone just got a significant upgrade this week, adding a search engine for emoji, GIFs, and more from Bing.
The new search feature will copy GIFs to your clipboard so you can paste them into messages and can even choose GIFs from what you type for contextual searches. If you type something like “yaaaas!” or something inane like that, you can search for matching GIFs of that nature.
Microsoft is planning on adding in additional themes, support for iOS text replacement and cursor placement using 3D Touch.
The new built-in search is obviously meant to compete with Google, after Google previously released the Gboard keyboard for iPhone with built-in search.
There are several different keyboards available for iPhone to choose from, but now that giants like Microsoft and Google have made their own options available, the vanilla iPhone keyboard seems like an afterthought, especially considering Microsoft previously acquired SwiftKey.
It’ll all come down to personal preference, but Microsoft just shot ahead to Google’s level with its addition of these new features.
Microsoft sure loves it when research projects beget actual products, and it just released another for the masses to play with. Pix is a replacement camera app (what?) available for iPhones and iPads (what?), and in short, it promises better photos of the people around you without any extra work on your part. It’ll run on just about every iOS device from the iPhone 5S on, and an Android version is in the works too. (Microsoft didn’t have a firm answer when I asked if these features would make their way into the Windows 10 Mobile camera.) And you know what? In some ways, I wish this was the camera app that Apple built in the first place.
First, the basics. The most important thing you need to know about Pix is that it’s been tuned to make your pictures of people look better.
“There are things the Apple camera does that we don’t do and might not ever do,” said Josh Weisberg, GM of computational photography at Microsoft. “The goal was around people photos — can we make better people photos than the stock camera? And we succeeded.”
From the moment you start Pix, it’s capturing what your camera is pointed at -– you can never tell when something’s about to happen, after all. Once you press the shutter button, Pix snaps 10 frames, and Weisberg says that’s where the magic really kicks in. Algorithms evaluate those ten frames for obvious things like sharpness or exposure, but also underlying characteristics like whether a person in the shot seems happy or sad. When that near-instantaneous process is done, you’ll be given up to three “Best Images.” The image data from the leftover photos is used to enhance those winners before being deleted. All of this happens on the fly and without any extra fiddling, so you don’t need to be a photo buff to snap some great shots.
If the app detects a bunch of similar pictures, it’ll stitch them into a Live Image, but only when it thinks what’s going on in the photos is interesting. Oh, and the Hyperlapse feature that Microsoft has been working on for years is here too. This time, though, you can turn existing photos into time-lapses, or just use it to stabilize video you just shot.
Using Pix is very much a learning process, and I don’t just mean for you, the user. According to Weisberg, the app sends anonymized bits of “telemetry” — settings data and what Best Images people fave’d or deleted — back to the mothership, where human judges will examine them and adjust the image processing algorithms accordingly. Basically, the more you use Pix, the more insight it gains into what makes a photo good. All told, Weinberg was right: The app really is helpful for improving your photos of people. Usually, anyway.
In no time at all, I was snapping photos using Pix that came out punchier and with a greater emphasis on the people in the shot. When the testing period inevitably overlapped with post-work drinks at a local dive, Pix shined even brighter. I mean that literally too. Smartphone camera sensors often flounder in dim, dank conditions, leaving software to do the heavy lifting required to make a passable photo. Microsoft’s photo processing was both super-fast and mostly great at brightening up pictures of my colleagues and removing grain without making things look unnatural. I was utterly impressed… until I wasn’t.
Left: Microsoft Pix; Right: Apple’s camera app.
My biggest issue with Pix in its current form is that it’s inconsistent. Sometimes my shots were clear improvements over what Apple’s camera app was capable of. Other times, though, Apple’s softwaer had a clear edge. Take landscape photos, for instance: Even before Microsoft’s instantaneous image processing did its thing, the app had trouble exposing shots with bright backgrounds. Pix’s outdoor shots tended to be a little blown out, while Apple’s camera software was generally better at balancing exposure levels.
And for all the work that went into teaching Pix to enhance photos of people, it still struggles at times. A “Best Image” it suggested of a colleague in the dimly lit dive mentioned earlier was noticeably less crisp than the image the camera actually captured. In the app’s zeal to brighten up her face, it smoothed out her features a little too much. Long story short, the version of Pix I played with was still fairly hit or miss.
Left: Microsoft Pix; Right: Apple’s camera app.
But that doesn’t mean you shouldn’t use it. In fact, I’d strongly recommend giving this a download, even if you’re not the sort of person who already juggles multiple camera apps. The benefits of better image processing can be seen from the get-go, but the weightier, far more fascinating goal is to see how much Microsoft’s system can learn about good photographs. In a way, it’s almost as though we’re collectively training it to better understand art. The very nature of Microsoft’s algorithmic processing means these early issues will probably get ironed out over time, and I’m fascinated to see how long it takes before Pix becomes great in every situation.