46

Google teamed up with pro photographers to train its Clips lifelogging camera

Google debuted its $250 'Clips' lifelogging camera to the public during the October 2017 launch event for the Google Pixel 2. Now, with the camera becoming officially available in only a few weeks time, the company has published a blog post that explains how the Clips camera's underlying algorithms were trained to identify and keep the best shots, and discard the leftovers.

It turns out Google relied on the expertise of a documentary filmmaker, a photojournalist, and a fine arts photographer to train the AI and feed some high-quality photography into its machine learning model. The group collected and analyzed recorded footage from members of the development team to try and answer the question: "What makes a memorable moment?"

"We needed to train models on what bad looked like," said Josh Lovejoy, Senior Interaction Designer at Google. "By ruling out the stuff the camera wouldn't need to waste energy processing (because no one would find value in it), the overall baseline quality of captured clips rose significantly."

The learning process includes basic elements of photography, such as an understanding of focus and depth-of-field, or the rule of thirds, but also some things that are obvious to most humans but less so to an algorithm—for example: don't cover a lens with your finger and avoid abrupt movements while recording.

Google admits that there is still a ways to go before perfection. It says the AI has been trained to look at "stability, sharpness, and framing," but without careful calibration, a face at the edge of the frame will be appreciated just as much as one at the center, even if the focus of interest is really somewhere else in the image.

"Success with Clips isn't just about keeps, deletes, clicks, and edits (though those are important)," Lovejoy says. "It's about authorship, co-learning, and adaptation over time. We really hope users go out and play with it." More detail on the development and training process is available on the Google blog.

View Comments (46)

Comments

All (46)
Most popular (1)
Editors' picks (0)
DPR staff (0)
Oldest first
Photoman
Photoman

Will it stop taking photos when I'm in the toilet...

8 months ago
Alex Akai
Alex Akai

Welcome to the "Circle" and we know how that ended...

9 months ago
thenoilif

Google is showing that they are quite clueless when it comes to hardware. They continue to make products that are just downright creepy when it comes to invasion of privacy. They should have learned from google glass. They also release their products with very little marketing so I often wonder if they actually have any sort of business plan for their hardware. For example, the Google Pixel is actually a decent phone from a hardware standpoint but its a brick (compared to the competition) seems to have its odd user limitations and I don't know a single person who owns one. I think it will be the same for this camera.

9 months ago*
DoctorQ99
DoctorQ99

I own a Google Pixel phone. Pleased to meet you!

9 months ago
RDVR
RDVR

I own a Pixel 2, as do two close friends. What are the odd limitations you refer to?

9 months ago
Impulses

Three Pixel users in my immediate family, mine is an original (small, 128GB), my sister and in law have the Pixel 2 & XL. I'm not sure how it's a brick, either model is as thin and light as any other phone... The 2016/small models had larger bezels but I've never heard anyone describe that as brick-ish.

I've had 3 HTC phones, a Nexus/LG, and a Sony, and the Pixel has easily been the most trouble free and smoothest after a year or use. /shrug

9 months ago
Impulses

No plans to buy this camera btw, in case you think there's a crossover there or I just buy everything they make...

9 months ago
cosinaphile

i cannot too lightly dismiss my contempt for the horror that apple and google
effect on the consumer experience of cell phones

the inability to freely use storage to allow a phone-based solution to the need of a media repository for music film and books ... forcing users in a revenue extraction scheme to access their person i the marketplace ally owned media ... and the murder
of the audio jack standard with absolutely nothing posed to replace it

go to any store anywhere and take a look at wired headphones ... see 2 nd standards there ? nope ... there is no second standard !..., its either wired with a headphone jack .... or Bluetooth [ also with a headphone jack ]

put the apple\ google coolaid down to see this for what it is ... an attempt to kill a standard they cannot monetize apple and google are the leaders
of anti consumer conduct ... the users of these products are fools

9 months ago*
Impulses

My Pixel has a headphone jack and I keep about 30GB of music on it at all times with no issues... /shrug

9 months ago
cosinaphile

impulse im glad you are satisfied

the pixel 2 and 2xl lack both sd and audio jacks crippled for no purpose but corporate greed

9 months ago
Impulses

Ehh... That's a big assumption. I can almost get on board with it as far as the headphones jack, at least in Apple's case, but Google doesn't even have a big accessory ecosystem to profit from by "forcing" Bluetooth on people. Shoot, didn't they lower the price of the Type C to headphone adapter? I agree not having it still sucks but I don't buy the corporate greed conspiracy wholesale.

Lack of microSD isn't even remotely close to forcing anyone to use cloud storage, if they wanted to double down on that they wouldn't sell 128GB+ versions or they'd price the NAND upgrade at even more of a premium. Not having the slot IS more secure in many cases and DOES often allow the money to be spent on a faster internal solution that users without an option will definitely use.

9 months ago
Impulses

Sure having both would be great... TBH I had mSD slots on 3-4 phones for 3-5 years and the only time I ever took it out was when I upgraded. I'm willing to bet that's a fairly common occurrence with most people using cases, or needing to reboot to get at the card, etc. I don't feel like I've lost anything but I recognize there's advantages to having it like data recovery when the device is damaged.

This isn't even a smartphone/mobile site tho, I dunno what banging the mSD/3.5mm drum over and over in any article remotely related to the subject is really accomplishing...

9 months ago
justmeMN

Is it really: "Google teamed up advertisers to train its Clips lifelogging camera to collect information about you"

After all, that's Google's primary business.

9 months ago
Jack Jian

Imagine the day they join hands with Facebook.

9 months ago
noisephotographer

What I don't understand: Why is there such a large camera bump?Apparently it's a 1/2.3" sensor + f/2.4 lens, so just a smartphone camera. Does it have a huge battery?

9 months ago*
srados

I can see this being used with Dementia patients.On the end of a day they can review it...

9 months ago
Mike Davis

... after asking the nurse, "What is this thing on my gown?"

9 months ago
srados

Doesn't Google is doing this as part of NSA, PRISM already logging our lives via smartphones and social media?

9 months ago
KodaChrome25
KodaChrome25

I clicked on the blog to read further. I was doing the two-finger scroll on my trackpad to read the post. Later when I woke up, I found I had scrolled to the bottom of the blog. Just closed the tab and went on with my day.

9 months ago
cosinaphile

its like contracting the poets of the king james bible to write trump jokes for jimmy kimmel

they need to do something to make this 40 dollar video camera seem worth 250 ridiculous dollars

9 months ago*
tinternaut

Sounds like the device worn by some of the characters in The Circle. So who fancies going transparent?

9 months ago
h2k

"So who fancies going transparent?"
---
Well not me, i promise. But others wearing the thing will make anyone in their path transparent anyway.

9 months ago
Impulses

That movie was pretty awful tbh... I hope it's not gonna become part of the gestalt for this kinda discussions, there's way better examples even if you agree with some of it's ideas.

9 months ago
tinternaut

I wasn’t aware it had been made into movie...

9 months ago*
Impulses

I hope the book was better!

9 months ago
HDomDP

Like a lot of human jobs are replaced with automation, I still feel we may be witnessing the beginning of the end of professional event photography. It may not happen until this tech is better and the cameras are cheaper but down the road, I can totally see some people would choice having a few of these cameras at the kids party or even wedding, instead of hiring a pro for photos / video.

9 months ago
chrisno

such technology requires a lot of developments in automation, softwares, sensor and lens..

not gonna happen in the next 20 years.

9 months ago
damian5000

You might be massively surprised. Look how far tech has come in the last 20 years. Huge strides.

9 months ago
chrisno

Technology disrupt faster in areas that makes a lot of money, not areas making little money.

To replace total control of human takes billions of developments in areas above which doesn't make sense for cost efficiency at this moment of time.

9 months ago
cosinaphile

it will happen .. and be adequate for the artless souless consumer peasants it targets .... good enough like fake bokeh is good enough ... we will get less and less the more we accept less and less

9 months ago
Impulses

Ehh, they said the same about cheap disposable P&S cameras and handing dozens of those out still kept some human intelligence and skill in the loop... I don't see the tech getting the far that quickly (where you just have tiny AI controlled drones filming everything rather than weird wearables with weird angles), and a lot of people will never care for that approach regardless.

9 months ago
chrisno

yup exactly. drones also too loud to be everywhere
automation of driverless cars, customer services, delivery, cleaning and serving areas would be the early developments. not photography and videography.

9 months ago
damian5000

Too loud "now" is key. I'm not saying it WILL happen, but if you don't think it might happen you haven't been paying attention very well.

9 months ago
chrisno

you just gotta be realistic how much money it can generate
ok if it generate 500 billions a year, probably the tech will get there

and you forget the battery life
the drones cannot be small enough, not for the next 5 years

9 months ago
Impulses

I don't think anyone can reliably predict 20 years ahead... We look at old sci-fi now and we get a kick out of the stuff that acutally came true, but for every one of those achievements in tech there's like three that still haven't happened.

5 and maybe 10 years down the line is somewhat easier... None of this is happening in 5 years' time. It took like a dozen years of development and refinement for drones to *sort* of break into the mainstream. It took half a dozen years for smartphones to get to the kinda elegant state they now enjoy...

(and that's if you don't count any of the foundation laid by players no longer really in the game, Palm, Nokia, etc)

9 months ago
Impulses

The first iPhone came out over ten years ago and you could pretty much see where things were headed by then, even if things like apps and copy/paste were notoriously missing (they were there in other phones already).

If this kinda photog-replacing tech was right over the horizon we'd have more of an inkling by now. Alexa and Siri aren't ready to pilot a drone at any wedding I'd wanna go to. :P I'm sure it'll happen, eventually, maybe my kids or their kids will enjoy it.

Shoot if we're ever leaving this rock I'd want AI documenting it cause it's gonna be a long trip to anywhere else!

9 months ago
damian5000

Right. 3 that haven't happened and another 100,000 that weren't predicted that did. The reasons people give here are reasons why it's not done TODAY. Not 20 years from now. Obviously nobody can predict the future but you'd have to be a fool to not see it as a reasonable possibility. "They're too loud" 😂😂😂

9 months ago
Martin JC

Be aware, very aware. Google is interested in Google. Do you really want to log your life with 'clips' and hand over that data for free to Google?

9 months ago*
RED i

Seeing how many people are willing to post their personal lives on social media to use a free service shows you the majority doesn't care.

9 months ago
h2k

"Be aware, very aware. Google is interested in Google. Do you really want to log your life with 'clips' and hand over that data for free to Google?"
----
YES!!! YES!!!! YES!!!!!

I hear many a selfie enthusiast scream.

9 months ago
Richard Murdey
Richard Murdey

Google are welcome to it.

9 months ago
Mike Davis

Join the collective. Resistance is futile.

9 months ago
noisephotographer

Google, release an interchangeable lens camera!

9 months ago
noisephotographer

Or a HDR+ camera with multiple cameras for even better image quality and more fields of view.

9 months ago
chrisno

for $250, this will probably be the worst selling product in google lineup

9 months ago
Mike Davis

I think you're right - it will be a flash in the pan. People who were insensitive enough to wear Google Glass in public places took a lot of heat from strangers who didn't want to be photographed. What's changed? People will get rid of them after getting called "creep" a few times.

9 months ago