22

Microsoft introduces AI-powered audio and video transcription for OneDrive

Managing media files, especially when sharing them across a group of people is not always an easy task. This is especially true for video files. Today Microsoft has launched a new AI-powered media search function for OneDrive that should help simplify things.

The new feature is capable of transcribing audio and video files and displaying timestamped quotes alongside the media viewer, which is compatible with 320 file types. The new feature is an addition to Microsoft's already available AI-powered photo screening system that can detect location, objects and text in image files.

As an end result all media files become fully searchable. For example, you could search for a scanned receipt by typing one of the items listed on it, or use a piece of dialogue or voiceover to search in video or audio files.

The new feature is designed to facilitate collaboration across larger groups but could certainly also provide benefits for photographers and especially videographers with large archives. The automated transcription services will be natively available for video and audio files in OneDrive later this year.

View Comments (22)

Comments

All (22)
Most popular (0)
Editors' picks (0)
DPR staff (0)
Oldest first
BobT3218

Perhaps they could use their AI to work out how to make OneDrive work as intuitively as Dropbox. Until they do, I'm not interested.

2 months ago
Glen Barrington
Glen Barrington

Not sure what you are talking about.

This website: https://www.cnet.com/how-to/save-space-with-onedrives-new-files-on-demand-feature/

Describes how in Win 10, and I believe, in Win 7 and 8, you can offload files to OneDrive and, in effect, use OneDrive similar to that of a local external drive. Basically, once set up, the local HD maintains the file names in the local folder, but the actual files reside on OneDrive. You just navigate to the local HD folder and read/save as one would normally do. I'm using it for my photos in ACDSee (I assume it would work with other DAM tools), and my MS Office and Libre Office documents. No synch folder, just seamless integration.

It was easy to set up and works like a champ, I don't think it gets any more intuitive than that!

2 months ago*
BobT3218

Glen,
Thanks for that. That's not at all as I envisioned it. Little wonder I couldn't get it to work. Also, our internet speed is terribly slow. It took three days to upload my documents directory last time I tried. I might have to wait till they upgrade the speed.

2 months ago
huyzer
huyzer

This would be great for captioning for the hearing impaired, if they let us download the text, or automatically time it for us, especially for YouTube. Then again, I think YouTube does do this. I'm not sure how well though.

2 months ago
HowaboutRAW

And another reason I don't keep my files on "the cloud". And yeah, I've even used an FTP server once in a while--once even sent someone a 900MB MP4 file via my free Google Drive account with my Gmail account.

But I avoid the cloud for just this kind of reason.

3 months ago
DMakogon

What reason? To avoid transcription services? Not sure what the issue is here. The transcription is for you (or the people you share your content with), for searching. Just like any other metadata you add to content, but generated and extracted for you. This isn't a public-consumption search facility; it's for you, an account owner.

Our team uses Microsoft Stream for our team calls, which greatly eases ability to find specific segments within a video.

3 months ago
HowaboutRAW

You're giving over data (conversations) on videos held on the Microsoft cloud to Microsoft.

Don't suppose for a second that Microsoft, or some other party, won't be able to access that data for all sorts of purposes, outright spying, marketing, "profiling", etc.

Never said it were a "public consumption search facility".

3 months ago
Vince P
Vince P

They can't do that legally. if Microsoft etc were willing/could be bothered to break the law to get at your "really important" stuff then they could do it on your desktop laptop or phone.

2 months ago
HowaboutRAW

Vince P:

What law precludes a cloud service from not scanning your data held on the cloud?

Until 2017, Gmail did just that with emails.

Assuming you know what you're doing in fact Microsoft would have a very hard time getting the contents of your PC.

Now, right Google's Android already has some creepy big brother aspects, but Google has gone quite that far yet.

2 months ago
Vince P
Vince P

It depends on the jurisdiction but the principal throughout is that you own the information not the storage provider so any use contrary to an explicit agreement you have with them is no different than if it was on paper in a bank safety deposit.
Microsoft Apple et al could have unfettered access to everything on your PC if they wanted to abuse their OS.

2 months ago
HowaboutRAW

Vince P:

"It depends on the jurisdiction but the principal throughout is that you own the information not the storage provider so any use contrary to an explicit agreement you have with them is no different than if it was on paper in a bank safety deposit. "

This is simply untrue. And banks really don't have access to safety deposit box contents; they are physically prevented by the fact of the box renter holding a second key.

"Microsoft Apple et al could have unfettered access to everything on your PC if they wanted to abuse their OS."

Not really. You're making assumptions about continuous very fast internet connections. The only way to do such is to push for cloud computing. Then, note how you've said on your PC or Mac. Files on an external hard drive on the floor at my feet are NOT on my PC--references to said files can be found on my PC, but those can be erased.

2 months ago
HowaboutRAW

Vince:

Also I see you've not managed to cite any law to support your claim that Microsoft, or say Google, after transcribing your video in which you describe some terrorist attack, say a real one but in the past, classifies you as suspect, and reports you to various governments.

Now where there is a law, Microsoft, et al, can't use your video that you store on an MS cloud for its own commercial purposes. There you are entirely correct.

2 months ago
Vince P
Vince P

Not sure what you mean, they or others can't search that data without your permission or some other lawful authority. It's not Metadata so under the same controls as any other personal data. Doing so would definitely contravene data protection laws in Europe specifically GDPR and I am sure most of the world.

They use a program to transcribe it, if you let them, you still have control over what permissions you give them to do with the output data. Even AI algorithms that use swarm data to learn, can and do, use anonymised information, even then you can decline to allow that.
Big brother can get into your home, clone your computer, watch you at your keyboard etc, if they have a reason and lawful authority to do so.
The cloud is just another way of storing stuff.
The ongoing debate is about controlling what level of intrusion we are willing to allow in the name of stopping bad people doing bad things, rather than confusing that with where that intrusion takes place.

2 months ago
HowaboutRAW

V:

"Not sure what you mean, they or others can't search that data without your permission or some other lawful authority."

That's simply not true in most countries. I can't speak to every country in Europe. But both France and England are not good about privacy restrictions. Germany has a better reputation. Italy not so much.

"Big brother can get into your home, clone your computer, watch you at your keyboard etc, if they have a reason and lawful authority to do so. "

Sure with a search warrant. Otherwise it's real illegal in the USA and many other countries.

Story documents, not just video transcriptions, on "the cloud" is most certainly one way big brother, or big corporate morelike, can intrude upon your private thoughts, and musings. Right, this is a big reason to avoid GoogleDocs, and MS 360. Hard drives are reasonably inexpensive.

2 months ago*
Mnemon

What is or isn't legal now can change. That governments - and any other organisation - is / could be tempted to abuse and bend what and how they use your data should be obvious for anyone not absolutely naive. At the very latest since Snowdon's leaks. If it gives them an advantage it is far, far too tempting.

The problem with all these aspects is that you hand quite a bit of power and information to an organisation that - whether it is a government or a private company - doesn't have your interests as their major motivation. If they can use the data to further their knowledge about you they will. Even if it's just about better advertisment targetting they are still building up a profile of you as a user.

I'd be much happier if we'd limit how much of that profiling and analysis is happening.

2 months ago
Reactive

Transcription of audio files sounds very interesting. Have you ever found an application that allows you to open a spoken voice audio file, and turn out a text transcription of the speech? It's the missing link as far as I know. Despite all of Google's incredible work on speech recognition and translation, I've never found an app that can take a recorded file of somebody speaking Spanish (for example) and give me a transcript in Spanish and English.

3 months ago
TheClueless
TheClueless

I think the real key will be execution. Microsoft has a raft of competing features which all do the same thing and all it does is to confuse users.

OneNote for example has an audio transcription feature that seems to have gone away / become unusable, and since they now seem to abandon products even faster than Google while leaving anything new in a semi working state for so long, you really have to question yourself before you get too reliant on anything new they roll out.

3 months ago
arguros

You can use youtube

3 months ago
kkx

https://otter.ai is pretty neat. There are many rave reviews about this: https://www.fastcompany.com/40570158/theres-no-perfect-transcription-app-but-otter-is-getting-there

You can upload audio files or record directly using their app.

The translation (Spanish to English) part is missing, but the search, share and other feature is really nicely done.

2 months ago
Reactive

Thanks kkx, I'll keep an eye on otter.ai. But at the moment it only understands English, so that missing link still hasn't been made.

2 months ago
Ron Poelman
Ron Poelman

Pretty much the same tool they've used for sucking the last drop of information out of us.
Obviously, their server load got too high and this will enable them to use our clockwork to hold all those indexes and links for them.

3 months ago*
Vince P
Vince P

Not me I have a special hat that stops this.

2 months ago