Software Matches Images with Music for Great Slideshows

Scientists, movie producers create Picasso software which matches images to movie scenes and their corresponding soundtracks. Result is a slideshow with just the right accompaniment.

Software Matches Images with Music for Great Slideshows
Picasso recommends songs as soundtracks for given text, images and videos. Developed by Saarland University, the software exploits the knowledge of movie directors to match photos with movie scenes and their soundtracks.
Julie Jacobson · February 16, 2012

Say you have this great photo album of spring break in Florida and you want to include it in a musical slideshow. Wouldn’t it be nice if your computer knew intuitively to play “Don’t Worry Be Happy” instead of “September Morn?”

That’s exactly what Picasso software does.

Developed by scientists at Saarland University in Germany, Picasso matches your images with movie scenes and their corresponding soundtracks. The soundtrack genres are compared to music in some database – yours or an online service, it isn’t clear.

From there, a couple of tracks are offered to the user, who picks one to accompany the pics.

The demo of the system was spot-on: Elliott Smith’s serene “High Times” accompanies a series of nostalgic photos; a spunky orchestral accompanies a bike race; and the hip-hoppy “When (and if) the Big One Hits” by The Paper Chase is the backdrop for majestic cityscapes and ruins.

By contrast the “random” slide show played rap music with the nostalgic slideshow (uncomfortable); reggae for the bike race (buzz kill); and jazz for the cityscapes (which actually worked well).

Try the Picasso demo here.

Selecting the music is an “elaborate process,” the University says. Two researchers split 50 movies into screenshots and their related soundtracks. The software ranks the scenes that look most like the user’s image(s) and assembles a list of tracks, which are narrowed down to a few selections through a “mathematical calculation.”

The user selects the most suitable tracks because, “Some people might connect a picture of a little house surrounded by an idyllic landscape with a romantic weekend for two, while others might think about loneliness,” says Sebastian Michel, head of a junior research group at the Cluster of Excellence on “Multimodal Computing and Interaction” at Saarland.

Next up: music to support automatic sound recording of audiobooks, using text instead of pictures for context.

A free smartphone app called PicasSound is available for iOS and Android. It is programmed to select tracks from music stored on the user’s device.

Picasso will be presented at the CeBIT 2012 tradeshow in Hanover, Germany, March 6-10.

  About the Author

Julie Jacobson, recipient of the 2014 CEA TechHome Leadership Award, is co-founder of EH Publishing, producer of CE Pro, Electronic House, Commercial Integrator, Security Sales and other leading technology publications. She currently spends most of her time writing for CE Pro in the areas of home automation, security, networked A/V and the business of home systems integration. Julie majored in Economics at the University of Michigan, spent a year abroad at Cambridge University, earned an MBA from the University of Texas at Austin, and has never taken a journalism class in her life. She's a washed-up Ultimate Frisbee player currently residing in Carlsbad, Calif. Email Julie at [email protected]

Follow Julie on social media:
Twitter · LinkedIn · Google+

Julie also participates in these groups:
LinkedIn · Google+

View Julie Jacobson's complete profile.

  Article Topics

News · All Topics
CE Pro Magazine

Not a Magazine Subscriber?
Subscribe Today...It's FREE!!