Organisations worldwide are using software and hardware developed at the University of Southampton for the capture, processing and analysis of visual and multimedia data. Around 1,000 companies, including Wikipedia and Booking.com, are benefiting from the efficiencies these advanced technologies offer in terms of computer processing time and cloud storage. Techniques developed at Southampton are also helping institutions such as The British Museum and The Louvre to capture specialist images of important cultural heritage objects.
Research challenge and context
Whenever we browse the Web, we set off a chain of complex processes that allow images to be retrieved and displayed on our screens. Over the last decade, one aspect of image processing research in Electronics and Computer Science at Southampton has focused on enabling computers to keep up with the ever-growing demand for – and volume of – high-resolution images on the internet. Another stream of research has looked at ways to use machine learning and mathematical tools to analyse vast libraries of images.
Southampton’s image capture research has aimed to provide specialist imaging systems for museums and other cultural heritage institutions. While there have been huge advances in digital photography over recent years, traditional techniques still result in ‘flat’ 2D images. Southampton researchers aimed to develop a system to quickly capture the fine 3D detail of artefacts, creating a practical way for museums to take pictures of, and study, large collections.
Our solution
In collaboration with end-users and industry, the researchers developed two open-source software tools: libVIPS for image processing and OpenIMAJ which extracts information from multimedia data.
Most image processing systems load the entire image into the computer's memory and then transform it in a series of steps; each of these steps needs another complete copy of the image being worked on. libVIPS takes a different approach, chopping images into small tiles which are processed by separate cores, before being reassembled into their final form at the end. This enables it to run quickly with much smaller memory requirements. This software has evolved over decades of research projects and is now maintained by an open source community.
OpenIMAJ uses advanced methods to carry out specific tasks such as feature extraction, which uses machine learning to recognise particular shapes or patterns in digital images. The software also incorporates new approaches for the analysis of multimedia data such as videos.
The researchers worked on developing an image capture method known as reflectance transform imaging (RTI). This involves placing an object under a dome, in which it is photographed around 100 times with different light angles. The resulting data allows the viewer to digitally ‘re-light’ the object in order to investigate small surface details.
What was the impact?
These tools are contributing to more cost-effective and efficient business processes, helping organisations to innovate, and enabling researchers to gain new insights into ancient artefacts.
libVIPS is being used by around 1,000 companies to help them develop websites and systems that handle millions of images, using less cloud storage, speeding up processing times and saving more than 50 per cent of computing capacity. libVIPS has enabled Wikipedia to use high-resolution images for the first time, and Amazon Web Services uses it for efficient image handling in its content distribution networks. Other organisations that have adopted libVIPS include The National Gallery, US Food and Drug Administration, Booking.com, The New York Times and Capital One. It is also being used in medical settings to quickly process large diagnostic images.
Similarly, OpenIMAJ is being used by developers across the world, including in large organisations such as Yahoo and Comcast, to quickly create applications that extract information from multimedia data.
The researchers have worked with Ordnance Survey (OS) to develop new ways to exploit aerial imagery. Previous methods for extracting data from OS images were largely manual, making them costly and limited in scope. Southampton’s work on the use of machine learning to extract specific features from aerial images has enabled OS to prototype new data products, for example tools to classify building types and agricultural land use.
Imaging systems designed and built by the Southampton team are being used for museum, archive and university research in institutions worldwide to capture highly detailed images of historical artefacts. For example, researchers at the University of Oxford are using Southampton systems to study and decode fragile tablets inscribed with the world’s oldest undeciphered script. The British Museum uses a similar system for studies where conventional photography is insufficient to capture surface details, and at The Louvre the technology has been used to capture an entire collection of ancient seals for the first time.
Photo credit: Banner image courtesy of Biblioteque Nationale de France.
Related publications
Detection of Social Events in Streams of Social Multimedia
Practical scalable image analysis and indexing using Hadoop
Joint deep learning for land cover and land use classification
Archaeological applications of polynomial texture mapping: analysis, conservation and representation
A structured light approach to imaging ancient Near Eastern cylinder seals
Find out more
Talk to our research team and find out more about this work. Professor Kirk Martinez and Dr Jonathan Hare led the research on this project.