Spotlight on Metadata

An old adage about economics says that ‘when America sneezes, the world catches a cold.’  Today the saying could just as easily be used to refer to technology, considering the domicile of most of the world’s biggest IT companies.

In light of recent revelations – and speculation – about the US National Security Agency’s use of metadata, it is little wonder the ripple effects are being felt around the world. As far afield as Australia, proposed legislation requiring internet companies to retain email and phone data for two years has been shelved pending further consideration. Similar responses are being seen in Europe, where privacy laws are both stringent and complex.


The scrutiny of metadata about calls, such as time and duration, allowed the NSA to determine a considerable amount of information, in particular helping to identify where to direct its considerable intelligence gathering powers. In effect, it helped to narrow down where to find that needle in a haystack information that could potentially alert authorities to terrorism activity before the worst happens.

Putting this process into the mainstream media raised legitimate concerns about use of information, as well as outraging the conspiracy theorists.

Of course, when handling such sensitive information, the issue of privacy is immensely important. Most of us accept the need to track security risks but balancing that with individual rights is potentially the stuff of nightmares for executives of phone companies, government organisations, legislators – anyone with sensitive data in fact. Especially when that balancing act is performed in the spotlight of the world’s media. Those same executives are under equal pressure to extract the value from data and share it with chosen audiences.

On the plus side, the value of metadata has achieved a far greater awareness, yet even the definition of metadata has been subject to scrutiny. Businessdictionary.com defines metadata as:

Data that serves to provide context or additional information about other data. For example, information about the title, subject, author, typeface, enhancements, and size of the data file of a document constitute metadata about that document. It may also describe the conditions under which the data stored in a database was acquired, its accuracy, data, time, method of compilation and processing etc.’

One positive aspect of the fallout from the NSA situation is that far more organisations are aware of the potential benefits of managing metadata as well as the imperative for a well thought out plan. While the NSA is used to explaining and defending its practices to parliament and media, the phone company CEOs are unlikely to have relished the attention this saga attracted. Even in businesses and government departments far removed from the topic of phone calls, the awareness is prompting executives to consider the opportunities and ramifications presented to them – or in other words, to protect themselves.

This really validates the work we’ve done at Space-Time Research to take a lead on data lifecycle management based on our experience of the Generic Statistical Business Process Model (GSBPM), an international best practice model. The data lifecycle management process guides organisations to get the most from data – and metadata – while determining appropriate use and retention to safely navigate the various legislative minefields.

Inevitably, part of the sudden attention has diverted to the need for confidentialisation. From individuals to commercial and government organisations, the thirst for greater knowledge and demand for transparency has never been so great. Of course, in spite of a population disclosing more personal information than ever on the likes of Twitter and Facebook, we conversely place a greater priority on confidentiality than ever before.

Through our SuperSTAR products, Space-Time Research is the only organisation to provide confidentialisation of unit record data so that our customers can share a far greater level of insight without being added to the speed dial list of every investigative journalist on the planet. Used by many of the world’s leading statistics organisations as well as government departments and, increasingly, businesses, SuperSTAR ensures the headlines are for all the right reasons.