Big data is watching you

Data!data!data!” he cried impatiently. “I can’t make bricks without clay.” So said Sherlock Holmes in The Adventure of the Copper Beeches.

He may have found a latter-day ally in Attorney-General George Brandis, who is seeking to force telcos to keep data on your telephone calls and internet usage for up to two years, for the potential use of law enforcement agencies.

Technically we now have the ability to do what the Attorney-General wants. It’s been made possible by the tsunami we call ‘big data’. But is this revolution a friend or foe?

Data is the lifeblood of research and more recently it has gained similar importance for business and industry.

Technologies like sensor networks enable us to gather enormous volumes of data at prodigious speeds and often in complex forms. Where research data was once a set of meticulously measured points, for example in an agricultural experiment, it now might be a sequence of images or a video being captured in real time.