Big Data: You Have Been Measured

Expect to hear the term "Big Data" more and more over the next few years.

It's the term being used to describe the way companies are harvesting and using the enormous amounts of data that are stored about our behaviour on the Internet, and through shopping online. The amount of information is so large (and doubling every 3 or so years), that it has spawned a whole new science for the way it is analysed.

A couple of examples to help get your head around it – both come from a new book on the subject to be published next month, Big Data: A Revolution That Will Transform How We Live, Work and Think

When I walk into a bookshop, I will spend a lot of time browsing - picking up books, flicking through them, putting them down again, before I settle on the book i want to buy. If a bookshop manager were to follow me around, noting these things down, he would learn a lot about me, and may even be able to suggest a book that I might like to look at. Impossible, of course.

But when I am doing the same thing on Amazon, all that information is being collected – what I look at , for how long, what pages I visit before I go onto Amazon, where I move onto afterwards. And over time, and collected together with data from other shoppers, Amazon has a monstrous data set that describes not just my behaviour and preferences, but the behaviour of huge populations. The number crunching machines at Amazon are able to analyse and test millions of hypotheses which can then be used to do a number of things: Work out what kind of covers I and other respond to. Work out the kinds of things I might also be interested in buying etc. Determine how driven I am by price and discount labels. Their primary aim is to work out how to sell me more.

Google works with similar data sets which have logged every search term you have ever typed into its search engine. It did a study it did on big data sets it looked at search terms during the winter when the flu virus was moving around, and discovered that it was able to successfully predict what regions and cities of the US would have a flu epidemic, days and even weeks before the conventional medical reporting system had registered the problem.

Christians often focus on the "scare story" behind the use of Big Data. But there is a very significant upside to be thankful for. There seems little doubt that there will be many uses of Big Data that will push forward medical science and technology to make all our lives better.

But there remain huge concerns - because this Big Data - about you and me, and the rest of humankind - has both moral and immoral uses. On one hand, working out what to offer me is what companies are all about. Amazon may show me something that I am really grateful for. But that same data could be used to manipulate me to spend money on something that I really don't want to buy.

Put these same tools into the hands of politicians, governments and organisations with a desire to shape and mould our thinking, and you have a whole set of moral questions to wrestle with. In the hands of the unscrupulous, it is an extension of the propaganda machine that has influenced our lives unseen for years through advertising, newspaper articles and other forms of social messaging. The data may be neutral - but those who want to use it may not be. And we are right to be concerned that there are legal checks and controls on how this data is used and shared. It needs to be used fairly.

Big data is nothing new to Christian believers. We serve an omniscient God who has an intimate knowledge of all the data about our lives. God measures our lives against his perfect standard - Jesus. And in that regard we are all found wanting. What the Lord said to Belshazzer at his feast is true for us: We have been measured and found wanting (Daniel 5 v 27).

But Jesus told us that the hairs our my heads are numbered (Luke 12 v 7) - not to make me feel uncomfortable about a prying judge, but to reassure me of my value before the God who made me, and to teach me to respect, love and serve Him.