What data journalism means to me

So I told someone recently I was a data journalist.

"Define 'data journalism'," they said.

"No," I replied.

"Why not?" they asked.

"Because it's an incredibly tedious thing to do," I said. "And anyone who thinks it is important to define data journalism precisely is likely to be doing it really, really badly."

I'm paraphrasing. In real life I'm not that much of an asshole. But it's basically what I said, and certainly what I believe.

The fact is, I am not interested in precisely defining data journalism. I am interested in practising data journalism, and how other people practise it. 

I am interested in brilliant content - content that is valuable; content that is unique; content that people actually want to read, share and engage with.

I am interested in how we harvest the sea of information that is now available to anyone with a computer, tablet or mobile phone. 

Information being pumped out, in real time, in the guise of data feeds. 

Information being published by governments in the form of spreadsheets that no one seems to bother to read. 

Information - stories, scandals even - hiding in plain sight.

Now, okay. If we can't construct any kind of definition of data journalism, then it becomes a meaningless concept. 

I wouldn't want that. 

But I'm with Wittgenstein on these things. I think data journalism is defined primarily by practice. I also believe it is a classic Wittgensteinian 'vague concept'. 

Like sport. 

Some things are clearly sports (rugby, say). Some things are clearly not sports (having a bath, for example). And then there are things in between. 

Is darts a sport? It has certain 'sporty' features, like precision improved with training, competition, etc etc. 

It has 'non-sporty' features, too, in that it isn't physically taxing, and (at least if you are an amateur) drinking beer can make you better at it.

So is darts a sport? 

Dunno. Don't care. There is no categorical answer because the concept 'sport' isn't precisely defined.

That doesn't mean there is no such thing as sport. Or that darts is any greater, or lesser, an activity, for being, or not being, a sport.

Is Phil Taylor a sportsman?

Dunno.

Is Phil Taylor amazing?

Yes.

Same with data journalism. Some things are data journalism. Some things are not data journalism. Some things are like darts: borderline. Are they data journalism? Are they not?

I'm not interested in that. I'm interested in whether they are amazing.

Having said all that, I am interested in the ways people try to over-define data journalism; to exclude things as not really data journalism at all.

I'm interested in this because I think it can teach us valuable lessons.

One the one hand, you have those people who say: "That isn't really data journalism."

We got this, for example, when we did our 'Pick your horse with data' gadget for the Grand National. This was, clearly, a bit of fun: a bit of fun aimed at people (like me) who don't just want to stick a pin in a list of names, but also don't want to spend hours and hours poring over every horse's performance in every race on every type of going etc etc.

It was a bit of fun.

It involved programming. And algorithms. And visualisation.

But it was definitely a bit of fun.

Was it 'journalism'?

Dunno.

Was it fun? Was it original? Did people love it?

Yes.

Then, at the other end, you have those who say: "This isn't really data journalism."

Often this is simply code for: it wasn't visibly complicated enough for me. 

Or: there weren't enough really difficult maths in it for me.

These seem odd complaints. It seems to me that if we want to connect with a mass audience then the difficult stuff should happen in the background; swans' frantic feet below the water, if you like, allowing for the serene progress above.

Also, I think people confuse 'statistics journalism' with 'data journalism'. 

I define 'data' the classical way: as information. There is nothing intrinsically mathematical about it.

For me, data journalism is essentially about applying advanced techniques for finding and interrogating information in the service of journalism in a digital age. 

That might involve scraping websites, knowing how to properly use spreadsheets, having programming skills, visualising data, and using and understanding data feeds.

Data journalism might involve all of these at once. Or more often some. Or sometimes none at all.

A final point. When Claire Miller and I set up the data unit, we defined two different work streams.

One was 'news', as classically understood. A perishable commodity with a 'line' or hook. We wanted to use data journalism techniques to create front-page news.

The other was 'resources'. By this we meant data journalism projects or tools which would be enduringly useful and which our readers could use to find and explore relevant information about their local schools, hospitals, crime rates or whatever.

I said at the start of this post I defined data journalism by practice.

For me, these splashes - based on some quite sophisticated statistical analysis by Claire of datasets that are not usually put together - were very much 'data journalism':

 

But then so was this one, based on a round-robin FOI that generated responses collated into a master spreadsheet that could then be sorted to generate a very simple but newsworthy line:

And so was this data scrape of sheet music by the data unit's Patrick Scott to find the modern singer with the greatest vocal range:

And so was Rob Grant's painstaking data-processing of more than a million records held by the Commonwealth War Graves Commission, which technical savvy allowed him to turn into such incredibly detailed (and moving) insights into local losses in World War One as this, for Liverpool:

All of these are 'data journalism'. And I'm proud of every one. 

Only I'm not proud of them because they are data journalism. I'm proud because they are bloody good.

Maps that tell a story: Is Labour finished in Scotland?

A really good map - like any really good data visualisation - tends to tell a story at a glance.

Sure, you might have to glance at a key to understand colours, or gradients, or whatever. But once you do, the best maps are often those where the weight, the import, hits you like a hammer.

So today, in the light of the Labour leadership contest, I decided to have a closer look at what went wrong for the party at the general election this year.

To do this, I simply looked at the vote share Labour achieved in each constituency in 2010, and the vote share it achieved in 2015.

Where Labour's vote share had gone up, I decided to colour the constituency red. Pale red for an increase of up to five percentage points. Mid-red for an increase of five to 10 percentage points. Bright red for an increase of more than 10 percentage points.

Where Labour's vote share had gone down, I decided to colour the constituency grey. Light grey for a loss of up to five percentage points, mid-grey for five to 10 percentage points, dark grey for more than 10 percentage points.

Here's the result (best viewed on desktop I'm afraid):

 

When I tweeted a static image of this map, I was struck by a correspondent who pointed out that it made Scotland look like a cancerous lung on a medical chart.

Certainly, from a Labour perspective, it shows that what happened in Scotland was not part of a general UK-wide pattern. What happened in Scotland was a uniquely Scottish trauma.

That's a problem for Labour. Normally, after taking a battering, you'd reflect on previous electoral history and assume you could find ways of righting the ship. But look at the map. The rest of the UK shows what a poor performance is supposed to look like: up in some places, down in others.

That sort of pattern gives you something to go on: clues about how to finesse your message.

Scotland, by contrast, was an utter wipeout. No one responded to anything Labour had to say. So what then?

Okay, so it's just one map. Just one way of looking at the figures. It certainly doesn't prove there is no way back for Labour in Scotland.

But it does suggest what happened was unique and decisive on a scale that political parties find hard to understand. Partly because they have never experienced it before. And partly, I daresay, because they'd rather live in denial.

Sketch for a memoir: five mildly embarrassing meetings with politicians

There's something about political journalism that reeks of seriousness. Of interviews in studies that smell of beeswax polish and are lined with leather-bound books. Of decisive nodding. Of important papers being removed from well-used briefcases. Of furrow-browed grey men whispering in corridors. Of throats being cleared and water being decanted into glasses. Of hyperactive Oxbridge Amelias and Ruperts shouting into their mobile phones: “Arrival in three. Arrival in three! Is the ordinary voter in position? Get the ordinary voter in position right now!”

Of entire stories, too, based on the thinnest sliver of nuance. A sliver so thin, sometimes, you could easily mistake it for absolutely nothing.

“Ah yes, Jeremy, but it isn't what he said – it's what he didn't say. Or more importantly, the tone in which he didn't say it.”

I loved my decade or so as a political journalist, actually. I also like to think I uncovered some decent exclusives, from time to time, in among the nonsense.

I broke the story about Greater Manchester's planned (and later aborted) congestion charge. I got the first interviews with a tearful Hazel Blears and a much-less-tearful James Purnell after they tried (and failed) to force Gordon Brown's resignation by quitting the cabinet.

Me, in a vintage shirt, about to make Hazel Blears cry. 

I don't know what's happening with my left hand

Through sheer force of whining, I got Tony Blair to write a pledge to fund the Metrolink tram expansion into the 2005 Labour manifesto.

You're welcome, tram fans.

But let us put the seriousness to one side. It is the nonsense I'd like to celebrate today. Because it is important for politicians and political hacks alike to remember just how much nonsense abounds.

So here are five embarrassing things I saw, did, and indeed had done to me as a political journalist.

There may well have been worse, but these are the ones that spring to mind. These are the ones I am ready to talk about, outside of the therapist's room.


1. THE TIME I MET JOHN PRESCOTT FOR THE FIRST TIME AND WITHIN SECONDS HE HAD PUT ME IN A WWE-STYLE HEADLOCK

This was in the very early days of my career as a political journalist.

Mr Prescott was launching some 'Let's Make The North The New London Through The Power Of Ideas'-type scheme.

You know the sort of scheme I mean: the sort politicians of all parties are far better at launching than delivering.

The launch was in Leeds. I was in Manchester. The M62 was completely rammed, as always.

I had been penciled in for an interview with Mr Prescott. I was late.

I dumped my car near the venue, ran the last 500 yards, found the Prescott entourage, announced I was David from the Manchester Evening News, and started a breathless apology.

I didn't get very far.

(Some important background. At the time, the MEN was running a fairly aggressive campaign to force the Labour government to reinstate some vital Metrolink funding. Mr Prescott, who had scrapped the grant, was very much in the paper's line of fire.)

No sooner had I said “Manchester Evening News” than the deputy prime minister had put an arm around my neck and pulled my head down to somewhere near his waistline.

He then rumpled my hair with his free hand and said: “Manchester Evening News, eh? Are you going to ask me about trams? Are you? Are you? Ha! You're obsessed with those trams!”

At the time, I hadn't done many interviews. I wondered whether in some strange way this was normal; some sort of weird fake bonhomie that was assumed to exist between political interviewer and political interviewee.

What has life taught me since? No, It wasn't normal. Not normal at all.


2. THE TIME ANN WIDDECOMBE ATE A HUGE PLATE OF SANDWICHES AND DIDN'T OFFER ME A SINGLE ONE, EVEN THOUGH I WAS OBVIOUSLY VERY HUNGRY AND IT WAS BASICALLY HER FAULT I HADN'T EATEN

I was working in Carlisle. Ms Widdecombe was making a visit to rally Cumbrian Conservative troops. She was running late when I met her along with the local Tory 'fixer'. Evening had become late evening, and we hadn't had time to eat.

When Ms Widdecombe finally arrived we went to a local pub. A round of sandwiches was ordered. It arrived on a large plate.

I was hungry. I made a rough calculation that there were plenty of sandwiches for the three of us.

“Oh good,” I thought.

I was just about to dive in when Ms Widdecombe pulled the plate towards her, expertly but unobtrusively shielding it with one arm, while picking at sandwiches with the other.

She then proceeded to wolf down every single one, without offering either me or the fixer so much as a crust.

To make it worse, I had to interview her as she ate. I believe it was the most sulky, passive-aggressive interview of my career.

What I said was: “Do you think the local Tory mood has been boosted by news of a two-point blah blah blah?”

What I wanted to say was: “Here's a question for you, Ann Widdecombe: why don't you give me just one sandwich when you have so very many sandwiches?”


3 THE TIME I MET GORDON BROWN FOR THE FIRST TIME AND WE SAT IN UNBEARABLY AWKWARD SILENCE FOR 10 MINUTES BEFORE HE EVEN SAID HELLO

I was pretty excited the first time I interviewed Gordon Brown. A Labour party press officer had managed to get me 'car time' with the then-chancellor.

This was a good thing. Mr Brown famously didn't do lengthy interviews, least of all with the regional press.

On the day in question he was visiting a factory. I was to join him in his car for the drive there. I would be able to interview him during the journey, which was scheduled to take about 20 minutes.

I drove to the pick up point. I got out of my own car. I walked towards Mr Brown's car.

The press officer emerged from the back seat, grinning, and gave me a thumbs-up.

I got in the seat the press officer had just vacated.

I looked over. There was Mr Brown, flicking through some briefing papers.

I decided a cheery greeting was in order.

“Hello,” I said brightly, “I'm Da----”

The chancellor raised a finger in my direction and almost imperceptibly shook his head. I trailed off.

“Okay,” I thought. “He's just going to finish reading the paragraph.”

Mr Brown continued reading to the end of the paragraph.

He continued reading to the end of the page.

He continued reading to the end of the next page.

I looked more closely at his pile of papers.

It looked suspiciously thick.

Mr Brown read another page.

And another.

And another.

And another.

I waited just under ten minutes in total. I know this, because I surreptitiously timed it. I had absolutely nothing else to do, and no one else to talk to.

(I mean, I guess I could have tapped the driver on the shoulder and said, “Excuse me, is this normal for him?” But it didn't seem like a good idea.)

Ten minutes might not sound like a long time to you. In those exact circumstances, it felt like an age.

Occasionally I wondered if I shouldn't, in fact, be saying something.

Was the finger thing really a 'shushing' gesture? Had I been mistaken? Was I just wasting my own time?

Was Mr Brown sitting there thinking: “This is odd. I've given this joker car time. And he's not asking me anything at all!”

Eventually the chancellor finished. He carefully shuffled the pages together, and put them down by his side. He rubbed his eyes. He made a noise that sounded like a sigh.

Then he suddenly turned to me.

“Oh hi there!” he said brightly - exactly as if the previous ten minutes had simply never happened. "How are you?"

What I said was: “Oh hi! I'm fine thanks! I'm David from the Manchester Evening News.”

What I wanted to say was: “I'm really quite freaked out right now. You have freaked me out, Gordon Brown, and I need to open a window and take a few deep breaths.”


4. THE TIME I MADE A JOKE ABOUT SCREENWASH THAT DIDN'T GO DOWN VERY WELL

I was riding in the passenger seat of Tony Blair's car. The prime minister himself wasn't in it. He was waiting somewhere else.

I had been put in his car to get me to the interview location on time.

We had a police escort. Motorcycle outriders. Blue lights. Traffic parting, melting away, as we sailed down Princess Parkway.

It was a lot of fun.

In truth – in my defence - I was probably a bit giddy with it all.

I didn't know anyone in the car.

I assumed the driver was some sort of special forces bodyguard type.

There were also a couple of members of the prime minister's entourage in the back, but they were chattering away into their mobile phones.

I became aware that I hadn't spoken a word since I got in the car. That seemed wrong. With the blue lights and the scattering traffic, it felt like we were all in this together. Part of something important.

I felt I should say something. But what?

Then I noticed that the driver had been obsessively squirting screenwash throughout the entire journey. Not once or twice, but time and time again.

The outlines of an ice-breaking joke began to form in my mind.

(At this point, I should point out that I am terrible at jokes. Terrible at constructing jokes. Terrible at telling jokes. Terrible at knowing when not to tell jokes, too.)

I waiting until the next squirt. Then I went for it.

“I hope," I said, "that isn't taxpayers' screenwash!”

Silence. Complete, agonising, terrible silence. Then a tap on the shoulder.

“David? It is David, isn't it?"

I nodded.

"Please don't speak to the driver again.”

It seems we weren't all in it together after all.


5 THE TIME I WAS CHATTING WITH LIAM FOX IN A CHINESE RESTAURANT WHEN I HAD TERRIBLE CRAMP AND ENDED UP WRITHING AROUND ON THE FLOOR

Liam Fox was in town to have a Chinese meal with Manchester-based journalists, including me. I had not met him before.

I had been playing football earlier that evening.

Occasionally, after football, I get terrible cramp in my upper legs.

I do mean 'terrible'. The cramp I get is the kind where I simply have to roll around on the floor, leg ramrod straight, while crying out in pain.

This usually continues until someone is kind enough to grab my foot and push the toes up towards my head.

I'm sure you can figure out the details of this story for yourself.

Suffice to say, Mr Fox looked pretty horrified and shuffled off in the other direction.

I remember ruefully recounting this story to a friend a couple of days later.

"I couldn't believe it," I said. "He is a doctor. Surely he's seen cramp before?"

"Did you tell him it was cramp?" asked the friend.

"No," I admitted.

"Well then," she said. "To be fair, he might not have know it was cramp. He probably thought you were a terrorist."

Quite.