The league of love and hate: a virtuous data circle

I love the idea of generating useful and interesting content purely from data supplied by readers.

It's the ultimate virtuous circle. Readers provide content. That content generates interest. That interest stimulates more readers to provide content.

It's what makes TripAdvisor so successful. It isn't like anyone's getting cheated; people like giving their opinions, just as they like reading the opinions of others.

And the more people give their opinion, the more valuable the dataset becomes.

I remember when we first set up systems to collect fans' player ratings for football matches. A couple of people would object along the lines of: "Is this really worth it if sometimes on a couple of hundred people are actually filling in the ratings forms?"

The answer is yes. Because how many people submit ratings is far less significant than how many people engage with those ratings. Far more people want to read ratings than want to supply them.

Plus, once you've started collecting the data, there are all manner of other types of content you can create, questions you can answer.

Which players only play well at home? Which players only play well against 'easy' teams? Which players only play well in the warmer months?

One of my favourite user-generated data projects is the League of Love and Hate. It's a glorified survey we run every year in the off-season. Essentially we ask people to tell us which Premier League team they support (if any), then tell us what they think of other teams in the league.

The overall result - most liked and disliked - is done on a 'Eurovision-style' voting system. The views of fans of all clubs carry the same weight, no matter how many of them actually choose to vote.

This year, as things stand, it isn't looking great for Manchester United:

Ah well.

We can also tell fans who complete the survey what fans of other clubs think about their team.

In the case of Liverpool, for example, Everton fans dislike their neighbours the most. Leicester fans, for whatever reason, seem to have a soft spot for Liverpool:

Finally we can also tell Liverpool fans what their fellow Reds supporters think. Surprisingly, perhaps, they hate not just Manchester United but also Chelsea more than Everton:

The League of Love and Hate was only ever intended as a bit of off-season fun. What's interesting, though, is the amount of coverage this rather light-hearted poll has gathered in the past. Even our media rivals pounced on the results in 2013, which showed Chelsea (rather than Manchester United) at the bottom of the popularity pile. In terms of Trinity Mirror titles, headlines like this were some of the best read that month.

All of which goes to show: harnessed correctly, and with the right caveats, there's something enduringly fascinating about the wisdom of the crowd.

When did Labour stop caring about winning?

"A meme entered the Labour mind that identified its time in office with the Iraq war and overindulgence of the rich. The progressive achievements of 13 years in power – the minimum wage, equality legislation, record spending on public services, the greatest amount of redistribution from rich to poor of any period of Labour government – all that was gradually effaced." 

So wrote Andrew Rawnsley in his column this weekend.

I don't know exactly when it was that I started to find the collective voice of the modern Labour party ever-so-slightly annoying. When it began to sound a bit less like a kind, open, optimistic friend, and a bit more like a hectoring A-level sociology student at the dinner table.

When it seemed to have became more important to win arguments than elections; to have the smug satisfaction of being 'right' rather than helping millions of people live slightly better, fairer, healthier lives.

Who is this collective voice? Honestly I don't know; but I don't recognise the voice of the Labour councillors, party members and MPs that I dealt with during my years at a political editor. I hear a younger, angrier, less nuanced voice. I hear a voice less rooted in the struggles of everyday politics. I hear a voice that may take defeat very personally, but is metropolitan, reasonably well off, and doesn't have to personally live with the consequences of that defeat.

It is a voice that sees the world in black and white; where it isn't enough for the Tories to be wrong. They have to be evil, too.

And so, it seems, does Tony Blair.

Because nothing is so utterly perplexing to me than the attitude of this current Labour voice to the last man to actually win them an election.

Now I understand what happened in Iraq. Honestly, I do. And I understand why many people will find what the government did over that issue utterly unforgivable.

But surely, apart from that, we should still be able to ask the question: what else happened during the Blair years? And was it good or bad for the country, from a Labour-supporter's point of view?

Surely we should be able to hold those two things in our heads at the same time? Surely we should be capable of allowing for the possibility that the world is a morally complex place, where one person can do some bad things, and some good things, and the scales simply hang in the balance?

Because when I think of the Blair years I also think of the very large sums of money that were ploughed into our public services. I think of the thousands of schools that were completely rebuilt; the fact investment in the NHS rose to something like what it should have been for a Western European country.

I think of the many lives that were undoubtedly saved as a result. Lives of family members, friends.

When I think of the Blair years I think of the minimum wage. I think of tax credits. I think of SureStart centres.

I think of the Human Rights Act, the abolition of hereditary peers, the Freedom of Information Act. I think of devolution.

I think of the completion of a sustainable peace in Northern Ireland.

I remember discussions with Conservative friends and contacts who were genuinely worried that their party had been pushed to the margins and rendered obsolete.

None of the things are minor details.

None of them can be brushed out of moral existence with an impatient snort, a wave of the hand, and an angry: "Yes but Iraq!"

They did happen. They did change the political discourse.

They did make a huge difference to people's lives.

And I think that should matter.

I'm not arguing for a second that the Blair years, domestically, were above reproach. That would be ridiculous. You can question the levels of spending once the 'golden rule' was ditched. You can question the efficacy of much of the spending: PFI, for example, or the love of expensive bureaucracy, watchdogs and scrutineers.

And you certainly question the fact that our economy was driven to a dangerously large extent by flyaway house prices stoked by dodgy lending, and an under-regulated financial sector that no one wanted to ask too many searching questions about.

The fact remains: from a Labour perspective there was an enormous amount of money spent, with the majority targeted at the poor. There was a real push to improve education and work towards a meritocracy.

I think that should matter.

If you want to do these things, you need to win elections. If you want to win, you need to take people's problems seriously. Not just the very poorest, but other people, too.

People who are working hard and trying to support a family on the median wage, for example.

People who vote. People who decide elections.

You can't just tell those people that if they don't care about the 'bedroom tax' or food banks then they are plain selfish.

They have concerns of their own, legitimate concerns that you need to understand and address.

You need to win their trust, and build a coalition which gives you the right to change the world.

You need to listen, not lecture.

You need to admit you might be wrong, or at the very least consider this: isn't it possible that rightness is measured, ultimately, not by the number of like-minded keyboard warriors who share and endorse your views on social media, but rather by the number of lives that you actually get to improve?

Mapped: where are people most likely to take anti-depressants?

This is a map I made today as part of some analysis of a new parliamentary research paper looking at different prescription rates around the country in 2014.

I chose to focus on anti-depressants, and specifically look at those areas where prescribing rates were significantly higher than the national average.

Prescribing rates are expressed as prescription items (the thing you get when you hand in your prescription - a box or course of anti-depressants, say) per 1,000 patients registered with a GP.

The average prescription rate for anti-depressants is about 1,000 per 1,000 across the year. Or, to put it another way, one boxful of anti-depressants for every many, woman and child in the country.

Certain places, though, have much higher rates. Here's the map; figures are only available for England, so ignore Wales and Scotland. Grey places have above-average prescription rates for anti-depressants, The darker the grey, the higher the rate. You can click on an area for precise figures; the number is the percentage by which anti-depressant prescriptions are higher than the national average:

 

Straight away, you can see some sort of visual pattern. There is definitely more grey in the north than the south. There is more grey on the coast than inland. There is no grey at all in or around London.

The areas with the highest rates of anti-depressant prescriptions are in County Durham, Blackpool, the Lincolnshire coast, and Norfolk. There are also clusters in Cornwall, Merseyside, Cumbria and the Yorkshire coast.

Now I haven't done detailed correlation work, and I am not in a position to draw any firm conclusions on the reasons for these patterns.

You could argue it has something to do with poverty; something to do with unemployment. I'm pretty sure the darker grey areas will have above-average deprivation. I'm less sure the correlation will be particularly precise. There is an awful lot of deprivation in London, for example, and in Manchester. Yet these areas are not top of the 'depression' list. London, in fact, is right at the bottom.

At the very least, therefore, there must be other factors in play. One might be age: some areas are younger than others, and younger people might be less likely to be on anti-depressants. I don't know; it's certainly possible.

Another factor might be GP access. Being registered with a GP doesn't necessarily make it is easy to see a GP. A lower level of anti-depressant prescriptions might not mean a lower level of depression; it might just mean people who need tablets aren't getting them.

Other demographic factors might be relevant, too. It's notable to me that the most culturally and ethnically mixed communities in England seem to have relatively low rates of anti-depressant prescriptions. I'm thinking of London and Birmingham, in particular. Could it be that white Britons are more likely to get depressed? Or rather, more likely to seek and obtain medication for it?

The most interesting theory put to me on Twitter today was none of these. Rather, a correspondent  pointed out that all the areas with the most grey seemed to be the most physically isolated. And there does seem something in that. I don't know how good the public transport links are in County Durham, Cornwall, Norfolk or West Cumbria. But I wouldn't be surprised to hear they weren't great.

I wonder if as sociable creatures it isn't important to our well-being to feel plugged in to a wider community. I also wonder if we don't need to feel free, not trapped, and moving forward, not standing still.

The original data was lodged in the House of Commons library as a research paper this week. I'd be interested to hear if anyone come up with some precise correlations, or even just wacky theories.

Ideology, values and politics

You could ask it as a GCSE maths question, probably.

Candidate A will make life 20 per cent better and fairer than now. There is a 20 per cent chance he will be elected as Prime Minister by the British people.

Candidate B will make life 80 per cent better and fairer than now. There is a 1 per cent chance she will be elected as Prime Minister by the British people.

Candidate C will make life 10 per cent better and fairer than now. There is a 60 per cent chance she will be elected as Prime Minister by the British people.

Let's say A, B, and C are standing for the leadership of a political party.

Which one should you support, if you want the best and fairest outcome for the country?

I mean if you really, really care?

I mean you're not just grandstanding, or point-scoring in some abstract political debate.

I mean you actually want to make a difference to the lives of other people.

Which one should you choose?

And if your answer isn't C, then why?

Yes, I do call this news

"You call this news?"

Anyone who has ever worked on, or for, or even just near a news website will recognise this.

It is The Great Comment. It is Comment Number One. It is the comment equivalent of the Ford Focus: it crops up everywhere, all the time, so much so that you almost stop noticing it.

But it is there.

You might think it looks pretty harmless. You would be wrong.

You should know, as every journalist knows, it has been written by someone frothing with rage and indignation. So much rage and indignation that if you asked him to italicize the sentence, he wouldn't know where to start. He couldn't choose between

"You call this news?"

and 

"You call this news?"

or maybe even

"You call this news?"

because what he really wants is to italicise every word in the sentence; to say it with such savage irony that any journalist who read it must surely burst into tears and immediately resolve to mend their ways.

Well I'm sorry, but no.

For one thing, the comments are almost always left on stories which patently are news, of a particular sort. It might, for example, be entertainment news - Justin Bieber visiting a nightclub in Birmingham while wearing a funny hat. But how is that 'not news'? If you actually saw Justin Bieber in a nightclub in Birmingham, wearing a funny hat, I daresay it would be all you'd talk about for the next few days. 

It might not be Watergate. That doesn't mean it isn't news. 

For another thing, who ever said all the content on a newspaper website had to be news? Newspapers themselves contain TV listings, cartoons, weather forecasts, sudoku, crosswords, horoscopes and opinion columns. They have TV reviews. They have leader articles. Call any of that news? Nope. So what? It's just interesting content that complements news.

My team, for example, does data journalism. And the scope for genuinely useful, interactive data-based content is huge. Only it isn't necessarily news in the conventional sense. To take just a couple of simple examples: this gadget tracks car parking spaces in Manchester using live data. And this one lets you find out how good your local GP surgeries are.

News? Maybe not. But it is using technology to tell people something useful about their local communities. And surely that is something newspaper sites should be doing.

Actually, though, what annoys me most about 'call this news?' comments isn't either of these things. It is the implied moral criticism. It is the implication that newspapers should only cover 'worthy' news.

Now I don't put 'worthy' in quotation marks to sneer: great investigative journalism that holds power to account still exists, and is still a wonderful thing. I put it in quotation marks because elsewhere in the news landscape, it is less clear to me what is 'worthy' and what is not.

Is covering a court case involving a murder more or less 'worthy' than writing a story about someone doing a fun run to raise money for charity? How much more 'worthy' is a report of a council sub-committee rubbing stamping a decision without dissent, which perhaps a couple of dozen people read, compared to a video of a swearing parrot that makes tens of thousands of people laugh?

The former is somehow more worthy, no doubt. But how much? And, bluntly: how much should newspapers care about these more marginal cases, if the difference in potential traffic is so great?

For the implication of the 'call this news?' brigade seems to be that newspaper organisations - commercial newspaper organisations, whose very existence depends on getting eyes on pages and copies on doormats - should put more stock in 'worthiness' than anything else.

Why? We never did before. Newspapers were always about things that interested the public as well as things that were truly in the public interest. We are just better now at measuring what that is.

And maybe Mr 'Call This News?' doesn't like that. Maybe he doesn't like a mirror being held up to society, only for it to show that people aren't all clamouring for detailed line-by-line scrutiny of party manifestos.

But that isn't my fault. It isn't the fault of the newspapers. And it isn't the newspapers' job to fix it, if it really needs fixing.

I think people should take a rather different view. I think people should be amazed, as I am, by the lengths to which newspapers have gone to adapt their coverage of politics, civic affairs, health, education, and other 'worthy' topics in order to keep people reading, and caring.

You see more sharp analysis than before, more punchy exclusives, more campaigning journalism, fewer meandering opinion pieces and fewer verbatim reports of meetings. 

And you know what? That kind of journalism is harder, not easier. Anyone can sit through a meeting all day, then write 300 words about it. Getting exclusives people want to read takes contacts, judgement, research, work. It is fulfilling a moral duty journalists and editors impose on themselves; because you could quite easily fill a website with swearing parrots and probably do pretty well for yourself in traffic terms.

One day - one wonderful day - I will open one of these investigative stories and the first comment will say: "Thank you - I call this news!" One day. But not yet.

On certainty

In a previous life, I used to write a newspaper column. And here's the thing: I used to hate it.

I used to hate it not because I hate newspapers columns per se. There is nothing wrong with a well-expressed opinion, a well-crafted argument, a piece that makes you think about the world a different way.

There is nothing wrong with a column that tells you something you didn't know. There is nothing wrong with a column that makes you happy, or sad, or angry, or whatever.

Neither did I hate it just because I was particularly bad at it.

No; what I hated about writing a column - and what I often dislike about columns I read - is the presumption of moral certainty.

For it isn't enough that columnists write as if they are correct on all relevant ethical or political points. Oh, no; they also have to write as if they quite obviously correct, and that only a particularly stupid person would fail to see it. It is why so many columns are written in that annoying, smug, semi-exasperated tone of a parent explaining to a small child why eating too much chocolate is bad for you, or why you shouldn't poke your fingers into a plug socket.

And yet, this is all wildly implausible. When it comes to ethical and political judgments, I daresay none of us get things 100 per cent right, 100 per cent of the time. It's like that weird fiction always and for ever accepted in party politics: that in any and every argument, our guy is absolutely on the side of right, and their guy is absolutely on the side of wrong. Because our guy is a saint and their guy is evil. (This came up recently during the Budget when some commentators, talking about the  Conservatives' decision to mandate a living wage, pointed out it was a 'curveball' that Labour would find 'very difficult' to respond to. But why? Why couldn't Labour, and Labour supporters, just say: "Yep. This bit - paying people more money - is good.")

A lot of ethical issues are really quite complicated. That's why - unlike the sum of the internal angles of a triangle or the best technique for the high jump - we are still disputing them, passionately and with no clear resolution. Like abortion, say. Or euthanasia. Or animal rights. Or the limits (assuming there are at least some limits) on freedom of speech.

I have opinions on all those things. But they are complex opinions that don't really lend themselves to the traditional column structure. No one wants to read a column along the lines of "Euthanasia? Dunno. It's complicated." And yet maybe they should. It's probably a much better way of approaching such issues - with an open mind, a readiness to think hard and change your mind, and a sense of humility about what kind of conclusion you are after. The world is a complex place full of complex issues. Like it or not, they can't all be solved conclusively on deadline, in exactly 500 words.

On writing

Ask a journalist why they decided to become a journalist.

Honest answers might include:

I'm really nosy

or

I'm a rampant narcissist who needs to feel all world events somehow inextricably involve me

or

I fear structure in my life (because of my father/mother/Catholic upbringing/unexpected death of a much-loved pet dog at an early age) 

or

I don't know; it just seemed cool before, you know, the internet and that.

Journalists never actually say any of those things. Instead, backed into a corner, they tend to say something like:

 I just wanted to write.

Now this is a terrible reason to become a journalist. The kind of writing people have in mind when they say 'I just wanted to write' is absolutely not the kind of writing that 90 per cent of journalists do 90 per cent of the time.

When people say 'I just wanted to write...', that sentence implicitly continues '...about the meaning of life'. Or '...about love and loss'. 

No one ever says, or means:

I just wanted to write about chip-pan fires being tackled using positive pressure ventilation fans

Or

I just wanted to write about a procession of crackheads appearing in magistrates' court charged with a string of depressing low-level crimes.

Journalism isn't really about writing as expression. Nor should it be, not in essence. It is about writing as communication. 

It is about finding out what has happened, and telling people as clearly, efficiently and fairly as possible what has happened. 

This isn't glamorous, but it is hugely important; much more important than most people realise. 

It is what the legendary Guardian editor CP Scott had in mind when he said, "Facts are sacred, but comment is really annoying."**

What it isn't, though, is romantic. It doesn't fire the soul.

So here's the thing. I became a journalist because I'm a nosy narcissist who fears structure and wanted to be cool. And yet part of me still just wants to write.

Which is partly why I've started this blog. I might write about data journalism, or politics, or innovation, or anything that I feel I want to say. I honestly don't know.

I just feel I want a space to write.

(** This may be a slight misquote.)

Preludes IV

His soul stretched tight across the skies
That fade behind a city block,
Or trampled by insistent feet
At four and five and six o’clock;
And short square fingers stuffing pipes,
And evening newspapers, and eyes
Assured of certain certainties,
The conscience of a blackened street
Impatient to assume the world.

 I am moved by fancies that are curled
Around these images, and cling:
The notion of some infinitely gentle
Infinitely suffering thing.

 Wipe your hand across your mouth, and laugh;
The worlds revolve like ancient women
Gathering fuel in vacant lots.