Speaking truth with numbers

siteadmin — Tue, 25 Jul 2017 05:27:13 +0000

Data visualization has made journalism more objective

“A picture is worth a thousand words” is a well-repeated cliché. But it is not any more in journalism, where the increasing use of data visualization to tell stories has revolutionized the field, making journalism more objective, more interpretative and bringing authenticity to storytelling.

The workflow in data journalism has three separate processes. The first is data sourcing and preparation. This can be done in various ways — either through direct sourcing from public documents, or from surveys or indirectly through methods such as “web scraping” and creation of data sets from digital resources. Web scraping requires a lot of refining and cleaning up of data from various sources like PDF documents, HTML pages and text files. There are several free tools available for this job. At an advanced level, a working knowledge of the python programming language and various libraries which aid in HTML scraping is useful.

Also read this: Essential tips and tools for beginning data journalists

Analyzing data for journalistic purposes does not require one to be a trained statistician but one needs to be at least familiar with simple statistical concepts.

Some document caches from which data is to be created are so large that it is difficult to parse or prepare useful tables out of them without the help of a much larger team than what newspapers typically have. Simon Rogers (who was formerly The Guardian’s data editor), in his book on data journalism, Facts are Sacred writes how The Guardian used techniques such as crowdsourcing to obtain big data used to come up with stories, like the MP expenses scandal in the United Kingdom. Mr. Rogers rightly points out in his book that data journalism is “80% perspiration, 10% great idea and 10% output” — a statement that rings true and puts emphasis on the first process of data preparation.

Also read this: Not Numbers, but Numbers Which Matter

The next step in data journalism is analyzing the data and looking for patterns, rules, exceptions, in order to tell a coherent story. For non-coders — most data journalists come under this category — this typically involves a lot of work with spreadsheets, pivoting tables, simple statistical analyses and so on. Analyzing data for journalistic purposes does not require one to be a trained statistician but one needs to be at least familiar with simple statistical concepts (for example, correlation does not amount to causation). If one requires a crash course in basic econometrics, D.N. Gujarati’s book (of the same name) is a good place to start.

The third part in data journalism is data visualization, the most exciting feature that has galvanized the digital medium in particular. The journalist needs an intuitive feel of how to present a data graphic that explains the story in an effective manner. Various software tools — like fusion tables, chart wrappers and the D3 (dynamic document design) javascript library — are freely available but to get a familiarity with graphic design, statistician and political scientist Edward Tufte’s books The Visual Display of Quantitative Information and Envisioning Information are very useful guides.

This post was originally published on The Hindu and is reproduced here with permission.

Main Image: FoxBusiness

The post Speaking truth with numbers appeared first on Data Stories.

Not Numbers, but Numbers Which Matter

siteadmin — Fri, 21 Jul 2017 06:59:34 +0000

By Zahid Asghar

We are living in a world where data are ubiquitous. It is estimated that by 2020 there will be 1.75GB data generated for each individual living in this world in every minute. Companies/organizations which have explored the potential of data are ruling the world. Facebook, Google, Amazon, IBM etc have huge market share as these have been very successful in utilizing data at its best.

Facebook’s alone market value is over 450 billion US dollars, more than Pakistan’s total GDP. Data is new oil but data pose serious challenges particularly in developing countries where data literacy is not very high. We are misled by numbers as we try to link everything to numbers. We ignore its not numbers but numbers which matter.

Getting data insight by simply using statistics without deeply exploring the phenomenon is normally misleading. This holds true from government to the game of cricket.

Pakistan has won ICC Champion Trophy 2017 and has beaten comprehensively both England and India in the semi-final and final matches, respectively. Hasan Ali’s 3 wickets in semi-final and any other bowler’s 3 wickets or Hasan Ali 3-wickets in some other match are equal. So, result one will draw from this statistics that both bowlers or same bowler performance in both matches is the same. But this is not true when one explores the issue in detail. Hasan Ali’s 3-wickets in semi-final includes all in-form top English batsmen while the other bowler might have 3-wickets of tail-enders only. Same is true for final match when Amir gets 3 top Indian Batsman out and gets 3-wickets in his record. Those who have watched the match will not think Amir performance in numbers but in numbers which matter. This list goes on. A century in home ground against a weak team is counted the same as century away from home country against a strong team.

When we try to measure or link everything to numbers, data becomes a double edge sword. On one hand numbers are highly necessary to get insight about the issue but at the same time, we have incentives to increase the number without taking hard path. Government increases its tax-revenue by simply increasing tax-rate on a sector which is already documented and easy to cap. For example, banking transactions, cell phones and other such services are easy to tax and government can show increase in tax collection without bringing undocumented economy into the tax-net. This will show an increase in tax-revenue without taking hard reforms and no effort will be made to bring new sectors into tax-net.

Same is the case when we measure progress in research in terms of research papers produced in the universities. HEC is showing its performance by telling policy makers that research papers produced overtime have increased manifold.

In order to get more funding and to improve ranking, universities are putting pressure on their faculty to increase research papers even if it is at the cost of quality of teaching.

Since ranking, recognition of individuals and different awards are linked with simple numbers, therefore, many faculty members are trying to publish in journals which have high impact factor by paying hefty amount as submission fees in dollars. They get personal promotions and other benefits at the cost of producing quality graduates. A research paper on exploring national issue of great importance may remain a working paper but has great significance while an impact factor paper in a journal based on technical analysis (many a times after paying around $1000) is a paper for the sake of publication.

Former will be given zero weightage, while latter will get all the benefits in terms of promotion and other monetary gains.

Getting data insight requires the task to explore the issue in detail instead of simply looking at numbers. For this we must enhance data literacy at all levels of the society among journalists, academia and bureaucracy. Without evidence based policy analysis, we may misallocate our limited resources and it will be difficult to have development where no one is left behind.

Zahid Asghar is a faculty member at Quaid-i-Azam University (QAU), Islamabad, Pakistan.

Main image: TalktheTalk

The post Not Numbers, but Numbers Which Matter appeared first on Data Stories.

reportingwithnumbers Archives - Data Stories

Speaking truth with numbers

Data visualization has made journalism more objective

Not Numbers, but Numbers Which Matter

Zahid Asghar is a faculty member at Quaid-i-Azam University (QAU), Islamabad, Pakistan.