Stephen Few left a comment in my post “Is Data Visualization Useful? You’ll Have to Prove it“. We all have much to learn with Steve, so instead of leaving the discussion buried in an old post, I thought it would be interesting to make it more visible. Please read the comment then come here and join the discussion. Here is my answer.
Steve, sorry if I sound provocative, that’s not my intention. You are the leading expert in data visualization for business, you are doing a remarkable work with your books, with your blog, with your forum, with your patience to answer posts like mine. I have to be thankful for that. And I do agree with 95% of what you write. But you don’t want to be surrounded by people who fully agree with you, do you?
The Effectiveness of Data Visualization
You say “the effectiveness of data visualization is well established by a large body of empirical evidence”. I want to believe that too. However in this study Jarvenpaa writes:
“Graphical charts are generally thought to be a superior reporting technique compared to more traditional tabular representations in organizational decision making. The experimental literature, however, demonstrates only partial support for this hypothesis.”
And J.-A. Mayer adds:
“This study refutes the general superiority of visual information in improving the decision quality (‘naive superiority hypothesis’). The choice and design of visual presentation is determined by information structure, decision environment, the decision-maker and the task decision. (…) The successful use of visual information depends substantially on its acceptance by the manager and the environment.”
What do these authors tell us? First, we cannot be 100% sure about the effectiveness of data visualization. Second, there are many other variables at play. And third, managers must accept it. This is a critical factor. Managers love impression management, and making a good impression using the dreaded “professional-looking charts” is the path of least resistance.
Data Visualization Success Stories
I have no doubts that you could share with us many success stories. When I write about an “admission of impotence” I am not questioning your ability to create/lead/mentor successful data visualization projects. But if you want to use those projects to inspire the average person I think you’ll fail most of the time, unfortunately.
Let me tell you how the layman looks like in my part of the world. He makes charts like this:
He believes that a 3D pie chart “looks more precise” and he doesn’t know that Excel chart defaults can be changed (more advanced laymen are able to switch to more “impactful” colors like reds, yellows and bright greens). In my part of the world, a layman doesn’t even know what “data visualization” is about (and they don’t even care). (Here are some more profiles.)
If you are preaching to the choir your conversion rate may be high. But the layman is not easily impressed. You must convert one at a time, and that’s something many of us can’t afford. Can you? He’ll keep making those pie charts because that’s what his manager requires him to do, he doesn’t know better, he’s lazy or you fail to convince him of a causality effect between better charts and better results.
The Layman Must Like Your Charts
In a business environment, charts don’t have to be memorable, only results do. But if you want to change behaviors, your audience must like the new behavior and accept the unavoidable pain. Likable charts help conversion.
You say “I do not discount people’s emotions”. I don’t see it, I’m sorry. The way I see it, you sacrifice everything to the altar of “chart effectiveness”. I don’t find a single one of your charts where the use of color is not purely functional. You say “you should support your claim with concrete examples”. I do have lots of examples: all your charts!
Let me reemphasized this: I agree with you. Chart effectiveness is what we should aim at. But I’m part of the choir. I’m not the layman. I don’t use pie charts.
Pie Charts Again
Unlike most people, I don’t think pie chart addiction is a disease. It is a symptom of a much more serious problem: low numeracy and poor data management skills. Address this problem and pie charts will virtually disappear.
How do you address this problem? “I don’t use pie charts, and I strongly recommend that you abandon them as well.” Researchers like Ian Spence and Stephen Kosslyn don’t think pie charts are as bad as you paint them. Even if they are, it’s very hard to talk people out of an addiction with purely rational arguments.
Perhaps this is my European soul speaking, but I do prefer a gradual approach (“this is acceptable, for the time being”) whereby people (hopefully) start to develop a sensibility to the perceptual issues.
By the way, how come we keep telling people that charts are about trends and patterns, not about the precise figures and then we argue that pie charts are bad because we can’t tell the difference between a 13% slice and a 14% slice? It doesn’t make sense (I’m exaggerating).
We must find more compelling arguments. I don’t like pie charts just because they are a waste of space (low data density) and can only answer very basic questions, better answered using a table. These arguments are good enough for me. I don’t care if we humans are bad at calculating areas and angles. That’s an academic argument that is irrelevant in the real world (I’m being provocative now…).
To Sum Up
You have a very consistent approach to data visualization and you practice what you preach. You believe that you can convince people using rational arguments.
Mine is a much more comfortable place. I know that eye-candy is a can of worms that shouldn’t be opened. I know that we should protect the layman from himself. I know that simple rules with no exceptions work better than complex rules no one bothers to learn or understand.
But I like the gray areas. I like to protect the poor and the oppressed pies and I try to find their small role in the world of data visualization. The same with eye-candy. The same with emotions. The right amount can get your foot in the door. What is “the right amount”? I don’t know. I’m still searching.
The empirical evidence that Stephen Few wants to cite goes back to Bill Cleveland and others pointing out that am individual’s precision of perception of a pie charts less than that of other charts. Based on the primacy of precision, Stephen is wandering out through the wilderness of corporate visualization preaching the gospel that pie charts are evil. But like our English grammar teacher who did not like passive voice sentences and would not want us having sentences end with a preposition, sometimes sentence variety is desired. It is not something we can do without.
Like a diet without dessert, there are times when my colleagues cajole me into doing comparative pie charts. They know why I don’t like them, but as part of a broader story the pie charts (2d, no drop shadow, thematically coded, …) are something they need. Hopefully I can get a concession out of them (like to remind them that not every row in a table needs a line in it) and move towards a happy and more readable medium.
In a classroom or as the owner of a business, you have have hard and fast rules–which many of us agree with 100%. As a practitioner who has to work with colleagues (and bosses) with a more classical perspective, some of us pick our battles and hope that habituation will move the organization along to a better place than where we started.
Chris: There are many studies that test Cleveland’s findings. I would recommend Simkin & Hastie’s ‘An Information-Processing Analysis of Graph Perception’, and Hollands & Spence’s ‘The discrimination of graphical elements’. These authors claim that precision is not generic, but task-dependent (pie charts are more precise than bar charts when judging proportions but not in a comparison judgement).
Speaking of Cleveland, he writes in The Elements of Graphing Data: “Many people will find colors in Plates 1 and 2 unesthetic, garish, and clashing. This was done on purpose to maximize the visual discrimination. Pleasing colors that blend well tend not to provide as good visual discrimination.” I don’t agree with him, but I like this kinda anti-Tufte quote 🙂
Jorge,
The thought that some of your readers might assume that they understand my position based on your comments alone is rather unpleasant. Those who know my work know better. You represent my position in simplistic ways that gloss over vital truths. One example is your statement: “I know that simple rules with no exceptions work better than complex rules no one bothers to learn or understand.” Although most of what I teach is quite simple to understand, I don’t teach simple rules with no exceptions. Rather, I help people understand effective graph design at the conceptual level (how things work and why they work as they do), which allows them to apply their understanding flexibly, which includes bending the rules and even breaking them when doing so will produce a better result. People learn by tackling simple concepts in the beginning and then by building their understanding to greater and greater levels of complexity over time. Although I give most of my attention to beginners, and therefore focus on simple concepts most often, when I work with those who are more advanced, I teach greater levels of complexity.
My opinion of the “layman” appears to be different from yours. Most of the people who attend the many courses that I teach each year are novices in data visualization—not already members of “the choir”—and they hunger to learn better ways of making sense of and communicating data. They respond quite well to rational explanations and examples of what works, what doesn’t, and why. They are grateful for the help. I teach hundreds of people, face to face, each year, and work with many more one-on-one as an adviser, helping them apply the skills that I teach to real problems in the workplace. I have a fairly good understanding of the “layman”.
In response to my claim that “I do not discount emotions,” you wrote:
“I don’t see it, I’m sorry. The way I see it, you sacrifice everything to the altar of ‘chart effectiveness’. I don’t find a single one of your charts where the use of color is not purely functional. You say ‘you should support your claim with concrete examples’. I do have lots of examples: all your charts!”
Is this really your idea of an adequate response? The colors that I use are not emotional enough for you? I’m not even sure what that means. When I teach the use of bright, fully saturated colors in graphs and on dashboards to attract attention or to shout the warning “Look here!”, am I not appealing to emotion? Yes, my appeal to emotion is for a purpose, and is therefore functional, but so is yours. You are promoting the use of visual effects to engage the viewer’s interest? Is this not functional in purpose?
What I teach is that a graph’s design should promote its effectiveness. In a case when you need to do something extra to get someone to look at a graph in the first place, you can certainly add something to its design to accomplish this, but you are not forced to do this in a way that undermines the graph’s effectiveness. Graphs have stories to tell, which requires that they sometimes raise their voices, but they needn’t ever speak in tongues that the audience cannot understand or stick gags in their mouths that garble their words. A data visualization always has a purpose (or at least it should). What I teach is that its design should support that purpose as effectively as possible. Whatever works best is what should be done. I advocate design principles that work effectively in most circumstances. In situations when these principles should be bent or ignored altogether to serve the visualization’s purpose, then that’s what we should do.
Jorge—you claim that your position is “more comfortable” than mine, apparently because you are a pragmatist and I am an unbending idealist. You are mistaken. Like it or not, we all live in that world of gray where nothing is purely black or white. When you represent my position as unbendingly idealist, you resort to a misleading over-simplification, characterizing as black and white what I daily recognize very well as a world of gray.
On the matter of Jarvenpaa’s research, the statements that you quote appear naïve. No expert in data visualization that I know would make the general claim that “Graphical charts are…a superior reporting technique compared to more traditional tabular representations in organizational decision making.” We claim that particular graphical representations are superior for particular purposes in particular circumstances. Graphical representations support the decision-making process extremely well. In this we are confident, based on empirical findings. However, tables have their place also. Quoting Jarvenpaa as you have unjustly casts data visualization in a bad light.
In your effort to be provocative, I believe that you sometimes create confusion. I invite you to disagree with me to your hearts delight. I hope to learn from you when you challenge me. All I ask is that, when you do, represent my position accurately and responsibly.
Jorge:
Just to say that I completely agree with you, and 90% with Sthepen. I like simpler, (very) easy to read charts, that shows trends quickly to managers.
In a bussiness environmet, a good chart must tell us a story, and it must do it quickly. If a chart don’t reveals information quicker than a data table, I better delete it and rather show the table.
A good chart, most of the times, don’t need impactfull colors to show a trend.
Congratulations for your blog.
Stephen & Jorge, you both make some very good points.
Unfortunately Stephen, your long winded disputes of what you deem right in the world of visualization is turning me off to your work. As for what you consider a layman – if someone has sought you out and is attending your course, they have already been converted. While they may be a novice, they already have understood the big picture of data visualization and recognize you as a thought leader in the field.
I mention your name in my “BI” department at work, and the response I get is “who?”
Maybe a relevant example to wedge into this little debate is the lack of color in Bullet Graphs. You insist on having no color in them for the color blind but I think it severely inhibits the general report consumer to understand what it is the graph is telling them. (I’d like to see a test done on this – I dare you to do one and put it the results on your site!)
I work in a very large company and I just found out that the executives want a dashboard. Do you know what visualization they want front and center? Gauges!
Jorge – yes indeed congrats on your blog! Your work is such that Stephen Few is reading and feels compelled to discuss.
L. Quezada,
You’re response to Jorge’s comments is an example of what I fear. You seem to believe that my charts are not simple and easy to read. Nothing could be further from the truth. In fact, if you were to sample my charts vs. Jorge’s, I believe you would find that on average, Jorge’s are more complex than mine. This is not a criticism of Jorge’s work. The fact is that, on average, more of Jorge’s examples address the needs of people with a greater level of charting sophistication than mine, so a greater level of complexity is appropriate. The goal is to keep data displays as simple as possible without oversimplifying them. Jorge and I both strive for this goal.
R. Bergfalk,
The tone of your comments suggests that you support some of the practices that I warn people against, or perhaps are committed to a product that I’ve disparaged.
My “long winded disputes” are attempt to correct misinformation. As someone who works hard to support data sensemaking and communication, I am a stickler for the truth.
Regarding my understanding of the “layman”, you are mistaken in your assumption that I only work with people who have chosen to attend my courses or read my books and are therefore already converted. Perhaps half of the people who attend my courses were sent to them by their managers. They represent the full spectrum of people who need to make sense of data and present it to others. They approach what I do freshly, without any prior commitment to or even knowledge of my work. Regardless of where they stand before taking the course, with rare exception they end the day with a clear understanding of the simple design practices that I teach and a desire to put them to work for themselves. Assuming, from your comment, that you work in a BI department, you might be interested in knowing that I worked in and managed BI departments for many years. Like you, I supported real people in real organizations in the real world. I still do, even though I now work as a consultant, writer, and teacher.
Your statement that the executives in your company want gauges on their dashboards is not an argument for their effectiveness. I’m well aware of their popularity. I’m also well aware of the popularity of cotton candy. Regarding bullet graphs, I actually don’t insist on having no color in them. What I advocate is that the background fill colors, which represent a sequential series of qualitative states such as good, satisfactory, and bad, consist of sequential intensities of a single hue, which need not be restricted to grayscale. Respect for those who are colorblind is not my only reason for advocating this. Another is the fact that if there are too many colors on a dashboard, the dashboard becomes visually overwhelming, and thus hard to look at, and you lose the ability to use color to attract attention to particular information, because the many colors that appear on the dashboard work perceptually as distractors.
Regarding your congratulations to Jorge for his success, I share your appreciation for Jorge’s work, except in those rare cases when he misrepresents my position, as he’s done here. The fact that I read and respond to his blog is not a measure of his success, however. I respond to almost any blog about data visualization when the content, in my opinion, is either exceptionally good, needs correction, or simply catches my interest. In an effort to remain in touch with everything related to data visualization, I use Google Alerts to search for and send me everything new that pops up on the Web about this topic.
Stephen,
I only support some of the practices that you warn people against because I am told to do so by management. Same goes for the Xcelsius license we are about to buy. I’m the young new guy in the department and I can’t seem to persuade my own coworkers that it’s not the product we want/need, never mind the multiple layers of management above me.
This is partially the reason for my comment regarding the executives and gauges. Both the requirements AND the method of implementation is coming from the top of the organization. A mistake in my opinion, but unless I am the consultant hired to give advice, my word is meaningless. Jorge has written about these kinds of situations many times, and it is somewhat relevant to his talk of pie charts. Sometimes these poor practices are what’s required and it is beyond our control to do anything about it.
I apologize for assuming that you only work with the converted – I was not aware you taught such courses. However, outside the geographical areas you teach, who do you think would follow your work? I suppose a curious individual might come across your book or get pointed to your work from a blog like Jorge’s, but if that’s the case, they are already possibly questioning their current methodology and seeking out a better way of doing things.
As for the color in bullet charts, I still stand that red/yellow/green would be more intuitive than multiple hues of a single color. Personally, I don’t see why one would need any other colors on a dashboard. Use black and grey for data points and simply reserve color for a quick status indicator.
I think for the most part, all of us here are on the same side 🙂
Hi All,
Paraphrasing the infamous Rodney King, “Can’t we all just get along?”
I must say I generally enjoy healthy debate about topics related to my profession, and for the most part I found this thread interesting and enjoyable. But I think it’s time to let it go. We’re now approaching a “You say po-tay-to and I say po-tah-to” type of discussion, whereas I think, in truth both parties share a great deal of common ground.
What is my take away from this? As professionals in the field of data visualization, it is our role to do the best job we can to present usable information to the data consumers, while also recognizing that the (nominally) ‘best’ method for doing so is not always going to be the one we’re told to use. Sound about right?
Our firm deploys BI solutions to smaller companies, and we’re often compelled to present information in a glitzy way, even though, from an informational perspective, there are other better options. But what we try to do is fulfill stakeholder expectations at the front-end of the project, and slowly migrate them to a more effective, albeit slightly less flashy presentation in rounds two and three. As they start to use the dashboards, they begin to realize that they can convey information better if they trim the fat from the visuals. But in truth, while we can educate and recommend, the ultimate decision lies with the information consumer.
Stephen, I definitely respect your views. I’m a long-time owner of your book, and often recommend it to my clients (no commission required 🙂 ). But what I understand from Jorge’s comments is that sometimes you have to bend a bit to accommodate the consumer. And on that, I must agree with him.
Just like with email, it’s really easy to get sucked into reading between the lines of someones blog post. It’s a venue where inflection is impossible to interpret, and there’s no eye contact to help understand intent. That’s why I use so many smiley faces 🙂 Might I humbly suggest that both parties are right in their own context, no gross misrepresentation of another’s work was intended, and we should all continue to strive for the best possible results for our stakeholders, accommodating their wishes to some degree even if we feel they are in error? Please? 🙂
That’s my 2 cents.
Regards,
Jim Payton
Interesting and hot topic.
I think we as BI specialists (be it a consultant or a regular employee) focusing on visualization information can do our share to design better dashboards, for example: using subdued colors when it makes sense, getting rid of fluff (and those wonderful animated charts Xcelsius offers as a default), etc.
Showing the customer how much more information can be displayed in the same space a gauge or pie chart occupies with a different visualization such as a ranked bar chart.
We should try to make better dashboards (which the customer sometimes confuses with detailed reports by the way), and true sometimes the customer will despite our efforts and arguments insist on a gauge or pie chart, so be it.
Not showing a customer better visualizations than a gauge for example is a disservice IMHO.
Sure there is eye-candy, but would you like cotton candy for breakfast… EVERY DAY? 🙂
Take it away! Oh it’s so ugly.
Re: tabular data. Tufte spends time in VDQD talking about tabular data and advocating for using real numbers.
The Tukey attitude (e.g. stem plots) advocates for tabular data too.
Nobody said visual display has to be with colored bars, lines, or patches.
PS Just imagine a histogram with 5 observations. Is it better to put an axis on the left or a number over the top of each? I’d go with the latter. Colorize the important number.
Re: J.A. Myer’s quote.
Who would ever suggest that presentation could make ALL the difference? That is an obvious straw man.
I keep thinking each comment is going to be my last.
Either the DATA is compelling, or it is not. Don’t blame yourself if the chart fails to “wow”.
If you feel you need to “convert” people in your office, why not buy one or more of Tufte’s very expensive prints and hang them up on your wall. They wowed most of us and got us interested in the subject, it should work on other sensible people.
Another potential approach is this: take a case where the correct result WILL BE OBSCURED by an ugly-#ss chart as above. Maybe some sales data that should be obvious or famous to someone in your firm. Then remove the labels, spin the chart around, and colour it so as to confuse them.
Say, “OK, which piece of the pie is [biggest client] ?” They guess wrong. You say, A-ha! and show them a flat, representative chart.
I have to agree with Steven on the use of gauges. I not only find them a waste of space, but also tacky. I can understand that executives like them, as they are showing there success (or failure). Just because we call something a dashboard, doesn’t mean you need to literally but gauges on it. What’s next, a steering wheel and a hula dancer bobble head? With that said, the color in the graphs, the fonts and the layout can all add to something that is not only informative, but also visually pleasing.
But if the client wants a gauge, recommend against it once, and then give it to him. You can’t help him if he fires you.