Turning data into information, better decisions, and stronger organizations

The Real Work of Data Science

Preface

It is a great time for data science! The Economist proudly proclaims that data is “the world’s most valuable resource,” 1 and Hal Varian and Tom Davenport 2 have variously called statistics and data science “the sexiest job of the twentieth century.” In searching the web for the term data scientist , we find the following definition, “‘Data Scientist’ means a professional who uses scientific methods to liberate and create meaning from raw data.” 3 Similar definitions have been offered for statisticians and data analysts. 4 Yet we believe the work is more involved and requires skills far beyond those needed to create meaning from raw data.

A Higher Calling

The difference between a good data scientist and a great one is like the difference between a lightning bug and lightning.* Indeed, they are two separate beasts.

The Difference Between a Good Data Scientist and a Great One

To be effective, data scientists should learn all they can about the company or agency that employs them, including its customers, products, and services; the overall industry, including major partners and competitors; and management's vision for the future. Some companies provide good orientation programs for new employees. Even so, learning the business is a lifelong proposition. There are many ways to learn about a business – here are some options.

Learn the Business

Understand the Real Problem

Lawrence Peter (Yogi) Berra once observed, “You can learn a lot just by watching.” It is especially poignant for data scientists who must “get out there” and learn all they can about everything surrounding the data they use.

Get Out There

This chapter is about data quality, which is a plague on data scientists and CAOs (and practically everyone else for that matter). It takes up to 80% of data scientists' time (Wilder‐James 2016) and is the problem they complain about most (Kaggle 2017). Worse, you can never be sure you've found all the errors and, worse still, the issues grow larger and more impactful with AI. The data needed for data science can come from primary sources (created with similar objectives) or secondary sources (someone else collected it, often for a different objective, and processed it before it reaches you).

Sorry, but You Can't Trust the Data

Most of us learn more from our mistakes than we do from our successes. One of us, Redman, learned a lesson that has stuck with him for 30 years.* The story concerns his first big presentation at AT&T headquarters. He completed his preparation well in advance and rehearsed carefully. Then off to the big meeting.

Make It Easy for People to Understand Your Insights

It may seem trite to mention, but the power in data science lies not so much in uncovering what the data reveals about itself but what it reveals beyond itself. In this chapter, we focus on the process of moving from numbers to data, to information and insights (Kenett 2008), what we call “generalizability” in the information quality framework to be introduced in Chapter 13.

When the Data Leaves Off and Your Intuition Takes Over

The wide‐angle perspective of data science includes activities as diverse as building trust so you are asked to contribute to really important problems, clearly stating the problem, conducting the analyses, teaching, supporting decisions in practice, and so forth. This chapter focuses on one activity that is too often ignored – impact assessment.

Take Accountability for Results

Over the past several years, the term data‐driven has penetrated the business lexicon and appears to be here to stay.* Data Driven is the title of one of Redman's books, and academic work shows that companies that rate themselves as “data‐driven” are measurably more profitable (McAfee and Brynjolffson 2012) than those that don't. Health care is undergoing a similar transformation, with evidence‐based medicine becoming the worldwide standard for improved care delivery (Masic et al. 2008). So, becoming data‐driven is clearly a worthwhile endeavor. But what exactly does it mean?

What It Means to Be "Data‐driven"

Biased decision‐making is the enemy of data science. We have all experienced the disappointment when an important decision does not go our way.* The feeling is far worse when you feel that the decision was somehow rigged – that the decision‐maker did not pay your results their due or only used some of the data. You can accept a fair decision that goes the other way, but a rigged decision feels much worse. And the ill will festers.

Root Out Bias in Decision‐making

Nothing improves data science like a demanding decision‐maker, one who is striving to become data‐driven (Chapter 10), who wants to bring as much data and data science as possible to bear and constantly expects data scientists to deliver more. At the same time, we have already noted that many people's least favorite course in college was statistics (and they have not forgotten!). Don't be surprised when your colleagues and decision‐makers are skeptical of p ‐values, logistic regressions, and ANOVA. Your only recourse to overcome these hurdles is to “teach, teach, teach.”

Teach, Teach, Teach

In the last chapter, we focused on teaching your colleagues some basics and providing a starter set of questions for decision‐makers. Of course, this business of helping decision‐makers become increasingly better consumers of data science never ends. As they gain experience, you need to provide them a more formal template based on the eight dimensions of the information quality model (Kenett and Shmueli 2016a). This will help them go deeper, facilitate discussions regarding trade‐offs, and help them improve the quality of information generated in their organizations. Breiman (2001) depicts two cultures in the use of statistical modeling to reach conclusions from data, data modeling, and algorithmic analysis. The InfoQ framework addresses outputs from both approaches, in the context of business, academic, services, and industrial work.

Evaluating Data Science Outputs More Formally

Imagine senior executives with no particular interest in data and data science. Yet, every day, they receive a cacophony of grandiose proclamations that “data is the new oil,” extravagant claims about analytics and artificial intelligence, exhortations to become data‐driven, heated claims that they must digitize, dire warnings about privacy and the reputational damage that results from data breaches. Yes, about five tech firms are profiting, but no important competitor has truly embraced data and data science. The cover of the May 6, 2017, issue of the Economist proclaimed that data is now “the world's most valuable asset,” but these claims have been around a long time. Further, the failure rate on analytics projects is high (Demurkian and Dai 2014). The messages conflict and none of them fit together.

Educating Senior Leaders

Maintenance is a big area for analytics. One data scientist, employed at an international semiconductor company, develops optimal “split maintenance schedules.” The idea is to perform maintenance tasks in several steps, separated in time, replacing major maintenance shutdowns. The advantage is less total downtime, which translates into huge savings in wafer fabrication. He enjoyed great success within the maintenance organization and hoped to build upon that success by combining it with CBM, which employs production systems and raw material data. But this data is only available from the operations department's databases. And he couldn't get access. As the story illustrates, “Silos are the enemy of data sharing.” They are particularly damaging to data science because so many opportunities lie in combining data.

Putting Data Science, and Data Scientists, in the Right Spots

Let us say you work for a bank, car manufacturer, or cellular operator. Your company completes many tasks and transactions designed to meet the needs of its customers and accumulates written reports, call center transcripts and recordings, engineering specs, financial data, inventory levels, and many other types of data. What you and the company do with this data reflects the maturity level of your company's management approach and analytic capabilities.

Moving Up the Analytics Maturity Ladder

This chapter provides an industrial context into the advance of data science. This context is important because it illustrates an important role for CAOs and senior managers – namely, identifying macrotrends based on evidence from disparate sources and positioning their companies to take advantage of them.

The Industrial Revolutions and Data Science

“Much fine work in statistics involves minimal mathematics; some bad work in statistics gets by because of its apparent mathematical content” (Cox 1981, p. 295). This down‐to‐earth description of work done in statistics, by one of the most important statisticians of this century, is quite telling. Sir David Cox started his career as a statistician at the Wool Industries Research Association in Leeds, England. The experiences he gained in this work environment shaped his extensive groundbreaking work in statistical methodology.

Epilogue

This list builds on Joiner (1985) and Kenett and Thyregod (2006), mapping skills to the life cycle introduced in Chapter 1.

Appendix A: Skills of a Data Scientist

There are many approaches to defining data . Here we use the one that best corresponds to the way data is created and used in organizations and, in our view, best supports data science (Redman 2008). Thus, data consists of a data model and a data value .

Appendix B: Data Defined

Kenett and Shmueli (2014) defined information quality (InfoQ) as the utility derived from a specific analysis of a specific data set, conditioned on the analysis goals. InfoQ is determined by eight dimensions discussed in Chapter 13. Questions to help assess these dimensions for a specific report are listed below (see also Kenett and Shmueli 2016a).

Appendix C: Questions to Help Evaluate the Outputs of Data Science

Privacy and ethical considerations have always been of concern for data scientists, but the increasing breadth and depth of available data raises these concerns to new levels. This is especially true for behavioral big data (BBD) that captures human and social actions and interactions in increasing breadth and level of detail. Such concerns include:

Appendix D: Ethical Considerations and Today's Data Scientist

We have taken the position throughout this book that data scientists must do much more than technical work. Still in our view, it is critically important that data science builds on solid theoretical and technical foundations. So, while a full review is beyond scope, a few remarks on technical aspects of data science are in order.

Appendix E: Recent Technical Advances in Data Science

Aigner, M. and Ziegler, G. (2000). Proofs from the Book , third edition, Springer‐Verlag Berlin Heidelberg, Germany. American Statistical Association (ASA). (2016). Ethical guidelines for statistical practice. http://www.amstat.org/ASA/Your‐Career/Ethical‐Guidelines‐for‐Statistical‐Practice.aspx . Baggaley, K. (2017). Your memories are less accurate than you think. https://www.popsci.com/accurate‐memories‐from‐eyewitnesses . Bapna, R., Jank, W., and Shmueli, G. (2008). Consumer surplus in online auctions. Information Systems Research 19: 400–416. Barkai, J. (2018). Predictive maintenance: myths, promises, and reality. http://joebarkai.com/predictive‐maintenance‐myths‐promises‐and‐reality . Barrett, L. (2003). Hospital revives its dead patients. Baseline . http://www.baselinemag.com/c/a/Projects‐Networks‐and‐Storage/Hospital‐Revives‐Its‐QTEDeadQTE‐Patients . February 10. BBC Scotland. (2017). Voice recognition lift. https://www.youtube.com/watch?v=J3lYLphzAnw . Bernard, T.S. (2011). Are serious errors lurking in your credit report? New York Times . https://bucks.blogs.nytimes.com/2011/06/07/are‐serious‐errors‐lurking‐in‐your‐credit‐report . June 7. Box, G.E.P. (1980). Beer and statistics Monday night seminar (reported from memory by first author). Box, G.E.P. (1997). Scientific method: the generation of knowledge and quality. Quality Progress 30: 47–50. Box, G.E.P. (2001). An interview with the International Journal of Forecasting. International Journal of Forecasting 17: 1–9. Breiman, L. (2001). Statistical modeling: the two cultures. Statistical Science 16(3): 199–231. Camoes, J. (2017). 12 ideas to become a competent data visualization thinker. https://excelcharts.com/12‐ideas‐become‐competent‐data‐visualization‐thinker . Carr, N. (2003). IT doesn't matter. Harvard Business Review , May, pp. 41–49. Chandler, A. (1993). The Visible Hand: The Managerial Revolution in American Business . Belknap Press. Cobb, G.W. and Moore, D.S. (1997). Mathematics, statistics, and teaching. American Mathematical Monthly 104: 801–823. Coleman, S. and Kenett, R.S. (2017). The information quality framework for evaluating data science programs. In: Encyclopedia with Semantic Computing and Robotic Intelligence (ed. P. Sheu), 125–138. World Scientific Press. Cox, D.R. (1981). Theory and general principle in statistics. Journal of the Royal Statistical Society, Series A 144(2): 289–297. Data Science Association. (2018). Code of conduct. http://www.datascienceassn.org/code‐of‐conduct.html . Davenport, R.H. and Patil, D.J. (2012). Data scientist: the sexiest job of the 21st century. Harvard Business Review. https://hbr.org/2012/10/data‐scientist‐the‐sexiest‐job‐of‐the‐21st‐century . Davis, S. (1997). Future Perfect . Basic Books. De Veaux, R., Agarwal, M., Averett, M. et al. (2017). Curriculum guidelines for undergraduate programs in data science. Annual Review of Statistics and Its Applications 4: 15–30. https://www.annualreviews.org/doi/abs/10.1146/annurev‐statistics‐060116‐053930 . Deming, W.E. (1931). Dedication to The Economic Control of Quality of Manufactured Product by W. Shewhart, pp. i–iii. D. Van Nostrand Company, Inc. Deming, W.E. (1982). Quality, Productivity and the Competitive Position . Cambridge, MA: MIT, Center for Advanced Engineering Study. Deming, W.E. (1986). Out of the Crisis . Cambridge, MA: MIT Press. Demurkian, H. and Dai, B. (2014). Why so many analytics' projects still fail. Analytics . http://analytics‐magazine.org/the‐data‐economy‐why‐do‐so‐many‐analytics‐projects‐fail . July/August. Donoho, D. (2017). 50 years of data science. Journal of Computational and Graphical Statistics 26(4): 745–766. Doumont, J.L. (2013). Creating effective slides: design, construction, and use in science. https://www.youtube.com/watch?v=meBXuTIPJQk . Duhigg, C. (2012). How companies learn your secrets. New York Times Magazine . https://www.nytimes.com/2012/02/19/magazine/shopping‐habits.html?pagewanted=all&_r=0 . February 16. Economist . (2017). Data is giving rise to a new economy. May 6. Efron, B. and Hastie, T. (2016). Computer Age Statistical Inference: Algorithms, Evidence and Data Science . Cambridge University Press. Einstein, A. n.d. Quote from http://www.azquotes.com/quote/811850 . Fienberg, S. (1979). Graphical methods in statistics. American Statistician 33(4): 165–178. Fisher, R.A. (1922). On the mathematical foundations of theoretical statistics. Philosophical Transactions of the Royal Society, Series A 222: 309–368. Godfrey, B. and Kenett, R.S. (2007). Joseph M. Juran, a perspective on past contributions and future impact. Quality and Reliability Engineering International 23: 653–663. Greene, R. and Elffers, J. (1998). The 48 Laws of Power . Viking Penguin. Hahn, G. (2003). The embedded statistician. Youden address. http://rube.asq.org/statistics/design‐of‐experiments/the‐embedded‐statistician.pdf . Hahn, G.J. (2007). The business and industrial statistician: past, present and future. Quality and Reliability Engineering International 23: 643–650. Hahn, G.J. and Doganaksoy, N. (2011). A Career in Statistics: Beyond the Numbers . Wiley. Hartman, E., Grieve, R., Ramsahai, R. et al. (2015). From SATE to PATT: combining experimental with observational studies to estimate population treatment effects. Journal of the Royal Statistical Society, Series A 178: 757–778. Harvard Business Review. (2013). Data and organizational issues reduce confidence. http://go.qlik.com/rs/qliktech/images/HBR_Report_Data_Confidence.PDF?sourceID1=mkto‐2014‐H1 . Harvard Business Review (2018). HBR Guide to Data Analytics Basics for Managers . Harvard Business Review Publishing. Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2e. Springer. Henke, N., Bughnn, J., Chui, M. et al. (2016). The Age of Analytics: Competing in a Data Driven World . McKinsey. Huber, P. (2011). Data Analysis: What Can Be Learned from the Past 50 Years . Wiley. Hunter, W.G. (1979). Private communication to first author, Madison, WI. IDC. (2018). www.idc.com . James, G., Witten, D., Hastie, T. et al. (2013). An Introduction to Statistical Learning: With Applications in R . Springer. Joiner, B.L. (1985). The key role of statisticians in the transformation of North American industry. American Statistician 39(3): 233–234. Joiner, B.L. (1994). Fourth Generation Management: The New Business Consciousness . New York: McGraw Hill. Joint Commission International (2018). JCI accreditation standards for hospitals, 6e. https://www.jointcommissioninternational.org/jci‐accreditation‐standards‐for‐hospitals‐6th‐edition . Juran, J.M. (1988). Juran on Planning for Quality . New York: Free Press. Kaggle. (2017). The state of data science and machine learning. https://www.kaggle.com/surveys/2017 . Kahneman, D., Lovallo, D., and Sibony, O. (2011). The big idea: before you make that big decision. Harvard Business Review , February, pp. 50–60. Kaplan, R. and Norton, D. (1996). Using the balanced scorecard as a strategic management system. Harvard Business Review , January–February, pp. 75–85. Katkar, R. and Reiley, D.H. (2006). Public versus secret reserve prices in eBay auctions: results from a Pokémon field experiment. Advances in Economic Analysis and Policy 6(2): Article 7. Kenett, R.S. (2008). From data to information to knowledge. Six Sigma Forum Magazine , November, pp. 32–33. Kenett, R.S. (2015). Statistics: a life cycle view (with discussion). Quality Engineering 27(1): 111–129. Kenett, R.S. (2017). The Information Quality Framework for Evaluating Manufacturing 4.0 Analytics Distinguished Lecture. TUE Data Science Center Eindhoven. https://assets.tue.nl/fileadmin/content/faculteiten/win/DSCE/Research/5Lecture_series/20170601_DSCe_Lecture_Ron_Kenett.pdf . June 1. Kenett, R.S. and Baker, E. (2010). Process Improvement and CMMI for Systems and Software: Planning, Implementation, and Management . Taylor and Francis, Auerbach Publications. Kenett, R.S., Coleman, S.Y., and Stewardson, D. (2003). Statistical efficiency: the practical perspective. Quality and Reliability Engineering International 19: 265–272. Kenett, R.S., de Frenne, A., and Tort‐Martotell, X. (2008). The statistical efficiency conjecture. In: Statistical Practice in Business and Industry (ed. S. Coleman, T. Greenfield, D. Stewardson, et al.), 61–95. Wiley. Kenett, R.S., Pfeffermann, D., and Steinberg, D.M. (2018). Election polls – a survey, critique and proposals. Annual Review of Statistics and Its Application 5: 1–24. http://www.annualreviews.org/doi/abs/10.1146/annurev‐statistics‐031017‐100204 . Kenett, R.S. and Raanan, Y. (2011). Operational Risk Management: A Practical Approach to Intelligent Data Analysis . Chichester, UK: Wiley. Kenett, R.S. and Salini, S. (2011). Modern Analysis of Customer Surveys with Applications Using R . Wiley. Kenett, R.S. and Shmueli, G. (2014). On information quality (with discussion). Kenett, R.S. and Shmueli, G. (2016a). Information Quality: The Potential of Data and Analytics to Generate Knowledge . Wiley. Kenett, R.S. and Shmueli, G. (2016b). Helping authors and reviewers ask the right questions: the InfoQ framework for reviewing applied research. Statistical Journal of the International Association for Official Statistics (IAOS) 32: 11–19. Kenett, R.S. and Thyregod, P. (2006). Aspects of statistical consulting not taught by academia. Statistica Neerlandica 60(3): 396–412. Kenett, R.S., Zacks, S., and contributions by Amberti, D. (2014). Modern Industrial Statistics: With Applications in R, MINITAB and JMP , 2e. Wiley. Kohavi, R. and Thomke, S. (2017). The surprising power of online experiments. Harvard Business Review . https://hbr.org/2017/09/the‐surprising‐power‐of‐online‐experiments . September–October. Laney, D. (2017). Infonomics: How to Monetize, Manage, and Measure Information as an Asset for Competitive Advantage . Routledge. Lavy, V. (2010). Effects of free choice among public schools. Review of Economic Studies 77(3): 1164–1191. https://academic.oup.com/restud/article‐abstract/77/3/1164/1570661 . Lewis, M. (2017). The Undoing Project: A Friendship That Changed Our Minds . W.W. Norton. Loftus, E.F. (2013). Eyewitness testimony in the Lockerbie bombing case. Memory 21: 584–590. Lohr, S. (2018). Facial recognition is accurate, if you're a white guy. New York Times . https://www.nytimes.com/2018/02/09/technology/facial‐recognition‐race‐artificial‐intelligence.html . February 9. Masic, I., Miokvic, M., and Muhamedagic, B. (2008). Evidence based medicine – new approaches and challenges. Journal of Academy of Medical Sciences of Bosnia and Herzegovina 16(4): 219–225. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3789163 . McAfee, A. and Brynjolffson, E. (2012). Big data: the management revolution. Harvard Business Review . https://hbr.org/2012/10/big‐data‐the‐management‐revolution . October. McGarvie, M. and McElheran, K. (2018). Pitfalls of data‐driven decisions. In: HBR Guide to Data Analytics Basics for Managers , 155–164. Harvard Business Review Press. Nagle, T., Redman, T., and Sammon, D. (2017). Only 3% of companies' data meets basic quality standards. Harvard Business Review . https://hbr.org/2017/09/only‐3‐of‐companies‐data‐meets‐basic‐quality‐standards . O'Keefe, K. and O'Brien, D. (2018). Ethical Data and Information Management Concepts, Tools and Methods . Kogan Page. O'Neil, C. (2016). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy . Crown. Pearl, J. and Bareinboim, E. (2011). Transportability across studies: a formal approach. Technical Report R‐372, Cognitive Systems Laboratory, Dept. Computer Science, Univerisity of California, Los Angeles. Pearl, J. and Bareinboim, E. (2014). External validity: from do‐calculus to transportability across populations. Statistical Science 29(4): 579–595. Pirelli. (2016). How Pirelli is becoming data driven. https://www.pirelli.com/global/en‐ww/life/how‐pirelli‐is‐becoming‐data‐driven . March 23. Pollack, A. (1999). Two teams, two measures equaled one lost spacecraft. New York Times . https://archive.nytimes.com/www.nytimes.com/library/national/science/100199sci‐nasa‐mars.html?scp=2 . October 1. Rao, C.R. (1985). Weighted distributions arising out of methods of ascertainment: what population does a sample represent? In: A Celebration of Statistics: The ISI Centenary Volume (ed. A.C. Atkinson and S.E. Fienberg), 543–569. New York: Springer. Rasch, G. (1977). On specific objectivity: an attempt at formalizing the request for generality and validity of scientific statements. Danish Yearbook of Philosophy 14: 58–93. Redman, T. (2008). Data Driven: Profiting from Your Most Important Business Asset . Harvard Business Review Press. Redman, T. (2013a). What separates a good data scientist from a great one. Harvard Business Review . http://blogs.hbr.org/2013/01/the‐great‐data‐scientist‐in‐fo . January 28. Redman, T. (2013b). Are you data‐driven? Take a hard look in the mirror. Harvard Business Review . http://blogs.hbr.org/2013/07/are‐you‐data‐driven‐take‐a‐har . July 11. Redman, T. (2013c). Become more data‐driven by breaking these bad habits. Harvard Business Review . http://blogs.hbr.org/2013/08/becoming‐data‐driven‐breaking . August 12. Redman, T. (2013d). Are you ready for a chief data officer? Harvard Business Review . https://hbr.org/2013/10/are‐you‐ready‐for‐a‐chief‐data‐officer . October 30. Redman, T. (2013e). How to start thinking like a data scientist. Harvard Business Review . https://hbr.org/2013/11/how‐to‐start‐thinking‐like‐a‐data‐scientist . November 29. Redman, T. (2014). Data doesn't speak for itself. Harvard Business Review . https://hbr.org/2014/04/data‐doesnt‐speak‐for‐itself . April 29. Redman, T. (2015). Can your data be trusted? Harvard Business Review . https://hbr.org/2015/10/can‐your‐data‐be‐trusted . October 29. Redman, T. (2016). Getting in Front on Data: Who Does What . Technics. Redman, T. (2017a). The best data scientists get out and talk to people. Harvard Business Review . https://hbr.org/2017/01/the‐best‐data‐scientists‐get‐out‐and‐talk‐to‐people . January 26. Redman, T. (2017b). Root out bias from your decision‐making process. Harvard Business Review . https://hbr.org/2017/03/root‐out‐bias‐from‐your‐decision‐making‐process . March 17. Redman, T. (2017c). Seizing opportunity in data quality. MIT Sloan Management Review. https://sloanreview.mit.edu/article/seizing‐opportunity‐in‐data‐quality . November 29. Redman, T. (2018a). Are you setting your data scientists up to fail? Harvard Business Review . https://hbr.org/2018/01/are‐you‐setting‐your‐data‐scientists‐up‐to‐fail . January 25. Redman, T. (2018b). If your data is bad, your machine learning tools are useless. Harvard Business Review . https://hbr.org/2018/04/if‐your‐data‐is‐bad‐your‐machine‐learning‐tools‐are‐useless . April 2. Redman, T. and Sweeney, W. (2013a). To work with data, you need a lab and a factory. Harvard Business Review . https://hbr.org/2013/04/two‐departments‐for‐data‐succe . April 24. Redman, T. and Sweeney, W. (2013b). Seven questions to ask your data geeks. Harvard Business Review . https://hbr.org/2013/06/seven‐questions‐to‐ask‐your‐da . June 10. Rosling, H. (2007). The best stats you’ve ever seen. https://www.youtube.com/watch?v=hVimVzgtD6w&t=43s . Ross, C. (2017). IBM pitched its Watson supercomputer as a revolution in cancer care. It's nowhere close. https://www.statnews.com/2017/09/05/watson‐ibm‐cancer . September 5. Rubin, D. (1987). Multiple Imputation for Nonresponse in Surveys . New York: Wiley. Schmarzo, B. (2017). Unintended consequences of the wrong measures. https://www.datasciencecentral.com/profiles/blogs/unintended‐consequences‐of‐the‐wrong‐measures?_ga=2.21159096.226186582.1522766955‐1679466071.1522672939 . October 19. Senge, P. (1990). The Fifth Discipline: The Art and Practice of the Learning Organization . New York: Doubleday. Shannon, C. (1948). A mathematical theory of communication. Bell System Technical Journal 27: 379–423 and 623–656. Shewhart, W.A. (1926). Quality control charts. Bell System Technical Journal 5: 593–603. Shmueli, G. (2010). To explain or to predict? Statistical Science 25(3): 289–310. Shmueli, G. (2017). Analyzing behavioral big data: methodological, practical, ethical, and moral issues. Quality Engineering 29(1): 57–74. Shmueli, G. (2018). Data ethics regulation: two key updates in 2018. http://www.bzst.com/2018/02/data‐ethics‐regulation‐two‐key‐updates.html . Silver, N. (2012). The Signal and the Noise: Why So Many Predictions Fail . Penguin Press. Simon, D. (2010). Selective attention test. https://www.youtube.com/watch?v=vJG698U2Mvo . Singer, M. (2018). Tech's ethical “dark side”: Harvard, Stanford and others want to address it. New York Times . https://www.nytimes.com/2018/02/12/business/computer‐science‐ethics‐courses.html . February 12. Srivastiva, D., Caannapieco, M., and Redman, T. (2019). Ensuring high‐quality private data for responsible data science: vision and challenges. Journal of Data and Information Quality . Accepted for publication. Surveytown. (2016). 10 examples of biased survey questions. https://surveytown.com/10‐examples‐of‐biased‐survey‐questions . April 12. Taguchi, G. (1987). Systems of Experimental Design , vol. 1–2 (ed. D. Clausing). New York: UNIPUB/Kraus International Publications. Tashea, J. (2017). Courts are using AI to sentence criminals. That must stop now. Wired . https://www.wired.com/2017/04/courts‐using‐ai‐sentence‐criminals‐must‐stop‐now . April 17. Tufte, E.R. (1997). Visual Explanations: Images and Quantities, Evidence and Narrative . Cheshire, CT: Graphics Press. Tukey, J.W. (1962). The future of data analysis. Annals of Mathematical Statistics 33: 1–67. Tversky, A. and Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science 211(4481): 453–458. van Buuren, S. (2012). Flexible Imputation of Missing Data . Chapman & Hall/CRC. Wang, S., Jank, W., and Shmueli, G. (2008). Explaining and forecasting online auction prices and their dynamics using functional data analysis. Journal of Business Economics and Statistics 26: 144–160. Wikipedia. (2018a). Imputation (statistics). https://en.wikipedia.org/wiki/Imputation_(statistics ). Wikipedia. (2018b). Response bias. https://en.wikipedia.org/wiki/Response_bias . Wikipedia. (2018c). Truthiness. https://en.wikipedia.org/wiki/Truthiness . Wilder‐James, E. (2016). Breaking down data silos. Harvard Business Review . https://hbr.org/2016/12/breaking‐down‐data‐silos . December 5. Zhang, M.X., Van Den Brakel, J., Honchar, O. et al. (2018). Using state space models for measuring statistical impacts of survey redesigns. Australian Bureau of Statistics Research Paper. 1351.0.55.160. www.abs.gov.au/ausstats/abs@.nsf/mf/1351.0.55.160?OpenDocument . 

References

A List of Useful Links

A/B testing, 41 Advent of the factory, second industrial revolution, 84 American Statistical Association, 98 analytics, 4–5 ecosystem, 19 maturity ladder, 80 Antis, traits of, 46 Arkwright, Richard, 84 artisanal system, 83 assessing information quality, 63–64 Australian Bureau of Statistics, 41 automated factory, 83 automobile suspension system, 85 

Community Questions