Goodbye data silos, hello ‘self-service’ analytics

Business silos have kept data guarded for years, but new tools are allowing everyone across the enterprise to extract more value

Every night, two million travelers rest their heads in an Airbnb. Collectively, those bookings represent a treasure trove of data for the company—demographic insights for marketing, rental metrics for sales, details about app performance for IT, and much more.

Who gets to share in those riches? Everyone, it turns out, thanks to Airbnb’s new Dataportal, a searchable repository of actionable company data that is accessible to every worker.

Airbnb and other large companies, including Walmart, Google, Uber, and IBM, are making their data more widely accessible outside centralized business-intelligence teams. Data democratization, as the trend is called, pulls information out of its traditional silos and makes it available through self-service data tools across the organization.

“With self-service tools, anyone can pull a report and build data models,” says Sam Ransbotham, associate professor of information systems at Boston College. “But more importantly, they can become savvier consumers of the analytics capabilities of people who create more advanced models.”

New technologies are making it possible for teams in different departments to work with the same data.

Ever-growing volumes of data flood into businesses every day. But access isn’t easy to come by—and not just for security reasons. Nearly half of employees in a recent MIT Sloan Management survey say their companies are too secretive about sharing internal data, while nearly 1 in 3 executives in a PwC study say organizational silos are the main obstacle to their company extracting business value from data.

Partly this is because of confusion in the C-suite about data management. The rise of new roles such as chief data officer, chief digital officer, and chief analytics officer alongside traditional CIO and CTO positions often only encourages business units to hoard their data.

“Because data stewardship is poorly specified, often people assume, ‘If I gathered and maintain it, then I own it,’” says Tom Davenport, professor of information technology at Babson College and author of Competing on Analytics: The New Science of Winning.

Breaking down data silos

The data silos have been well-fortified over the years. For decades, business units have managed data in a Babel of different formats: customer relationship management (CRM) software for sales and support, human capital management (HCM) software for HR, and so on.

But the mass adoption of cloud services has given businesses greater amounts of data storage and processing power within easy reach. At the same time, new data-management platforms with machine learning capabilities have sprung up to serve every major business function, from IT to facilities. Both trends are helping break down the silos and putting data in the hands of more people who can leverage its value.

Financial analysts and marketers can use drag-and-drop interfaces to build datasets or predictive models.

Master data management (MDM) software, for example, can analyze data up to 100 times faster than its predecessors. It also allows data “sharding,” which partitions very large databases into smaller pieces for improved search. Enterprise adoption of MDM is expected to more than double over the next five years.

New customer data platforms are also replacing old-school CRM software, making it possible to combine structured data (such as formal surveys and purchase records) and unstructured data (such as chat and social media chatter) into a single intelligent system. More than 3 in 4 companies have adopted or are developing a CDP, according to a Forbes Insights/Treasure Data survey.

Together, the technologies make it possible for teams in different departments to work with the same data.

Walmart has put vast amounts of its data into the hands of managers across its retail empire with its analytics hub, the Data Café. (“Café” stands for Collaborative Analytics Facilities for Enterprise.) It collects 200 streams of internal and external information, including 40 petabytes of up-to-the-minute data from the chain’s more than 20,000 stores. Assisted by in-house data experts, business teams can use the hub to get real-time insights on stocking, pricing, and in-store marketing.

This can pay off in on-the-spot decisions that boost sales. In one instance, a grocery team went to the hub to find out why sales of a particular product had suddenly dropped. They found that in some regions the products were listed at a higher price than they should have been and were able to quickly fix the problem.

Self-service analytics

New, user-friendly data-analytics tools that require little or no knowledge of coding are empowering business analysts and others to take on roles traditionally designated for data scientists. Financial analysts, marketers, or HR specialists—who want to tap the power of machine learning or other data tools but lack the advanced skills to build and train algorithms—can use simple drag-and-drop interfaces to build datasets, create predictive models or prepare graphic data visualizations.

These new “self service” analytics platforms are growing so rapidly that their combined output will soon surpass that of professional data scientists, according to Gartner research.

Making data more widely accessible across the enterprise has its challenges. One is ensuring the quality of the data that’s being shared—that it’s accurate, reliable, relevant, and up to date. Yet 96% of companies admit having difficulties maintaining data quality, according to Alegion and Dimensional Research. That takes a hefty toll: Up to 35% of a company’s lost revenue is attributable to bad data, according to a 2019 PwC study. “Raw data is not good for anyone,” says Abhi Yadav, co-founder and CEO of Zylotech, a Boston-based customer analytics platform.

Many companies must go through extensive data cleanups, such as standardizing terms, before they can leverage the benefits of the new tools. “At American Airlines, they told me they had 11 different meanings of the term ‘airport,’” Davenport says. “You can imagine how much confusion that caused.”

Equally crucial is making sure that the data’s context is understandable to all users. Airbnb’s Dataportal avoids this pitfall by including metadata fields on each content page that shows who created it, when, and in what context, establishing a clear provenance if questions arise.

Google Goods, the tech giant’s internal data management platform for employees, uses similar metadata tools, but it goes further by employing AI to rank which data is most useful and relevant for an employee’s search.

Security challenge

Making data more accessible, of course, makes it more vulnerable to hackers and other security breaches. Nearly one in three companies have highly sensitive data accessible to every employee, and security experts warn that only 3% of most companies’ data is fully secure, according to a 2018 Varonis Global Data Risk Report.

Even worse, 1 in 4 data breaches are caused by insiders, including employee errors, data theft, stolen credentials, or phishing emails, Verizon’s 2018 Data Breach Investigations Report found.

Business leaders must tread a fine line to balance the need for data security and the growing internal demand for greater access to actionable data for better decision-making. The goal is to create a data-driven culture—but one where that data is well protected.


Related Articles:

Rise of the chief data officer

Why data strategy is becoming business strategy
Column

Care and feeding tips for your new data scientist

Data scientists are in big demand—as are management models that make the most of their prized skills
Staffing a data enterprise needing IT pros

How to staff a data factory

As data becomes the lifeblood of business, CIOs and CISOs need to cultivate talent that can maximize its value