Using Predictive Analytics within Business Intelligence: A Primer

Predictive analytics has helped drive business intelligence (BI) towards business performance management (BPM). Traditionally, predictive analytics and models have been used to identify patterns in consumer oriented businesses, such as identifying potential credit risk when issuing credit cards, or analyzing the buying habits of retail consumers. The BI industry has shifted from identifying and comparing data patterns over time (based on batch processing of monthly or weekly data) to providing performance management solutions with right-time data loads in order to allow accurate decision making in real time. Thus, the emergence of predictive analytics within BI has become an extension of general performance management functionality. For organizations to compete in the market place, taking a forward-looking approach is essential. BI can provide the framework for organizations focused on driving their business based on predictive models and other aspects of performance management.

We'll define predictive analytics and identify its different applications inside and outside BI. We'll also look at the components of predictive analytics and its evolution from data mining, and at how they interrelate. Finally, we'll examine the use of predictive analytics and how they can be leveraged to drive performance management.

Overview of Analytics and Their General Business Application

Analytical tools enable greater transparency within an organization, and can identify and analyze past and present trends, as well as discover the hidden nature of data. However, past and present trend analysis and identification alone are not enough to gain competitive advantage. Organizations need to identify future patterns, trends, and customer behavior to better understand and anticipate their markets.

Traditional analytical tools claim to have a 360-degree view of the organization, but they actually only analyze historical data, which may be stale, incomplete, or corrupted. Traditional analytics can help gain insight based on past decision making, which can be beneficial; however, predictive analytics allows organizations to take a forward-looking approach to the same types of analytical capabilities.

Credit card providers offer a first-rate example of the application of analytics (specifically, predictive analytics) in their identification of credit card risk, customer retention, and loyalty programs. Credit card companies attempt to retain their existing customers through loyalty programs, and need to take into account the factors that cause customers to choose other credit card providers. The challenge is predicting customer loss. In this case, a model which uses three predictors can be used to help predict customer loyalty: frequency of use, personal financial situations, and lower annual percentage rate (APR) offered by competitors. The combination of these predictors can be used to create a predictive model. The predictive model can then be applied and customers can be put into categories based on the resulting data. Any changes in user classification will flag the customer. That customer will then be targeted for the loyalty program. Financial institutions, on the other hand, use predictive analytics to identify the lifetime value of their customers. Whether this translates into increased benefits, lower interest rates, or other benefits for the customer, classifying and applying patterns to different customer segmentations allows the financial institutions to best benefit from (and provide benefit to) their customers.

What's Really Driving Business Intelligence

If you follow the logic of the major analysts covering the business intelligence (BI) market, the market drivers for business intelligence software are based on fairly simple environmental factors. The most commonly cited market drivers are the following:

1. Increasing Regulation—New laws in both the US and Europe are requiring companies to make their external reporting more transparent, forcing business to develop better systems for storing and retrieving the most current and detailed information on operations.

2. Information Overload—Having invested heavily in CRM, ERP, and SCM systems, many businesses are awash in data, but short on actionable intelligence. Being able to aggregate, mine, and analyze data in order to prepare for and respond to business and market events is the next step in making IT investments pay off.

3. Demand for Accountability and Metrics—A slow economic recovery has forced many businesses to continue trimming budgets, while requiring greater accountability for every area of spending. Business intelligence, and its associated data-mining, analytics and scorecards, provides the tools necessary to track performance metrics tied directly to strategic corporate goals.

4. Need to Improve Competitive Responsiveness—With markets exposed to increasing competition, customer demand and pricing pressure, businesses need to reduce cycles by accelerating processes that support aggressive competitive strategies. BI initiatives provide real-time information that can help businesses eliminate process delays and streamline management to improve decision-making and market response.

There's nothing wrong with these descriptions of existing market conditions. Each of them is a correct and compelling reason for businesses to support BI initiatives. However, they don't tell the whole story. In fact none of these market drivers, taken individually or taken as a whole, are enough to explain the level of investment being made in business intelligence software.

Think about it. Businesses have found ways to skirt or delay the impact of increasing regulations for decades. Why would they suddenly respond so rapidly to new regulations today? While information overload is acute, many businesses took a soaking in IS investments over the past few years. What smart CEO would throw good money after bad to try and rescue a previous investment? Metrics and accountability are certainly in high demand while budgets are tight, but how many businesses would invest millions of dollars just to be confident their million-dollar investments are sound? That kind of long-term thinking doesn't move markets in our quarterly-driven world. And finally, yes, businesses are being compelled to be more efficient and effective in order to compete, but that battle has been shaping up for decades among TQM, Six-Sigma, lean production methodologies—does anyone really believe the end of the rainbow is only a dashboard away?

While, each of these market drivers is accurate, they're only symptoms of a much deeper drive—a drive that is shaped by a concern far greater than the threat of regulation, information overload, accountability or competitive response. It's a drive that reflects the deepest fears of a CFO. It's a drive that shapes the search for CEOs that can move the businesses that move markets. It's a drive that cuts straight to the bottom line of the corporation, because it's about the single, all-important factor that defines the success of every business today.

It's all about how the value of a business is measured.

How BI Addresses the Needs of SOX Compliance

Traditionally, BI software has targeted the needs of financial decision makers. BI tools initially enabled organizations to analyze financial data, to identify trends, and to drill down on report data to reveal operational transactions, as well as to assign tasks to individual employees, in order to give management the ability to implement robust auditing processes. The driver behind these functions is the ability to capture data from several data sources across an organization, and to centralize them in a data warehouse. Aside from data centralization across the organization, data warehouses allow organizations to implement and monitor data quality activities to ensure accurate data. This reduces the potential for accidental data errors.

BI tools help vendors to meet the demands of organizations that need to comply with SOX regulations, scorecards, and business activity monitoring (BAM). General reporting and analysis functionality permits organizations to take a top-down approach to management, yet still meet SOX compliance. CEOs and CFOs who are responsible for assuring compliancy and who are accountable to the SEC often aren't directly responsible for actual report generation or in-depth budgeting. Task assignment and management of processes are internal driving forces within BI, and help companies manage employee tasks and responsibilities for each financial report and function, as well as ensuring data quality. Basically, BI allows the CEO to manage internal processes and data to meet SOX compliance, and gives CEOs the ability to micromanage tasks at each level to ensure compliance, and to identify any potential errors (as well as identifying who made them, and when they were made within the process). If proper data quality processes are implemented, organizations can guarantee that data errors do not occur within the data warehouse itself and that any key stroke errors and the like are cleansed as they enter the data warehouse, before financial analyses and reporting functions are performed to meet SOX requirements.

Although, as mentioned above, BI software can help organizations meet SOX compliancy, vendors have also taken SOX issues into account when upgrading their product suites to make sure that required standards can be met on an ongoing basis. Even though many other forms of financial reporting software meet SOX compliancy, BI solutions have the added bonus of built-in workflow processes and data integration features to ensure long-term compliancy. Data within spreadsheets can be changed, and structures are not always put in place to manage those changes. However, BI software suites have built-in task assignment and audit functions for managing, distributing, and auditing data (based on where the data comes from, who has ownership of the data, and how the data has been processed).

Using Business Intelligence Infrastructure to Ensure Compliancy with the Sarbanes-Oxley Act

The US Sarbanes-Oxley Act (SOX) of 2002 was established to protect investors from the potential for fraudulent accounting. After the exposure of several corporate scandals, such as the Enron and WorldCom affairs, the US government was compelled to pass legislation ensuring accurate financial reporting and auditing from organizations publicly traded in the United States. SOX affects any public corporation competing in the American marketplace. As a result of SOX, not only have financial controls and reporting schedules become stricter, but responsibility for accurately reporting financial results has been placed in the hands of organizational heads, namely the chief executive officers (CEOs) and chief financial officers (CFOs), to provide accurate financial and auditing data.

This means that financial departments have had to reevaluate the way they manage their controls and reporting. It is no longer possible for organizations to change data without accounting for these changes to shareholders. Now that the responsibility for accurate financial reporting has been placed on upper management, with heavy fines and potential prison terms being imposed for noncompliance, financial analysis tools, such as those provided by business intelligence (BI) vendors, are becoming increasingly important to the financial auditing process. Ensuring proper data controls, proper reporting and auditing structures, and the accurate capture of the ensuing data, are important aspects of SOX compliance and make up the essential elements of BI solutions.

There are three sections of SOX that deal directly with the use of information technology (IT). Section 302 requires management certification that procedures have been put in place to address accurate financial conditions and disclosure controls for all financial statements. Section 404 requires management certification that effective internal controls and procedures have been developed for financial report preparation. Finally, section 409 requires that timely reports be provided to investors, the US Securities and Exchange Commission (SEC), and other corporate stakeholders.