Hi Folks,
This post is part of Series Business Intelligence – Tools & Theory
Currently running topic for this series is listed as below :
Series Business Intelligence – Tools & Theory
>>Chapter 1 : Business Intelligence an Introduction
>>Chapter 2 : Business Intelligence Essentials
>>Chapter 3 : Business Intelligence Types
>>Chapter 4 : Architecting the Data
>>Chapter 5 : Introduction of Data Mining
>>Chapter 6 : Data Mining Techniques
>>Chapter 7 : Introduction to Data Warehousing
>>Chapter 8 : Different Ways of Data Warehousing
>>Chapter 9 : Knowledge Management
>>Chapter 10 : Data Extraction
>>Chapter 11 : Business Intelligence Life Cycle
>>Chapter 12 : Business Intelligence User Model
>>Chapter 13 : Business Intelligence Issues and Challenges<You are here>
Continuing from my previous post on this series, If you have missed any link please visit link below
We are going to Cover the Following Points in this article
- Introduction
- Critical Challenges for Business Intelligence success
Introduction
By now you must be familiar with the history of Business Intelligence, the opportunity in BI and application of Dashboards and OLAP in BI.. Business Intelligence (BI) is not a product or a system. It is a continuously growing plan, vision and architecture which always seek to support an organization’s operations and direction with its well planned business aims. There are many challenges that the organizations face while using BI. There are 10 critical challenges that the organization should understand and address for the success of BI. Successful BI brings greater gain which is the true sign of business success. A BI project team lacking the BI application implementation experience will fail to deliver the required results in the first iteration. In BI projects, the business analysis issues are related to the source data, which is spread around the organization in different data stores and in different formats. The Customer Pain Point drives the customer business. It helps to identify the customer problem, the need that the business addresses. The business objectives of any BI project should be tied to the financial consequences such as the lost revenue and the reduced profit.
In this unit you will learn about customer pain points and the methods to develop cost effective enterprise friendly BI solutions.
Critical Challenges for Business Intelligence success
In this section we will discuss the challenges and issues faced in the path of BI success. There is lot of information readily available to those who require it but is not in a consistent format. Obtaining the required data quickly and accurately to make decisions is the biggest challenge and the solution to this is Business Intelligence. There are many challenges that organizations face. There are 10 critical challenges that organizations should understand and address for the success of BI. The BI projects fail because of the following:
· Failure to identify BI projects as cross-organizational business initiatives, and to recognize that they vary from the typical standalone solutions.
· Unengaged business supporters or supporters who have little or no power in the enterprise.
· Unavailable or unwilling business representation.
· Lack of skilled and readily available staff, or sub-optimal staff employment.
· No software release idea, that is, no iterative development method.
· No work breakdown arrangement, that is, no methodology.
· No business analysis or standardization activities.
· No regard for the impact of dirty data on business gain.
· No understanding of the requirement for and use of meta-data.
· Too much reliance on different methods and tools. (Silver bullet syndrome)
With BI, the business success is realized through fast, easy access to actionable data. This access is best achieved through suitable and correct insight into business conditions and customers, finances and markets. Successful BI is never an accident, the organizations attain it when they do the following:
· Make good decisions with better speed and confidence.
· Streamline the operations.
· Reduce the product development cycles.
· Maximize the value from the existing product lines and predict new opportunities.
· Create better and more focused marketing as well as better relationships with the customers and the suppliers alike.
· We will discuss the challenges which organizations face while implementing Business Intelligence.
Cross-Organizational Partnership
Any business plan including a decision-support project was traditionally focused on a particular goal that was restricted to a set of products or a region of business. Due to this narrow focus, organizations were not able to analyze the affect of the project on the business operations as a whole. As the organizations became more customer-focused, these initiatives began to combine customer information with product information. It is vital to realize that it is not manufacturing plants and the product managers, but customers and markets that, drive the business. It is best to rectify any of the customer issues before the customer realizes that the problem even existed. The enterprises have a better chance of attaining high customer loyalty if the customers pay when their problem is solved, and not when the product is shipped.
Earlier, integration used to occur in regional or departmental databases, with no cross-regional collaboration. The enterprise data warehouses were the
next step in the evolution towards cross-organizational integration of information for the decision-support purposes such as sales reporting, Key Performance Indicators1 (KPIs) and trends analysis. Customer Relationship Management (CRM) followed, bringing the promise of increased sales and profitability through personalization and customization. BI is the next step in attaining holistic cross-organizational observation. It has the potential to deliver huge payback, but demands unique collaboration. Where BI is concerned, association is not limited to the departments within the organization, as this needs the integration of knowledge about the customers, the competition, the market conditions, the vendors, the partners, the products and the employees at all levels.
To succeed at BI, an enterprise should care for a cross-organizational collaborative culture in which everyone grasps and works towards the planned vision. Next we will understand how the role of business sponsors impact BI.
Business Sponsors
Strong business sponsors actually believe in the importance of the BI project. They campaign for it by removing political roadblocks. Without a helpful and dedicated business sponsor, a BI project will struggle for support within the organization and will usually fail.
Business sponsors set up proper objectives for the BI application, making sure that they support the strategic idea. The sponsors also support the business case assessment which helps in setting up the project scope. If the scope is too big, the sponsors can priorities the deliverables. For most BI projects, the business sponsors also begin a data-quality campaign in the affected departments. This is a task for the business sponsors because it is the business users who have to identify the data. Finally, the business sponsors run a project review session at particular checkpoints to ensure that the BI application functionality maps properly to the strategic business aims, and also on the Return On Investment2 (ROI) which can be objectively measured.
Dedicated Business Representation
In this topic, we will see how the various aspects of a business affect business intelligence success. The basic focus of the BI projects is technical-oriented rather than business-oriented. The reason for this is that most of the BI projects are run by IT project managers with minimal business knowledge. These managers do not tend to involve in business communities. Therefore, it is not surprising that most of the projects fail to deliver the expected business profits. It is vital to note that around 20% of businessmen use BI applications 80% of the time. Therefore, it is important to recognize the main business and technical representatives at the starting of a BI project in order to keep them motivated throughout the project. A BI project team should have involved the stakeholders from the following areas:
· Business Executives: They are the visionaries who are aware of the organizational strategies. They have to help make main project decisions and have to be solicited for deciding the project’s direction at different stages.
· Finance Department: This department is responsible for accounting and can give insight into the organization’s efficiencies and development areas.
· Marketing Personnel: They have to be involved during all the phases of the project because they are the key users of the BI applications.
· Sales and Customer Support Representatives: They must have direct customer contact and need to able to present the customers’ view during a BI project. The customers can help recognize the final aims of the BI system. Acceptance of the products or the service strategies matters the most. The main business partners have to give a different view of the customer and solicit the information at the start of an ongoing basis.
· IT Personnel: They support the operational systems and gives awareness about the accumulation of BI requests from the various groups. In addition to giving technical expertise, the IT staff in the BI project team should analyze and give BI-related requests.
· Operations Managers and Staff: They can make planned business
decisions which will link between the strategic and the operational data which makes it important during some key phases of the BI project.
Availability of Skilled Team Members
Now that you know the role various aspects of a business plays in BI projects, let us discuss the need for skilled team members in BI projects. BI projects vary considerably from one another because at their outset, they lack concrete, well-defined deliverables. In addition, the business and the technical skills necessary to implement a BI application are relatively different from the other operational Online Transaction Processing (OLTP) projects. For example, while the operational projects normally focus on a specific area of the business, such as Enterprise Resource Planning (ERP), CRM or supply chain management (SCM), a BI project integrates analyses and delivers the information obtained from almost every area of the business.
The necessary technical expertise differs as well, for example, a database administrator’s focus is the efficient retrieval of data using OLTP systems. By contrast, where BI systems are concerned, it is important to focus on data storage in addition to data retrieval.
A BI project team lacking the BI application implementation experience will fail to deliver the required results in the first iteration. Since, most of the BI projects have aggressive timelines and short delivery cycles, an inexperienced and unskilled team will be a risk and it must be avoided. The mandatory skills required for BI projects include the following:
· BI business analysts who can do the cause-and-effect analysis to create the business process models for valuating the decision alternatives. These individuals should be able to perform the what-if analysis by following the proven BI methodology.
· The KPI experts skilled in creating the balanced scorecards. These experts must be able to recognize the KPIs that can meet the business needs, calculate and report them and monitor the performance. They should also iteratively re-evaluate the KPI effectiveness and should also integrate these KPIs into the balanced scorecard.
· The Balanced scorecard experts always develop and fine-tune the scorecards. Measuring this success in a dynamic business environment requires a useful toolset. With a balanced scorecard, an organization’s idea and strategy can be translated to objectives, targets, metrics and incentives to meet the objectives and targets.
· Data warehouse architects with the experience in developing the BI- related logical and physical data models, includes both the star schemas and the OLAP. Preferably, these people may also have experience with technologies such as statistical tools and data mining algorithms.
· Cube developers and the implementers with experience in applying BI specific data models, OLAP servers and queries. These individuals should be able to create and deploy complex and intelligent cubes to conduct the multi-dimensional OLAP analysis for different users.
· Personalization experts experienced in developing Web-based generic BI applications that may not only meet the reporting needs of many users, but also give a personalized view to each user.
Business Intelligence Application Development Methodology You know that for BI projects to succeed, the team must stick to a plan which has clearly defined methodologies, objectives and milestones. Unlike other undertakings, the BI projects are not limited to a set of departmental requirements. Rather, their idea is to give cross-organizational applications. Therefore, the BI methodologies and deliverables vary. Like any project, the BI answers some of the basic questions such as the following:
· What will be delivered?
· What are the benefits and expected ROI?
· What is the total cost?
· When will it be delivered?
· Who will do it?
The answers jointly define the BI project as follows:
The project deliverables will map aims to the strategic business targets. These deliverables should be measured in business terms. For example, to boost the sales by 20%, the sales data combined with pipeline data should be available to the sales teams within three days of the month’s end. Project scope should align the deliverables with the BI application deployment phases and timelines. Unlike the traditional OLTP applications, the number of transactions the system can perform cannot be measures in the BI project scope. Transactions basically signify an organization’s processes, which in turn represent the functions. Since BI projects are data- intensive, and not function – intensive, their scope should be measured by the data they will convert to the target BI databases, and on how quickly this data can be available. Focus on the data is required because almost 80% of the effort in a typical BI project is spent on data-related activities.
ROI for a BI project should be derived from the project deliverables. The project sponsors should measure the usefulness of the delivered BI applications after the completion of each phase to decide if the project is delivering the promised ROI. If it is not, then improvements should be made.
Planning BI Projects
In this topic we will understand why BI projects tend to hit the unknowns more often than OLTP projects? OLTP projects relate the processes of an organization, which in turn signify the functions. On the other hand the BI projects provide data which will be converted into information, which in turn is converted into action. Therefore, the BI project planning is not a one-time activity, but an iterative process in which the resources, timelines, scope, deliverables and plans are constantly adjusted. Figure depicts the planning processes in BI project. The planning processes are plan, execute, monitor and enhance.
Figure BI Project Planning Process
Although it is an iterative process, the primary project plan should be created with as much detail as possible. Figure shows the BI planning activities.
Figure : BI Project Planning Activities
BI project planning activities include the following:
· Determine Project Requirements: According to this, the existing high level data, functionality and infrastructure requirement should be reviewed and revised to include more detail and remove ambiguity.
· Determine the Condition of Source Files and Databases: Before
finishing the project plan, the operational data stores should be reviewed to account for any issues that may come up during the data analysis phase.
· Determine or Revise Cost Estimates: During this activity, the organization performs detailed analysis to determine purchase and maintenance cost estimates for hardware, software, network equipment, business analysts, IT staff members, implementation, training and consultants.
· Determine or Revise Risk Assessment: The enterprise should perform a thorough risk assessment to exactly decide and rank the BI project risks which is based on the severity and the likelihood of their occurrence.
· Identify Critical Success Factors: In this activity, the organization should determine what conditions should exist for the project to be successful. The factors include supportive business sponsors, realistic time frames and the availability of resources.
· Prepare Project Charter: This is a complete Memorandum of Understanding (MoU) which has to be prepared by the project team. The MoU should be approved by the business sponsors and the key business representatives.
· Creating a High-Level Project Plan: These are complete breakdown of tasks, resources, timelines, task dependencies and resource dependencies mapped on to a calendar.
· Project Kick-off: On completing the plan, the project is kicked off in an orientation session where all the team members, business representatives and BI sponsors are present.
Next we will discuss about the importance of data standardization.
Business Analysis and Data Standardization
BI projects are data-intensive and ’data out’ is as vital as ‘data in’. It is important to scrutinize the source data. In most of the BI projects, the business analysis issues are related to the source data, which is spread around the organization in different data stores and in different formats. Some of the issues as listed as follows:
· Recognizing Information Needs: Most of the business analysts face challenges when it comes to recognizing business issues related to the
BI application aims. They must evaluate how addressing these issues can help in obtaining answers to business questions like ‘Why is there a reduction in the sales revenue in the fourth quarter on the West Coast?’ Once these issues are identified, business analysts can easily decide on the related data requirements, and these requirements can help identify the data sources for the necessary data.
· Data Merge and Standardization: The major challenge faced by every BI project is the team’s ability to understand the scope, the effort and the importance of making the required data available for knowledge workers. This data consists of fragments in different internal systems and must be combined into a common data warehouse which is not a small task. The data requirements normally extend beyond internal sources, to private and external data. Therefore, the data merge and the standardization activities should be planned and started at the beginning of the BI project.
Affect of Dirty Data on Business Profitability
You now know the data requirements in BI projects. In this topic, you will learn how dirty data poses a challenge to BI. Inaccurate and inconsistent data costs the enterprises millions. It is important to recognize which data is crucial and then find out how to clean it. Any dirty data must be recognized. A data-cleansing plan should be developed and also implemented. The business objectives of any BI project should be tied to the financial consequences such as the lost revenue and the reduced profit. The financial consequences are basically the result of a business problem related to the inaccuracies in the reports due to the dependence on invalid, inaccurate or inconsistent data. However, most of the BI projects fail to tie financial consequences to dirty data through the monetary expressions such as losing Rupees ten crore in the quarterly revenue due to the enterprise’s inability to up-sell. Even the best of BI applications become worthless if driven by dirty data. Therefore, it is crucial for every BI project to employ knowledgeable business analysts who can understand the meaning of the source data and also ensure its quality.
Underestimating the data-cleansing process is one of the main reasons for the BI failure. The inexperienced BI project managers often support their estimates on the number of technical data conversions necessary. The project managers fail to take into account the vast number of transformations necessary to enforce the business data domain rules and the business data integrity rules.
There are some big organizations with many old file structures. Here, the ratio of a particular data transformation effort can be predicted to be as high as 85% effort in data cleansing and only 15% in implementing the technical data conversion rules. Therefore, even if the estimates appear to be realistic at the beginning of a project, data-cleansing efforts have to be factored in. Note that the full-time involvement of the right business representatives is necessary for data cleansing activity.
Importance of Meta-Data
You are now familiar with the affects of dirty data and the need to cleanse data before even starting a BI project. Now we will learn about meta-data. Clean data is useless to knowledge workers if they cannot understand the context of the data. Valid business data will be meaningless unless tied to its meaning. Hence, it is important for all the BI applications to create and also to handle the meaning of each data element. This data is known as the meta-data. The management of meta-data is an important activity in BI projects. Meta-data explains an organization in terms of its business activities and the business objects on which it has to be performed. It helps transform the business data into information which is important for every BI environment. For example, what is the meaning of profit? Will every business person understand profit in the same way? Does profit have only one calculation? Are there are any different interpretations of profit and are all interpretations correct? If there are many legitimate versions of profit, then multiple data elements should be created, each with its own exclusive name, definition, content rules and relationships. All this data is called meta- data. Meta-data helps business executives navigate the BI target databases and also helps IT personnel manage the BI applications.
There are two types of meta-data. They are the following:
· Technical meta-data which gives information about the BI applications and databases, and also helps the IT staff in managing these applications.
· Business meta-data gives business users with information on the data stored in the BI applications and databases.
Both types are important for BI success and have to be mapped to each other and kept in the meta-data repositories.
Silver Bullet Syndrome
Earlier in the unit, we familiarized with the term Silver Bullet syndrome. Let us now discuss it in detail. There is neither a single technology nor a technique which can determine all the challenges to achieve the aim of a successful BI environment. That is, there is no silver bullet. The BI projects have a huge scope and cover the several environments and technologies. The BI environment consists of the following:
· Tool for Extracting, Transforming and Loading the data from different source systems into the targeted BI data warehouse.
· Data warehouse which stores the historical and the current business data, as well as an OLAP server gives the analytic services.
· Front-end BI applications used to give querying, reporting and analytic functions to the organization’s knowledge workers.
In most organizations, the BI components are implemented in many different phases and by the project teams. Each team uses the product that satisfies most of its functional requirements. More tools can create bigger complexity and increased interoperability issues, and also needs more administration involvement. The BI project teams should always consciously attempt for the lowest possible number of tools which will allow different BI activities to map to the same overall roadmap.
Hope you will like Series Business Intelligence – Tools & Theory series !
If you have not yet subscribe this Blog , Please subscribe it from “follow me” tab !
So that you will be updated @ real time and all updated knowledge in your mail daily for free without any RSS subscription OR news reading !!
Happy Learning and Sharing !!
For More information related to BI World visit our all Mentalist networks Blog
SQL Server Mentalist … SQL Learning Blog
Business Intelligence Mentalist … Business Intelligence World
Microsoft Mentalist … MVC,ASP.NET, WCF & LinQ
MSBI Mentalist … MS BI and SQL Server
NMUG Bloggers …Navi Mumbai User Group Blog
Architectural Shack … Architectural implementation and design patterns
DBA Mentalist …Advance SQL Server Blog
MVC Mentalist … MVC Learning Blog
Link Mentalist … Daily Best link @ your email
Infographics Mentalist … Image worth explaining thousand Words
Hadoop Mentalist … Blog on Big Data
BI Tools Analysis … BI Tools
Connect With me on
| Facebook |Twitter | LinkedIn| Google+ | Word Press | RSS | About Me |
(B)usiness (I)ntelligence Mentalist 