MSBI : BI # 29 : Business Intelligence – Tools & Theory # 21 : Data Mining Techniques #3 : Decision Tree & Neural Networks

Hi Folks,

This post is part of Series Business Intelligence – Tools & Theory

Currently running topic for this series is listed as below :

Series Business Intelligence – Tools & Theory

>>Chapter 1 : Business Intelligence an Introduction

>>Chapter 2 : Business Intelligence Essentials

>>Chapter 3 : Business Intelligence Types

>>Chapter 4 : Architecting the Data

>>Chapter 5 : Introduction of Data Mining

>>Chapter 6 : Data Mining Techniques<You are here>

Continuing from my previous post on this series, If you have missed any link please visit link below

We are going to Cover the Following Points in this article

• Decision Tree
• Neural Networks
• Introduction to Neural Network

Decision Tree

Decision tree is a style of representation which depicts learning from a set of independent instances. Decision tree adopts „Divide and conquer‟ approach to the problem of learning. Decision tree is a simple and powerful form of multiple variable analyses. It is an algorithm that identifies various ways of splitting data set into branch like segments. The segments form an inverted decision tree with root node at the top.

Decision tree is a possible representation of a decision function. It is useful when the complete knowledge of data is not necessary for making appropriate decisions and when the process of gaining data is expensive. Decision tree determines which data and in which order one should collect to achieve the effective decision with minimal average cost. Decision tree thus represents knowledge and can be used for effective decision-making. There exist algorithms for automatic construction of decision trees. Automatic construction of decision trees is the traditional part of artificial intelligence.

The decision tree is an analytical tool of business intelligence system that helps business managers to resolve the uncertainties in making investment decisions. It clarifies the choices, risks, objectives and monetary gains. It also helps to gain the information required in a business application. The decision-tree helps management to determine an alternative point in business that yields the greatest monetary gain.

Decision tree is a widely used learning method. It is easy to interpret and can be represented in “if then else” rule. Decision tree algorithm does not require any prior knowledge of data distribution. Decision tree has the ability to work well on noisy data. This algorithm is useful in diverse field like classifying patients based on the disease in a hospital, to find loan applicant by likelihood of payment in banks and so on.

Decision tree has the following advantages:

· It has reasonable training time.

· It allows fast application and is easy to interpret and deduce.

· It is easy to implement.

· It has the ability to handle large number of features and data.

Apart from having various advantages, the decision algorithm also has some drawbacks which are mentioned below:

· It cannot handle complicated relationship between features.

· It has simple decision boundaries.

· It has lot of missing data problem.

Illustration of Decision Tree

In the general form of decision tree modeling approach, a relationship is extracted to which one or more decision rules can be derived that describes the relationship between the input and target. Every rule assigns a record or observation from the data set to a node in a branch. The assigned record/observation is based on the value of one of the fields or columns in the data set. Fields or columns that are used to create values are called inputs. Splitting rules applied on data results in a hierarchy of branches within branches that form an inverted decision tree. Rules can be selected and used to display the decision tree. Selected rule provides a means to visually examine and describe the tree-like network of relationships that characterize the input and target values.

Decision tree is a nested hierarchy of branches. Each branch in the tree is referred as a node. A node with all its descendent segments forms an additional segment or a branch of that node. The bottom nodes of the decision tree are referred as “terminal nodes” or “leaves”. The decision rule provides each leaf with a unique path for data to enter the class that is defined as the leaf.

The figure shows the illustration of general decision tree.

General Form of Decision Tree

Neural Networks

Neural networks simulations seem to be recent development. Neural network has significant impact on business intelligence. It was established before the computer came into existence. Warren McCulloch, the neurophysiologist and Walter Pits2, the logician invented the first artificial neuron in 1943. The technology and the fund during those days did not allow them to progress much in their invention. Today the neural network field is benefited with a resurgence of interest and a corresponding increase in funding.

Neural networks operates differently and offers a much more powerful and deep data analysis. It is a very powerful tool of business intelligence. Implementing neural network, user does not have to locate interesting patterns and relations manually. This will “mine” the multidimensional data intelligently in a semi or full automatic process and extract the useful findings for business applications. Neural network is meant to learn and apply the knowledge to improve the business process.

Introduction to Neural Network

Neural network, also referred as an Artificial Neural Network (ANN) is an information processing system. Neural network is an inspiration from the way the biological systems like brain, process the information. It has the novel structure of information processing system.

There is no one perfect tool to use in data mining. It is question as what is available and what gives the best results. Many researches on data mining have confirmed that the neural network to be a promising data mining tool. Neural networks offer a qualitative method for business and economic systems that traditional quantitative tools in statistics and econometrics cannot quantify due to the complexity involved in translating the systems into precise mathematical functions. Neural network has the ability to detect and assimilate relationships between a large numbers of variables in a large mass of data.

Neural networks are among the data mining practitioners, particularly in medical research, finance and marketing. This is because they have proven their predictive power through comparison with other statistical techniques using real data sets. The design problem in neural network system need more research before they are widely accepted in industries.

Neural networks are programmed or trained to store, recognize, and associatively retrieve patterns or database entries to do the following:

· Solve combinatorial optimization problems

· Filter noise from measurement data

· Control ill-defined problems

Neural network on the whole estimates the sampled function when a user does not know the form of function. The two abilities of the neural network that is pattern recognition and function estimation makes it a very important utility in data mining. The need for automated processing becomes very clear when the data sets grow to massive sizes. A neural network with their model estimators and dual nature serves data mining in numerous ways.

Neural network is made up of a huge number of highly interconnected processing elements also known as neurones, working together to solve specific problem. Like humans, Artificial Neural Network also learns from examples. It can be configured for any specific applications through learning process. The learning process in Artificial Neural Network is similar to that in the biological system of human beings, which involves tuning to the synaptic connections that exists between the neurones.

Neural network has the ability to derive meaning from complicated or imprecise data. It can be used to extract and detect the complex patterns and trends that are very difficult to be noticed by either human or computer techniques. The trained neural network is considered as an expert in analyzing information given in any field.

A neural network must be configured such that the application of a set of inputs produces the desired set of outputs either by a direct method or through relaxation process. Neural network is referred as a non-linear statistical data modeling tool. They can be used to model complex relationships between inputs and outputs or to find patterns in data. Data warehousing firms are harvesting information from datasets in the process known as data mining using the neural network as a tool. The process of manipulation and cross-fertilization of the data helps the users to make more informed decision. Neural network consists of three essential components namely:

· The architecture or model

· The learning algorithm

· The activation function

Neural network based on the architecture provides associations, classifications, clusters, prediction and forecasting to the data mining industry. Financial forecasting on e of the application of data mining and it is significant practical interest. Due to the ability of neural network to mine the valuable information from a mass of history information and be efficiently used in financial areas and hence the applications of neural networks to financial forecasting have been very popular over the last few years. Some researchers have confirmed that neural network performs better than conventional statistical approach in financial forecasting and an excellent data mining tool. Neural network is not just a data mining tool, it can also be used to find patterns in the data and infer rules from them. It is also useful in providing information on associations, classifications, clusters, and forecasting.

Feed forward neural network consists of three layers. Therefore, it also referred as multilayered feed forward neural network. The three layers of the feed forward neural network are an input layer, hidden layer and output layer. Each layer consists of one or more processing element. Processing elements are the one that simulates neurons in the brain. It is often referred as neurons or nodes. Input to the processing element is given either from the outside world or the previous layer. The connection is established between the processing elements in each layer that have weight or parameters associated with them. The weights are adjusted to the processing elements during training. In this neural network, information travels only in the forward direction and there are no feedback loops. Figure

Shows neutral network technology.

Neural Network Topology

The process of training a feed forward neural network is given below:

· Input data is given to the network. It is propagated through the network until it reaches the output layer. This forward process produces a predicted output.

· The obtained predicted output is subtracted from the actual output and an error value for the networks is calculated.

· The neural network then uses supervised learning, in most of the cases back propagation is used to train the network. Back propagation is a learning algorithm for adjusting the weights. It starts with the weights between the output layer processing element and the last hidden layer processing element. It works backwards through the network.

· The forward process begins again when the back propagation is finished. This cycle is continued till the error between predicted and actual outputs is minimized.

Back propagation is a common method used for teaching artificial neural networks how to perform a given task. This is used in layered feed forward neural networks in which the neurons send signals in forward direction and errors are propagated backwards. This algorithm uses a supervised learning method which means that the user provides the algorithm with an example of inputs and outputs that network has to compute and then error which is the difference between the actual and expected results is calculated. The main idea of the back propagation algorithm is to reduce this error until the neural network learns the training data.

Advantages of Neural Network

The following are some of the advantages of neutral networks:

· Self organization: A neural network is capable of creating its own representation of the information given during the learning process.

· Real time operation: A neural network is capable of multitasking nearly in real time. A special hardware device is designed and manufactured which takes the real time operation advantage of neural network.

· Adaptive learning: A neural network is capable of learning how to perform tasks based on the data given in training or by initial experience.

· Fault Tolerance by Redundant Information Coding: Partial fault/damage in a network leads to the corresponding degradation of performance.

However, some network capabilities may possibly be retained even with major network damage.

Neural Network versus Conventional computers

Neural network adopts a different approach to solve the problem when compared to that of conventional computers. A conventional computer uses an algorithmic approach, where in a computer is given the steps to solve the problem. The computer will not be able to solve the problem until the steps are given. The operations of conventional computers are predictable as it follows the known and stated instructions to solve the given problem.

In neural network, information is processed in a similar way as in a human brain. It consists of numbers of highly interconnected processing neurones which work in parallel to solve the given problem. Neural network cannot be programmed, it learns from the example given to it. It should be noted that the examples given to the neural network should be chosen carefully; else the network will function incorrectly. The operations of neural network are unpredictable, as it finds out the way or procedure to solve the given problem by itself.

Neural networks and conventional algorithmic computers are not in contest but complement each other. Some tasks like arithmetic operations can be performed better using algorithmic approach where in conventional computers are useful. Some tasks are more suitable to neural networks. It is also possible to solve some tasks by the combinational usage of neural network and computer, where the computer monitors the neural network.

Hope you will like Series Business Intelligence – Tools & Theory series !

If you have not yet subscribe this Blog , Please subscribe it from “follow me” tab !

So that you will be updated @ real time and all updated knowledge in your mail daily for free without any RSS subscription OR news reading !!

Happy Learning and Sharing !!

For More information related to BI World visit our all Mentalist networks Blog

SQL Server Mentalist … SQL Learning Blog

Business Intelligence Mentalist … Business Intelligence World

Microsoft Mentalist … MVC,ASP.NET, WCF & LinQ

MSBI Mentalist … MS BI and SQL Server

NMUG Bloggers …Navi Mumbai User Group Blog

Architectural Shack … Architectural implementation and design patterns

DBA Mentalist …Advance SQL Server Blog

MVC Mentalist … MVC Learning Blog

Link Mentalist … Daily Best link @ your email

Infographics Mentalist … Image worth explaining thousand Words

Hadoop Mentalist … Blog on Big Data

BI Tools Analysis … BI Tools

Connect With me on

| Facebook |Twitter | LinkedIn| Google+ | Word Press | RSS | About Me |