The data and decision analytic process is a path leading from the larva of data to the butterfly of knowledge, understanding, and insight.
Before starting to work on data analytic and associated decision analytic projects it is necessary, in order to ensure the quality of the results, to
- define an orderly data analytic and decision analytic process
- select methods for executing the process
- select software packages for implementing the methods
In order to ensure the reliability of the answers/solutions and the quality of the decisions made and actions taken, it is necessary to adhere to the analytic process in an orderly manner and apply the methods and the software packages in a competent manner
Before starting an analytic process it is necessary to state the question/problem under consideration and ask the following preliminary questions:
- Is the answer/solution considered known?
- Is the the answer/solution based on sufficiently recent/reliable data?
- Was the analysis performed in a competent/reliable manner?
- Is the results of the analysis presented/visualized in such a way that it sufficiently increases the understanding and insight of the target group ?
- Do the results of the analysis, their presentation/visualization, and the resulting understanding and insight form a sufficently firm basis for decision making and action?
If any of the answers are no there may be a reason to go ahead with the analytic and decision analytic process. If all the the answers are yes it is unnecessary to go ahead with the process unless you are confident that you can improve the results materially or introduce your particular results to a new or wider audience. But beware of hubris.
The main stages of a combined data analytic and decision analytic process
- State an important question/problem
- Data analysis
- Select data relevant to answering the questions or solving the problems
- Prepare the data for analysis. Employ visualization during preparation
- Analyze the data – Increase knowledge about the past, present, and future state of the system generating the data – Increase knowledge about individual variables and the relationship between variables. Employ visualization extensively during analysis
- Descriptive data analysis
- Exploratory data analysis
- Confirmatory data analysis
- Predictive data analysis
- Present/visualize the results of the analysis
- Evaluate the results of the analysis – Have the original questions been answered?
- Decision analysis
- Make decisions based on the results of the analysis
- Implement decisions – Act
- Present/visualize the results of the actions
- Evaluate the results of the actions – Have the original problems originally posed been solved?
- Reiterate the process or its individual stages as necessary