Data Mining and Predictive Analysis

• Data Mining Tasks in Discovering Knowledge in Data
• Statistical Approaches to Estimation and Prediction
• Statistical Inference
• How Confident are We in Our Estimates?
• Confidence Interval Estimation of the Mean
• How to Reduce the Margin of Error
• Confidence Interval Estimation of the Proportion
• Hypothesis Testing for the Mean
• Assessing The Strength of Evidence Against The Null Hypothesis
• Using Confidence Intervals to Perform Hypothesis Tests
• Hypothesis Testing for The Proportion
• Reference
• The R Zone
• R Reference
• Exercises

2 LiveLab in this lesson — see the labs panel →

• Two-Sample t-Test for Difference in Means
• Two-Sample Z-Test for Difference in Proportions
• Test for the Homogeneity of Proportions
• Chi-Square Test for Goodness of Fit of Multinomial Data
• Analysis of Variance
• Reference
• The R Zone
• R Reference
• Exercises

5 LiveLab in this lesson — see the labs panel →

• Supervised Versus Unsupervised Methods
• Statistical Methodology and Data Mining Methodology
• Cross-Validation
• Overfitting
• Bias–Variance Trade-Off
• Balancing The Training Data Set
• Establishing Baseline Performance
• The R Zone
• R Reference
• Exercises

1 LiveLab in this lesson — see the labs panel →

• An Example of Simple Linear Regression
• Dangers of Extrapolation
• How Useful is the Regression? The Coefficient of Determination, r2
• Standard Error of the Estimate, s
• Correlation Coefficient r
• Anova Table for Simple Linear Regression
• Outliers, High Leverage Points, and Influential Observations
• Population Regression Equation
• Verifying The Regression Assumptions
• Inference in Regression
• t-Test for the Relationship Between x and y
• Confidence Interval for the Slope of the Regression Line
• Confidence Interval for the Correlation Coefficient ρ
• Confidence Interval for the Mean Value of y Given x
• Prediction Interval for a Randomly Chosen Value of y Given x
• Transformations to Achieve Linearity
• Box–Cox Transformations
• The R Zone
• R References
• Exercises

7 LiveLab in this lesson — see the labs panel →

• An Example of Multiple Regression
• The Population Multiple Regression Equation
• Inference in Multiple Regression
• Regression With Categorical Predictors, Using Indicator Variables
• Adjusting R2: Penalizing Models For Including Predictors That Are Not Useful
• Sequential Sums of Squares
• Multicollinearity
• Variable Selection Methods
• Gas Mileage Data Set
• An Application of Variable Selection Methods
• Using the Principal Components as Predictors in Multiple Regression
• The R Zone
• R References
• Exercises

10 LiveLab in this lesson — see the labs panel →

• Classification Task
• k-Nearest Neighbor Algorithm
• Distance Function
• Combination Function
• Quantifying Attribute Relevance: Stretching the Axes
• Database Considerations
• k-Nearest Neighbor Algorithm for Estimation and Prediction
• Choosing k
• Application of k-Nearest Neighbor Algorithm Using IBM/SPSS Modeler
• The R Zone
• R References
• Exercises

2 LiveLab in this lesson — see the labs panel →

• What is a Decision Tree?
• Requirements for Using Decision Trees
• Classification and Regression Trees
• C4.5 Algorithm
• Decision Rules
• Comparison of the C5.0 and CART Algorithms Applied to Real Data
• The R Zone
• R References
• Exercises

1 LiveLab in this lesson — see the labs panel →

• Input and Output Encoding
• Neural Networks for Estimation and Prediction
• Simple Example of a Neural Network
• Sigmoid Activation Function
• Back-Propagation
• Gradient-Descent Method
• Back-Propagation Rules
• Example of Back-Propagation
• Termination Criteria
• Learning Rate
• Momentum Term
• Sensitivity Analysis
• Application of Neural Network Modeling
• The R Zone
• R References
• Exercises

1 LiveLab in this lesson — see the labs panel →

• Simple Example of Logistic Regression
• Maximum Likelihood Estimation
• Interpreting Logistic Regression Output
• Inference: Are the Predictors Significant?
• Odds Ratio and Relative Risk
• Interpreting Logistic Regression for a Dichotomous Predictor
• Interpreting Logistic Regression for a Polychotomous Predictor
• Interpreting Logistic Regression for a Continuous Predictor
• Assumption of Linearity
• Zero-Cell Problem
• Multiple Logistic Regression
• Introducing Higher Order Terms to Handle Nonlinearity
• Validating the Logistic Regression Model
• WEKA: Hands-On Analysis Using Logistic Regression
• The R Zone
• R References
• Exercises

2 LiveLab in this lesson — see the labs panel →

• Bayesian Approach
• Maximum A Posteriori (MAP) Classification
• Posterior Odds Ratio
• Balancing The Data
• Naïve Bayes Classification
• Interpreting The Log Posterior Odds Ratio
• Zero-Cell Problem
• Numeric Predictors for Naïve Bayes Classification
• WEKA: Hands-on Analysis Using Naïve Bayes
• Bayesian Belief Networks
• Clothing Purchase Example
• Using The Bayesian Network to Find Probabilities
• The R Zone
• R References
• Exercises

3 LiveLab in this lesson — see the labs panel →

• Model Evaluation Techniques for the Description Task
• Model Evaluation Techniques for the Estimation and Prediction Tasks
• Model Evaluation Measures for the Classification Task
• Accuracy and Overall Error Rate
• Sensitivity and Specificity
• False-Positive Rate and False-Negative Rate
• Proportions of True Positives, True Negatives, False Positives, and False Negatives
• Misclassification Cost Adjustment to Reflect Real-World Concerns
• Decision Cost/Benefit Analysis
• Lift Charts and Gains Charts
• Interweaving Model Evaluation with Model Building
• Confluence of Results: Applying a Suite of Models
• The R Zone
• R References
• Exercises
• Hands-On Analysis

1 LiveLab in this lesson — see the labs panel →

• Decision Invariance Under Row Adjustment
• Positive Classification Criterion
• Demonstration Of The Positive Classification Criterion
• Constructing The Cost Matrix
• Decision Invariance Under Scaling
• Direct Costs and Opportunity Costs
• Case Study: Cost-Benefit Analysis Using Data-Driven Misclassification Costs
• Rebalancing as a Surrogate for Misclassification Costs
• The R Zone
• R References
• Exercises

1 LiveLab in this lesson — see the labs panel →

• Classification Evaluation Measures for a Generic Trinary Target
• Application of Evaluation Measures for Trinary Classification to the Loan Approval Problem
• Data-Driven Cost-Benefit Analysis for Trinary Loan Classification Problem
• Comparing Cart Models With and Without Data-Driven Misclassification Costs
• Classification Evaluation Measures for a Generic k-Nary Target
• Example of Evaluation Measures and Data-Driven Misclassification Costs for k-Nary Classification
• The R Zone
• R References
• Exercises

1 LiveLab in this lesson — see the labs panel →

• Review of Lift Charts and Gains Charts
• Lift Charts and Gains Charts Using Misclassification Costs
• Response Charts
• Profits Charts
• Return on Investment (ROI) Charts
• The R Zone
• R References
• Exercises
• Hands-On Exercises

• The Clustering Task
• Hierarchical Clustering Methods
• Single-Linkage Clustering
• Complete-Linkage Clustering
• k-Means Clustering
• Example of k-Means Clustering at Work
• Behavior of MSB, MSE, and Pseudo-F as the k-Means Algorithm Proceeds
• Application of k-Means Clustering Using SAS Enterprise Miner
• Using Cluster Membership to Predict Churn
• The R Zone
• R References
• Exercises
• Hands-On Analysis

3 LiveLab in this lesson — see the labs panel →

• Self-Organizing Maps
• Kohonen Networks
• Example of a Kohonen Network Study
• Cluster Validity
• Application of Clustering Using Kohonen Networks
• Interpreting The Clusters
• Using Cluster Membership as Input to Downstream Data Mining Models
• The R Zone
• R References
• Exercises

2 LiveLab in this lesson — see the labs panel →

• Rationale for BIRCH Clustering
• Cluster Features
• Cluster Feature TREE
• Phase 1: Building The CF Tree
• Phase 2: Clustering The Sub-Clusters
• Example of Birch Clustering, Phase 1: Building The CF Tree
• Example of BIRCH Clustering, Phase 2: Clustering The Sub-Clusters
• Evaluating The Candidate Cluster Solutions
• Case Study: Applying BIRCH Clustering to The Bank Loans Data Set
• The R Zone
• R References
• Exercises

• Rationale for Measuring Cluster Goodness
• The Silhouette Method
• Silhouette Example
• Silhouette Analysis of the IRIS Data Set
• The Pseudo-F Statistic
• Example of the Pseudo-F Statistic
• Pseudo-F Statistic Applied to the IRIS Data Set
• Cluster Validation
• Cluster Validation Applied to the Loans Data Set
• The R Zone
• R References
• Exercises

2 LiveLab in this lesson — see the labs panel →

• Affinity Analysis and Market Basket Analysis
• Support, Confidence, Frequent Itemsets, and the A Priori Property
• How Does The A Priori Algorithm Work (Part 1)? Generating Frequent Itemsets
• How Does The A Priori Algorithm Work (Part 2)? Generating Association Rules
• Extension From Flag Data to General Categorical Data
• Information-Theoretic Approach: Generalized Rule Induction Method
• Association Rules are Easy to do Badly
• How Can We Measure the Usefulness of Association Rules?
• Do Association Rules Represent Supervised or Unsupervised Learning?
• Local Patterns Versus Global Models
• The R Zone
• R References
• Exercises

1 LiveLab in this lesson — see the labs panel →

• The Segmentation Modeling Process
• Segmentation Modeling Using EDA to Identify the Segments
• Segmentation Modeling using Clustering to Identify the Segments
• The R Zone
• R References
• Exercises

1 LiveLab in this lesson — see the labs panel →

• Rationale for Using an Ensemble of Classification Models
• Bias, Variance, and Noise
• When to Apply, and not to apply, Bagging
• Bagging
• Boosting
• Application of Bagging and Boosting Using IBM/SPSS Modeler
• References
• The R Zone
• R Reference
• Exercises

• Simple Model Voting
• Alternative Voting Methods
• Model Voting Process
• An Application of Model Voting
• What is Propensity Averaging?
• Propensity Averaging Process
• An Application of Propensity Averaging
• The R Zone
• R References
• Exercises
• Hands-On Analysis

• Introduction To Genetic Algorithms
• Basic Framework of a Genetic Algorithm
• Simple Example of a Genetic Algorithm at Work
• Modifications and Enhancements: Selection
• Modifications and Enhancements: Crossover
• Genetic Algorithms for Real-Valued Variables
• Using Genetic Algorithms to Train a Neural Network
• WEKA: Hands-On Analysis Using Genetic Algorithms
• The R Zone
• R References
• Exercises

1 LiveLab in this lesson — see the labs panel →

• Need for Imputation of Missing Data
• Imputation of Missing Data: Continuous Variables
• Standard Error of the Imputation
• Imputation of Missing Data: Categorical Variables
• Handling Patterns in Missingness
• Reference
• The R Zone
• R References

• Cross-Industry Standard Practice for Data Mining
• Business Understanding Phase
• Data Understanding Phase, Part 1: Getting a Feel for the Data Set
• Data Preparation Phase
• Data Understanding Phase, Part 2: Exploratory Data Analysis

• Partitioning the Data
• Developing the Principal Components
• Validating the Principal Components
• Profiling the Principal Components
• Choosing the Optimal Number of Clusters Using Birch Clustering
• Choosing the Optimal Number of Clusters Using k-Means Clustering
• Application of k-Means Clustering
• Validating the Clusters
• Profiling the Clusters

• Do You Prefer The Best Model Performance, Or A Combination Of Performance And Interpretability?
• Modeling And Evaluation Overview
• Cost-Benefit Analysis Using Data-Driven Costs
• Variables to be Input To The Models
• Establishing The Baseline Model Performance
• Models That Use Misclassification Costs
• Models That Need Rebalancing as a Surrogate for Misclassification Costs
• Combining Models Using Voting and Propensity Averaging
• Interpreting The Most Profitable Model

• Variables to be Input to the Models
• Models that use Misclassification Costs
• Models that Need Rebalancing as a Surrogate for Misclassification Costs
• Combining Models using Voting and Propensity Averaging
• Lessons Learned
• Conclusions

• Data Summarization and Visualization
• Part 1: Summarization 1: Building Blocks Of Data Analysis
• Part 2: Visualization: Graphs and Tables For Summarizing And Organizing Data
• Part 3: Summarization 2: Measures Of Center, Variability, and Position
• Part 4: Summarization And Visualization Of Bivariate Relationships