Neural Networks for Applied Sciences and Engineering: From Fundamentals to Complex Pattern Recognition

FROM DATA TO MODELS: COMPLEXITY AND CHALLENGES IN UNDERSTANDING BIOLOGICAL, ECOLOGICAL, AND NATURAL SYSTEMS Introduction Layout of the Book FUNDAMENTALS OF NEURAL NETWORKS AND MODELS FOR LINEAR DATA ANALYSIS Introduction and Overview Neural Networks and Their Capabilities Inspirations from Biology Modeling Information Processing in Neurons Neuron Models and Learning Strategies Models for Prediction and Classification Practical Examples of Linear Neuron Models on Real Data Comparison with Linear Statistical Methods Summary Problems NEURAL NETWORKS FOR NONLINEAR PATTERN RECOGNITION Overview and Introduction Nonlinear Neurons Practical Example of Modeling with Nonlinear Neurons Comparison with Nonlinear Regression One-Input Multilayer Nonlinear Networks Two-Input Multilayer Perceptron Network Case Studies on Nonlinear Classification and Prediction with Nonlinear Networks Multidimensional Data Modeling with Nonlinear Multilayer Perceptron Networks Summary Problems LEARNING OF NONLINEAR PATTERNS BY NEURAL NETWORKS Introduction and Overview Supervised Training of Networks for Nonlinear Pattern Recognition Gradient Descent and Error Minimization Backpropagation Learning and Illustration with an Example and Case Study Delta-Bar-Delta Learning and Illustration with an Example and Case Study Steepest Descent Method Presented with an Example Comparison of First Order Learning Methods Second-Order Methods of Error Minimization and Weight Optimization Comparison of First Order and Second Order Learning Methods Illustrated through an Example Summary Problems IMPLEMENTATION OF NEURAL NETWORK MODELS FOR EXTRACTING RELIABLE PATTERNS FROM DATA Introduction and Overview Bias-Variance Tradeoff Illustration of Early Stopping and Regularization Improving Generalization of Neural Networks Network structure Optimization and Illustration with Examples Reducing Structural Complexity of Networks by Pruning Demonstration of Pruning with Examples Robustness of a Network to Perturbation of Weights Illustrated Using an Example Summary Problems DATA EXPLORATION, DIMENSIONALITY REDUCTION, AND FEATURE EXTRACTION Introduction and Overview Data Visualization Presented on Example Data Correlation and Covariance between Variables Normalization of Data Example Illustrating Correlation, Covariance and Normalization Selecting Relevant Inputs Dimensionality Reduction and Feature Extraction Example Illustrating Input Selection and Feature Extraction Outlier Detection Noise Case Study: Illustrating Input Selection and Dimensionality Reduction for a Practical Problem Summary Problems ASSESSMENT OF UNCERTAINTY OF NEURAL NETWORK MODELS USING BAYESIAN STATISTICS Introduction and Overview Estimating Weight Uncertainty Using Bayesian Statistics Case study Illustrating Weight Probability Distribution Assessing Uncertainty of Neural Network Outputs Using Bayesian Statistics Case Study Illustrating Uncertainty Assessment of Output Errors Assessing the Sensitivity of Network Outputs to Inputs Case Study Illustrating Uncertainty Assessment of Network Sensitivity to Inputs Summary Problems DISCOVERING UNKNOWN CLUSTERS IN DATA WITH SELF-ORGANIZING MAPS Introduction and Overview Structure of Unsupervised Networks for Clustering Multidimensional Data Learning in Unsupervised Networks Implementation of Competitive Learning Illustrated Through Examples Self-Organizing Feature Maps Examples and Case Studies Using Self-Organizing Maps on Multi-Dimensional Data Map Quality and Features Presented through Examples Illustration of Forming Clusters on the Map and Cluster Characteristics Map Validation and an Example Evolving Self-Organizing Maps Examples Illustrating Various Evolving Self Organizing Maps Summary Problems NEURAL NETWORKS FOR TIME-SERIES FORECASTING Introduction and Overview Linear Forecasting of Time-Series with Statistical and Neural Network Models Example Case Study Neural Networks for Nonlinear Time-Series Forecasting Example Case Study Hybrid Linear (ARIMA) and Nonlinear Neural Network Models Example Case Study Automatic Generation of Network Structure Using Simplest Structure Concept-Illustrated Through Practical Application Case Study Generalized Neuron Network and Illustration Through Practical Application Case Study Dynamically Driven Recurrent Networks Practical Application Case Studies Bias and Variance in Time-Series Forecasting Illustrated Through an Example Long-Term Forecasting and a Case study Input Selection for Time-Series Forecasting Case study for Input Selection Summary Problems