Introduction to

Functional Networks

With Applications.

A Neural Based Paradigm.

E. Castillo,A. Cobo, J.M. Gutiérrez, and E. Pruneda.

Kluwer International Publishers

More information on Functional Networks

---

Table of Contents

1 Introduction to Neural Networks 1

• 1.1 Introduction 1
  1.2 Inspiration from Neuroscience 2
  1.3 Neural Network Components 3
  1.3.1 Network Architectures 4
  1.3.2 Neurons: Processing Units 5
  1.4 Learning 9
  1.4.1 Validation 9
  1.5 The Hopfield Neural Network 11
  1.6 Example: Storing and Retrieving Patterns 13
  1.7 Feed Forward Networks: Perceptrons 14
  1.7.1 Learning Algorithms 17
  1.7.2 Improvements and Modifications 18
  1.7.3 Examples 19
  1.8 Multi-layer Perceptrons 23
  1.8.1 The Backpropagation Learning Algorithm 25
  1.8.2 Backpropagation Algorithm 27
  1.9 Examples 28
  1.9.1 A Nonlinear Classification Problem 28
  1.9.2 Estimating a Function From Data 30
  1.9.3 Electric power consumption 32
  1.9.4 Chaotic Time Series: the Henon Map 32
  1.10 Competitive Neural Networks 36
  1.10.1 Example 37
  Exercises 39
  

2 Introduction to Functional Networks 43

• 2.1 Introduction 43
  2.2 Motivating Functional Networks 44
  2.3 Elements of a Functional Network 46
  2.4 Differences Between Neural and Functional Networks 49
  2.5 Working With Functional Networks 50
  2.6 An Introductory Example 52
  Exercises 57
  

3 Functional Equations 59

• 3.1 Introduction 59
  3.2 A First Definition and Some Examples of Functional Equations 60
  3.2.1 The Main Cauchy's Functional Equation 60
  3.2.2 The Main Pexider's Functional Equation 60
  3.3 Some Motivating Examples of Functional Equations 61
  3.3.1 Area of a rectangle 61
  3.3.2 Area of a trapezoid 62
  3.3.3 Simple Interest 64
  3.3.4 A Comparison of the Rectangle and the Interest Problems 65
  3.3.5 The Sum of Products Functional Equation 65
  3.3.6 The Synthesis of Judgements Functional Example 70
  3.4 Some Functional Equations in Functions of a Single Variable 72
  3.4.1 The Cauchy's Equations 72
  3.4.2 The Pexider's Equations 74
  3.4.3 Uniqueness 75
  3.4.4 The Uniqueness Model Functional Equation 76
  3.5 Some Functional Equations in Functions of a Several Variables 76
  3.6 The Generalized Associativity Functional Equation 76
  3.7 The Generalized Bisymmetry Functional Equation 77
  3.8 The Translation Equation 78
  3.9 Some General Methods for Solving Functional Equations 79
  3.9.1 Replacing Variables by Given Values 79
  3.9.2 Transforming one or several Variables 80
  3.9.3 Transforming one or several Functions 80
  3.9.4 Using a More General Functional Equation 81
  3.9.5 Treating Variables as Constants 82
  3.9.6 Separating Variables 83
  3.9.7 By Differentiation 83
  3.9.8 Mixed Methods 84
  Exercises 84
  

4 Some Functional Network Models 87

• 4.1 Introduction 87
  4.2 One-Layer Functional Networks 87
  4.3 The Uniqueness Model 93
  4.3.1 Simplification of the Model 93
  4.3.2 Uniqueness of Representation 94
  4.3.3 Learning the Model 95
  4.4 The Generalized Associativity Model 98
  4.4.1 Simplification of the Model 99
  4.4.2 Uniqueness of Representation 99
  4.4.3 Learning the Model 100
  4.5 The Separable Model 101
  4.5.1 Simplification of the Model 101
  4.5.2 Uniqueness of Representation 102
  4.5.3 Learning the Model 103
  4.5.4 An Example 103
  4.6 The Generalized bisymmetry Model 104
  4.6.1 Simplification of the Model 104
  4.6.2 Uniqueness of Representation 105
  4.7 Serial Functional Model 106
  4.7.1 Simplification of the Model 106
  4.7.2 Uniqueness of Representation 107
  4.7.3 Learning the Model 107
  4.7.4 An Iterator Example 108
  4.8 Independent Multiple Output Models 109
  4.8.1 Simplification of the Model 110
  4.9 Dependent Multiple Output Network I 111
  4.9.1 Simplification of the Model 111
  4.9.2 Uniqueness of Representation 113
  4.10 Dependent Multiple Output Network II 113
  4.10.1 Simplification of the Model 113
  4.10.2 An Example of Application 115
  Exercises 116
  

5 Model Selection 121

• 5.1 Introduction 121
  5.2 The Minimum Description Length Principle 122
  5.3 Encoding Integer and Real Numbers 123
  5.4 Selection of Models 125
  5.5 Application to Functional Networks 126
  Exercises 127
  

6 Applications to Time Series 129

• 6.1 Introduction 129
  6.2 Univariate Box-Jenkins Time Series Models 130
  6.3 Functional Networks and Univariate Models 134
  6.4 Applications to Box-Jenkins Models 136
  6.4.1 The AR(2) series 136
  6.5 Applications to Economic Problems 137
  6.5.1 The Consumer Price Index 138
  6.5.2 The Electric Power Consumption Data 139
  6.5.3 The International Airlines' Passengers Data 143
  6.5.4 The Spanish Data Example 144
  6.6 Applications to Chaotic Series 145
  6.6.1 The H\accent 19 e non series 146
  6.6.2 The Lozi Map 150
  6.6.3 The Holmes Map 152
  6.6.4 The Burger Map 156
  6.7 Applications to Noise Reduction and Information Masking 159
  6.7.1 Noise Reduction: The H\accent 19 e non Model Revisited 160
  6.7.2 Extracting Information Masked by Chaos 160
  6.8 Multivariate Box-Jenkins Time Series Models 165
  6.8.1 A two Dimensional Model: the Double Logistic Map. 166
  Exercises 168
  

7 Applications to Differential Equations 173

• 7.1 Introduction 173
  7.2 Differential, functional and difference equations. The equivalence problem 176
  7.2.1 From differential equations to functional equations 176
  7.2.2 From functional equations to difference equations 180
  7.2.3 From difference to differential equations 181
  7.3 Approximations using functional networks 184
  7.3.1 Equally spaced data 184
  7.3.2 Unequally spaced data 188
  7.4 Example of Application: The Beam Problem 190
  7.5 Appendix 197
  

8 Applications to CAD 199

• 8.1 Introduction 199
  8.2 Surfaces in Implicit Form 200
  8.2.1 Uniqueness of Representation 202
  8.2.2 Learning 204
  8.3 Surfaces in Explicit Form 206
  8.4 Surfaces in Parametric Form 211
  8.5 Further Remarks 215
  Exercises 216
  

9 Applications to Regression 219

• 9.1 Introduction 219
  9.2 Linear Regression Model 219
  9.2.1 Assumptions of the Linear Regression Model 220
  9.2.2 Linear Regression Model in Matrix Form 221
  9.2.3 The Extra Sum of Squares Principle 225
  9.2.4 Mallows' $C_p$ statistic 226
  9.3 Non-Linear Regression 230
  9.3.1 Assumptions of the Non-Linear Regression Model 231
  9.3.2 Estimation of the Model 232
  9.4 Functional Networks and Regression Models 232
  9.4.1 One example:Life Expectation 234
  Exercises 234
  

References 239 Index