Neural Network Article Index for
Neural 		Shopping
Neural 		Articles about
Neural Network 		Website Links For
Neural 		 

Information About

Neural Network
  CATEGORIES ABOUT NEURAL NETWORK
   
computational neuroscience
   
psychology
   
neural networks
   
network architecture
   
networks
   
information, knowledge, and uncertainty
   
APPAREL
BABY
BEAUTY
BOOKS
CAR TOYS
CELL PHONES
DVD'S
ELECTRONICS
GOURMET FOOD
GROCERIES
HEALTH & PERSONAL
HOME & GARDEN
JEWELRY
MUSIC
MUSIC INSTRUMENTS
OFFICE PRODUCTS
SOFTWARE
SPORTING GOODS
TOOLS & HARDWARE
TOYS
VIDEO GAMES
SHOPPING HOME
MORE SHOPPING...



A neural network is an interconnected group of Biological Neurons . In modern usage the term can also refer to Artificial Neural Network s, which are constituted of Artificial Neuron s. Thus the term 'Neural Network' specifies two distinct concepts:
# A Biological Neural Network is a Plexus of connected or functionally related neurons in the Peripheral Nervous System or the Central Nervous System . In the field of Neuroscience , it most often refers to a group of neurons from a nervous system that are suited for laboratory analysis.
# Artificial Neural Network s were designed to model some properties of biological neural networks, though most of the applications are of technical nature as opposed to Cognitive Model s.

Please see the corresponding articles for details on artificial neural networks or biological neural networks. This article focuses on the relationship between the two concepts.

Neural networks are made of units that are often assumed to be simple in the sense that their State can be described by single numbers, their "activation" values. Each unit generates an output signal based on its activation. Units are connected to each other very specifically, each connection having an individual "weight" (again described by a single number). Each unit sends its output value to all other units to which they have an outgoing connection. Through these connections, the output of one unit can influence the activations of other units. The unit receiving the connections calculates its activation by taking a Weighted Sum of the input signals (i.e. it multiplies each input signal with the weight that corresponds to that connection and adds these products). The output is determined by the activation function based on this activation (e.g. the unit generates output or "fires" if the activation is above a threshold value). Networks learn by changing the weights of the connections.


CHARACTERIZATION

In general, a neural network is composed of a group or groups of physically connected or functionally associated neurons. A single neuron can be connected to many other neurons and the total number of neurons and connections in a network can be extremely large. Connections, called Synapses are usually formed from Axons to Dendrites , though dendrodentritic microcircuits p.666 and other connections are possible. Apart from the electrical signalling, there are other forms of signaling that arise from Neurotransmitter diffusion, which have an effect on electrical signaling. Thus, like other Biological Networks , neural networks are extremely complex. While a detailed description of neural systems seems currently unattainable, progress is made towards a better understanding of basic mechanisms.

Artificial Intelligence and Cognitive Modeling try to simulate some properties of neural networks. While similar in their techniques, the former has the aim of solving particular tasks, while the latter aims to build mathematical models of biological neural systems.

In the Artificial Intelligence field, artificial neural networks have been applied successfully to Speech Recognition , Image Analysis and adaptive Control , in order to construct Software Agents (in Computer And Video Games ) or Autonomous Robot s. Most of the currently employed artificial neural networks for artificial intelligence are based on Statistical Estimation , Optimisation and Control Theory .

The Cognitive Modelling field is the physical or mathematical modelling of the behaviour of neural systems; ranging from the individual neural level (e.g. modelling the spike response curves of neurons to a stimulus), through the neural cluster level (e.g. modelling the release and effects of dopamine in the basal ganglia) to the complete organism (e.g. behavioural modelling of the organism's response to stimuli).


THE BRAIN, NEURAL NETWORKS AND COMPUTERS

While historically the brain has been viewed as a type of computer, and vice-versa, this is true only in the loosest sense. Computers are not models of the brain (even though it is possible to describe a logical process as a computer program, or to simulate a brain using a computer) as they were not created with that purpose in mind.

However, neural networks used in artificial intelligence have traditionally been viewed as simplified models of neural processing in the brain. The question of what is the degree of complexity and the properties that individual neural elements should have in order to reproduce something resembling animal intelligence is a subject of current research in theoretical neuroscience.


NEURAL NETWORKS AND ARTIFICIAL INTELLIGENCE

''Main article: Artificial Neural Network ''

Background


Neural network models in artificial intelligence are usually referred to as artificial neural networks (ANNs); these are essentially simple mathematical models defining a function f : X ightarrow {Y} . The epithet ''network'' is used because this function is decomposable into a number of simpler, interconnected elements.

A particular type of ANN model corresponds to a ''class'' of such functions. What has attracted the most interest in neural networks is the possibility of ''learning''.

  • \in F which solves the task in an ''optimal sense''.


  • , C(f^---) \leq C(f) orall f \in F


The Cost Function C is an important concept in learning, as it is a measure of how far away we are from an optimal solution to the problem that we want to solve. Learning algorithms search through the solution space in order to find a function that has the smallest possible cost.


Learning paradigms

There are three major learning paradigms, each corresponding to a particular abstract learning task. These are Supervised Learning , Unsupervised Learning and Reinforcement Learning . Usually any given type of network architecture can be employed in any of those tasks.


Supervised learning

In Supervised Learning , we are given a set of example pairs (x, y), x \in X, y \in Y and the aim is to find a function f in the allowed class of functions that matches the examples. In other words, we wish to ''infer'' the mapping implied by the data and the cost function is related to the mismatch between our mapping and the data.


Unsupervised learning

In Unsupervised Learning we are given some data x, and the cost function to be minimised can be any function of the data x and the network's output, f. The cost function is determined by the task formulation. Most applications fall within the domain of Estimation Problems such as Statistical Modelling , Compression , Filtering , Blind Source Separation and Clustering .


Reinforcement learning

In Reinforcement Learning , data x is usually not given, but generated by an agent's interactions with the environment. At each point in time t, the agent performs an action y_t and the environment generates an observation x_t and an instantaneous cost c_t, according to some (usually unknown) dynamics. The aim is to discover a ''policy'' for selecting actions that minimises some measure of a long-term cost, i.e. the expected cumulative cost. The environment's dynamics and the long-term cost for each policy are usually unknown, but can be estimated. ANNs are frequently used in reinforcement learning as part of the overall algorithm. Tasks that fall within the paradigm of reinforcement learning are Control problems, Games and other Sequential Decision Making tasks.


Learning algorithms

There are numerous algorithms available for training neural network models; most of them can be viewed as a straightforward application of Optimization theory and Statistical Estimation .

Most of the algorithms used in training artificial neural networks are employing some form of Gradient Descent . This is done by simply taking the derivative of the cost function with respect to the network parameters and then changing those parameters in a Gradient-related direction.

Evolutionary Computation methods, Simulated Annealing , Expectation Maximization and Non-parametric Methods are among other commonly used methods for training neural networks. See also Machine Learning .


Theoretical properties


Capacity

Certain theoretical models of neural networks have been analyzed in a way that allows properties such as their maximum storage capacity to be calculated independently of any learning algorithm. Various techniques originally developed for studying disordered magnetic systems ( Spin Glasses ) have been successfully applied to simple neural network architectures, such as the perceptron. Influential work by E. Gardner and B. Derrida has revealed many interesting properties about perceptrons with real-valued synaptic weights, while later work by W. Krauth and M. Mezard has extended these principles to binary-valued synapses.


Types of Artificial Neural Network s

See Artificial Neural Network for a discussion on the various types of neural networks.


NEURAL NETWORKS AND NEUROSCIENCE

Theoretical and computational neuroscience is the field concerned with the theoretical analysis and computational modeling of biological neural systems.
Since neural systems are intimately related to cognitive processes and behaviour, the field is closely related to cognitive and behavioural modeling.

The aim of the field is to create models of biological neural systems in order to understand how biological systems work. To gain this understanding, neuroscientists strive to make a link between observed biological processes (data), biologically plausible mechanisms for neural processing and learning (biological neural network models) and theory (statistical learning theory and information theory).


Types of models

Many models are used in the field, each defined at a different level of abstraction and trying to model different aspects of neural systems. They range from models of the short-term behaviour of individual neurons, through models of how the dynamics of neural circuitry arise from interactions between individual neurons, to models of how behaviour can arise from abstract neural modules that represent complete subsystems. These include models of the long-term and short-term plasticity of neural systems and its relation to learning and memory, from the individual neuron to the system level.


Current research

While initially research had been concerned mostly with the electrical characteristics of neurons, a particularly important part of the investigation in recent years has been the exploration of the role of Neuromodulators such as Dopamine , Acetylcholine , and Serotonin on behaviour and learning.

References