Home Contact Sitemap Bookmark us

• Home
• Services
• Products
• Biometrics
• eFinance
• Company

Neural Network Frequently Asked Questions (FAQ)

The information displayed here is part of the FAQ monthly posted to comp.ai.neural-nets
Last modification: 12-01-1994

1. What is the comp.ai.neural-nets newsgroup for ? How should it be used ?
2. What is a neural network (NN) ?
3. What can you do with a Neural Network and what not ?
4. Who is concerned with Neural Networks ?
   
5. What does 'backprop' mean ?
6. How many learning methods for NNs exist ? Which ?
7. What about Genetic Algorithms ?
8. What about Fuzzy Logic ?
   
9. Good introductory literature about Neural Networks ?
10. Any journals and magazines about Neural Networks ?
11. The most important conferences concerned with Neural Networks ?
12. Neural Network Associations ?
13. Other sources of information about NNs ?
    
14. Freely available software packages for NN simulation ?
    • Rochester Connectionist Simulator
    • UCLA-SFINX
    • NeurDS
    • PlaNet5.7
    • GENESIS
    • Mactivation
    • Cascade Correlation Simulator
    • Quickprop
    • DartNet
    • SNNS
    • Aspirin/MIGRAINES
    • Adaptive Logic Network kit
    • NeuralShell
    • PDP
    • Xerion
    • Neocognitron simulator
    • Multi-Module Neural Computing Environment (MUME)
    • LVQ_PAK, SOM_PAK
    • SESAME
    • Nevada Backpropagation (NevProp)
    • Fuzzy ARTmap
    • PYGMALION
    • Basis of AI backprop
15. Commercial software packages for NN simulation ?
16. Neural Network hardware ?
    
17. Databases for experimentation with NNs ?
    
18. Acknowledgements

A1) What is the comp.ai.neural-nets newsgroup for ?

1. Requests.
   Requests are articles of the form

   "I am looking for X"

   where X is something public like a book, an article, a piece of software. The most important about such a request is to be as specific as possible!
   If multiple different answers can be expected, the person making the request should prepare to make a summary of the answers he/she got and announce to do so with a phrase like

   "Please reply by email, I'll summarize to the group"

   at the end of the posting.
   The Subject line of the posting should then be something like

   "Request: X"

2. Questions.
   As opposed to requests, question ask for a larger piece of information or a more or less detailed explanation of something. To avoid lots of redundant traffic it is important that the poster provides with the question all information s/he already has about the subject asked and state the actual question as precise and narrow as possible.
   The poster should prepare to make a summary of the answers he/she got and announce to do so with a phrase like

   "Please reply by email, I'll summarize to the group"

   at the end of the posting.
   The Subject line of the posting should be something like

   "Question: this-and-that"

   or have the form of a question (i.e., end with a question mark)
   
3. Answers
   These are reactions to questions or requests. As a rule of thumb articles of type "answer" should be rare. Ideally, in most cases either the answer is too specific to be of general interest (and should thus be e-mailed to the poster) or a summary was announced with the question or request (and answers should thus be e-mailed to the poster).
   The subject lines of answers are automatically adjusted by the news software. Note that sometimes longer threads of discussion evolve from an answer to a question or request. In this case posters should change the subject line suitably as soon as the topic goes too far away from the one announced in the original subject line. You can still carry along the old subject in parentheses in the form

   "Subject: <...new subject...> (was: <...old subject...>)

4. Summaries
   In all cases of requests or questions the answers for which can be assumed to be of some general interest, the poster of the request or question shall summarize the ansers he/she received. Such a summary should be announced in the original posting of the question or request with a phrase like

   "Please answer by email, I'll summarize"

   In such a case, people who answer to a question should NOT post their answer to the newsgroup but instead mail them to the poster of the question who collects and reviews them. After about 5 to 20 days after the original posting, its poster should make the summary of answers and post it to the newsgroup.
   Some care should be invested into a summary:
   1. simple concatenation of all the answers is not enough: instead, redundancies, irrelevancies, verbosities, and errors should be filtered out (as good as possible)
   2. the answers should be separated clearly
   3. the contributors of the individual answers should be identifiable (unless they requested to remain anonymous [yes, that happens])
   4. the summary should start with the "quintessence" of the answers, as seen by the original poster
   5. A summary should, when posted, clearly be indicated to be one by giving it a Subject line starting with "SUMMARY:"
   Note that a good summary is pure gold for the rest of the newsgroup community, so summary work will be most appreciated by all of us. (Good summaries are more valuable than any moderator ! :-> )
   
5. Announcements
   Some articles never need any public reaction. These are called announcements (for instance for a workshop, conference or the availability of some technical report or software system).
   Announcements should be clearly indicated to be such by giving them a subject line of the form

   "Announcement: this-and-that"

6. Reports
   Sometimes people spontaneously want to report something to the newsgroup. This might be special experiences with some software, results of own experiments or conceptual work, or especially interesting information from somewhere else.
   Reports should be clearly indicated to be such by giving them a subject line of the form

   "Report: this-and-that"

7. Discussions
   An especially valuable possibility of Usenet is of course that of discussing a certain topic with hundreds of potential participants. All traffic in the newsgroup that can not be subsumed under one of the above categories should belong to a discussion.
   If somebody explicitly wants to start a discussion, he/she can do so by giving the posting a subject line of the form

   "Subject: Discussion: this-and-that"

   It is quite difficult to keep a discussion from drifting into chaos, but, unfortunately, as many many other newsgroups show there seems to be no secure way to avoid this. On the other hand, comp.ai.neural-nets has not had many problems with this effect in the past, so let's just go and hope... :->

A2) What is a neural network (NN) ?

A vague description is as follows:

An ANN is a network of many very simple processors ("units"), each possibly having a (small amount of) local memory. The units are connected by unidirectional communication channels ("connections"), which carry numeric (as opposed to symbolic) data. The units operate only on their local data and on the inputs they receive via the connections.

The design motivation is what distinguishes neural networks from other mathematical techniques:

A neural network is a processing device, either an algorithm, or actual hardware, whose design was motivated by the design and functioning of human brains and components thereof.

A3) What can you do with a Neural Network and what not ?

A4) Who is concerned with Neural Networks ?

• Computer scientists want to find out about the properties of non-symbolic information processing with neural nets and about learning systems in general.
  
• Engineers of many kinds want to exploit the capabilities of neural networks on many areas (e.g. signal processing) to solve their application problems.
  
• Cognitive scientists view neural networks as a possible apparatus to describe models of thinking and conscience (High-level brain function).
  
• Neuro-physiologists use neural networks to describe and explore medium-level brain function (e.g. memory, sensory system, motorics).
  
• Physicists use neural networks to model phenomena in statistical mechanics and for a lot of other tasks.
  
• Biologists use Neural Networks to interpret nucleotide sequences.
  
• Philosophers and some other people may also be interested in Neural Networks for various reasons.
  

A5) What does 'backprop' mean ?

"You are almost not knowing what you are actually doing when using backpropagation" :-)

Rumelhart, D. E. and McClelland, J. L. (1986): Parallel Distributed Processing: Explorations in the Microstructure of Cognition (volume 1, pp 318-362). The MIT Press.

A6) How many learning methods for NNs exist ? Which ?

• In supervised learning, there is a "teacher" who in the learning phase "tells" the net how well it performs ("reinforcement learning") or what the correct behavior would have been ("fully supervised learning").
• In unsupervised learning the net is autonomous: it just looks at the data it is presented with, finds out about some of the properties of the data set and learns to reflect these properties in its output. What exactly these properties are, that the network can learn to recognise, depends on the particular network model and learning method.

1. UNSUPERVISED LEARNING (i.e. without a "teacher"):
   • Feedback Nets:
     • Additive Grossberg (AG)
     • Shunting Grossberg (SG)
     • Binary Adaptive Resonance Theory (ART1)
     • Analog Adaptive Resonance Theory (ART2, ART2a)
     • Discrete Hopfield (DH)
     • Continuous Hopfield (CH)
     • Discrete Bidirectional Associative Memory (BAM)
     • Temporal Associative Memory (TAM)
     • Adaptive Bidirectional Associative Memory (ABAM)
     • Kohonen Self-organizing Map (SOM)
     • Kohonen Topology-preserving Map (TPM)
   • Feedforward-only Nets:
     • Learning Matrix (LM)
     • Driver-Reinforcement Learning (DR)
     • Linear Associative Memory (LAM)
     • Optimal Linear Associative Memory (OLAM)
     • Sparse Distributed Associative Memory (SDM)
     • Fuzzy Associative Memory (FAM)
     • Counterprogation (CPN)
2. SUPERVISED LEARNING (i.e. with a "teacher"):
   • Feedback Nets:
     • Brain-State-in-a-Box (BSB)
     • Fuzzy Congitive Map (FCM)
     • Boltzmann Machine (BM)
     • Mean Field Annealing (MFT)
     • Recurrent Cascade Correlation (RCC)
     • Learning Vector Quantization (LVQ)
   • Feedforward-only Nets:
     • Perceptron
     • Adaline, Madaline
     • Backpropagation (BP)
     • Cauchy Machine (CM)
     • Adaptive Heuristic Critic (AHC)
     • Time Delay Neural Network (TDNN)
     • Associative Reward Penalty (ARP)
     • Avalanche Matched Filter (AMF)
     • Backpercolation (Perc)
     • Artmap
     • Adaptive Logic Network (ALN)
     • Cascade Correlation (CasCor)

A7) What about Genetic Algorithms ?

A GA is an optimization program
that starts with some encoded procedure, (Creation of Life)
mutates it stochastically, (Get cancer or so)
and uses a selection process (Darwinism)
to prefer the mutants with high fitness
and perhaps a recombination process (Make babies)
to combine properties of (preferably) the succesful mutants.

A8) What about Fuzzy Logic ?

• Klir, G.J. and Folger, T.A., Fuzzy Sets, Uncertainty, and Information, Prentice-Hall, Englewood Cliffs, N.J., 1988.
• Kosko, B., Neural Networks and Fuzzy Systems, Prentice Hall, Englewood Cliffs, NJ, 1992.

A9) Good introductory literature about Neural Networks ?

1. The best:
   • Hecht-Nielsen, R. (1990). Neurocomputing. Addison Wesley.
     Comments: "A good book", "comprises a nice historical overview and a chapter about NN hardware. Well structured prose. Makes important concepts clear."
     
   • Hertz, J., Krogh, A., and Palmer, R. (1991). Introduction to the Theory of Neural Computation. Addison-Wesley: Redwood City, California. ISBN 0-201-50395-6 (hardbound) and 0-201-51560-1 (paperbound)
     Comments: "My first impression is that this one is by far the best book on the topic. And it's below $30 for the paperback."; "Well written, theoretical (but not overwhelming)"; It provides a good balance of model development, computational algorithms, and applications. The mathematical derivations are especially well done"; "Nice mathematical analysis on the mechanism of different learning algorithms"; "It is NOT for mathematical beginner. If you don't have a good grasp of higher level math, this book can be really tough to get through."
   
   
2. Books for the beginner:
   • Aleksander, I. and Morton, H. (1990). An Introduction to Neural Computing. Chapman and Hall. (ISBN 0-412-37780-2).
     Comments: "This book seems to be intended for the first year of university education."
     
   • Beale, R. and Jackson, T. (1990). Neural Computing, an Introduction. Adam Hilger, IOP Publishing Ltd : Bristol. (ISBN 0-85274-262-2).
     Comments: "It's clearly written. Lots of hints as to how to get the adaptive models covered to work (not always well explained in the original sources). Consistent mathematical terminology. Covers perceptrons, error-backpropagation, Kohonen self-org model, Hopfield type models, ART, and associative memories."
     
   • Dayhoff, J. E. (1990). Neural Network Architectures: An Introduction. Van Nostrand Reinhold: New York.
     Comments: "Like Wasserman's book, Dayhoff's book is also very easy to understand".
     
   • McClelland, J. L. and Rumelhart, D. E. (1988). Explorations in Parallel Distributed Processing: Computational Models of Cognition and Perception (software manual). The MIT Press.
     Comments: "Written in a tutorial style, and includes 2 diskettes of NN simulation programs that can be compiled on MS-DOS or Unix (and they do too !)"; "The programs are pretty reasonable as an introduction to some of the things that NNs can do."; "There are two editions of this book. One comes with disks for the IBM PC, the other comes with disks for the Macintosh".
     
   • McCord Nelson, M. and Illingworth, W.T. (1990). A Practical Guide to Neural Nets. Addison-Wesley Publishing Company, Inc. (ISBN 0-201-52376-0).
     Comments: "No formulas at all( ==> no good)"; "It does not have much detailed model development (very few equations), but it does present many areas of application. It includes a chapter on current areas of research. A variety of commercial applications is discussed in chapter 1. It also includes a program diskette with a fancy graphical interface (unlike the PDP diskette)".
     
   • Orchard, G.A. & Phillips, W.A. (1991). Neural Computation: A Beginner's Guide. Lawrence Earlbaum Associates: London.
     Comments: "Short user-friendly introduction to the area, with a non-technical flavour. Apparently accompanies a software package, but I haven't seen that yet".
     
   • Wasserman, P. D. (1989). Neural Computing: Theory & Practice. Van Nostrand Reinhold: New York. (ISBN 0-442-20743-3).
     Comments: "Wasserman flatly enumerates some common architectures from an engineer's perspective ('how it works') without ever addressing the underlying fundamentals ('why it works') - important basic concepts such as clustering, principal components or gradient descent are not treated. It's also full of errors, and unhelpful diagrams drawn with what appears to be PCB board layout software from the '70s. For anyone who wants to do active research in the field I consider it quite inadequate"; "Okay, but too shallow"; "Quite easy to understand"; "The best bedtime reading for Neural Networks. I have given this book to numerous collegues who want to know NN basics, but who never plan to implement anything. An excellent book to give your manager."
   
   
3. The classics:
   • Kohonen, T. (1984). Self-organization and Associative Memory. Springer-Verlag: New York. (2nd Edition: 1988; 3rd edition: 1989).
     Comments: "The section on Pattern mathematics is excellent."
     
   • Rumelhart, D. E. and McClelland, J. L. (1986). Parallel Distributed Processing: Explorations in the Microstructure of Cognition (volumes 1 & 2). The MIT Press.
     Comments: "As a computer scientist I found the two Rumelhart and McClelland books really heavy going and definitely not the sort of thing to read if you are a beginner."; "It's quite readable, and affordable (about $65 for both volumes)."; "THE Connectionist bible.".
   
   
4. Introductory journal articles:
   • Hinton, G. E. (1989). Connectionist learning procedures. Artificial Intelligence, Vol. 40, pp. 185--234.
     Comments: "One of the better neural networks overview papers, although the distinction between network topology and learning algorithm is not always very clear. Could very well be used as an introduction to neural networks."
     
   • Knight, K. (1990). Connectionist, Ideas and Algorithms. Communications of the ACM. November 1990. Vol.33 nr.11, pp 59-74.
     Comments:"A good article, while it is for most people easy to find a copy of this journal."
     
   • Kohonen, T. (1988). An Introduction to Neural Computing. Neural Networks, vol. 1, no. 1. pp. 3-16.
     Comments: "A general review".
   
   
5. Not-quite-so-introductory literature:
   • Anderson, J. A. and Rosenfeld, E. (Eds). (1988). Neurocomputing: Foundations of Research. The MIT Press: Cambridge, MA.
     Comments: "An expensive book, but excellent for reference. It is a collection of reprints of most of the major papers in the field."
     
   • Anderson, J. A., Pellionisz, A. and Rosenfeld, E. (Eds). (1990). Neurocomputing 2: Directions for Research. The MIT Press: Cambridge, MA.
     Comments: "The sequel to their well-known Neurocomputing book."
     
   • Caudill, M. and Butler, C. (1990). Naturally Intelligent Systems. MIT Press: Cambridge, Massachusetts. (ISBN 0-262-03156-6).
     Comments: "I guess one of the best books I read"; "May not be suited for people who want to do some research in the area".
     
   • Khanna, T. (1990). Foundations of Neural Networks. Addison-Wesley: New York.
     Comments: "Not so bad (with a page of erroneous formulas (if I remember well), and #hidden layers isn't well described)."; "Khanna's intention in writing his book with math analysis should be commended but he made several mistakes in the math part".
     
   • Levine, D. S. (1990). Introduction to Neural and Cognitive Modeling. Lawrence Erlbaum: Hillsdale, N.J.
     Comments: "Highly recommended".
     
   • Lippmann, R. P. (April 1987). An introduction to computing with neural nets. IEEE Acoustics, Speech, and Signal Processing Magazine. vol. 2, no. 4, pp 4-22.
     Comments: "Much acclaimed as an overview of neural networks, but rather inaccurate on several points. The categorization into binary and continuous- valued input neural networks is rather arbitrary, and may work confusing for the unexperienced reader. Not all networks discussed are of equal importance."
     
   • Maren, A., Harston, C. and Pap, R., (1990). Handbook of Neural Computing Applications. Academic Press. ISBN: 0-12-471260-6. (451 pages).
     Comments: "They cover a broad area"; "Introductory with suggested applications implementation".
     
   • Pao, Y. H. (1989). Adaptive Pattern Recognition and Neural Networks Addison-Wesley Publishing Company, Inc. (ISBN 0-201-12584-6).
     Comments: "An excellent book that ties together classical approaches to pattern recognition with Neural Nets. Most other NN books do not even mention conventional approaches."
     
   • Rumelhart, D. E., Hinton, G. E. and Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, vol 323 (9 October), pp. 533-536.
     Comments: "Gives a very good potted explanation of backprop NN's. It gives sufficient detail to write your own NN simulation."
     
   • Simpson, P. K. (1990). Artificial Neural Systems: Foundations, Paradigms, Applications and Implementations. Pergamon Press: New York.
     Comments: "Contains a very useful 37 page bibliography. A large number of paradigms are presented. On the negative side the book is very shallow. Best used as a complement to other books".
     
   • Zeidenberg. M. (1990). Neural Networks in Artificial Intelligence. Ellis Horwood, Ltd., Chichester.
     Comments: "Gives the AI point of view".
     
   • Zornetzer, S. F., Davis, J. L. and Lau, C. (1990). An Introduction to Neural and Electronic Networks. Academic Press. (ISBN 0-12-781881-2).
     Comments: "Covers quite a broad range of topics (collection of articles/papers )."; "Provides a primer-like introduction and overview for a broad audience, and employs a strong interdisciplinary emphasis".

A10) Any journals and magazines about Neural Networks ?

1. Neuron Digest
   Internet Mailing List. From the welcome blurb: "Neuron-Digest is a list (in digest form) dealing with all aspects of neural networks (and any type of network or neuromorphic system)"
   Moderated by Peter Marvit. To subscribe, send email to neuron-request@cattell.psych.upenn.edu comp.ai.neural-net readers also find the messages in that newsgroup in the form of digests.
2. Usenet groups comp.ai.neural-nets and comp.theory.self-org-sys.
   There is a periodic posting on comp.ai.neural-nets sent by srctran@world.std.com (Gregory Aharonian) about Neural Network patents.
3. Central Neural System Electronic Bulletin Board
   Modem: 509-627-6CNS; Sysop: Wesley R. Elsberry; P.O. Box 1187, Richland, WA 99352; welsberr@sandbox.kenn.wa.us Available thrugh FidoNet, RBBS-Net, and other EchoMail compatible bulletin board systems as NEURAL_NET echo.
4. Neural ftp archive site ftp://ftp.funet.fi/pub/sci/neural
   Is administrating a large collection of neural network papers and software at the Finnish University Network file archive site ftp.funet.fi in directory /pub/sci/neural Contains all the public domain software and papers that they have been able to find. All of these files have been transferred from FTP sites in U.S. and are mirrored about every 3 months at fastest. Contact: neural-adm@ftp.funet.fi
5. USENET newsgroup comp.org.issnnet
   Forum for discussion of academic/student-related issues in NNs, as well as information on ISSNNet and its activities.
6. AI CD-ROM
   Network Cybernetics Corporation produces the "AI CD-ROM". It is an ISO-9660 format CD-ROM and contains a large assortment of software related to artificial intelligence, artificial life, virtual reality, and other topics. Programs for OS/2, MS-DOS, Macintosh, UNIX, and other operating systems are included. Research papers, tutorials, and other text files are included in ASCII, RTF, and other universal formats. The files have been collected from AI bulletin boards, Internet archive sites, University computer deptartments, and other government and civilian AI research organizations. Network Cybernetics Corporation intends to release annual revisions to the AI CD-ROM to keep it up to date with current developments in the field. The AI CD-ROM includes collections of files that address many specific AI/AL topics including:
   - Neural Networks: Source code and executables for many different platforms including Unix, DOS, and Macintosh. ANN development tools, example networks, sample data, and tutorials are included. A complete collection of Neural Digest is included as well.
   The AI CD-ROM may be ordered directly by check, money order, bank draft, or credit card from:

   Network Cybernetics Corporation
   4201 Wingren Road Suite 202
   Irving, TX 75062-2763
   Tel 214/650-2002
   Fax 214/650-1929
   

   The cost is $129 per disc + shipping ($5/disc domestic or $10/disc foreign) [See the comp.ai FAQ for further details]
7. http://www.eeb.ele.tue.nl/ In World-Wide-Web (WWW, for example via the xmosaic program) you can read neural network information by opening the universal resource locator (URL) http://www.eeb.ele.tue.nl/ It contains a hypertext version of this FAQ and other NN-related information.

1. Rochester Connectionist Simulator
   A quite versatile simulator program for arbitrary types of neural nets. Comes with a backprop package and a X11/Sunview interface.
   anonymous FTP from cs.rochester.edu (192.5.53.209)
   Directory : pub/simulator
   Files:

   README (8 KB)
   rcs_v4.2.justdoc.tar.Z (documentation) (1.6 MB)
   rcs_v4.2.justsrc.tar.Z (source code) (1.4 MB)
   

2. UCLA-SFINX
   ftp 131.179.16.6 (retina.cs.ucla.edu)
   Name: sfinxftp
   Password: joshua (currently not working)
   Directory: pub/
   Files :

   README
   sfinx_v2.0.tar.Z
   

   Email info request : sfinx@retina.cs.ucla.edu
   
3. NeurDS
   request from mcclanahan%cookie.dec.com@decwrl.dec.com simulator for DEC systems supporting VT100 terminal.
   OR
   anonymous ftp gatekeeper.dec.com [16.1.0.2]
   Directory: pub/DEC
   File: NeurDS031.tar.Z
   
4. PlaNet5.7 (also known as SunNet)
   ftp 133.15.240.3 (tutserver.tut.ac.jp)
   pub/misc/PlaNet5.7.tar.Z
   or
   ftp 128.138.240.1 (boulder.colorado.edu)
   pub/generic-sources/PlaNet5.7.tar.Z (also the old PlaNet5.6.tar.Z)
   A popular connectionist simulator with versions to run under X Windows, and non-graphics terminals created by Yoshiro Miyata (Chukyo Univ., Japan). 60-page User's Guide in Postscript. Send any questions to miyata@sccs.chukyo-u.ac.jp
   
5. GENESIS
   GENESIS 1.4.1 (GEneral NEural SImulation System) is a general purpose simulation platform which was developed to support the simulation of neural systems ranging from complex models of single neurons to simulations of large networks made up of more abstract neuronal components. Most current GENESIS applications involve realistic simulations of biological neural systems. Although the software can also model more abstract networks, other simulators are more suitable for backpropagation and similar connectionist modeling. May be obtained via FTP from genesis.cns.caltech.edu [131.215.137.64]. Use 'telnet' to genesis.cns.caltech.edu beforehand and login as the user "genesis" (no password required). If you answer all the questions asked of you an 'ftp' account will automatically be created for you. You can then 'ftp' back to the machine and download the software (ca. 3 MB). Contact: genesis@cns.caltech.edu.
   
6. Mactivation
   anonymous ftp from bruno.cs.colorado.edu [128.138.243.151]
   Directory: /pub/cs/misc
   File: Mactivation-3.3.sea.hqx
   
7. Cascade Correlation Simulator
   A simulator based on Scott Fahlman's Cascade Correlation algorithm.
   Anonymous ftp from ftp.cs.cmu.edu [128.2.206.173]
   Directory: /afs/cs/project/connect/code
   File: cascor1a.shar (206 KB)
   There is also a version of recurrent cascade correlation in the same
   directory in file rcc1.c (107 KB).
   
8. Quickprop
   A variation of the back-propagation algorithm developed by Scott Fahlman. A simulator is available in the same directory as the cascade correlation simulator above
   File: nevprop116.shar (137 KB)
   
9. DartNet
   DartNet is a Macintosh-based Neural Network Simulator. It makes full use of the Mac's graphical interface, and provides a number of powerful tools for building, editing, training, testing and examining networks.
   This program is available by anonymous ftp from dartvax.dartmouth.edu [129.170.16.4]
   Directory: /pub/mac
   File: dartnet.sit.hqx (124 KB)
   Copies may also be obtained through email from bharucha@dartmouth.edu. Along with a number of interface improvements and feature additions, v2.0 is an extensible simulator. That is, new network architectures and learning algorithms can be added to the system by writing small XCMD-like CODE resources called nDEF's ("Network Definitions"). A number of such architectures are included with v2.0, as well as header files for creating new nDEF's. Contact: sean@coos.dartmouth.edu (Sean P. Nolan)
   
10. SNNS
    "Stuttgart Neural Network Simulator" from the University of Stuttgart, Germany.
    A luxurious simulator for many types of nets; with X11 interface: Graphical 2D and 3D topology editor/visualizer, training visualisation, etc. Currently supports backpropagation (vanilla, online, with momentum term and flat spot elimination, batch, time delay), counterpropagation, quickprop, backpercolation 1, generalized radial basis functions (RBF), RProp, ART1, ART2, ARTMAP, Cascade Correlation, Recurrent Cascade Correlation, Dynamic LVQ, Backpropagation through time (for recurrent networks), batch backpropagation through time (for recurrent networks), Quickpropagation through time (for recurrent networks), and is user-extendable.
    Available through anonymous ftp from ftp.informatik.uni-stuttgart.de [129.69.211.2]
    Directory: /pub/SNNS
    Files:

    SNNSv3.0.tar.Z (1530 KB)
    SNNSv3.1.Manual.ps.Z (1296 KB)
    SNNSv3.1.Readme (7744 Bytes)

11. Aspirin/MIGRAINES
    Aspirin/MIGRAINES 6.0 consists of a code generator that builds neural network simulations by reading a network description (written in a language called "Aspirin") and generates a C simulation. An interface (called "MIGRAINES") is provided to export data from the neural network to visualization tools.
    The system has been ported to a large number of platforms. The goal of Aspirin is to provide a common extendible front-end language and parser for different network paradigms. The MIGRAINES interface is a terminal based interface that allows you to open Unix pipes to data in the neural network. This replaces the NeWS1.1 graphical interface in version 4.0 of the Aspirin/MIGRAINES software. The new interface is not a simple to use as the version 4.0 interface but is much more portable and flexible. The MIGRAINES interface allows users to output neural network weight and node vectors to disk or to other Unix processes. Users can display the data using either public or commercial graphics/analysis tools. Example filters are included that convert data exported through MIGRAINES to formats readable by Gnuplot 3.0, Matlab, Mathematica, and xgobi.
    The software is available from two FTP sites:
    CMU's simulator collection on pt.cs.cmu.edu [128.2.254.155]
    Directory: /afs/cs/project/connect/code
    File: am6.tar.Z
    and UCLA's cognitive science machine ftp.cognet.ucla.edu [128.97.50.19]
    Directory: /alexis
    File: am6.tar.Z
    The compressed tar file is a little less than 2 megabytes.
    
12. Adaptive Logic Network kit
    Available from menaik.cs.ualberta.ca. This package differs from the traditional nets in that it uses logic functions rather than floating point; for many tasks, ALN's can show many orders of magnitude gain in training and performance speed.
    Anonymous ftp from menaik.cs.ualberta.ca [129.128.4.241]
    Directory: /pub/atree
    Files:

    atree.readme: README (7 KB)
    atree2.tar.Z: UNIX source code and examples (145 KB)
    atree2.ps.Z: Postscript documentation (76 KB)
    a27exe.exe: MS-Windows 3.x executable (412 KB)
    atre27.exe: MS-Windows 3.x source code (572 KB)
    

13. NeuralShell
    Available from FTP site quanta.eng.ohio-state.edu [128.146.35.1]
    Directory: /pub/NeuralShell
    File: NeuralShell.tar
    
14. PDP
    The PDP simulator package is available via anonymous FTP at nic.funet.fi [128.214.6.100]
    Directory: /pub/sci/neural/sims
    File: pdp.tar.Z (0.2 MB)
    The simulator is also available with the book

    "Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises" by McClelland and Rumelhart. MIT Press, 1988.

    Comment: "This book is often referred to as PDP vol III which is a very misleading practice! The book comes with software on an IBM disk but includes a makefile for compiling on UNIX systems. The version of PDP available at nic.funet.fi seems identical to the one with the book except for a bug in bp.c which occurs when you try to run a script of PDP commands using the DO command. This can be found and fixed easily."
    
15. Xerion
    Xerion is available via anonymous ftp from ftp.cs.toronto.edu [128.100.3.6]
    Directory: /pub/xerion
    Files:

    xerion-3.1.ps.Z (153 kB)
    xerion-3.1.tar.Z (1322 kB)

    plus several concrete simulators built with xerion (about 40 kB each). Xerion runs on SGI and Sun machines and uses X Windows for graphics. The software contains modules that implement Back Propagation, Recurrent Back Propagation, Boltzmann Machine, Mean Field Theory, Free Energy Manipulation, Hard and Soft Competitive Learning, and Kohonen Networks. Sample networks built for each of the modules are also included. Contact: xerion@ai.toronto.edu
    
16. Neocognitron simulator
    An implementation is available through anonymous ftp at unix.hensa.ac.uk [129.12.21.7]
    Directory: /pub/uunet/pub/ai/neural
    File: neocognitron.tar.Z
    The simulator is written in C and comes with a list of references which are necessary to read to understand the specifics of the implementation. The unsupervised version is coded without (!) C-cell inhibition.
    
17. Multi-Module Neural Computing Environment (MUME)
    MUME is a simulation environment for multi-modules neural computing. It provides an object oriented facility for the simulation and training of multiple nets with various architectures and learning algorithms. MUME includes a library of network architectures including feedforward, simple recurrent, and continuously running recurrent neural networks. Each architecture is supported by a variety of learning algorithms. MUME can be used for large scale neural network simulations as it provides support for learning in multi-net environments. It also provide pre- and post-processing facilities. The modules are provided in a library. Several "front-ends" or clients are also available. X-Window support by editor/visualization tool Xmume. MUME can be used to include non-neural computing modules (decision trees, ...) in applications.
    Version 0.73 of MUME has been deposited for anonymous ftp on mickey.sedal.su.oz.au [129.78.24.170]
    Directory: /mume
    File:
    Contact: Marwan Jabri, SEDAL, Sydney University Electrical Engineering, NSW 2006 Australia, marwan@sedal.su.oz.au
    
18. LVQ_PAK and SOM_PAK
    These are packages for Learning Vector Quantization and Self-Organizing Maps, respectively. They have been built by the LVQ/SOM Programming Team of the Helsinki University of Technology, Laboratory of Computer and Information Science, Rakentajanaukio 2 C, SF-02150 Espoo, FINLAND There are versions for Unix and MS-DOS available from cochlea.hut.fi [130.233.168.48]
    Directory: pub/lvq_pak
    Files:

    lvq_pak-2.1.tar.Z (340 kB, Unix)
    lvq_p2r1.exe (310 kB, MS-DOS self-extract archive)

    Directory: pub/som_pak
    Files:

    som_pak-1.1.tar.Z (246 kB, Unix)
    som_p1r1.exe (215 kB, MS-DOS self-extract archive)

19. SESAME
    (Software Environment for the Simulation of Adaptive Modular Systems) SESAME is a prototypical software implementation which facilitates:
    • Object-oriented building blocks approach.
    • Contains a large set of C++ classes useful for neural nets, neurocontrol and pattern recognition. No C++ classes can be used as stand alone, though!
    • C++ classes include CartPole, nondynamic two-robot arms, Lunar Lander, Backpropagation, Feature Maps, Radial Basis Functions, TimeWindows, Fuzzy Set Coding, Potential Fields, Pandemonium, and diverse utility building blocks.
    • A kernel which is the framework for the C++ classes and allows run-time manipulation, construction, and integration of arbitrary complex and hybrid experiments.
    • Currently no graphic interface for construction, only for visualization.
    • Platform is SUN4, XWindows.
    Unfortunately no reasonable good introduction has been written until now. We hope to have something soon. For now we provide papers (eg. NIPS-92), a reference manual (>220 pages), source code (ca. 35.000 lines of code), and a SUN4-executable by ftp only. Sesame and its description is available for anonymous ftp on ftp ftp.gmd.de [129.26.8.90]
    Directory: gmd/as/sesame
    Files:

    sesame-4.5.tar.Z
    sesame-4.5-doc.ps.Z

    gmd/as/sesame Questions please to sesame-request@gmd.de There is only very limited support available. Currently we can not handle many users.
    
20. Nevada Backpropagation (NevProp)
    NevProp is a user-friendly backpropagation program written in C for UNIX, Macintosh, and DOS. The original version was Quickprop 1.0 by Scott Fahlman, as translated from Common Lisp into C by Terry Regier. The quickprop algorithm itself was not substantively changed, but we inserted options to force gradient descent (per-epoch or per-pattern) and added generalization & stopped training, c index, and interface enhancements.
    FEATURES: NevProp version 1.15...
    • UNLIMITED (except by machine memory) number of input PATTERNS;
    • UNLIMITED number of input, hidden, and output UNITS;
    • Arbitrary CONNECTIONS among the various layers' units;
    • Clock-time or user-specified RANDOM SEED for initial random weights;
    • Choice of regular GRADIENT DESCENT or QUICKPROP;
    • Choice of LOGISTIC or TANH activation functions;
    • Choice of PER-EPOCH or PER-PATTERN (stochastic) weight updating;
    • GENERALIZATION to a test dataset;
    • AUTOMATICALLY STOPPED TRAINING based on generalization;
    • RETENTION of best-generalizing weights and predictions;
    • Simple but useful bar GRAPH to show smoothness of generalization;
    • SAVING of results to a file while working interactively;
    • SAVING of weights file and reloading for continued training;
    • PREDICTION-only on datasets by applying an existing weights file;
    • In addition to RMS error, the concordance, or c index is displayed. The c index shows the correctness of the RELATIVE ordering of predictions AMONG the cases; ie, it considers all possible PAIRS of vectors. This statistic is identical to the area under the receiver operating characteristic (ROC) curve, widely used in technology assessment.
    The most updated version of NevProp will be made available by anonymous ftp from the University of Nevada, Reno: unssun.scs.unr.edu [134.197.10.128]
    Directory: pub/goodman/nevpropdir
    File: nevprop1.16.shar
    Limited support is available from Phil Goodman (goodman@unr.edu), University of Nevada Center for Biomedical Research.
    
21. Fuzzy ARTmap
    Available through anonymous ftp from park.bu.edu [128.176.121.56]
    Directory: /pub
    File: fuzzy-artmap.tar.Z (44 kB)
    (This is just a small example program.)
    
22. PYGMALION
    This is a prototype that stems from an ESPRIT project. It implements back-propagation, self organising map, and Hopfield nets. On imag.imag.fr [129.88.32.1]
    Directory: /archive/neural/pygmalion
    File: pygmalion.tar.Z (1534 kb)
    
23. Basis-of-AI-backprop
    Here are some of the details of a set of back-propagation programs I have been working on. Earlier versions have been posted in comp.sources.misc and people around the world have used them and liked them. This package is free for ordinary users but shareware for businesses and government agencies ($200/copy, but then for this you get the professional version as well). I do support this package via email.
    Some of the highlights are:
    • in C for UNIX and DOS and DOS binaries
    • gradient descent, delta-bar-delta and quickprop
    • extra fast 16-bit fixed point weight version as well as a conventional floating point version
    • recurrent networks
    • numerous sample problems
    To get this version simply ftp to ftp.mcs.com where you will land in the directory /work/public/mcsnet.users. Then cd to drt and read readme.1st. The expanded professional version is $30/copy for ordinary individuals including academics and $200/copy for businesses and government agencies. Prices and contents subject to change without notice. Some of the highlights are an improved user interface, more activation functions, networks can be read into your own programs, dynamic node creation, weight decay, SuperSAB
    Contact: Don Tveter; 5228 N. Nashville Ave.; Chicago, Illinois 60656 drt@mcs.com