\newpage
\begin{appendix}

\section*{Appendix}

\section{Adding Activation Functions to SNNS in Four Steps}
\label{appendix:add}

\subsection{First Step - Updating the Function Table}
	As a first step the new activation function has to be entered into 
	the function table of the kernel, located in the file 
	\verb|<SNNS-DIR>/kernel/sources/func_tbl.c|\,.\\
	For one function three entries are necessary:
	\begin{enumerate}
		\item The actual activtaion function.
		\item First derivate function of the activation function.
		\item Second derivate function of the activation function.
	\end{enumerate}
	These entries have to be created, even if there is no first or second 
	derivate function. \\
	The three entries for the sine activation function look like this:
	\begin{enumerate}
		\item activation function : \\
			\verb+{"ACT_Sinus", ACT_FUNC, 0, 0, (FunctionPtr) ACT_SIN},+
		\item first derivate function :\\
			\verb|{"ACT_Sinus", ACT_DERIV_FUNC, 0, 0, (FunctionPtr) ACT_DERIV_SIN},|
		\item first derivate function : \\
			\verb|{"ACT_Sinus", ACT_2_DERIV_FUNC, 0, 0, (FunctionPtr) ACT_2_DERIV_SIN},|
	\end{enumerate}
	For a better understanding I included the table description of the
	func\_tbl.c file of the SNNSv4.1:
{
\small
\begin{verbatim}

/*  The function table contains 5 entries:
     - the XGUI-Identifier of this function (a pointer to a string constant),
       note that the identifier have to match [A-Za-z]^[|, ]* (i.e. the first
       character of the symbol must be a letter and all other characters must
       not match [|, ])

     - an integer constant that determine the class of the function.
       8 different classes are possible:
       OUT_FUNC, ACT_FUNC, ACT_DERIV_FUNC, SITE_FUNC, UPDATE_FUNC, LEARN_FUNC,
       INIT_FUNC, PRUNING_FUNC and TEST_FUNC

     - the number of input parameters for this function (only needed for Update,
       Init, Learning and Test functions)

     - the number of output parameters returned by this function (only needed
       for Update, Learning and Test functions)

     - a pointer to the (user defined) function.
*/

\end{verbatim}
}
	If the activation function does not have a first or second derivate function then 
	the appropriate entry must point to the dummy functions ACT\_DERIV\_DUMMY or 
	ACT\_2\_DERIV\_DUMMY respectively.






%% SECOND STEP %%
\subsection{Second Step - Implementing the Activation Function}	
	The activation functions, and the first and second derivate functions of 
	the activation functions reside in
	\verb|<SNNS-DIR>/kernel/sources/trans_f.c|\,. \\
	The best idea is probably to copy/paste the declaration from another activation 
	function:
\begin{verbatim}

FlintType ACT_SIN(struct Unit *unit_ptr)
{
	ACT_FUNC_DEFS
	register FlintType sum;
	
}

\end{verbatim}
	Here it gets a bit vague, as I did not look up how they declared the types`
	used here, but I do have some pretty neat ideas:
	\begin{description}
		\item[FlintType] Float.
		
		\item[struct Unit] Double linked list with some ID stuff that sparates between
			site and unit\footnote{Unit is the SNNS expression for neuron}.
		
		\item[ACT\_FUNC\_DEFS] Some macros that allow access to the list
			\verb+struct Unit *unit_ptr+. To see how this macro works either 
			search for them in the SNNS source files or read the code of the 
			other activation functions.
	\end{description}
	
	Implementing the sine is quit simple: get the bias, get the inputs for 
	every connection, add them up and return sin(sum).
\begin{verbatim}

FlintType ACT_SIN(struct Unit *unit_ptr)
{
  ACT_FUNC_DEFS
  register FlintType  sum;


  sum =  GET_UNIT_BIAS(unit_ptr);
  if (GET_FIRST_UNIT_LINK( unit_ptr ))
    do
      sum += GET_WEIGHTED_OUTPUT;
    while (GET_NEXT_LINK);
  else
    if (GET_FIRST_SITE( unit_ptr ))
      do
			sum += GET_SITE_VALUE;
      while (GET_NEXT_SITE);

  return(sin(SIN_FAKTOR*sum));
}

\end{verbatim}
	Repeat this process for the first and second derivation function.






%% THIRD STEP %%	
\subsection{Third Step - Defining the Function Prototypes}
	Declare the function prototypes in the file 
	\verb|<SNNS-DIR>/kernel/sources/trans_f.h|\,.\\
	In case of the sine activation function and its derivations it looks like this:
\begin{verbatim}

		extern FlintType ACT_SIN(struct Unit *unit_ptr);
		extern FlintType ACT_DERIV_SIN(struct Unit *unit_ptr);
		extern FlintType ACT_2_DERIV_SIN(struct Unit *unit_ptr);

\end{verbatim}






	
%% FOURTH STEP %%
\subsection{Fourth Step - Recompiling the Kernel and Xgui}
	How this is done depends on version and installation, but one of these
	should work:
	\begin{itemize}
	\item make kernel; make xgui
	\item make all
	\item build sim
	\item build compile
	\end{itemize}






\section{Implementation of NetGEN Cubic Spline}
\label{appendix:addcs}
All files referenced in this section are located in the \verb=<SNNS-DIR>/kernel/sources/= directory.




\subsection{func\_tbl.c}
We added the following to the function table:
{
\small
\begin{verbatim}

{"Act_NetGEN_CubicSpline", ACT_FUNC, 0, 0, (FunctionPtr) NetGEN_CS},
{"Act_NetGEN_CubicSpline", ACT_DERIV_FUNC, 0, 0, (FunctionPtr) NetGEN_DERIV_CS},
{"Act_NetGEN_CubicSpline", ACT_2_DERIV_FUNC, 0, 0, (FunctionPtr) NetGEN_2_DERIV_CS},

\end{verbatim}
}






\subsection{trans\_f.c}
We added the function bodies of the NetGEN CubicSpline functions.

We also added a general function to get a FILE pointer to the files with the general function parameters (gfp). 
This functions was done with respect to future additions of other functions that require gfp.

Additionally we added a function that reads the coresponding file and calculates the spline function in the 
correct intervall or the requested derivation of the spline.





\subsection{trans\_f.h}
We added the declarations for NetGEN and sine second derivation functions.





\subsection{func\_mac.h}
We added the following macro to get the current unit's name:
\begin{verbatim}
#define  GET_UNIT_NAME( u_ptr )    ((u_ptr)->unit_name) 
\end{verbatim}
return value of this function is a \verb+char *+ .





\subsection{Location and Layout of the GFP-Files}
\label{appendix:gfp}

GFP stands for general function parameters. For every type of function 
used there is an individual directory, thus allowing future additions.
The following two subsections deal with the GFP in regards to the 
cubic spline function.



\subsubsection{GFP-Path}
The path to the GFP files is given either by the environmental variable 
SNNS\_GFP or, when it is not defined, a subdirectory \verb+.SNNS_GFP+ in 
the user's homedirectory (as defined by the envrionmental variable HOME)
is used.\\
In case of the cubic spline the files are located in a subdirectory /CS.\\
\\
e.g.:
When \verb+SNNS_GFP=/usr/share/snns-gfp+ the files are located in the directory
\verb=/usr/share/snns-gfp/CS=.
If the SNNS\_GFP is undefined the HOME variable is used. Assuming that 
\verb+HOME=/home/stud1/hhofbaue+ the the files are located in 
\verb=/home/stud1/hhofbaue/.SNNS_GFP/CS=.\\
\\
The file name is identical to the unit name.\\
More detailed information can be obtained from the source code (section \ref{get:GFP:file}).




\subsubsection{GFP File Layout}
The first entry in the file is the number of entries. Followed by the given number 
of entries of the following format:
\begin{verbatim}

struct cs_gfp{
  float x;
  float y;
  float a;
  float b;
  float c;
}

\end{verbatim}
Where the formula of the cubic spline function is 
$$f(x')=y + a*z + b*z^2 + c*z^3,\,\,z=x'-x$$
\\
Thus the file looks like this:
\begin{verbatim}

  int n
  struct cs_gfp(1)
  struct cs_gfp(2)
  .
  .
  .
  struct cs_gfp(n-1)
  struct cs_gfp(n)

\end{verbatim}
The cs\_gfp entries are sorted from the highest x to the lowest x (going from right 
to left in the functions graph), such that 
$$cs\_gfp(1).x > cs\_gfp(2).x > \dots > cs\_gfp(n).x$$




\subsection{Function Source Code}
For the readers convenience we included the code of the added functions.




\subsubsection{Source for get\_GFP\_file}
\label{get:GFP:file}
This function returns a file pointer to the given function and filename.
{
\small
\begin{verbatim}

FILE *get_GFP_file(char *func, char *filename)
{
    char buf[1024];
    char *env;
    int home = 0;

    if( (env = getenv("SNNS_GFP")) == NULL) 
    {
        if( (env = getenv("HOME")) == NULL) 
        {
            fprintf(stderr, "get_GFP_file : neither SNNS_GFP nor 
                             HOME is set -> cannot generate GFP 
                             path \n");
            return NULL;
        }
        home=1;
    }

    strcpy(buf, env);
    if(home) strcat(buf, "/.SNNS_GFP");

    strcat(buf, "/");
    strcat(buf, func);
    strcat(buf, "/");
    strcat(buf, filename);
    return fopen(buf, "r");
}

\end{verbatim}
}





\subsubsection{Source for cs\_f}

This function calculates the value of the cubic spline (in the given derivative)
for a specific unit.\\
Note that the function maps to the identity transfer function if an error occures.
{
\small
\begin{verbatim}

float cs_f( float x, char *name, int derivation)
{
    FILE *fp;
    struct {
        float x;
        float y;
        float a;
        float b;
        float c;
    }gfp;
    
    int rv;
    float f_ret;
    int entries = 0;

    if( derivation < 0 || derivation > 2 )
    {
        fprintf(stderr, "cs_f : wrong derivation number (%d) -> 
                         remap to identity", derivation);
        return x;
    }

    if ( (fp = get_GFP_file("CS", name)) == NULL ) return x;  
    
    rv = fread( &entries, sizeof(entries), 1, fp);
    if ( !rv || (entries < 2)) 
    {
        fclose(fp);
        fprintf(stderr, "cs_f : entry missmatch (%d) -> 
                         remap function to identity\n", entries);
        return x; /* no entries or to few enries -> id*/
    }
    
    rv = fread( &gfp,sizeof(gfp), 1, fp);
    if( !rv ) 
    {
        fclose(fp);
        fprintf(stderr, "cs_f : gfp read error (%s) -> 
                         remap function to identity\n", name);
        return x;
    }
    
    if( x >= gfp.x ) 
    {
        fclose(fp);
        switch(derivation)
        {
            case 0  : return gfp.y;
            case 1  : return 0;            
            case 2  : return 0;
            default : break;  
        }
        fprintf(stderr, "cs_f : impossible error (wrong derivative 
                         ???) -> remap to idetity\n");
        return x;
    }
    entries--;
    
    while(entries > 0 && x < gfp.x ) 
    {
        rv = fread( &gfp, sizeof(gfp), 1, fp);
        if( !rv )  
        {
            fclose(fp);
            fprintf(stderr, "cs_f : gfp read error 2 (%s) -> 
                             remap to identity\n", name);
            return x;        
        }
        entries--;
    };
    
    if( (entries == 0) && (x < gfp.x) )
    {
        fclose(fp);
        switch(derivation)
        {
            case 0  : return gfp.y;
            case 1  : return 0;
            case 2  : return 0;
            default : break;  
        }
        fprintf(stderr, "cs_f : impossible error 2 (wrong derivative 
                         ???) -> remap to identity\n");
        return x;    
    }
    
    fclose(fp);
    f_ret = x - gfp.x;
    
    switch(derivation)
    {
        case 0 : 
            f_ret = gfp.y + gfp.a*f_ret + gfp.b*f_ret*f_ret + gfp.c*f_ret*f_ret*f_ret;
            break;
        case 1 :
            f_ret = gfp.a + 2*gfp.b*f_ret + 3*gfp.c*f_ret*f_ret;
            break;
        case 2 :
            f_ret = 2*gfp.b + 6*gfp.c*f_ret;
            break;
        default : 
            fprintf(stderr, "cs_f : impossible error 3 (wrong derivative 
                             ???) -> remap to identity\n");
            f_ret = x;   
    };
    
    return f_ret;
}

\end{verbatim}
}


\end{appendix}
