Y 4 (301Z)

Esther Mondragn, Eduardo Alonso, Alberto Fernndez & Jonathan Gray

RESCORLA & WAGNER SIMULATOR ver. 4

TABLE OF CONTENTS

1. Introduction............................................................................................. 1 2. Whats new in version 4? ......................................................................... 1 3. Before you start ....................................................................................... 1 4. Running the application ........................................................................... 2
4.1. Saving the experiment design ............................................................................ 6

4.5. Compound stimuli with and without configural cues .............................. 9

5. Terms of use .......................................................................................... 16 6. Feedback ............................................................................................... 17 7. References ............................................................................................. 17

4.6 Context simulation .................................................................................................12

4.3. Exporting results into a spreadsheet .............................................................. 6 4.4. Figures display.......................................................................................................... 7

4.2. Opening a file to retrieve your experiment design .................................... 6

1. Introduction

RESCORLA & WAGNER SIMULATOR ver. 4

This document is a quick introduction for installing and using the Rescorla & Wagner Simulator ver. 4. Executable versions (.exe for Windows and .app for Apple) are available. Once downloaded to your computer, the file will run without installation. The software can also run on UNIX, Linux, etc. as long as there is a Java Runtime Environment installed. R&W Simulator ver. 4 simulates the Rescorla & Wagner (1972) model and extends version 3.1., which incorporated compound stimuli and configural cues (Alonso, Mondragn & Fernndez, 2012), to work with different contexts. This document doesn't refer to any programming details or to the psychological theory behind the Rescorla and Wagner model.

2. Whats new in version 4?

R&W Simulator ver. 4 incorporates a method to simulate contexts within experimental designs. It maintains the functionalities of previous versions, importantly, the ability to calculate stimulus compounds and configural cues. As a result, version 4 also allows computing context compounds as well as configural cues with contexts. In addition, we have added the possibility of processing probe trials. The graphic environment has been improved and bugs cleaned. The description of the new functionalities in this guide will appear flanked by two red asterisks (). Following useful feedback, errors have been corrected. Thanks to all.

3. Before you start

You will need to go to http://www.cal-r.org/index.php?id=R-Wsim and choose your computer platform to download the program. 1

A file called RW_Simulator.exe will start to download to your PC or, alternatively, a disk image (RW_Simulator_Image.dmg) containing the application will be downloaded to your MAC. 1 PC users just need to save the RW_Simulator.exe file into their desired location. It does not require any installation; it is now ready to go.

RESCORLA & WAGNER SIMULATOR ver. 4

Users that wish to access the code should also download the RW_Simulator.jar file. They would need to uncompress it to retrieve the .class files. The content of these files can be viewed using a Java editor such as Eclipse (http://www.eclipse.org/downloads/) or NetBeans, and the Java Development Kit (JDK). Java programs and NetBeans are available for free download at http://www.oracle.com/technetwork/java/javase/downloads/index.html. All the above programs are free to download and use.

Users of other platforms should select the JAVA button to download the RW_Simulator.jar file. This file will run in any platform provided that the Java Runtime Environment (JRE) is installed. Most popular Linux distributions such as Fedora, Debian, Ubuntu, Arch, and CentOS already include JRE.

MAC users should double-click the downloaded image file to mount the disk. The RW_Sim_Disk will appear on your desktop and open automatically. Drag the RW icon into the Applications folder located in the same window at its right. A file called RW_Simulator.app will be saved into your Application folder. No further installation is required. Thats it! You can now trash the RW_Sim_Disk and the RW_Simulator_Image.dmg file 2.

4. Running the application

To start the application you need to navigate to the directory in which you stored the file and double click on the file's icon. The opening screen should look like in Figure 1 (PC version, MACs GUI varies slightly).
Ignore security warnings and accept the files. The RW icon will show in the Dock Applications Folder Display as a generic yet fully functional icon until the computer is restarted.
1 2

This screen consists of the main menu (File, Settings, and Help), followed by a section in which Groups and Phases are input; at the bottom left the Set Parameters

button precedes a table to enter the CS and US parameters and, at its right, a section in which the resulting data will be displayed.

RESCORLA & WAGNER SIMULATOR ver. 4

You can directly add or subtract groups and phases with the corresponding + and buttons. Now, we select the cell on the first row next to Group 1 and input the experimental design for Phase 1 describing each trial type as follows: 3

Now, you can choose between to create a new experiment design or to load one that you have previously saved. Suppose that this is your first time with the application, so we are creating a new design. We are entering a biconditional discrimination design as a guiding example to make it interesting.

Figure 1: Opening screen PC GUI.

RESCORLA & WAGNER SIMULATOR ver. 4

Different trial types should be separated by a slash symbol (/). Note that no space should appear between the characters. For instance, Phase 1 in Group 1 of our biconditional discrimination example would read:

Number of trials followed by Stimuli followed by Reinforcer (+ or )

60AX+/60AY-/60BX-/60BY+
WARNING: Once you have written each design cell down you must press Enter for the simulator to process the data. If you dont, it will consider the cell empty. Trying to input data in a different cell or to modify the cell size will delete what you have just input. We can also work with probe trials, which must be specified adding a hat symbol (^) immediately after the cue (e.g., 60AX^+). The associative strength values calculated on those trials will then be copied to the output and to the figures as a snapshot of the chosen cue. The reminder phases and groups can be added following Figure 2.

Figure 2: Group and Phase Design table.

Group 1, Group 2, etc. are the names given by default; we have changed them to more illustrative BICOND and PR BICOND to show you that they can be renamed as you please. The order of the trials is defined by the order they are entered; alternatively, you can tick the Random box to randomize the order of presentation within phases. NOTE: If you input a large number, the processing speed will be affected. The time remaining will be shown as a progress bar. The number of different random combinations that the simulator shall execute will be determined in the Settings/Number of Random Combinations menu. By default this number is set to 1,000.

RESCORLA & WAGNER SIMULATOR ver. 4

Now we input the parameters, by pressing the Set Parameters button. Two tables appear. In the top table, values for each of the stimuli must be entered. The bottom table contains a set of default values given to the US (+ is set to 0.5, to 0.45, + to 1 and to 0); of course, the user can modify these parameters. Moreover, the user can set different US per phase in the Settings menu if required. After completing the tables, press the Run button to start the simulation.

This will produce a text output on the data area, located on the right hand side, like in Figure 3. These are the results of the simulation for our design. The user can roll up and down to check the stimulus mean V values per Trial, Group and Phase.

Figure 3: Parameters and results tables.

The parameters entered in the Group and Phase tables can be changed anytime; of course, the values will have to be amended in the Parameters table accordingly. The Clear All button resets the application deleting all data.

RESCORLA & WAGNER SIMULATOR ver. 4

4.1. Saving the experiment design

If you would like to save your design so you can retrieve it another time (for instance, to modify it), all you have to do is to click Save on the File menu bar or press Alt+S (S, in Mac). This will pop up a window, which asks you where you would like to save your file. You need to choose a directory and a filename. Click OK to store the values. A file with the extension .rws will appear in the chosen directory.

NOTE: Design files (.res) from previous versions of the R&W Simulator will not work in this version. The new functionalities included in version 4 required a new input allocation format.

4.2. Opening a file to retrieve your experiment design

Once you have saved your design, you can re-use it by clicking Open from the File menu or the key combination Alt+O (O). The application will update both the design and its corresponding parameters.

4.3. Exporting results into a spreadsheet

The application has the ability to export the results in .xlsx type spreadsheets, like the ones used by Excel, Microsoft Office. It creates a workbook that has a different sheet per group. Phases are presented individually on a separate table. Each sheet contains the name of the file followed by CS and US parameters. A line showing its design precedes each Phase table. Figure 4 shows the exported data. To export the results select File/Export or Alt+E (E).

 The time remaining will be shown as a progress bar.

WARNING: When exporting a large file, please allow some time for the file to be saved before attempting to open it; otherwise you will get an exporting error.

RESCORLA & WAGNER SIMULATOR ver. 4

Figure 4: Excel spreadsheet as exported by the simulator.

4.4. Figures display

We can see a graphical representation of the results by pressing the Display Figures button. It retrieves the calculated mean values per stimulus and trial plotting each phase of the design on a different panel. The user can enable and 7

disable Groups and cues from displaying. The window may remain open while the user chooses to do a new simulation to facilitate comparing results.

RESCORLA & WAGNER SIMULATOR ver. 4

While pointing over the figure, right button click (or, in Mac, Ctrl+Click) opens a menu that allows the figures to be saved, copied, printed, zoomed and modified (Figure 5). For instance, to facilitate data comparison you may wish to fix the Y-axis limits set by default to auto-range. Right click (or Ctrl+Click) and choose Properties. Then, select 'Plot' at the first row of tabs; 'Range Axis' (for Y-axis) at the second, and 'Range' at bottom row of tabs. Unmark the Auto-adjust range' tick-box and modify the range values as required.

Figure 5: Graphical display of the results showing the figure menu.

Figure 6 shows these Properties menus. 8

RESCORLA & WAGNER SIMULATOR ver. 4

Figure 6: Figure properties menu.

4.5. Compound stimuli with and without configural cues

This simulator generates associative strength values for standard additive compound stimuli. It also computes compound values using added configural cues. Hence, it simulates not only the original Rescorla and Wagner model (Rescorla & Wagner, 1972) but also later configural assumptions proposed to explain certain complex discriminations (Wagner & Rescorla, 1972). It also allows to set negative values to -, that can be useful in simulating categorization experiments in humans involving symmetrical outcomes. To calculate compound values, you must select Settings/Show Compound Results. Once ticked, you must press again the Run button. Now, the associative strength for the compound stimuli, represented as AX, BX, etc., will be shown. As before, the data can be exported and graphically displayed. See Figure 7.

RESCORLA & WAGNER SIMULATOR ver. 4

Figure 7: Compound stimulus associative strength during the biconditional discrimination.

If you wish to calculate stimulus compounds assuming configural cues you must also tick Settings/Use Configural Cues in the main menu 3 (Figure 8). Press Set Parameters to input the alpha values for the configural cues. By default, the product of the elemental alpha values will appear but the user can modify these values. Configural cues are represented as c(AX), c(AY), etc..

Now, you will need to press Run and the data will be updated. Configural cue compounds are represented as [AX], [BX], etc.
3 Both, Compound Results and Use Configural Cues should be marked to display the compound values with added configural cues.

10

RESCORLA & WAGNER SIMULATOR ver. 4

The displayed figures will now show configural cue compounds instead of standard compounds (Figure 9).
Figure 8: Settings for calculating configural cues and parameters table.

Figure 9: Associative strength values for the configural cue compounds during the biconditional discrimination training.

11

Should you require it, you can choose to display the calculated mean associative strength values for the configural cues. You just need to select the corresponding tick-boxes. In the simulator, the user can choose to work with contexts by selecting Context Simulation in the Settings menu (Figure 10).

RESCORLA & WAGNER SIMULATOR ver. 4

4.6 Context simulation

Next, we enter the salience of the context, 0.15 by default, and the context identity. Six distinct contexts are available to be assigned and their salience configured per phase, with a maximum of one context in each phase (Figure 11).

Figure 10: Context simulation selection.

Figure 11: Context simulation specific settings.

12

The context is treated as an always present background cue, implemented like any other stimulus but with the particularity that it repeats itself throughout the session, acquiring associative strength during reinforced trials and losing it during non-reinforced trials. In addition, unlike a standard stimulus, contextual cues lose associative strength during the inter-trial interval. To determine the amount of loss during ITIs, an ITI/CS ratio is assumed. The corresponding functionality ITI/CS ratio appears at the left of the Context with a default value of 5. As with any other cue, the simulator also allows the formation of context-cue compounds. If the user also wishes to assume context-configural cues, it can do so by choosing in the Settings menu the corresponding option along with Context Simulation. Figure 12 shows a design used to exemplify the context functionality. Context parameters and ITI/CS ratio can be changed anytime by clicking on the corresponding cell.

RESCORLA & WAGNER SIMULATOR ver. 4

Figure 12: Context conditioning and context and cue conditioning.

We run 12 context conditioning trials in Phase 1. During Phase 2, a cue A is given 4 conditioning trials. In Group Blk-Sum Phase 1 and Phase 2 are given in the same context . Group Control Phase 1 is run in context , while Phase 2 is run in context . WARNING: When simulating contexts if you require an empty Group/Phase do not leave the corresponding cell blank or with a 0 value, instead insert a sign. Otherwise, you will experience interferences with the cue selector in that Phase figure display. 13

RESCORLA & WAGNER SIMULATOR ver. 4

Figure 13 shows the simulated results during the 12 context conditioning trials in Phase 1.

Figure 13: Context conditioning through training in Phase 1.

14

RESCORLA & WAGNER SIMULATOR ver. 4

Figure 14 shows an example of cue (A) blocking by the context and stimulus context summation, as a compound context-cue, following context conditioning.

Figure 13: Cue blocking by the context and context cue summation.

15

5. Terms of use

RESCORLA & WAGNER SIMULATOR ver. 4

Please, read carefully the following license agreement. If you don't accept the terms of the agreement, please delete immediately the software from your computer.

R&W Simulator ver. 4 and its software and documentation is copyrighted by Esther Mondragn, Eduardo Alonso, Alberto Fernndez and Jonathan Gray (The author). The following terms apply to R&W Simulator ver. 4 and to previous versions unless explicitly disclaimed. The author hereby grants permission to use, copy and distribute, (but NOT sell or modify) this software and its documentation, provided that it is retained unchanged in all copies and that this notice is included verbatim in any distributions. No written agreement, license, or royalty fee is required to use or distribute this software.

Use of this software and its authorship must be acknowledged in oral (for example, lectures, tutorials, laboratory sessions, demonstrations, conferences) or written communication (for example, books, articles, proceedings). The author is not liable for any misuse or misleading use of the software.

IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

THE AUTHOR SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOTLIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. THIS SOFTWARE AND ITS DOCUMENTATION ARE PROVIDED ON AN "AS IS" BASIS, AND THE AUTHORS HAVE NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. 16

6. Feedback

RESCORLA & WAGNER SIMULATOR ver. 4

If you have any questions or comments about the application or this document, please feel free to email Esther Mondragn, at e.mondragon@cal-r.org. We welcome any suggestions or criticisms. If there is an inaccuracy somewhere, please let us know where it occurred and what values were used. It would be very helpful if a saved file from the simulator, which contains the values, is sent as an attachment.

7. References
Alonso, E., Mondragn, E., & Fernndez, A. (2012). A Java simulator of Rescorla and Wagner's prediction error model and configural cue extensions, Computer Methods and Programs in Biomedicine, 108, 346-355.

Wagner, A.R., & Rescorla, R.A. (1972). Inhibition in Pavlovian conditioning: Application of a theory. In R.A. Boakes & M.S. Halliday (Eds.), Inhibition and Learning (pp. 301-336). New York: Academic Press.

Rescorla, R.A., & Wagner, A.R. (1972). A theory of Pavlovian conditioning: The effectiveness of reinforcement and non-reinforcement. In A.H. Black & W.F. Prokasy (Eds.), Classical Conditioning II: Current Research and Theory (pp. 6499). New York: Appleton-Century-Crofts.

17