Formulas for Learning Direct Learning, 1st-Order Occasion Setting, and 2nd-Order Occasion Setting.

In the table, subscripts are stimulus names (e.g., A, B, C; "sum" for all stimuli present on a trial); superscripts are trial numbers. For "R" formula, superscript "n" for all variables, and subscript "sum" for all variables except "R.” Formulas are arranged in column format for readability. Responding (R) formula ultimately predicts behavioral responding and learning. R operates by adding excitation and subtracting inhibition (formula in dark gray). Light gray columns highlight the similar variables used in the "recipe" across our learning formulas. Hierarchical control of a) 2nd-order occasion setting (2nd OS) on 1st-order occasion setting (1st OS) and b) 1st OS on direct associative learning (i.e., CSs) is accomplished with modulation, in which the higher-order stimulus affects the lower-order stimuli’s signal of US (non)occurrence. The gateway to higher-order learning (from direct learning to 1st OS, and from 1st OS to 2nd OS) is lower-order stimulus ambiguity. Mechanism through which 1st OS is learned is γ1, which is the degree to which the CS is ambiguous (i.e., that the CS has both direct excitation and direct inhibition). If the present CS is unambiguous (e.g., only excitatory or only inhibitory), γ1 remains at 0, and no 1st OS is learned. Once a given CS is ambiguous (i.e., the CS is both excitatory and inhibitory), γ1 becomes positive, allowing P and N to increase from zero and for 1st OS to be learned. CS ambiguity is necessary but not sufficient for 1st-order occasion setting to be learned. In order for 1st OS to be learned, the individual must also learn that the CS can be modulated by a 1st-order positive occasion setter () or 1st-order negative occasion setter (). This will occur if a stimulus/context (i.e., the 1st-order occasion setter) provides information about the CS’s (non)reinforcement and if the stimulus/context is less salient than the CS e.g., 43. and are values contained by the CS (rather than the 1st-order occasion setter), indicating the CS can be modulated by a 1st-order positive or negative occasion setter, respectively. Thus, a CS must be both ambiguous and trained with a 1st-order occasion setter in order to be modulated by other 1st-order occasion setters (e.g., a simple partially reinforced CS will not be affected by a 1st-order occasion setter because its and will equal 0, causing the 1st OS terms in the R formula to equal 0).