Discussion
We found that the level of bottom-up and top-down attention allocated to stimuli during encoding affected the impact of the very same stimuli, when used as distractors, during subsequent memory retrieval. First, the distractors extracted from boundaries reduced the retrieval time more than those coming from non-boundary locations, when the temporal distance between Im1/Im2 was long in the movie. In line with our previous proposal of contingent attention capture within memory representation, such behavioural advantage was linked with increased activation in the right SMG26, and here more specifically localised in the PFt and PFop subregions. Second, we showed that the level of activity in the middle rSMG (a ventral part of the PF subregion) was modulated according to whether an active segmentation task was performed during encoding. The latter indicates that retrieval related activity in this part of the PF was affected by a combination of bottom-up and top-down factors of the encoding experience. These results demonstrate that the anterior and middle regions of the rSMG play different functional roles in mediating the effect of task-irrelevant distractors during temporal order retrieval, reflecting dissociable contribution of bottom-up signals generated by the external input (boundary vs. non-boundary frames) and participant’s prior task-related experiences (segmentation vs. viewing). We discuss the dissociable involvements of the anterior and middle rSMG in light of the fractionation view of the ventral parietal cortex.
The mere presence of repeated pictures can initiate episodic retrieval without having the participants to try to remember the information intentionally, indicating that retrieval processes can be incidental and stimulus-driven43. As the attention-to-memory model posits a role of IPL in bottom-up attentional capture to a previously experienced stimulus25, one may suppose that the anterior SMG activity observed here was driven by the pure stimulus-driven attention capture and/or incidental retrieval by the preferentially processed DBORDER distractors36. Nonetheless, this is unlikely given that the DBORDER distractors by themselves did not activate any of the SMG subregions significantly (and without any behavioural effect either). Instead, the effects of the DBORDER distractors manifested as an interaction with the temporal distance separated by Im1/Im2 in the movie.
In long distance trials, the retrieval related activity in the anterior SMG (including the PFt and PFop, see Fig. 2) was larger - and accompanied with faster RT - when the trial included a border than a non-border distractor, indicating that any stimulus-driven capturing effects by DBORDER distractors took place specifically when the two memory probes were temporally far away in the movie. We previously reported distance-dependent modulation related both to short trials (in the superior temporal sulcus) and to long trials in the supramarginal gyrus26, as here. In that study, we suggested that the distractor effects on long trials may relate to shifts of attention within memory representations24. Here, a related effect may occur, with the mnemonic status of the distractor images impacting on the retrieval of the memory probes, only when the latter involved substantial (“long”) shifts of attention between the temporal positions of the memory probes. The link between stimulus-driven attention control and episodic memory retrieval is also substantiated by previous studies that have implicated the anterior SMG in being strongly connected with the mnemonic related MTL23, ascribing this part of the IPL to memory based attentional processes. Here, the interaction between the effect of boundary (attention-related) and the effect of Im1/Im2 temporal distance (memory-related) supports the attention-to-memory hypothesis, showing that the bottom-up contribution, common to the segmentation and the passive viewing groups, dominated processing in the anterior part of rSMG.
Conversely, a ventral part of area PF in the middle SMG was not sensitive to the “Distance × Boundary” interaction when both groups were considered together. Rather, this area showed a pattern analogous to that found in the anterior SMG subregions only in the segmentation group (see Fig. 2; with a significant interaction between “Distance”, “Boundary” and “Group”). This showed that in this area any bottom-up effect of the distractors during retrieval was dependent on the participant’s prior task-related experiences, in line with a recent theoretical notion in the attention literature to account for the role of the IPL in mediating the joint contribution of endogenous and stimulus driven factors3132. One possibility is that the middle SMG may encompass a higher-level processing node, together with frontal cortices (e.g., lateral prefrontal areas28), in a putative hierarchical network. Within this hierarchy, the higher order nodes represent the expected signal, which consists of knowledge/expectations stored in internal models determined by the top-down influences44. The participant’s prior task-related experiences determined how the distractors were initially encoded and accordingly, and to what extent they would influence attention allocation during retrieval. So far, evidence for such hierarchical organisations has been discussed in the context of sensory paradigms45 and attention to the external environment46. Our current results raise the possibility of extending this framework to attention capture within memory and point to the middle SMG as a possible neural correlate.
At first sight, our study seems to share some similarity with previous studies that explicitly manipulated attention at encoding4748. These previous studies used variants of the Posner spatial cueing paradigm to direct participants attention towards (or away from) a target picture. Turk-Browne et al.48 showed that top-down attention to memory-targets can help memory formation, and associated this facilitation with activity in regions such as parahippocampal cortex. In a different study, Uncapher et al.47 showed that top-down attention to memory-targets activated the dorsal parietal cortex and improved memory success. Common to both studies was that bottom-up attention to non-targets activated the ventral parietal cortex, but diminished later memory of the targets. These studies demonstrated that different types of attention signals during encoding influence memory formation and treated attention as a scalar modifier for the memory success: more attention to the memory targets leads to better memory (see also49 for a review). Analogous mechanisms may have contributed to our current results. However, we further reason that memories for complex events may be formed, and stored differently into the memory engram, depending on whether these events had received processing derived from stimulus-driven attention, or a combination of both stimulus-driven and task/goal-related attention. In this sense, we consider the effects of top-down/bottom-up attention on memory encoding not to be merely “quantitative/scalar”, but rather specific and “qualitative”.
Here we propose that the task-related experience of performing the event-transition detection during encoding generated an internal construct, rendering the attended/identified distractors-stimuli a special mnemonic status (segmentation group), compared to when the very same events were not task-relevant (passive viewing group). Because of their particular status, these distractors subsequently gave rise to specific processes – and brain activation – during retrieval. In particular, it should be emphasised that our encoding material was a long dynamic movie (i.e., more complex in terms of temporal-spatial and semantic details than a single picture or scene) and that the temporal order judgement task demanded a faithful reconstruction of sequences of events for reaching correct decisions. In these settings, the memory retrieval may comprise the inclusion of rich contextual details and episodic associations, including the distractors-events. At this point, the mnemonic status of the different types of distractors would provide different levels of contextual information, modulating their impact on retrieval.
Such context-related processes may be interpreted also in the framework of other accounts of parietal activity during episodic memory retrieval. According to the output buffer hypothesis50, the IPL supports post-retrieval representations of the retrieved information. One specific model, the episodic buffer model, suggests that the IPL acts as a temporary storage system, supporting the maintenance and representation of the contextual details of episodic memories while a memory decision is being made51. In our case, the different sets of distractor stimuli – which had received different attention signals at encoding – might have provided different levels of contextual richness for the post-retrieval representation of the retrieved information, which in turn tapped into different parts of the IPL at retrieval. Other authors hypothesised that the IPL enables the working memory buffer through interactions with other cortical systems involved in memory storage, such as the MTL. In this context, the IPL retrieval activity reflects the engagement of processes that operate directly on the retrieved information, through cortical binding of relational activity (CoBRA)52. Based on these accounts, we attribute the different attentional signals at encoding – which resulted in different mnemonic statuses of the distractors – to causing different modulatory effects on the reconstruction-based processes during retrieval.
Concerning the parcellation of the IPL, it should be noted that schemes other than the one adopted here10 are also available, such as in1112. These different schemes do not necessarily fully agree with each other, and thus the existing delineation of IPL may not be definitive (cf. Nelson et al., 2010). For instance, with respect to the “Distance × Boundary” interaction, here we found significant peaks in two adjacent VOIs (PFt and PFop), but the whole-brain analysis showed a single cluster of activation extending across these two cytoarchitectonic regions. Also, the activation peak in area PF was located at the border between the temporo-parietal junction (TPJ) and the posterior superior temporal gyrus (pSTG)20, in an region that is labelled differently in the literature31153. Some studies showed that this IPL/STG boundary region is functionally different from other parts of the IPL. For example, the pattern of lesion damage in the pSTG (but not other parts of the parietal cortex) is highly predictive of the incidence of spatial neglect20, suggesting that this part of the IPL may represent a convergence zone of attentional and semantic streams19. Other authors noted that the ventral part of the PF area often reaches to the superior temporal cortex9. Results based on a large sample of human brains revealed that several caudal branches of the STS ascend into the ventral segment of the IPL54, highlighting the importance of relating functional findings to specific gyri or sulci rather than merely ascribing the effects to an overall region. Considering that there might be smaller “sub-units” within area PF, the effects by the group-related manipulation should be interpreted more definitively with respect to the ventral IPL/STG boundary region, rather than the area PF as a whole.
We conclude that task-irrelevant stimuli presented during retrieval can influence temporal order judgment, as a function of their specific mnemonic status determined by both bottom-up and top-down influences during encoding. Task-irrelevant boundary distractors that had received processing derived from stimulus-driven attention at encoding were found to modulate the activity in the anterior rSMG (PFt and PFop subregions) during retrieval, whereas distractors that were encoded under a combination of both bottom-up and goal-related signals were found to affect retrieval activity in a ventral part of the middle rSMG. Our findings reflect a fine fractionation of parietal cortex functions, such that the SMG subregions play differing roles during memory retrieval, as a function the attentional signals allocated to the stimuli during encoding.