Posted comment on ´Neural Global Pattern Similarity Underlies True and False Memories` by Z. Ye, B. Zhu, L. Zhuang, Z. Lu, C. Chen and G. Xue and published in Journal of Neuroscience 22nd June 2016, 36 (25) 6792-6802; DOI: https://doi.org/10.1523/JNEUROSCI.0425-16.2016
Ye and colleagues used as a basis in their investigation of neuronal firing of memories the computational global matching model that hypothesizes that memory strength of a given item is derived from the match (measured as similarity) between its representation and all other studied items (i.e memories) at that time. They looked at the strengths of true and false memories that arose from global similarity of each item`s neural activation pattern during retrieval to that of the groups during encoding and called it ER-nGPS.
In their experiments, Ye and colleagues used fMRI and the participants were scanned during both the encoding and retrieval phases of the memory task, which was an adapted version of the Deese-Roediger-McDermott (DRM) paradigm. The participants, who were 35 healthy college students, were presented visually with 9 word lists each containing 12 words that related to one particular theme. Eight of the 12 words were part of the group study and the other 4 were used as ´critical lures` (words encoded, but not presented in the recall part of the test). Thirty-six semantically unrelated words were also used as ´foils` (non-studied words) in the recognition test.
For the encoding phase, each word was presented for 3 seconds. Then, the participant was asked to perform a perceptual orientation task for 8 secs to prevent further processing of the recently presented word. They were asked to judge each word as pleasant or not and to give each a value from 1-4 by pressing a button. In the retrieval phase, the participants were first given a 2-back working memory task for 10 minutes as a distraction. Then, they were asked on the presentation of each word (36 studied, 36 critical lures and 36 foils) if they judged the word being presented as being definitely new (given a rating of 1) to definitely old (4). These confidence responses rated memory strength. The similarity of the global neuronal firing patterns between encoding and retrieval of the test items was assessed for all 72 studied items.
The participants were also asked to rate semantic similarity. For each task given, they had to assess pairwise semantic similarity to the tested items. In this case only 8 words and 4 critical lures from each word list were used. Judgement for semantic similarity between the two words was tested by rating using a value of 1 for a very weak semantic association to 6 for a very strong semantic association.
The information obtained from the experiments underwent behavioural and univariate activation analyses. In the behavioural analysis, the differences in the endorsement rates of target, lure, and foil items and associated reaction times in the recognition test were examined.
In the univariate activation analysis, single-item response estimation, neural global pattern similarity between encoding and retrieval (ER-nGPS), ROI analysis, mixed-effects model and mediation analysis were carried out.
Ye and colleagues found that in their behavioural tests there was a mean endorsement rate of 90% for targets, 46% for lures, and 11% for foils showing that the participants exhibited high accuracy for true memories, but also showed a high level of false memories. In the rating of items, the authors found that for the targets and lures that the memory strengths of the studied items related to semantic similarity. Target items exhibited high memory strength and semantic similarity was higher for lures than the targets.
For their fMRI-based results, Ye and colleagues used the calculated ER-nGPSs for the individual items which were the neural activation pattern similarities between each item during its retrieval with all other items during their encoding. They examined whether the ER-nGPS was associated with memory strength and found agreement. There was high memory strength in high ER-nGPS areas such as left inferior frontal gyrus (LIFG), left inferior parietal lobule (LIPL), left superior parietal lobule (LSPL), and left ventral lateral occipital complex (LvLOC). This was confirmed by linear mixed-model analysis where increased true memory strength increased the ER-nGPS observed. However, not all categories produced the same results. Therefore, the authors tested the lures, but not the foils because they showed very low memory strength. In the case of the lures, the ER-nGPS increased with memory strength in the LIFG, LIPL, and the LSPL, but not in the LvLOC or right ventral lateral occipital complex (RvLOC). Within the word lists similarities produced similar results. Therefore, the authors concluded that the ER-nGPS of the frontoparietal regions was associated with the strength of both true and false memories, whereas the ER-nGPS in the visual cortex was only associated with the strength of a true memory.
Ye and colleagues also investigated the activity in the medial temporal lobe which is associated with memory. In these experiments they used whole brain searchlight analysis and found that the ER-nGPS was not associated with memory strength. Four regions of interest (ROI) were identified and they found that there was only a slight significant difference between a high memory strength item and one of low strength in the left hippocampus only.
In their experiments on sematic similarity, the ER-nGPS reflected the similarity and hence mediated the effect of semantic global ratings (sGS) on memory strength. They found the similarity in the LSPL, partially in the LIPL, but not in the LIFG and therefore, concluded that ER-nGPS is more sensitive to the content of episodic representation rather than univariate activation level.
In their experiments to investigate if ER-nGPS could differentiate between true and false memories, Ye and colleagues looked at the area which exhibited high strength for the lures, ie. the RvLOC. In this area they found greater activation for targets judged as old than lures judged as old. Two other areas showed the same results: the right intracalcarine cortex extending to the right lingual gyrus and a small cluster in the right superior parietal lobe. When the authors looked at activations for correctly rejected lures and foils judged as new, they found strong activation in a large cluster of the left lateral prefrontal cortex (responsible for cognitive control and conflict resolution) and in a small cluster in the medial frontal cortex (responsible for conflict processing). They also found stronger activation for the group of targets judged as old than that of foils judged as new in the left lateral prefrontal cortex (LPFC), but found no difference between lures judged as old and lures judged as new, or between targets judged as old and lures judged as old. Therefore, their experiments recorded strong positive association between the activation of LPFC and ER-nGPS in the LIPL for both true and false (ie. lure) items. This led to the authors concluding that the activation of the LPFC was associated with the discrepancy of the ER-nGPS in the LIPL and the visual cortex. When a mixed-effect model analysis was carried out, a strong positive association between the ER-nGPS difference (LIPL − RvLOC) and the left LPFC activation was found. Hence, it was concluded that the cognitive control process might result from a discrepancy between the ER-nGPS in the parietal and visual cortices.
The univariate activation level experiments in different brain areas also reflected the activation levels determined from the fMRI analyses. The authors used a mixed-effect regression model and found that after controlling for univariate activation levels, ER-nGPSs were still a significant predictor of true memory strength in all ROIs such as the LIFG, LIPL, LSPL, and LvROC and were a significant predictor for the strength of false memory in the LIPL and a marginally significant predictor in the LIFG. They also found using univariate analysis that there was greater activation for true memory than false memory in the left MFG, bilateral IPL, precuneus, and anterior and posterior cingulate cortices. However, these regions did not overlap with those showing differences in ER-nGPS between true and false memories.
In summary, Ye and colleagues showed in their experiments based on global matching computational model that memory strength of a given item depended on how it was encoded during learning and on its similarity of its neural activity pattern with other studied items. They showed multiple ER-nGPSs carried distinct information and contributed differentially to true and false memories. The location of the ER-nGPS was also found to be important. Parietal regions reflected semantic similarity and ER-nGPSs were scaled to the recognition strengths of both true and false memories, ie. to studied and unstudied items whereas activity in the visual cortex areas contributed solely to true memory. The differences between parietal and visual cortices correlated to frontal monitoring processes. Therefore, it was concluded that multiple neural mechanisms underlie memory strengths of events registered in the brain and this area requires further research and discussion.
Ye and colleagues looked at the neuronal activation patterns measured using fMRI that are associated with real, experienced objects (here, words) and correlated these to whether the participant has seen the word before (termed here a ´true` memory), or not (a ´false` memory) and to the degree of similarity the presented word had to others. They interpreted their results using mathematical modelling and statistical analysis. What makes this article interesting is first, that neuronal activation relating to encoding and retrieval is put on a mathematical modelling basis, and second how this type of measurement gives information about the neurochemical mechanisms involved in how objects are learnt and recalled. The experiments show that instead of researchers looking at neuronal activation patterns for single items, they can actually look at activation patterns for whole groups and see the difference when one member of that group is removed. Essentially, this is what is being used to determine where the default mode network of consciousness lies. Consciousness researchers take a neuronal activation picture of a conscious experience and keep removing the activity from the area with the lowest activation until only one area is left. This was said to be the root of the conscious experience. Ye`s experiments also confirm that activation patterns are strengthened by grouping items and that overall activity reflects the source of the stimulation as well as its actual content. This can be said since if only the content is reflected in the neuronal trace then there would be no discrepancy between the patterns achieved between the imagined or false words and the real images observed in past (ie. true).
So, how can the experiments be explained from the perspective of neurochemical mechanisms? The experiments begin with the encoding of the word lists. When the first word is shown to the participant, the visual pathways are stimulated leading to the activation of the sensory stores and then short term memory stores. The brain`s linguistic centres are activated since the word is recognized because all words used were known. Learning is achieved by repetition as the participant is given 3 seconds to commit the word to memory, essentially a reasonable time in learning terms. This results in the neurochemical mechanisms being activated for long term storage. (Long-term storage is assumed because recall takes over 10 minutes later.)
The first word presented can be said to be learnt ´pure`, ie. without processing because there is nothing to relate it to. Even if processing occurs it is likely to be at a low level and of a general nature inspired because of how the participant knows he will be tested in the future, ie. with the association of meaning. Learning is reinforced by the given task of rating the word according to ´pleasantness`. It is known that assigning an emotional value (the ´emotional tag`) to a memory can affect learning and its later retrieval. In Ye and colleagues experiment, learning is also reinforced by the multi-modality of the task, that is the participant sees the word (visual sense) and then must press a button to give the word the emotional value. Hence, different areas representing the visuomotor areas are activated and all added to the overall pattern of firing. The activity of these areas, however, remain the same for all the words, since only the visual information, emotional value and meaning of the word are different.
The presentation of the second word of the word list instigates essentially the same systems as for the first, ie. the same visual pathway, attentional, memory and motor mechanisms. Apart from the individual characteristics of the word, the presentation of the second word differs because of the level of processing carried out by the participant. It is known that the second word has an apparent link to the first and this link is not visual, but in word meaning. The participant knows that the word belongs to a group and it is likely the word is processed unconsciously since the words used are familiar. This accounts for the speed of whole process. The type of processing carried out is categorization and therefore, the psychologist models of relatedness and schema likely come into play. Accretion probably also applies which is where the addition of a new example to relevant information already in memory (in this case, the previously learnt words) leads to tuning and restructuring if necessary so the schema is more accurate. Inferences could also be used.
It is known that processing and linking to other words results in stronger neuronal cell patterns. The presentation of visual information leads to known patterns of connectivity in neuronal cell firing. Guidotti found that spontaneous brain activity could be evoked by previously presented stimuli. Task evoked patterns to trained stimuli versus novel found patterns in several cortical regions such as the visual cortex, V3, V3A, V7, DFN, precuneus, inferior parietal lobe, dorsal attentional network (intraparietal sulcus which discriminated between trained and novel stimulus). This agrees with areas demonstrating brain activity in categorization such as the V1, basal ganglia and bilateral intraparietal sulcus as shown by Seger. In Ye and colleagues experiments they found activity in areas such as the LIFG (an area associated with speech comprehension), LIPL (language, sensory motor control of writing – here probably the button pushing), LSPL (spatial orientation, sensory information from the hand – normally writing, but here again the button pushing) and LvLOC (associated with the visual process). Parietal lobe activity is also associated with attention, the visual perception-action WHERE model which fits in with word recognition and the required motor processes and working memory with the inferior area associated with multi-modality and the lateral inferior being highly sensitive to memory learning recency, but not repetition. According to theories on the neurochemical mechanisms linked with object recognition, activity in the medial temporal lobe is associated with encoding success and so this V5 area is linked to form, sleep, movement, visual perception, and visual working memory. Therefore, Ye and colleagues results of no activity in this particular area was a surprise.
Again repetition aids the learning process of the second presented word and the whole procedure is repeated until all the words in the list are learnt. In the recall part of the test, on presentation of a word the participant asks himself if the word has been seen before and hence, was one learnt in the previous stage. This is an easier test of recall than one of asking the participant to remember each word presented. It is unlikely that a participant could recall all 12 words from the beginning as a list because he would have needed to have employed mnemonic methods in the learning phase and that is not likely considering the time frame of presentation and learning and the distraction task. Instead, the category is sought out and the features that make up that category and the participant has to rate the level of certainty about whether he has seen this word previously, or not. Jang provides a neurochemical explanation for this as it was shown that the brain encodes experience in an integrative fashion by binding together various features of an event into what was termed an ´event file`. A subsequent reoccurrence of an event feature could then cue the retrieval of the memory file to ´prime` cognition and action. The ´event file` could also include attentional control states, emotional values etc. It was found that areas such as the hippocampus and putamen integrate event features across all these levels in conjunction with other regions representing concrete-feature-selective (primarily visual cortex) and category selective (posterior frontal cortex) and control demand selective (insula, caudate, anterior cingulate, parietal cortex) event information. Hence, according to Jahnke words are learnt as a group and the retrieval of one would mutually generate and support the rest of group. This was seen with sharp-wave ripple complexes (short episodes of increased activity with superimposed high frequency oscillations) occurring during rest and sleep which showed that replay and the SW were tightly interconnected. Such activity was attributed to dendritic sodium spikes found in the hippocampal CA3 and CA1 areas. Recognition of objects is also associated with activity in the perirhinal cortex (Malkora) and Ho showed that this area had a well-established role in familiarity based recognition of individual items. The area responds to novelty and familiarity by increasing or decreasing firing rates. Oscillatory activity occurs in the low beta and low gamma frequency bands in sensory detection, perception and recognition. Stimulation of this area at 30-40HZ causes old items to be treated as novel.
In Ye and colleagues experiments, presentation of each word in the recall part of the experiment requires the participant to make a decision of whether he has seen this word before, or not and perform a motor action. Therefore, neuronal traces also show activity in those areas involved in decision making, ie. the strength of the cortico-striatal pathway and prefrontal cortex (Daw, Chung-Chuan), parietal cortex (guidance of eye movements), basal ganglia, motor structures. This activation, just like those areas representing the visuomotor mechanisms of the button pushing, is the same whether the item has been encoded or not. Therefore, Ye and colleagues could conclude that the only difference between the neuronal traces observed, apart from visual attributes and meaning, was whether the word had been visually seen during encoding or not. Activity in the occipital cortex was observed for words presented during the encoding part of the test (´true` memories), whereas its absence denoted ´false` memories. This expands Fuentemilia`s observations that ´true` memories cause activity in the inferior longitudinal fascile (a major connective pathway of the medial temporal lobe), whereas ´false` memory relates to activity in the superior longitudinal fascile. In this case, the so-called ´false` items relate to visual imagery with the firing of multiple common features including general meaning, but not all are correct. Therefore, the task given is more difficult to get 100% correct and this was proven by the high number of false calls. Brascamp explains this by saying that when an individual knows he is faced with inconclusive or conflicting perception then there is a dominance of whatever perceptual interpretation was commonly reported on a previous encounter. In this case, by asking if a word had been seen before, the brain processed it as relating to the ´meaning` and the likelihood that it had been. Therefore, the word was deemed as being familiar. In order to achieve higher scores, the participant would need to divorce this feeling of familiarity with recognition of the word in a visual capacity only.
In Ye and colleagues experiment, there was no official feedback as to the level of correct or incorrect answers given either instantaneously or at a later date and therefore, with the former, no real-time feedback processing. The participants may have intuitively felt that an error had been made and firing activity would then be visible in the anterior cingulate cortices and amygdala areas. However, again likelihood of activity in these areas would be the same for all words and would enhance the overall neuronal activation pattern rather than be present and specific for either real or imagined word groups.
Therefore, it can be concluded that Ye and colleagues experiments show that neuronal activation relating to encoding and retrieval can be put on a mathematical modelling basis and activation is better if whole groups are considered with the required single object being removed from this group rather than just looking at the activation pattern of the object on its own. This type of measurement shows that retrieval appears to be improved when an object is learnt with its meaning and with others of the same category and not by word structure, or by order. This observation could lead to new learning techniques in the case of reminders for example which are important in prospective memory or for those suffering from forgetfulness.
Since we`re talking about the topic……………………
……does emotional attachment to words change global activation patterns? Does the level of anxiety shown by a participant during the test change the activity patterns during encoding and retrieval and does it change the categorization of the words?
…….would instigation of instant feedback during retrieval, eg. giving a positive or negative visual sign change the speed or accuracy of the following replies and change the global activation patterns achieved?
……would a concurrent testing of brain waves within the parietal cortex, perirhinal cortex and hippocampus show the theta, gamma brain wave synchronicity and would these change during the course of test or by presentation of encoded or novel words?
…. would tests with patients with ventral medial prefrontal cortical lesions show that the number of false results is decreased compared to the control group since this brain area is linked to increasing the influence of schematically congruent memories (Warren)?