Content area
Searching within natural scenes can induce incidental encoding of information about the scene and the target, particularly when the scene is complex or repeated. However, recent evidence from attribute amnesia (AA) suggests that in some situations, searchers can find a target without building a robust incidental memory of its task relevant features. Through drawingbased visual recall and an AA search task, we investigated whether search in natural scenes necessitates memory encoding. Participants repeatedly searched for and located an easily detected item in novel scenes for numerous trials before being unexpectedly prompted to draw either the entire scene (Experiment 1) or their search target (Experiment 2) directly after viewing the search image. Naïve raters assessed the similarity of the drawings to the original information. We found that surprise-trial drawings of the scene and search target were both poorly recognizable, but the same drawers produced highly recognizable drawings on the next trial when they had an expectation to draw the image. Experiment 3 further showed that the poor surprise trial memory could not merely be attributed to interference from the surprising event. Our findings suggest that even for searches done in natural scenes, it is possible to locate a target without creating a robust memory of either it or the scene it was in, even if attended to just a few seconds prior. This disconnection between attention and memory might reflect a fundamental property of cognitive computations designed to optimize task performance and minimize resource use.
Abstract
Searching within natural scenes can induce incidental encoding of information about the scene and the target, particularly when the scene is complex or repeated. However, recent evidence from attribute amnesia (AA) suggests that in some situations, searchers can find a target without building a robust incidental memory of its task relevant features. Through drawingbased visual recall and an AA search task, we investigated whether search in natural scenes necessitates memory encoding. Participants repeatedly searched for and located an easily detected item in novel scenes for numerous trials before being unexpectedly prompted to draw either the entire scene (Experiment 1) or their search target (Experiment 2) directly after viewing the search image. Naïve raters assessed the similarity of the drawings to the original information. We found that surprise-trial drawings of the scene and search target were both poorly recognizable, but the same drawers produced highly recognizable drawings on the next trial when they had an expectation to draw the image. Experiment 3 further showed that the poor surprise trial memory could not merely be attributed to interference from the surprising event. Our findings suggest that even for searches done in natural scenes, it is possible to locate a target without creating a robust memory of either it or the scene it was in, even if attended to just a few seconds prior. This disconnection between attention and memory might reflect a fundamental property of cognitive computations designed to optimize task performance and minimize resource use.
Keywords Attribute amnesia - Drawing - Memory recall - Scenes - Visual search
Many theories of visual processing assume or posit a strong connection between attention and memory, such that information we directly attend to, like the target of a search task, is stored in memory (Cowan, 2001; Hollingworth, 2005; Logan, 1988). However, the relationship between search behavior and subsequent memory for the scenes and the objects being searched for is complex. In general, searching for objects leads to memories of those objects and related elements in the scenes they inhabit (Castelhano & Henderson, 2005; Draschkow et al., 2014; Helbing et al., 2020; Williams et al., 2005). Scenes learned incidentally in a change blindness task (i.e., visual search for change) could be recognized as well as explicitly memorized scenes even 2 weeks after exposure (Utochkin & Wolfe, 2018). However, whether these memories can be accessed seems to be dependent on the way in which memory is used. For example, a repeatedly searched scene could speed up future searches, but only when the search target also repeated (Vo & Wolfe, 2012).
These studies primarily used recognition tasks or indirect measures of memory. However, in a recent series of studies (Bainbridge et al., 2019; Hall et al., 2021; see also Bainbridge, 2022, for a tutorial), it was demonstrated that observers can build richly detailed memories of multiple objects across multiple scenes, allowing them to recreate the appearance and spatial layout of those scenes through drawings with an astoundingly high quality. The drawing method has also been applied to assess memory for images seen during a visual search task. For example, Draschkow et al. (2014) found that participants were better able to draw objects they have searched for compared with objects they were instructed to memorize within natural scenes.
Though across the literature there is strong evidence that search can create a robust incidental memory trace for the target and related objects in a scene (Castelhano & Henderson, 2005; Draschkow et al., 2014; Helbing et al., 2020; Utochkin & Wolfe, 2018; Williams et al., 2005), a recent line of research has found evidence to the contrary. Specifically, Chen and Wyble (2015) asked participants to search for a letter among digits and report its location for tens of trials before presenting a surprise trial, in which participants were asked to report the identity of the letter instead of its location. Though the identity of the letter was relevant to search, participants had difficulty reporting the identity of the target-defining attribute in this surprise trial, a phenomenon termed attribute amnesia (AA). This finding extends to other attributes such as the color of a singleton (Born et al., 2019; Chen & Wyble, 2016; Harrison et al., 2021; Wang et al., 2021), a young face among old faces (Tam et al., 2021), a sad face among happy faces (Chen et al., 2019), the spatial configuration of the search array (O'Donnell et al., 2021), and even the color of a ball that is being carefully tracked in a video of ball-passers for up to a minute (Chen et al., 2016). Accuracy in recognizing the target is much higher on the very next trial after the surprise trial, suggesting that the attribute recognition task is easy when participants expect that they will have to report that information, prompting a switch to a more intentional encoding strategy. The memory failures demonstrated in AA have been theorized as an adaptive optimization process where only the response-relevant feature remains in the encoding set after repeated experience with the task (Chen et al., 2019; Hedayati et al., 2022; Wyble et al., 2019). Crucially however, most of these studies used simple stimuli (e.g., colored shapes; Chen & Wyble, 2016; Harrison et al., 2021) or complex stimuli without a scene context (e.g., isolated faces on a white background; Chen et al., 2019; Tam et al., 2021).
The findings of AA show that response-irrelevant details of search targets and environments are not always well encoded into memory. This is seemingly contradictory to previous studies showing that visual search in scenes induces a robust incidental memory of the search target and/ or scene. This difference could be due to various differences in the task design; for example, AA studies use a very small and simple search array, typically of four discrete objects in contrast to the richly detailed semantics of a natural scene.
In the current study, we adapted the standard AA paradigm to measure the incidental memory of scenes and targets formed from searches of real-world targets in naturalistic scenes. Moreover, we used visual recall techniques developed by Bainbridge et al. (2019) to not only collect drawings from participants via online data collection but also to evaluate the memory quality of produced drawings through crowd-sourced rater assessment (Bainbridge, 2022).
Specifically, we used a surprise drawing task to capture the mental representation of the attended-to scenes and search targets directly after search without presenting response alternatives as in a typical AA paradigm. Importantly, the "pre-surprise" search task was designed to embody design specificities in a typical AA search task that may work to minimize incentives to remember any information about the scene. Specifically, the search target was held constant across trials for each participant and was always in full view within the scene. Scenes also never repeated across trials, so remembering information about the scenes would not benefit future searches. Additionally, the response format of both the presurprise and surprise tasks used the same drawing tools to rule out the possibility of errors induced by unexpected response format changes (Cohen-Dallal et al., 2023; Swan et al., 2017), and the incidental memory probe appeared immediately after viewing the search display to minimize the opportunity for information to be lost due to decay and interference (Averell & Heathcote, 2011; Souza & Oberauer, 2015; Zhang & Luck. 2009). In each experiment, participants first completed 43 trials of search in which they reported locations of the search targets. They were then asked on a surprise trial to draw either the entire scene (Exp. 1) or the search target (Exp. 2). Should the act of search automatically create robust memories in naturalistic scenes, naive raters would be able to create easily recognizable drawings of the studied scene and target (Bainbridge et al., 2019). However, if the typical AA search task is able to deincentivize memory formation, surprise drawings of the search scene and target will be difficult to recognize.
Experiment 1
In this first experiment, we wanted to measure the quality of incidental working memory of the entire scene. Participants were tasked with finding targets from the same search category in nonrepeating indoor scenes and reporting their locations for tens of trials before being unexpectedly asked to draw the entire scene. Based on previous work with AA, we predicted that participants' memory of the scene would be poor, even though they were directly attending to it moments before.
Methods
Participants
Thirty-eight Pennsylvania State University undergraduates (Mage = 19.03 years, range: 18-22, 50% identify as female, 18% left-handed) participated in this online study after passing an exclusion criterion requiring sufficient engagement with the pre-surprise task (one excluded) and were compensated with course credit for their participation. Participants completed the study using their personal computer devices (36 desktops/laptops, one tablet, one smartphone). Additionally, participants reported their drawing ability (from a scale of 1 to 9), and the average drawing ability of participants was 1.95. This experiment (and all others) was approved by the Pennsylvania State University IRB, and all participants provided informed consent before participating.
Apparatus and stimuli
The study was designed with custom JavaScript and HTML code provided by Bainbridge (2022), wPaint code to design the drawing canvas (Websanova, 2014), and PsychoJS code (Version 2020.2) to host the experiment on Pavlovia (Peirce et al., 2019). We used 47 bedroom and living room scenes chosen from the FIGRIM image set (Bylinskii et al., 2015). Each scene possessed visible "pillows" and "wall-art" to correspond with our pre-surprise trial tasks. Furthermore, we made sure that the vicinity of the "pillows" was located generally in the bottom half of each image, while "wall-art" was located in the top half. The purpose of this consistency in location was to further aid in the ease of search. Each image was 500 by 500 pixels, which was the same size as the canvas. Four of the 47 images were reserved to be shown on only the surprise and control trials (Fig. 1C). To reduce variability of the critical surprise and control trials, the image shown on the surprise trial could be one of two specific images (counterbalanced across participants) with the other image shown on one of the control trials. This ensured that the image shown on the surprise trial for half of the subjects was also shown on the control trial for the other half. The other two images were reserved to only be shown on control trials.
Participants responded using a drawing canvas created through wPaint, an online paint function with various tools that allow free-hand drawing, pre-formed shape drawing, and the ability to change colors of the stencil. This drawing canvas was used to record participant responses on all trials (both target location and scene-drawing tasks), with only the instructions of what to draw changing across trial phases. This ensured that participants did not have to learn how to use a new response tool between the pre-surprise and surprise portion of the task.
Procedure
The procedure of Experiment 1 is shown in Fig. 1A. The experiment began with a practice trial in which participants were asked to draw a house using the drawing tools provided. As with all other trials, this canvas was shown without a time limit. The practice trial allowed participants to familiarize themselves with the various drawing tool functions.
Following the practice trial, participants read the instructions for the search task, and began the pre-surprise portion of the experiment.' In each trial, one scene appeared for 2,000 ms and was then followed by the blank drawing canvas. The participant was prompted to "circle the location of any wall-art," or "circle the location of any pillows." according to the condition they were in (randomized across participants). Participants circled the location in which they remembered their attended object on the canvas. This process repeated for 43 trials in total, with images presented in a randomly chosen sequence.
After 43 pre-surprise trials, participants were shown another scene for 2,000 ms and were then presented with a pop-up window (Fig. 1B). The pop-up alerted participants that the instructions were going to change, ensuring that they did not ignore or miss the surprise trial instructions." Once participants clicked out of the pop-up window, they were prompted to "draw the whole room." Additional instructions were provided below this new prompt encouraging them to draw the room using as much detail as possible, or to draw an X across the canvas if they remembered nothing about the scene. This surprise trial tested what participants could recreate from the scene when they did not have the expectation to remember it, although they were attending to their search target.
Finally, there were three control trials that were identical to the surprise trial except that there was no pop-up window. These trials served as a baseline measure of a participant's visual recall ability, since the participants recreated what they remembered from a scene but now with the expectation to remember.
Analysis
Drawings on the surprise and control trials were assessed by two groups of raters. The first group, naive raters recruited via Prolific (N = 279, 123 females, M = 27.2 years), participated in the scene recognition task. Raters had to recognize the drawings in a three-alternative forced-choice (3AFC) format (see Fig. 2A). Specifically, a drawing was shown on each trial and the scene that the drawing was based on was presented among two other category-matched scene foils that were never used in the drawing task. Raters were asked to pick the scene they thought the drawing was most likely based on. Each rater assessed the drawings from one or two drawers (i.e., four or eight trials) in a random order. They were compensated with approximately 10USD/hour (prorated per minute). Importantly, raters were not provided with any information about the task conditions in which the drawings were produced, meaning that they had no idea some drawings were produced on a surprise trial. Each participant drawing was assessed by a minimum of 10 raters.
The second group of raters (N = 5, 1 female, М = 30.8 years), consisting of the authors and a lab member, participated in the object detection task. Items in scenes used on the surprise and control trials were annotated with translucent color shades and each item was assigned a number. On each trial of the item recognition task, an annotated scene and a drawing based on that scene were shown side by side, and raters indicated the items they thought were present in the drawing by the items' numbers (see Fig. 2B). There was no lower or upper limit as to how many items a rater could indicate as present for each drawing. Each rater rated all drawings (152 trials). Although the raters knew some drawings were produced on a surprise trial, there were no cues in the rating paradigm to indicate which portion of the task the drawing was from.
Results
Overall, participant pre-surprise drawings reflected accurate memory for the location of the target object(s), indicating that they were sufficiently engaged in the pre-surprise task (see Fig. 1D)? Additionally of note, on the surprise trial, 10 of 38 (-26%) participants self-reported having no memory for the presented scene by drawing an X across the canvas. However, it is difficult to verify whether X responses indicated a loss of memory. Instead, X responses could indicate low effort responding or a hesitancy to draw due to a lack of confidence in their drawing ability or memory. Because of this, we removed the instructions about X drawings in the following experiments and where possible treat X drawers as an additional factor in the analyses of Experiment 1. The number of Xs drawn on the three control trials were 0, 1, and 4, respectively. Example participant drawings can be seen in Fig. 3A.
3AFC scene recognition task
Participant drawings were subdivided between those who drew Xs and those who did not (Fig. 3B). Rater recognition accuracy was assessed via a 2 X 4 mixed-factors analysis of variance (ANOVA), with between-subject factors drawing type (X vs. no X on surprise) and within-subjects factor trial order (surprise, Control 1, Control 2, Control 3). We observed a main effect of trial order, F(3, 108) = 26.74, р < .001, np = .43, with surprise trial drawings being significantly less accurate than the control trial drawings (all Bonferroni-corrected p values comparing surprise to each control trial < .001). There was no significant main effect of drawing type, F(1, 37) = 0.03, р > .8, п = .001, nor was there an interaction, F(3, 108) = 1.01,p = .39, п = .03.
Because we used two different surprise trial pictures, we also compared surprise versus control accuracy of each image when it was shown at the respective trial type (Fig. 3C). Thus, two independent-samples г tests were conducted using only ratings from non-X drawings, and we observed a significant increase in accuracy from surprise to control for both Image 1, #25) = -5.9, р < .001, and Image 2, (25) = -2.49, р = .02. For both Image 1 and Image 2, we additionally compared surprise accuracy with chance rater identification performance (33%) using both a one directional single-sample f test and an accompanying Bayes factor analysis, and found that surprise accuracy was no greater than chance for Image 1, 1(13) = -2.45, р > .9, BFui), = 10.25, but greater than chance for Image 2, 1(12) = 2.79, p = .008, BF,, = 0.13.
Object detection task
Our secondary task was to detect the prevalence of specific objects present in the participants" drawings (of those who did not draw an X). We tested pre-surprise targets drawn on the surprise trial with a 2 x 2 mixed-factor ANOVA, with factors pre-surprise target (pillow vs. wall art) and object drawn (also pillow vs. wall art). We observed no main effect of the pre-surprise target, F(1, 27) =.12,p = .73, Tp =.005, and no main effect of the object drawn, F(1, 27) = .19, р = .67, п" = .007. We observed a cross-over interaction of these two conditions, F(1, 27) = 52.6, р < .001, п = .67 (Fig. 4A). Post hoc analyses confirmed that pillows were more likely to be included in surprise trial drawings when the participants were given the pillow task, (13) = 4.82, p < .001, while wall art was more often drawn when participants were given the wall-art task, 1(13) = -5.43, р < .001.
We additionally analyzed the total number of objects drawn between surprise and control for the two surprise trial scenes via two paired samples t-tests: one for the full group and one using only the participants who did not draw an X at surprise (Fig. 4B). We observed a significant difference in number of objects drawn between surprise and control for the full group of participants, #37) = -2.09, р = .044, with more objects drawn on the control trial. However, when only analyzing participants who attempted to draw on the surprise trial, we observed no significant difference between surprise and control, 1(27) = -0.41, р = .68. We confirmed that this result was significant evidence in favor of the null hypothesis (no difference in objects detected between surprise and control) via a Bayesian paired-samples f test, BF), = 4.61.
Discussion
Our results suggest that search itself is not sufficient to form a robust incidental memory of the search scene, as drawings of scenes on the surprise trial could not be reliably recognized by raters, yet performance was dramatically improved on the following control trials. This pattern was consistent regardless of whether "X" drawings were included in the analyses or not. The results are consistent with previous studies showing that incidental memories for search non-targets are usually poor (Castelhano & Henderson, 2005; Draschkow et al., 2014; Williams et al., 2005). This poor incidental memory of the scenes was observed despite participants showing evidence of being able to report the location of the search targets, suggesting they were actively searching in the scenes. Furthermore, across all drawers, raters detected significantly fewer items from the surprise-trial drawings than the control ones, although this pattern disappeared when X drawers were excluded. This suggests that the quality, but not the quantity of the items being drawn was inferior in the surprise drawings. In the General Discussion, we return to this point. We also provide a detailed report of item-specific detection rates in the supplementary materials for reference (Table S1).
Another interesting observation from Experiment 1 is that the specific pre-surprise search target was often detected by raters in the surprise-trial drawings. Therefore, although there was not necessarily a robust incidental memory of the entire scene, participants might have built an incidental memory containing the visual details of the search target itself in addition to its location. However, it is also possible that since participants new the target type (pillow or all art) and its location, they could sketch a roughly square shape in the correct location, which would be recognized by the rater. To evaluate these two possibilities, in Experiment 2 we directly tasked participants to unexpectedly draw only the search target, in detail, on the surprise trial.
Experiment 2
In our second experiment, we investigated what details about the attended object the participants could recall. Our design was similar to Experiment 1, except that we asked participants to draw their target object in the surprise/control trials instead of the entire scene. Furthermore, all participants were given the task of circling the "wall-art" instead of being divided between wall-art and pillows, as the wall-art objects contained greater detail within them compared to the pillow targets, which made it easier for participants to draw identifying information.
Methods
Participants
A new sample of 40 Penn State undergraduates (Мо, = 18.75 years, range: 18-22, 80% identified as female, 12.5% left-handed) participated in this online study after passing an equivalent exclusion criterion to Experiment 1 (zero excluded). Again, participants completed the study using their personal devices (37 used desktops/laptops; two used a tablet/smartphone), and the average self-reported drawing ability of participants was 2.1 out of 9.
Apparatus and stimuli
All stimuli and materials are identical to Experiment 1 except for the following changes. The two surprise trial images with wall art were edited in Adobe Photoshop. The second smaller wall art in Image 1 was removed, and the wall art in Image 2 was changed to an abstract art piece with simple features found on Google Images. Additionally, there was one fewer control trial in this experiment; thus, Image 4 was never used as a scene in this study and Image 3 was always shown on Control Trial 1. Finally, to simplify the drawing canvas, drawers were only provided a handful of tools to recreate the wall art, leaving them with only the ability to free-draw using a stencil, change the color of the ink, erase lines, and undo whole edits if necessary.
Procedure
The procedures of Experiment 2 were identical to Experiment 1, except for the following changes. Participants were only instructed to search for wall art during the pre-surprise portion of the experiment. After 43 trials, participants were presented a pop-up window, and a surprise question that read "Surprise! Draw the wall art." Participants then completed two additional control trials where they drew the wall art from memory. Whichever of the two surprise images not shown at surprise was instead shown on Control Trial 2. Unlike Experiment 1, in all wall art drawing trials, participants were not instructed to draw an X across the canvas if they had no memory of the wall art.
Analysis
Drawings from the surprise and control trials were assessed by a group of naive raters from the Penn State participant pool (М = 204, 149 females, M = 18.6 years) in a wall art recognition task similar to the scene recognition task used in Experiment 1, except that the target wall art was now presented with three wall art foils (4AFC) selected from Google Images and edited to approximate the resolution and orientation of the target. One of the foils was a close match as the subject in the art was similar to the target's, whereas the other two foils were far matches with a different style of art (Fig. 5D). However, all foils were in the same art category as the target, being either abstract or scenery. Each rater assessed drawings from two participants (six trials) and were compensated with course credits. Each drawing was assessed by a minimum of 10 raters. Drawings were considered to have captured the gist of the wall art if the raters selected the close match foil (as well as the target). The order of the images was randomized on every trial.
Results
4AFC scene recognition task
Example wall art drawings can be found in Fig. SA. A one-way repeated-measures ANOVA assessed significance of surprise trial target recognition accuracy to control trial recognition accuracy. We observed a significant main effect of trial type, F(2, 78) = 13.95, р < .001, п = 26, with surprise trial performance being significantly worse than Control Trial 2, 1(39) = -5.06, р < .001, but not Control Trial 1, #39) = -1.21, р = 7. When analyzing gist recognition of the drawn scenes, we observed a significant main effect of trial type, F(2, 78) = 21.74, р < .001, Tp" = 36, and surprise trial performance was significantly worse than both Control Trial 1, (39) = -3.61, р = .001, and Control Trial 2, (39) = -6.58, р < .001. When analyzing accuracy changes between the two images shown at surprise (Fig. 5C), we observed that both the abstract wall art and scene wall art were less recognizable on the surprise compared to when shown on the control trial, both ps < .005. This pattern was also present for gist memory, both ps < .001. We additionally compared surprise accuracy with chance rater identification performance for both true target memory (25%) and gist memory (50%) using a one directional single-sample t test and accompanying Bayes factor analysis. We found that neither true target surprise accuracy, 1(19) = -1.16, р = .87, BF; = 8.41, nor gist accuracy, #19) = -2.51, р > .9, BF, = 12.78, were greater than chance for the scene wall art (i.e., Image 1), However, both true target surprise accuracy, #(19) = 1.98, р = .031, BF; = 0.44, and gist accuracy, 1(19) = 1.88, р = .038, BF; = 0.52, were greater than chance for the abstract wall art (i.e., Image 2; though the Bayes factors for both target and gist memory suggest only weak evidence in favor of the alternative hypothesis).
Discussion
In Experiment 2, we found that participant's drawings of the wall art on the surprise trial were oft much less recognizable in comparison with drawings of the same wall art on the control trials. These results complement the pattern of results observed for surprise whole scene drawings in Experiment 1. In addition, not only were naive raters unreliable when identifying the exact source image for the surprise trial drawings, but they were also unreliable at recognizing the gist information from those drawings. This suggests that incidental memory of the search target itself (beyond information related to pre-surprise response expectations) is low in quality following an AA search task. However, a remaining caveat is that in both Experiment 1 and 2, a surprise prompt appeared right before participants drew the scene or the wall art, which has been previously shown to impair memory performance (O'Donnell & Wyble, 2023). We attempted to rule out the competing hypothesis of surprise-related interference in the next experiment.
Experiment 3
In previous studies that explored the impact of surprise on working memory retention, it was shown that expectation violations may damage existing memory representations (O'Donnell & Wyble, 2023; Swan et al., 2017; Tam & Wyble, 2022; Wessel, 2018; Wessel et al., 2016). Thus, it is possible that surprise trial memory failures could be attributable to the presentation of an unexpected pop-up window.
To measure the effect of such interference on surprise trial accuracy we designed a third experiment to test whether the onset of a surprise prompt degraded the recognizability of participants' drawings. In this experiment, participants were explicitly told they would have to draw the whole scene on some trials, but on the last trial they were presented with an unexpected pop-up window identical to the one used in Experiments 1 and 2. If the presentation of the pop-up window interferes with the memory of the studied scene, then the participant's final drawing should be less recognizable by raters than any previous drawings.
Methods
Participants
A new sample of 38 Penn State undergraduates (M,ge = 18.89 years, range: 18-23, 87% identified as female, 3% left-handed) participated in this online study after passing an equivalent exclusion criterion to Experiment 1 (zero excluded). Again, participants completed the study using their personal computer devices (37 laptops/desktops, one tablet), and the average self-reported drawing ability of participants was 2.14 out of 9.
Apparatus and stimuli
All stimuli and materials were identical to Experiment 1, except for the following changes. The number of scenes shown on the non-drawing trials was reduced to 39, and one of the unused scenes was used as a drawing trial scene (five in total). The same two scenes that could appear on the surprise trial of Experiment 1 were counterbalanced across participants to appear on the pop-up trial. All other scenes were randomly selected to appear on the other four drawing trials. Like Experiment 2, the possible drawing tools participants could use were reduced to the free-draw, erase, ink selection, and undo buttons.
Procedure
The procedure of Experiment 3 was identical to Experiment 1, except for the following changes. Participants were initially informed that they would need to study a scene and draw it from memory, but they were told they would not need to draw every studied scene.' If the scene was not to be drawn, the participants would see text that read "Nothing to draw! Click to see the next image" and click a "Next Image" button during the drawing phase to start the next trial. If the scene was to be drawn, the canvas would appear. There were five drawing trials for each participant, which appeared at Trial 2, Trial 11 3, Trial 22 3, Trial 33 3, and Trial 44, the same trial number as was the surprise trial in Experiments 1 and 2. On the drawing screen, there was additional text reminding the participants to draw the scene with as much detail as they could remember. Crucially, prior to the onset of the last drawing trial, a pop-up appeared in front of the canvas that was identical to Experiment 1 which read: "Surprise! Remember the image you just now. Now read the instructions." Participants had to click it away before drawing. The experiment ended after the last drawing was submitted. Participants were not informed which trials would be drawing trials, nor were they told how many trials there were in the task (i.e., they did not know the pop-up trial was the last trial of the experiment). Like Experiment 2, participants were not told they could draw an X if they had no memory of the scene.
Analysis
All drawings were assessed by a group of naive raters from the Penn State participant pool (N = 197, 150 females, M = 18.9 years) in a scene recognition task similar to the one used in Experiment 1. Each rater assessed drawings from two participants (10 trials) and were compensated with course credits. Each drawing was rated by a minimum of 10 raters.
Results
3AFC scene recognition task
Example scene drawings can be found in Fig. 6a. We ran a one-way repeated-measures ANOVA, with factor trial order, and observed no significant main effect, F(4, 148) = 1.18, r = 32, p = .03. Additionally, we computed a repeated measures Bayes factor, and found strong evidence in favor of the null hypothesis, BF), = 10.22, r fixed effect = 0.5 (default JASP parameter; Rouder et al., 2012), suggesting no difference in rater accuracy between the pop-up trial and any other drawing trial (Fig. 6B). This lack of difference was present when analyzing accuracy for both scenes shown on the pop-up trial versus a no pop-up trial (Fig. 6C). There was minor evidence in favor of the null for Image 1, 1(36) = 0.73, р = .47, BF,, = 2.58, Cauchy scale = .707, and stronger evidence for the null for Image 2, #36) = -0.23, р = .82, BF, = 3.11.
Discussion
From Experiment 3, we found that there is little observable effect of interference induced by the surprise pop-up display. This lends support to the idea that surprise trial failures in Experiment 1 and 2 were due to weak incidental memory, rather than the memory being damaged due to surpriserelated interference. Participants learned to exploit task parameters and subsequently encode into memory only what was believed to be necessary to report post-search (March, 1991; Wyble et al., 2019).
General discussion
It has been previously demonstrated that search for objects in natural scenes creates incidental memory of those scenes and the searched objects (Castelhano & Henderson, 2005; Draschkow et al., 2014; Helbing et al., 2020; Utochkin & Wolfe, 2018; Williams et al., 2005). However, we demonstrated that this is not necessarily the case using an AA task coupled with visual recall. In our experiments participants had difficulty unexpectedly recalling both the scene and the target itself when the search task was simple. Participants searched for a specific target category in unique scenes for 43 trials and were then prompted to draw the scene they just searched in (Experiment 1) or the target they just located (Experiment 2). We found that surprise trial drawings from both experiments were significantly less likely to be correctly identified by naive raters compared with drawings from the subsequent control trials and in some cases, raters were near-chance-level performance in selecting the correct source image from the foils. Therefore, visual search does not necessitate encoding of the scene or even the target into memory. Finally, we wanted to determine whether the effect we saw was due to interference caused by the surprise prompt. The results of Experiment 3 provided no evidence that the surprise prompt was causing interference in participants' ability to recreate the scene.
These results suggest that search itself is not sufficient to form robust visual memories of the target or search environment even in scenes. Rather, the parameters of the search inform an individual's task set, perhaps incentivizing the formation of memories only if it aids them in completing a task. In other words, participants learn to exploit task parameters in order to optimize performance (March, 1991), which might serve as a means to minimize resource use. This exploitation is akin to the concept of a mental set, wherein an individual develops a specific means of completing a task, and executes that strategy even if it becomes suboptimal (Luchins, 1942); however, in this case participants learn to encode only what is necessary to complete the target localization task even if it leaves them exposed to unexpected task alterations (see also Hedayati et al., 2022; O" Donnell & Wyble, 2023; Wyble et al., 2019). Though this can be viewed as a susceptibility of the cognitive system to expectation biases, it also highlights the remarkable ability of the human mind to optimize task performance in mere minutes, both in terms of minimizing resource use as well as quickly correcting to not be caught by surprise a second time.
One factor that may play a role in explaining the discrepancy between our results and the results of previous studies is search difficulty. In most of the studies that found evidence of search induced memory formation, regardless of the response method used, the search task was relatively difficult, with targets that varied from trial to trial (Castelhano & Henderson, 2005), conjunction targets, such as a white telephone (Williams et al., 2005), or dense visual scenes with many overlapping objects (Vo & Wolfe, 2012). It has been found that increasing the difficulty of the search by increasing the search target memory load increases the strength of memory formation for search array elements following search (Guevara Pinto et al., 2020). Thus, search difficulty may create an incentive to remember both the scenes and objects in order to efficiently search within the scene itself. In contrast to harder searches, recent research and the results of this study suggest that when searching for targets in small arrays with clear differences relative to distractors, response-irrelevant details of targets are not well remembered. In other words, when memory is not incentivized, robust incidental working memories may not be formed, suggesting that memory formation may be adaptive, and strongly influenced by the parameters of the specific task.
These findings also expand our understanding of attribute amnesia. Since participants were unable to accurately recreate the details of the scene or object they attended to, we can conclude that AA is readily observed in a more natural setting and with visual recall on a blank canvas, as opposed to the forced-choice recognition paradigm that is typically used. This finding lends support to attribute amnesia as an encoding failure rather than a forgetting failure, as details of the target can be lost even when participants have the ability to visually recall information in whatever order they prefer, minimizing retrieval-induced forgetting (Anderson et al., 2000) that could occur when participants have to read the various options in a recognition question. Moreover, our results further suggest that attribute amnesia reflects a failure of explicit memory retrieval of the most recent trial, but does not preclude the formation of implicit memories (see also Harrison et al, 2021; Jiang et al., 2016; O'Donnell et al., 2021). Specifically, it has been previously shown that despite being unable to report response irrelevant features on a surprise trial, participants can still provide answers informed from learned summary statistics of previous trials (O'Donnell et al., 2021; Sasin et al., 2023). Similarly, in our observations, participants were not able to create drawings that were diagnostic of the memory source but were able to reconstruct memories with recognizable objects, perhaps through schematic reconstruction (Bainbridge et al., 2019; Brewer & Treyens, 1981), which requires only knowledge of the semantic category of the scene, or previously seen scenes.
As an aside, the results of Experiment 3 provide context for the role that proactive interference can play in the surprise trial accuracy in Experiments 1 and 2 (Greenberg & Underwood, 1950; Keppel & Underwood, 1962; Postman & Underwood, 1973), or the failure to recall newer memories due to retrieval-based competition with previously stored memories. Since the number of scenes viewed/studied in Experiments 1, 2, and 3 were equated prior to the onset of the surprise drawing or pop-up trial, and yet accuracy was high only in Experiment 3, it is likely that visual memory of the most recently viewed scene is not damaged by the mere repeated exposure to similar memoranda. We cannot, however, rule out the role of proactive interference given that scene images may be processed further after visual search as in Experiments 1 and 2, compared with explicit memorization as in Experiment 3 (see also Draschkow et al., 2014). Therefore, the amount of proactive interference accumulated through the pre-surprise trials might have been higher in Experiments 1 and 2. Further testing is required to determine any potential effect of proactive interference.
Finally, we wish to return to the discussion of unexpected results observed in Experiments 1 and 2. First, why could the raters detect target objects in Experiment 1, but could not match them to the correct foils in Experiment 2? We hypothesize that in the object detection task of Experiment 1, the drawers could use their knowledge of the location of the target to outline the pillow or wall art in the whole scene recreation. However, this wall art or pillow may have been generic and lacking in detail (perhaps a mere rectangle drawn in the relative position of where the target was located), which would have been enough information for a rater to identify the object as present in the drawn scene, but not enough detail to determine the exact wall art drawn. Previous research has suggested that participants can rely on knowledge of the pre-surprise target's location to intuit surprise trial answers (O' Donnell et al., 2021), and our results further exemplify this notion: participants had enough information to put the target in the correct location. This, coupled with the object detection task requiring raters only identify the outline of an object in close proximity to its location in the scene, may have led to the increased detection of search targets in the surprise trail drawing. Conversely, the rater recognition task of Experiment 2 removed location as a diagnostic feature for recognizing the target, making visual details necessary. That raters could not recognize targets supports our conclusions that little to no memory for the non-location details of the search target formed incidentally. They only encoded what was necessary to complete the pre-surprise task.
Second, why was the proportion of detectable objects between surprise and control in Experiment 1 near equivalent after filtering out participants that drew an X, although surprise drawings were still more difficult to recognize? In other words, among non-X drawers, although a similar number of objects was drawn in the surprise and control trials, the objects drawn in the surprise trial were less diagnostic for recognizing the studied source. While our current study cannot speak to the exact mechanism why this may be, inspecting the item-specific detection rate provides some insights (Table S1). First, the item detection rate in surprise trial drawings may be critically supported by the high detection rate of objects corresponding to the pre-surprise target type (pillow or wall art). Table S1 shows that the pre-surprise targets received uniformly high detection rates in the surprise trial, but some of these detection rates were drastically reduced in the control trials. This suggests that the pre-surprise search task induced reliable memory of the search targets (at least their outlines and locations) even when these items are not as diagnostic to the original scene and may not typically be remembered in explicit memorization. Another complementary possibility is that participants relied on schematic memories of living room and bedroom scenes to construct scenes rather than recall them (Brewer Treyens, 1981). Though reconstruction may have helped raters identify objects, it did not aid them in recognizing which specific bedroom/living room was studied on the surprise trial. Future research could assess this possibility by quantifying the spatial precision of objects drawn at surprise as well as the number of intrusive objects drawn at surprise versus control.
Overall, these results highlight that visual search can be highly successful without always engaging memory-forming processes. Juxtaposition of the current findings with the existing literature suggests that the difficulty of the search task may play a role in that difficult search tasks may incentivize memory formation. More importantly, the ability for attention and memory to dissociate reflects our cognitive system's ability to optimize resources by processing and attending to information in a scene without committing it to memory.
Supplementary Information The online version contains supplementary material available at https://doi.org/10.3758/s13421-023-01465-9.
Acknowledgements The authors would like to thank Taryn Green, Dheeraj Varghese, and Moussa Kousa for their assistance this research project. This work was supported by NSF Grant 1734220 awarded to author B.W.
Open Practices Statement All experimental scripts, drawer data files, drawings, rater data files, and analysis scripts for each experiment can be found on our OSF repository: https://osf.io/84edh/.
* Brad Wyble
Bwyble © gmail.com
1 The exact instructions for Experiment 1 were as follows: "Now for the main experiment. On each trial, you will see a picture of living room. Please look for the wall art in the image. After the image disappears a white canvas will appear. Using the paint tools, circle the exact location of each wall art in the image. Be as precise as possible when circling the location. Imagine you are encircling the whole WALL ART with the circle. This should take about 25 minutes. Click Start to begin."
2 Drexel University, Philadelphia, PA, USA
2 Previous piloting of the study confirmed that this pop-out was necessary, as without it, 28% of participants continued to engage in the pre-surprise task, even if the color of the submission button changed colors on this trial.
3 We observed a slight upwards bias in reporting the location of the target (Which is especially apparent for the pillows). We believe this bias is induced by the slight discrepancy in location of the studied scene versus the presented canvas, as the canvas appeared lower on screen due to the appearance of the submission button.
4 Note that a BF,, = .13, is the same as a BF,, = 7.67, which is strong evidence in favor of the alternative hypothesis.
5 The exact instructions of Experiment 3 were as follows: "Now for the main experiment. You will see series of living room and bedroom pictures. Please memorize each photo to the best of your ability. Sometimes, a white canvas will appear, and using the paint tools, you will draw the whole scene of the last image you just saw. Be as detailed and accurate as possible in your drawings. If no canvas appears, you do not need to hold onto the previous memory. This should take about 25 minutes. Click Start to begin."
6 We note that there was a typo in the text of this pop-up window; however, we have kept it in the manuscript to mimic the actual text participants were shown. It should have said, "Remember the image you just saw."
References
Anderson, М. C., Bjork, E. L., & Bjork, К. A. (2000). Retrieval-induced forgetting: Evidence for a recall-specific mechanism. Psychonomic Bulletin & Review, 7(3), 522-530. https://doi.org/ 10.3758/BF03214366
Averell, L., & Heathcote, A. (2011). The form of the forgetting curve and the fate of memories. Journal of Mathematical Psychology, 55(1), 25-35. https://doi.org/10.1016/j.jmp.2010.08. 009
Bainbridge, W. A. (2022). A tutorial on capturing mental representations through drawing and crowd-sourced scoring. Behavior Research Methods, 54, 1-13. https://doi.org/10.3758/ s13428-021-01672-9
Bainbridge, W. A., Hall, E. H., & Baker, C. I. (2019). Drawings of realworld scenes during free recall reveal detailed object and spatial information in memory. Nature Communications, 10(1), 1-13. https://doi.org/10.1038/s41467-018-07830-6
Born, S., Puntiroli, M., Jordan, D., & Kerzel, D. (2019). Saccadic selection does not eliminate attribute amnesia. Journal of Experimental Psychology: Learning, Memory, and Cognition, 45(12), Article 2165. https://doi.org/10.1037/xIm0000703
Brewer, W. F., & Treyens, J. С. (1981). Role of schemata in memory for places. Cognitive Psychology, 13(2), 207-230. https://doi.org/ 10.1016/0010-0285(81)90008-6
Bylinskii, Z., Isola, P., Bainbridge, C., Torralba, A., & Oliva, A. (2015). Intrinsic and extrinsic effects on image memorability. Vision Research, 116, 165-178. https://doi.org/10.1016/j.visres. 2015.03.005
Castelhano, M. S., & Henderson, J. M. (2005). Incidental visual memory for objects in scenes. Visual Cognition, 12(6), 1017-1040. https://doi.org/10.1080/13506280444000634
Chen, H., & Wyble, B. (2015). Amnesia for object attributes: Failure to report attended information that had just reached conscious awareness. Psychological Science, 26(2), 203-210. https://doi.org/ 10.1177/0956797614560648
Chen, H., & Wyble, B. (2016). Attribute amnesia reflects a lack of memory consolidation for attended information. Journal of Experimental Psychology: Human Perception and Performance, 42(2), 225-234. https://doi.org/10.1037/xhp0000133
Chen, H., Swan, G., & Wyble, B. (2016). Prolonged focal attention without binding: Tracking a ball for half a minute without remembering its color. Cognition, 147, 144-148. https://doi.org/10. 1016/j.cognition.2015.11.014
Chen, H., Yu, J., Fu, Y., Zhu, P., Li, W., Zhou, J., & Shen, М. (2019). Does attribute amnesia occur with the presentation of complex, meaningful stimuli? The answer is, "it depends". Memory & Cognition, 47(6), 1133-1144. https://doi.org/10.3758/ $13421-019-00923-7
Cohen-Dallal, H., Markus, O., & Pertzov, У. (2023). Adaptive visual working memory: Expecting a delayed estimation task enhances visual working memory precision. Journal of Experimental Psychology: Human Perception and Performance, 49(1), 7-21. https://doi.org/10.1037/xhp0001066
Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24(1), 87-114. https://doi.org/10.1017/S0140525X0 1003922
Draschkow, D., Wolfe, J. M., & Vo, M. L. H. (2014). Seek and you shall remember: Scene semantics interact with visual search to build better memories. Journal of Vision, 14(8), 10. https://doi. org/10.1167/14.8.10
Greenberg, R., & Underwood, B. J. (1950). Retention as a function of stage of practice. Journal of Experimental Psychology, 40, 452-457. https://doi.org/10.1037/h0062147
Guevara Pinto, J. D., Papesh, M. H., & Hout, M. C. (2020). The detail is in the difficulty: Challenging search facilitates rich incidental object encoding. Memory & Cognition, 48(7), 1214-1233. https:// doi.org/10.3758/s13421-020-01051-3
Hall, E. H., Bainbridge, W. A., & Baker, C. I. (2021). Highly similar and competing visual scenes lead to diminished object but not spatial detail in memory drawings. Memory, 1-14Advance online publication. https://doi.org/10.1080/09658211.2021.2010761.
Harrison, С. W., Kang, M., & Wilson, D. E. (2021). Remembering more than you can say: Re-examining "amnesia" of attended attributes. Acta Psychologica, 214, 103265. https://doi.org/10. 1016/j.actpsy.2021.103265
Hedayati, S., O'Donnell, Е. E., & Wyble, В. (2022). A model of working memory for latent representations. Nature Human Behaviour, 6(5), 709-719. https://doi.org/10.1038/s41562-021-01264-9
Helbing, J., Draschkow, D., & Vô, М. L. H. (2020). Search superiority: Goal-directed attentional allocation creates more reliable incidental identity and location memory than explicit encoding in naturalistic virtual environments. Cognition, 196(January), 104147. https://doi.org/10.1016/j.cognition.2019.104147
Hollingworth, A. (2005). The relationship between online visual representation of a scene and long-term scene memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(3), 396-411. https://doi.org/10.1037/0278-7393.31.3.396
Jiang, Y. V., Shupe, J. M., Swallow, К. M., & Tan, D. H. (2016). Memory for recently accessed visual attributes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 42(8), 1331-1337. https://doi.org/10.1037/x1m0000231
Keppel, G., & Underwood, B. J. (1962). Proactive inhibition in shortterm retention of single items. Journal of Verbal Learning and Verbal Behavior, 1, 153-161. https://doi.org/10.1016/S00225371(62)80023-1
Logan, G. D. (1988). Toward an instance theory of automatization. Psychological Review, 95(4), 492-527. https://doi.org/10.1037/ 0033-295X.95.4.492
Luchins, A. S. (1942). Mechanization in problem solving: The effect of Einstellung. Psychological Monographs, 54(6), i. https://doi. org/10.1037/h0093502
March, J. G. (1991). Exploration and exploitation in organizational learning. Organization Science, 2(1), 71-87. https://doi.org/10. 1287/orsc.2.1.71
O'Donnell, Е. E., Chen, H., & Wyble, В. (2021). No explicit memory for individual trial display configurations in a visual search task. Memory & Cognition, 49(8), 1705-1721. https://doi.org/10.3758/ s13421-021-01185-y
O'Donnell, R. E., & Wyble, B. (2023). Slipping through the cracks: The peril of unexpected interruption on the contents of working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition. Advance online publication. https://doi.org/ 10.1037/x1m0001214.
Peirce, J. W., Gray, J. R., Simpson, S., MacAskill, M. R., Hóchenberger, R., Sogo, H., Kastman, E., Lindelgv, J. (2019). PsychoPy2: experiments in behavior made easy. Behavior Research Methods. https://doi.org/10.3758/s13428-018-01193-y
Postman, L., & Underwood, B. J. (1973). Critical issues in interference theory. Memory & Cognition, 1, 19-40. https://doi.org/10.3758/ BF03198064
Rouder, J. N., Morey, R. D., Speckman, P. L., & Province, J. M. (2012). Default Bayes factors for ANOVA designs. Journal of Mathematical Psychology, 56(5), 356-374. https://doi.org/10.1016/j.jmp. 2012.08.001
Sasin, E., Markov, Y., & Fougnie, D. L. (2023). Meaningful objects avoid attribute amnesia due to incidental long-term memories. Scientific Reports, 13, 14464. https://doi.org/10.1038/ $41598-023-41642-7
Souza, А. S., & Oberauer, К. (2015). Time-based forgetting in visual working memory reflects temporal distinctiveness, not decay. Psychonomic Bulletin & Review, 22, 156-162. https://doi.org/10. 3758/s13423-014-0652-z
Swan, G., Wyble, B., & Chen, H. (2017). Working memory representations persist in the face of unexpected task alterations. Attention, Perception, & Psychophysics, 79, 1408-1414. https://doi.org/10. 3758/s13414-017-1318-5
Tam, J., Mugno, M. K., O'Donnell, К. E., & Wyble, В. (2021). And like that, they were gone: A failure to remember recently attended unique faces. Psychonomic Bulletin & Review, 28(6), 2027-2034. https://doi.org/10.3758/s13423-021-01965-2
Tam, J., & Wyble, B. (2022). Location has a privilege, but it is limited: Evidence from probing task-irrelevant location. Journal of Experimental Psychology: Learning, Memory, and Cognition, 9(7), 1051-1067. https://doi.org/10.1037/x1m000114710.1037/ x1m0001147
Utochkin, I. S., & Wolfe, J. M. (2018). Visual search for changes in scenes creates long-term, incidental memory traces. Attention, Perception, & Psychophysics, 80(4), 829-843. https://doi.org/10. 3758/s13414-018-1486-y
Vo, M., & Wolfe, J. (2012). When does repeated search in scenes involve memory? Looking at versus looking for objects in scenes. Journal of Experimental Psychology: Human Perception and Performance, 38(1), 23-41. https://doi.org/10.1037/a0024147
Wang, R., Fu, Y., Chen, L., Chen, Y., Zhou, J., & Chen, H. (2021). Consciousness can overflow report: Novel evidence from attribute amnesia of a single stimulus. Consciousness and Cognition, 87, 103052. https://doi.org/10.1016/j.concog.2020.103052
Websanova. (2014). wPaint. https://github.com/websanova/wPaint. Accessed 16 Sep 2023
Wessel, J. R. (2018). A neural mechanism for surprise-related interruptions of visuospatial working memory. Cerebral Cortex, 28(1), 199-212. https://doi.org/10.1093/cercor/bhw367
Wessel, J. R., Jenkinson, N., Brittain, J. S., Voets, S., Aziz, T. Z., & Aron, A. R. (2016). Surprise disrupts cognition via a frontobasal ganglia suppressive mechanism. Nature Communications, 7. https://doi.org/10.1038/ncomms11195
Williams, C. C., Henderson, J. M., & Zacks, R. T. (2005). Incidental visual memory for targets and distractors in visual search. Perception & Psychophysics, 67(5), 816-827. https://doi.org/10.3758/ BF03193535
Wyble, B., Hess, M., O" Donnell, Е. E., Chen, H., & Eitam, В. (2019). Learning how to exploit sources of information. Memory & Cognition, 47(4), 696-705. https://doi.org/10.3758/s13421-018-0881-x
Zhang, W., & Luck, S. J. (2009). Sudden death and gradual decay in visual working memory. Psychological Science, 20(4), 423-428. https://doi.org/10.1111/j.1467-9280.2009.02322.x
Copyright Springer Nature B.V. Jan 2025