Task epoch overview #

These are the epochs that are typically run for the movie task:

• Non-24 video watch (aka “control” or “concept” video)
• Non-24 free recall
• 24 video watch
• 24 free recall
• 24 recognition memory task
• 24 cued recall
• Sleep
• 24 free recall
• 24 recognition memory task
• 24 cued recall
• 24 concept fixation
• Non-24 free recall

 

A brief description of the epoch types:

Video watch

Participants watch a video, either an episode of a TV show or a portion of a movie. Typically ~40 mins long,

The non-24 video is currently a bespoke selection based on concept screening for each individual participant. This will eventually change to be another episode of TV (Merlin S04E11) when we start DBS experiments.

The 24 episode is season 6 episode 1.

Free recall

Participants are asked to verbally recount any and all details they can from the episode.

Recognition memory task

The participant is presented a series of short clips from the episode they  just watched, interspersed with clips from another episode from the same season (specifically S06E02 for 24), and they have to indicate after each clip if it it from the watched episode or not.

Cued recall

Participants are shown a series of images with associated questions and have to verbally answer the questions.

Concept fixation

Participants are shown a series of images with text labels and asked to hold that concept in mind until they hear a beep.

 

Requirements #

• Software: MATLAB
• Devices:
  • experiment laptop or computer,
  • microphone (Neuralynx),
  • backup microphone (Voice Memo app on Mac or any external microphone/voice recorder),
  • trackpad/mouse (prefer bluetooth mouse to prevent potential signal noise from touching laptop),
  • USB-C to USB-A adapter (for bluetooth mouse),
  • external speakers (wired not bluetooth),
  • audio cable splitter
  • audio amplifier (needs to be charged!)

 

Linux is the current preferred laptop to run the movie task, if this cannot be used then we will use the windows laptop.

Procedure #

Before going to the participant’s room #

Ideally before starting with the participant we would check that the non-24 video plays using the task code. As videos are prepped as mp4 as standard now this should not be an issue, but when possible it is best to check.

We should also prep the linux audio before starting with the participant (see below).

 

Properly Setting Up Audio On Linux #

Note: it is strongly suggested to use the external speaker setup. Otherwise, you will not be able to adjust the computer volume mid-experiment, nor will you have the ability to record the audio direct from the laptop.

 

With External Speakers/ Direct Audio Input to BNC: #

This setup will allow you to adjust volume mid-experiment and will provide a high-quality computer audio recording with the recorded data.

Setup the equipment as follows:

The direct-from-laptop audio should be plugged into Analogue 3 on the BNC board.

 

Log into the Linux computer and do the following steps:

1. Go to the desktop and double click the “Start_PulseAudio_17” icon:
   
2. Wait until the Speaker Icon appears in the top right corner of the computer screen:
   
3. Double click the “Open_Sound_Settings” icon on the desktop:
   
4. In the settings window that opens up, make sure that “Headphones – Built-in Audio” is selected as the Output Device
   
5. Close the sound settings window
6. Double click on the “Stop_PulseAudio_17” icon on the desktop:
   
7. Wait for the Speaker Icon to disappear from the top right corner of the computer screen:
   
8. Launch MATLAB by double clicking on the “Start_MATLAB_With_PTB” icon on the desktop:
   
9. You should now be able to navigate to the task code you wish to run.

 

The audio volume on the linux laptop should be set to max. The speaker volume can then be used for further adjustments.

Note: if you do wish to change the laptop output volume, you can also use the technique mentioned in the section below to “reset” audio and change the computer volume while MATLAB is open.

 

The amplifier can be turned on by twisting the dial. This should be set to 0.3, although you should monitor the Analogue3 signal in Pegasus to see if the amplitude is too low (so you would increase the amplification by turning the dial to a higher number) or if it is too high and clipping (then you would reduce the amplification).

 

 

 

Without External Speakers/ Direct Audio Input to BNC (not recommended): #

After logging in, double click on the “Run_24_Movie_Task” icon on the in the backup folder on the desktop:

A terminal window should appear, which will execute some commands related to starting/stopping PulseAudio 15/17 and then will finally conclude with MATLAB opening up.

Note: While by default this MATLAB window will open in the Movie_24 task, you can use the MATLAB file explorer to switch to a different task and run other task code from this window. Thus, this desktop shortcut can be used to run other tasks as well.

Adjusting volume while MATLAB is open #

If you are running MATLAB using the above method, you will notice that pressing the volume up/down buttons on the laptop do nothing. This is because the PulseAudio service is stopped to give PsychToolBox full control over audio output. If you want to adjust the volume without quitting MATLAB, you can do the following: (as long as no task code is currently running)

1. Go to the desktop and double click the “Reset_Audio_Only” icon:
   
2. Wait until the Speaker Icon appears in the top right corner of the computer screen:
   
3. You should now be able to use volume controls again. Adjust the volume to the desired level
4. Before running any task code, double click on the “Setup_Audio_for_24” icon on the desktop:
   
5. You should now be able to run task code properly again

 

Audio set up on windows #

The AUX connections with the splitter and amplifier should be the same as the diagram above for the linux. You should not need to do any additional software steps, just make sure the audio is selected to external speakers (as opposed to internal).

 

Neural recording prep  #

Standard prep applies, denoising, etc. Set up the laptop on the table with the external speakers. Make sure participant is comfortable – it’s a long task!

[Adi please add any details here you think necessary, particularly any movie24 specific ones]

Inputs from the laptop to the DAQ:

• USB TTL cable (must be plugged in before MATLAB opened or TTLs will not send)
• Aux out to record copy of speaker audio (into input ??? on IO board)

 

Other laptop prep (for first recording of pre- and post-sleep periods) #

Open MATLAB and test TTLs #

• You will likely already have MATLAB open from the above prep, but if not:
  • On linux: Double click the “Start_MATLAB_With_PTB” icon on the desktop.
  • On mac: Open MATLAB.
• In MATLAB’s Current Folder window navigate to Movie_24/key_functions/ParadigmUtilities
• In MATLAB’s Command Window type: testTTLs
• Look at DAQ screen to see if TTLs are coming through

 

Note: TTL tests only need to be done for first recording of the pre-sleep and post-sleep sessions.

 

Back-up audio prep #

On mac: if using Voice Memos for back-up audio, open app.

On linux: we currently do not have a laptop method for audio back up recordings. Please use the voice memo function on your phone.

 

Prep task code  #

Movie watch code #

• In MATLAB’s Current Folder window navigate to Movie_24/runfromhere_movies
• In MATLAB’s Command Window type: showmovie
• Select movie type from pop up window.
  • If “control” select specific file from resulting pop up window.
• Fill in run parameters (see below for more details).
  • Typically you should only have to input the subject number as the default parameters for the other parameters should be appropriate.
  • An exception to this is if you are restarting from an exited or crashed session, where you would set Resume session to 1.

Once the run parameters are filled in, you can leave the pop up window open, ready to click ‘OK’ when the everything is ready and the participant is ready to start the task.

Memory task code #

• In MATLAB’s Current Folder window navigate to Movie_24/runfromhere_memory
• In MATLAB’s Command Window type: run_movie_test_cnl
• Select task type, options are:
  • 24 S06E01 vs S06E02
  • Select movie file for recall
• Fill in run parameters (see below for more details)
  • Typically you should only have to input the subject number as the default parameters for the other parameters should be appropriate.
  • Exceptions to this include if you are restarting from an exited or crashed session, where you would set Resume session to 1, and if you are running the concept fixation task in the post-sleep session.

Once the run parameters are filled in, you can leave the pop up window open, ready to click ‘OK’ when the everything is ready and the participant is ready to start the task.

 

Running and recording the experiments #

The neural DAQ should always be the first device that starts recording, and then the back up audio, and then the MATLAB task should be started.

When ending the recording the reverse order should be taken: task off, back-up audio stop, then DAQ stop.

Preferably each run of a MATLAB task would have it’s own EXP folder.

 

The specifics of how to run the MATLAB code are given in the section “Running the MATLAB tasks“. The below is an outline guide for each portion of the task.

Pre-sleep movie watch and memory tasks #

Non-24 video watch and free recall:

• use showmovie.m
  • Select “Other video”.
  • Choose appropriate video file in pop up window.
• params:
  • ‘Free recall?’ set to 1.
• please do not change any video filenames.
• if the video does not work with psychtoolbox show with VLC.
• make a note in the bluebook how the video was shown.

 

24 video watch:

• use showmovie.m
  • Select ’24’

 

24 memory tasks:

• use run_movie_test_cnl
• select task type: ’24 S06E01 vs S06E02′
• params:
  • ‘Free recall?’ set to 1.
  • ‘Do recognition memory test?’ set to 1.
  • ‘Ask cued recall questions?’ set to 1.
  • ‘Do concept fixation?’ set to 0.

 

Sleep #

[Adi please fill in details for this, or make another wiki entry (if you think the sleep recording info is more general than just this task) and link to it here]

 

Post-sleep memory tasks #

24 memory tasks:

• use run_movie_test_cnl
• select task type: ’24 S06E01 vs S06E02′
• params:
  • ‘Free recall?’ set to 1.
  • ‘Do recognition memory test?’ set to 1.
  • ‘Ask cued recall questions?’ set to 1.
  • ‘Do concept fixation?’ set to 1.

 

Non-24 free recall:

• use run_movie_test_cnl
• select task type: ‘Select movie file for recall’
  • select appropriate video in pop up window
• params:
  • ‘Free recall?’ set to 1.
  • ‘Do recognition memory test?’ set to 0.
  • ‘Ask cued recall questions?’ set to 0.
  • ‘Do concept fixation?’ set to 0.

 

Running the MATLAB tasks #

More “click to continue” screens have been added to the most recent version of the code, and (I think) all instruction slides are on for approx 5 seconds without the ability to continue, and then some additional text will appear to indicate that you can click to continue off that slide.

showmovie.m: video playback (and optional free recall) #

Movie selection #

• Select the appropriate option from the pop up window: “Merlin”, “24”, or “Other video” .
• Selecting “Merlin” or “24” will take you to the run parameters pop up.
• Selecting “Other” will produce another pop up where you can select which video file to play, after which the run parameters pop up will appear.

Run parameters pop up #

The run parameters pop up includes:

• Subject number
  • input the 3 digit subject ID number.
• 24 episode number
  • Default for 24 is episode 1.
  • This is set to 0 for Merlin or other videos. Changing this will not have any effect on them.
• Full episode?
  • Default is 1 (full).
  • This is legacy functionality, not currently used to my knowledge.
  • Do not use any other value than 1 for Merlin or other video, effect on those is unknown.
• With pulses?
  • Default is 1 to have TTLs during task. Set to 0 if not connected to DAQ (eg when debugging without TTLs).
• Resume session?
  • Default 0. Set to 1 if continuing from a session that has just experienced a crash or was aborted.
• Debug mode?
  • Default 0. Set to 1 when debugging and want shortened sections and transparent window.
• Do free recall at end?
  • Set this value to 1 for free recall to occur at the end of movie watch.
  • For “Other video” this is default set to 1 and is recommended to be used.
  • For “Merlin” or “24” is set to 0 as recall is part of the memory task.
• Record spontaneous before movie?
  • Set this value to 1 to record 5 minutes of spontaneous activity before the movie starts.
  • Default is 1 for Merlin and 24.
  • Default 0 for “Other videos”. We do not need spontaneous recordings before the control videos.

Video playback controls #

During the video playback:

• you can pause the video with the space bar, and unpause by clicking on the trackpad (this will not exit out to matlab)
• you can exit with Esc any time (not currently working on linux, so if you need to pause use the above method)
  • will exit to matlab
  • can restart task and resume playback by re-running showmovie and inputting 1 for ‘Resume session?’ in params pop-up

Free recall #

The above controls do not work during free recall. There is no way to prematurely end the free recall portion. This is by design to promote the participant to think and hopefully recall more about the videos.

The free recall instructions given to the participant are:

We would like you to recall the episode you saw. Please recall in as much detail as possible and recount as many events or scenes from the episode as you can.

We would like the verbal instructions to emphasize giving specific details about the video. We specifically do not mention any particular order in which they have to recall.

The duration of free recall is related to the length of the video shown (minimum 2 minutes, maximum 5 minutes). Another instruction slide will appear to indicate how long the recall period will be, and that a timer will countdown each minute. It also informs the participant that they can continue after the timer is finished.

Once the recall period is over a screen appears encouraging the participant to continue if they can, and to click the mouse when they are done. This mouse click is now linked to a TTL pulse and will be a useful marker of when the recall period ends so please make sure it is clicked upon completion of recall. If the participant is done well before the end of the recall period and does not seem amenable to attempting to recall more it’s not critical to get the end of recall TTLs, but it is generally preferred.

Outputs #

The data will be saved in the subject folder within runfromhere_movies.

run_movie_test_cnl.m: recall, recognition, cued recall (and optional concept fixation) #

Test selection #

A pop up will appear giving the following options:

• 24 S06E01 vs S06E02
  • select this to run the 24 memory tests (incl. free recall).
• Select movie file for recall
  • this will then create another pop up where you can select the video file to be recalled (for calculation of recall duration, and so we have a log of which video is being recalled in the TTLs).
  • this is only for free recall (the run parameters default values reflect this) and will crash if other memory tasks are attempted.
  • typically this will be used for free recall of the non-24 video in the morning, but can be used for any recall as long as the appropriate video file is selected.

Soon we will be adding another test option (Merlin S04E10 vs S04E09/11) when we move to showing the same 2 videos to each participant (instead of a bespoke video).

 

Run parameters pop up #

The run parameters pop up includes:

• Subject number
  • input the 3 digit subject ID number.
• With pulses?
  • Default is 1 to have TTLs during task. Set to 0 if not connected to DAQ (eg when debugging without TTLs).
• Resume session?
  • Default 0. Set to 1 if continuing from a session that has just experienced a crash or was aborted.
• Debug mode?
  • Default 0. Set to 1 when debugging and want shortened sections and transparent window.
• Do free recall at beginning?
  • Set this value to 1 for free recall to occur at the beginning of the memory test.
  • After selecting  “24 S06E01 vs S06E02” this is set to 1. Set it to 0 if you ran free recall with the showmovie code.
  • After selecting “Select movie file for recall” this is set to 1.
• Do recognition memory test?
  • Set this value to 1 to run the recognition memory task.
  • Default is 1 for “24 S06E01 vs S06E02”
  • After selecting “Select movie file for recall” this is set to 0. The task will crash if you set this to 1.
• Ask cued recall questions?
  • Set this value to 1 to run the cued recall task.
  • Default is 1 for “24 S06E01 vs S06E02”
  • After selecting “Select movie file for recall” this is set to 0. The task will crash if you set this to 1.
• Do concept fixation?
  • Set this value to 1 to run the concept fixation task.
  • Default is 0 for “24 S06E01 vs S06E02”.
  • After selecting “Select movie file for recall” this is set to 0. The task will crash if you set this to 1.

 

Task controls #

During recognition task can exit with esc

• does not work during clip presentation, only after during response window
• will exit to matlab
• can be restarted from exited trial by re-running run_movie_test_cnl and inputting 1 for ‘Resume session?’ in params pop-up

 

Free recall #

Free recall in the memory task code is identical to that in the memory task code. See above for details.

 

Recognition memory task #

Participants are shown 150 short clips (0.8 to 3 seconds long).

• 75 are from the video they just watched (aka “target” clips).
• 75 are from another episode in the same season (“foil” clips).

 

After each clip they are asked:

‘Was this clip from the episode you watched?’

And have to respond ‘yes’ or ‘no’ by moving the cursor to click with the mouse on the answer on screen.

Trials are pseudorandomly ordered so that within every 20 trials there are 10 targets and 10 foils. Within each block of 20 trials the trial order is random. The order of presentation is different for each participant.

 

Cued recall #

Participants are shown 6 images with questions and must verbally answer the onscreen question. The image and question are onscreen for 10 seconds. The participant is given 45 seconds to respond, but encouraged to continue after that if possible. Once the participant has finished recalling they can click to move on to the next question.

 

Cued recall questions:

• Where does Chloe work and what upsets her about Jack?
• Who are these characters and how do they relate to terrorism?
• Who spoke to the President in the episode? What did they talk about?
• What was his story?
• Describe the times Jack was restrained. How did he get there or get free?
• Where are the various locations that the episode takes place?

 

The order of presentation is the same for all participants.

 

Concept fixation #

NOTE: This is only to be run in the morning as the final task epoch as it can bias memory.

Participants are shown an image with a text label on screen for 2 seconds. They are asked to hold this concept in memory until they hear a beep (which occurs 7 seconds after the image/text have left the screen). There is a “practice” first trial of an apple for you to walk through with the participant so they understand the task.

There are 36 fixations in total (excluding the apple practice trial).

• There are 12 ’24’ concepts
  • Ahmed Amar, Jack Bauer, Bomb, White House, Los Angeles, Bill Buchanan, Hostage, Chloe O’Brian, President Palmer, CTU (government agency), handcuffs, Abu Fayed
• and 6 non-24 concepts
  • Lady Gaga, New York City, Denzel Washington, Bruce Willis, Mount Rushmore, Apple Inc.

 

All concepts are shown twice. The order is pseudorandom so that within each block of 18 trials each concept is shown once. The order of presentation is the same for all participants.

 

Outputs #

The data will be saved in the subject folder within runfromhere_memory.

 

Crash handling #

If the task crashes and is unresponsive to exit with esc try the below:

1. ctl-c (x 2) then type ‘sca’ and hit enter
   • this should stop the task and close all psychtoolbox windows
   • matlab should remain open
2. cmd-q (for mac) or alt-F4 (windows)
   • this should close matlab
3. ctl-up (mac) or win-tab (windows) to try to change desktop
   • this should allow you to close the matlab window as it will exit the psychtoolbox fullscreen
   • there might be track pad gestures to achieve this also (eg three finger swipe up on mac)

If none of the above work then you will likely need to restart the laptop.

If you have to use any of the methods above then ‘Resume session’ will not work.