Cognitive Neurophysiology Laboratory

CNL Wiki

Docs: How to Process Screening

Updated on September 11, 2023

How to Process Screening #

For all these steps, please replace the path examples with your relevant path to Hoffman, etc

>>> patientDataManager

If this is the first experiment from a patient

A screenshot of a computer screen

Description automatically generated

1. Add New Patient to Database

-Enter or update patient information

-Enter in patient number on the GUI popup

-Enter in the patient information in the next popup

Start here if the patient is already in database

2. Add New Experiment to Database

Add or update session for existing patient

A screenshot of a computer screen

Description automatically generated

Enter in the patient number and experiment numbers on the next popups.

You want the Experiment Number to be the same as what the Lab Manager used/put in the Blue Folder.

Go to NAS > NLData > D567

Look for which experiment is Screening (example Experiment 3)

[For this example we use 99 so to not interfere with real screening]

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

A close-up of a document

Description automatically generated

Enter in the montage information in the next popup. Make sure to also select whether the experiment was recorded on Blackrock or Neuralynx, and the Experiment Type. For Screening it will be “Screening”, for the movie it will be “MovieParadigm”

A screenshot of a computer

Description automatically generated

3. Link to raw neural data

– If NLX, first run the command cleanUpNlxDirectories(‘/PATHTORAWNLXDATAONLTS’)

A screenshot of a computer

Description automatically generated

>> directoryToClean = ‘/Volumes/DATA/NLData/D567/EXP5_Screening_2/2023-08-22_17-16-55’

>> cleanUpNlxDirectories(directoryToClean);

– Click on “Link to raw neural data”

– If NLX, navigate to the folder on LTS with the data

– If Blackrock, navigate to the .ns5 file on LTS with the data

4. Enter Paradigm Parameters:

Screening:

click on “Paradigm Specific Parameters”, enter 1 for seconds before image onset, 2 for seconds after image offset

A screenshot of a message

Description automatically generated

click on “Stimulation Parameters”

click “None” for Stimulation Type and Submit

A screenshot of a computer

Description automatically generated

For Movie Paradigm, entering these parameters are not necessary

5.Click “Unpack Raw Data”

This will unpack the data to Hoffman. This step will take some time.

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

6.Process and Verify TTLs

Copy Screening folder from Box

A screenshot of a computer

Description automatically generated

Place in Screening folder on Hoffman

A screenshot of a computer

Description automatically generated

Click on “Process and Verify TTLs”

A screenshot of a computer

Description automatically generated

Only for Screening (and other paradigms with TTLs)

If it asks for an Events file, navigate to the directory on the LTS where the events are stored

It will ask for the ttlLog file. This is recorded on the experimental laptop while the paradigm is running. I usually download the “from laptop” folder from Box (put there by the Lab Manager) and then put it in “/home/chris/hoffman/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment2/from laptop”. The file it’s looking for is “from laptop/ttlLog.mat”.

A computer screen shot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

It will stop TTL parsing so that you can align the TTLs. The “ttlLog” variable is from the experimental laptop, while the “TTL” and “ts” variables are from the recording system. All 3 must have to have the same number of rows.

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

Often times there will be test TTLs sent in the ttlLog that don’t show up in the TTL/ts variables, so you can delete them from the ttlLog variable with a command like:

Need to remove 5 entries to match 1889 and 1894

ttlLog(1:5, 🙂 = [];

If it’s not there, then select the file with the highest room number, like “TTLs566-03-Aug-2023-12-45-25_room8.mat”

A white background with black and orange text

Description automatically generated

Most likely the recording will have been interrupted and there will be multiple files. The ttlLog will be off from the TS and TTLCode files by a lot:

A close-up of a computer screen

Description automatically generated

So then you have to piece together the different ttlLogs from the different recordings and remove the unwanted entries from the ts and ttlCode files.

This is the earliest recording:

‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/567_Screening/Experiment97/screening_1/567-21-Aug-2023-14-23-4/from laptop/TTLs567-21-Aug-2023-14-11-13_room2.mat’

#copy into ttlLog1

K>> ttlLog1 = ttlLog;

#load second recording

K>> load(‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/567_Screening/Experiment97/screening_1/567-21-Aug-2023-14-23-4/from laptop/TTLs567-21-Aug-2023-14-23-4_room2.mat’, ‘ttlLog’)

#copy into ttlLog2:

K>> ttlLog2 = ttlLog;

#load third recording

K>> load(‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/567_Screening/Experiment97/screening_1/567-21-Aug-2023-14-35-19/from laptop/TTLs567-21-Aug-2023-14-35-19_room2.mat’, ‘ttlLog’)

K>> ttlLog3 = ttlLog;

K>> load(‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/567_Screening/Experiment97/screening_1/567-21-Aug-2023-14-45-11/from laptop/TTLs567-21-Aug-2023-14-45-11_room2.mat’, ‘ttlLog’)

K>> ttlLog4 = ttlLog;

#combine all the ttlLog files into one file:

K>> ttlLog = [ttlLog1; ttlLog2; ttlLog3; ttlLog4];

K>> ttlLog(1:5, 🙂 = [];

A white and red line

Description automatically generated with medium confidence

#to look at the recording (1 -> 100) in a graph/plot

K>> figure; plot(ttlCode);

#find in ttlLog the ones that are wonky

K>> ttlsToRemove = 823:1027;

#remove those from ts and ttlCode

K>> ts(ttlsToRemove) = [];

K>> ttlCode(ttlsToRemove) = [];

K>> ttlsToRemove = 1650:2000;

K>> ts(ttlsToRemove) = [];

K>> ttlCode(ttlsToRemove) = [];

K>> ttlsToRemove = 2477:2564;

K>> ts(ttlsToRemove) = [];

K>> ttlCode(ttlsToRemove) = [];

Remove the last ones to make the ts, ttlCode, and ttlLog match

K>> ttlCode(3294)=[];

K>> ts(3294)=[];

Emily’s List of Commands:

A screenshot of a computer program

Description automatically generated

A screenshot of a computer

Description automatically generated

7. Run automatic spike detection/clustering

Currently I have the code at:

<path to Hoffman>PIPELINE_vc/ANALYSIS/AutomaticScreeningAnalysis/AutomaticScreeningAnalyze5

Add this folder to your path with:

addpath(genpath(‘/home/chris/hoffman/PIPELINE_vc/ANALYSIS/AutomaticScreeningAnalysis/AutomaticScreeningAnalyze5’))

To run, use the command

trialFolder = ‘/home/chris/hoffman/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment2/CSC_data’;

(Replace this with path to the CSC data folder on Hoffman)

automaticScreeningAnalyze5(trialFolder);

Note: One of my goals is to add this code to the rest of the database system and put it under version control as right now it’s somewhat independent

>> addpath(genpath(‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/AutomaticScreeningAnalysis/AutomaticScreeningAnalyze5’))

>> trialFolder = ‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment99/CSC_data’;

>> automaticScreeningAnalyze5(trialFolder);

OUTPUT

DETECTING SPIKES

Starting parallel pool (parpool) using the ‘local’ profile …

connected to 10 workers.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Warning: Error in SPC

> In do_clustering_single_AS (line 87)

In parallel_function>make_general_channel/channel_general (line 938)

In remoteParallelFunction (line 46)

Warning: Error in SPC

> In do_clustering_single_AS (line 87)

/bin/bash: ./cluster_maci.exe: cannot execute binary file

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Warning: Error in SPC

> In do_clustering_single_AS (line 87)

In parallel_function>make_general_channel/channel_general (line 938)

In remoteParallelFunction (line 46)

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

In parallel_function>make_general_channel/channel_general (line 938)

In remoteParallelFunction (line 46)

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

>>

8. Create channelMetaData

This step generates the channel metadata file necessary for subsequent analysis

updateChannelMetaData(patient,expNum)

This was the code I used

12:13

brs = {‘LA’, ‘LAH’, ‘LPHG’, ”, ‘RA’, ‘RAH’};
channelData = [];
for i = 1:48
channelData(i).brainRegion = brs{ floor((i-1)/8)+1};
end

12:14

And then you save this in ‘/home/chris/hoffman/PATIENT_DATABASE/Patient567/Experiment99’ as channelMetaData.mat

A screenshot of a computer

Description automatically generated

Save As

Replace channelMetaData.mat file in ‘/home/chris/hoffman/PATIENT_DATABASE/Patient567/Experiment99’ directory

Patient is the patient number, expNum is the experiment number. This will save the channel metadata file in the PATIENT_DATABASE

OUTPUT:

>> updateChannelMetaData(566,99)

Making patientInfo backup copy…saving changes to patientInfo…

Done.

Generating channel meta data

>>

9. Run calculateClusterCharacteristics

[clusterCharacteristics] = calculateClusterCharacteristics(subject, exp, screening)

Subject is the patient number in PDM, exp is the experiment number, and screening is 1 if it’s a screening experiment, 0 otherwise.

This step is only necessary for Screening

NOTE: This function uses a function called findpeaks in the Signal Processing Toolbox, however there is a conflicting function also called findpeaks that’s packaged in the PDM as part of the chronux toolkit. This may throw an error, so if it does you can remove chronux from path:

rmpath(genpath((‘/home/chris/research/PIPELINE/CODE/Utilities/chronux’));

10. Generate unit rasters

Generate the rasters by unit with the command:

rasters_by_unit(subject, exp, imageDirectory, plotResponsive)

Subject is the patient number, exp is the experiment number, imageDirectory is the path to where the raw image files are, plotResponsive is 1 for only responsive units, 0 for all units.

The raw image files are usually in a folder named “trial1” and uploaded to Box by the Lab Manager. I usually move it from Box to

A screenshot of a computer

Description automatically generated

“/home/chris/hoffman/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment2/trial1”. This path goes into the imageDirectory argument.

Hoffman: /u/project/ifried/data/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment2/trial1

I run both plotResponsive as 0 and 1 and upload both to Box.

The output of this function will be a series of individual pdf pages and Rasters_all.pdf within the CSC_data folder of the experiment folder in Hoffman. The individual pdf pages can be deleted once the Rasters_all is made. The Rasters_all.pdf is renamed to something like Rasters_p566_Screening1.pdf

11. Generate image rasters

Generate the rasters by image with the command:

rasters_by_image(subject, exp, imageDirectory);

This command works the same as the previous step.

How to Process Screening #

For all these steps, please replace the path examples with your relevant path to Hoffman, etc

>>> patientDataManager

If this is the first experiment from a patient

A screenshot of a computer screen

Description automatically generated

1. Add New Patient to Database

-Enter or update patient information

-Enter in patient number on the GUI popup

-Enter in the patient information in the next popup

Start here if the patient is already in database

2. Add New Experiment to Database

Add or update session for existing patient

A screenshot of a computer screen

Description automatically generated

Enter in the patient number and experiment numbers on the next popups.

You want the Experiment Number to be the same as what the Lab Manager used/put in the Blue Folder.

Go to NAS > NLData > D567

Look for which experiment is Screening (example Experiment 3)

[For this example we use 99 so to not interfere with real screening]

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

A close-up of a document

Description automatically generated

Enter in the montage information in the next popup. Make sure to also select whether the experiment was recorded on Blackrock or Neuralynx, and the Experiment Type. For Screening it will be “Screening”, for the movie it will be “MovieParadigm”

A screenshot of a computer

Description automatically generated

3. Link to raw neural data

– If NLX, first run the command cleanUpNlxDirectories(‘/PATHTORAWNLXDATAONLTS’)

A screenshot of a computer

Description automatically generated

>> directoryToClean = ‘/Volumes/DATA/NLData/D567/EXP5_Screening_2/2023-08-22_17-16-55’

>> cleanUpNlxDirectories(directoryToClean);

– Click on “Link to raw neural data”

– If NLX, navigate to the folder on LTS with the data

– If Blackrock, navigate to the .ns5 file on LTS with the data

4. Enter Paradigm Parameters:

Screening:

click on “Paradigm Specific Parameters”, enter 1 for seconds before image onset, 2 for seconds after image offset

A screenshot of a message

Description automatically generated

click on “Stimulation Parameters”

click “None” for Stimulation Type and Submit

A screenshot of a computer

Description automatically generated

For Movie Paradigm, entering these parameters are not necessary

5.Click “Unpack Raw Data”

This will unpack the data to Hoffman. This step will take some time.

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

6.Process and Verify TTLs

Copy Screening folder from Box

A screenshot of a computer

Description automatically generated

Place in Screening folder on Hoffman

A screenshot of a computer

Description automatically generated

Click on “Process and Verify TTLs”

A screenshot of a computer

Description automatically generated

Only for Screening (and other paradigms with TTLs)

If it asks for an Events file, navigate to the directory on the LTS where the events are stored

It will ask for the ttlLog file. This is recorded on the experimental laptop while the paradigm is running. I usually download the “from laptop” folder from Box (put there by the Lab Manager) and then put it in “/home/chris/hoffman/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment2/from laptop”. The file it’s looking for is “from laptop/ttlLog.mat”.

A computer screen shot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

It will stop TTL parsing so that you can align the TTLs. The “ttlLog” variable is from the experimental laptop, while the “TTL” and “ts” variables are from the recording system. All 3 must have to have the same number of rows.

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

Often times there will be test TTLs sent in the ttlLog that don’t show up in the TTL/ts variables, so you can delete them from the ttlLog variable with a command like:

Need to remove 5 entries to match 1889 and 1894

ttlLog(1:5, 🙂 = [];

If it’s not there, then select the file with the highest room number, like “TTLs566-03-Aug-2023-12-45-25_room8.mat”

A white background with black and orange text

Description automatically generated

Most likely the recording will have been interrupted and there will be multiple files. The ttlLog will be off from the TS and TTLCode files by a lot:

A close-up of a computer screen

Description automatically generated

So then you have to piece together the different ttlLogs from the different recordings and remove the unwanted entries from the ts and ttlCode files.

This is the earliest recording:

‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/567_Screening/Experiment97/screening_1/567-21-Aug-2023-14-23-4/from laptop/TTLs567-21-Aug-2023-14-11-13_room2.mat’

#copy into ttlLog1

K>> ttlLog1 = ttlLog;

#load second recording

K>> load(‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/567_Screening/Experiment97/screening_1/567-21-Aug-2023-14-23-4/from laptop/TTLs567-21-Aug-2023-14-23-4_room2.mat’, ‘ttlLog’)

#copy into ttlLog2:

K>> ttlLog2 = ttlLog;

#load third recording

K>> load(‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/567_Screening/Experiment97/screening_1/567-21-Aug-2023-14-35-19/from laptop/TTLs567-21-Aug-2023-14-35-19_room2.mat’, ‘ttlLog’)

K>> ttlLog3 = ttlLog;

K>> load(‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/567_Screening/Experiment97/screening_1/567-21-Aug-2023-14-45-11/from laptop/TTLs567-21-Aug-2023-14-45-11_room2.mat’, ‘ttlLog’)

K>> ttlLog4 = ttlLog;

#combine all the ttlLog files into one file:

K>> ttlLog = [ttlLog1; ttlLog2; ttlLog3; ttlLog4];

K>> ttlLog(1:5, 🙂 = [];

A white and red line

Description automatically generated with medium confidence

#to look at the recording (1 -> 100) in a graph/plot

K>> figure; plot(ttlCode);

#find in ttlLog the ones that are wonky

K>> ttlsToRemove = 823:1027;

#remove those from ts and ttlCode

K>> ts(ttlsToRemove) = [];

K>> ttlCode(ttlsToRemove) = [];

K>> ttlsToRemove = 1650:2000;

K>> ts(ttlsToRemove) = [];

K>> ttlCode(ttlsToRemove) = [];

K>> ttlsToRemove = 2477:2564;

K>> ts(ttlsToRemove) = [];

K>> ttlCode(ttlsToRemove) = [];

Remove the last ones to make the ts, ttlCode, and ttlLog match

K>> ttlCode(3294)=[];

K>> ts(3294)=[];

Emily’s List of Commands:

A screenshot of a computer program

Description automatically generated

A screenshot of a computer

Description automatically generated

7. Run automatic spike detection/clustering

Currently I have the code at:

<path to Hoffman>PIPELINE_vc/ANALYSIS/AutomaticScreeningAnalysis/AutomaticScreeningAnalyze5

Add this folder to your path with:

addpath(genpath(‘/home/chris/hoffman/PIPELINE_vc/ANALYSIS/AutomaticScreeningAnalysis/AutomaticScreeningAnalyze5’))

To run, use the command

trialFolder = ‘/home/chris/hoffman/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment2/CSC_data’;

(Replace this with path to the CSC data folder on Hoffman)

automaticScreeningAnalyze5(trialFolder);

Note: One of my goals is to add this code to the rest of the database system and put it under version control as right now it’s somewhat independent

>> addpath(genpath(‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/AutomaticScreeningAnalysis/AutomaticScreeningAnalyze5’))

>> trialFolder = ‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment99/CSC_data’;

>> automaticScreeningAnalyze5(trialFolder);

OUTPUT

DETECTING SPIKES

Starting parallel pool (parpool) using the ‘local’ profile …

connected to 10 workers.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Warning: Error in SPC

> In do_clustering_single_AS (line 87)

In parallel_function>make_general_channel/channel_general (line 938)

In remoteParallelFunction (line 46)

Warning: Error in SPC

> In do_clustering_single_AS (line 87)

/bin/bash: ./cluster_maci.exe: cannot execute binary file

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Warning: Error in SPC

> In do_clustering_single_AS (line 87)

In parallel_function>make_general_channel/channel_general (line 938)

In remoteParallelFunction (line 46)

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

In parallel_function>make_general_channel/channel_general (line 938)

In remoteParallelFunction (line 46)

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

>>

8. Create channelMetaData

This step generates the channel metadata file necessary for subsequent analysis

updateChannelMetaData(patient,expNum)

This was the code I used

12:13

brs = {‘LA’, ‘LAH’, ‘LPHG’, ”, ‘RA’, ‘RAH’};
channelData = [];
for i = 1:48
channelData(i).brainRegion = brs{ floor((i-1)/8)+1};
end

12:14

And then you save this in ‘/home/chris/hoffman/PATIENT_DATABASE/Patient567/Experiment99’ as channelMetaData.mat

A screenshot of a computer

Description automatically generated

Save As

Replace channelMetaData.mat file in ‘/home/chris/hoffman/PATIENT_DATABASE/Patient567/Experiment99’ directory

Patient is the patient number, expNum is the experiment number. This will save the channel metadata file in the PATIENT_DATABASE

OUTPUT:

>> updateChannelMetaData(566,99)

Making patientInfo backup copy…saving changes to patientInfo…

Done.

Generating channel meta data

>>

9. Run calculateClusterCharacteristics

[clusterCharacteristics] = calculateClusterCharacteristics(subject, exp, screening)

Subject is the patient number in PDM, exp is the experiment number, and screening is 1 if it’s a screening experiment, 0 otherwise.

This step is only necessary for Screening

NOTE: This function uses a function called findpeaks in the Signal Processing Toolbox, however there is a conflicting function also called findpeaks that’s packaged in the PDM as part of the chronux toolkit. This may throw an error, so if it does you can remove chronux from path:

rmpath(genpath((‘/home/chris/research/PIPELINE/CODE/Utilities/chronux’));

10. Generate unit rasters

Generate the rasters by unit with the command:

rasters_by_unit(subject, exp, imageDirectory, plotResponsive)

Subject is the patient number, exp is the experiment number, imageDirectory is the path to where the raw image files are, plotResponsive is 1 for only responsive units, 0 for all units.

The raw image files are usually in a folder named “trial1” and uploaded to Box by the Lab Manager. I usually move it from Box to

A screenshot of a computer

Description automatically generated

“/home/chris/hoffman/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment2/trial1”. This path goes into the imageDirectory argument.

Hoffman: /u/project/ifried/data/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment2/trial1

I run both plotResponsive as 0 and 1 and upload both to Box.

The output of this function will be a series of individual pdf pages and Rasters_all.pdf within the CSC_data folder of the experiment folder in Hoffman. The individual pdf pages can be deleted once the Rasters_all is made. The Rasters_all.pdf is renamed to something like Rasters_p566_Screening1.pdf

11. Generate image rasters

Generate the rasters by image with the command:

rasters_by_image(subject, exp, imageDirectory);

This command works the same as the previous step.

How to Process Screening #

For all these steps, please replace the path examples with your relevant path to Hoffman, etc

>>> patientDataManager

If this is the first experiment from a patient

A screenshot of a computer screen

Description automatically generated

1. Add New Patient to Database

-Enter or update patient information

-Enter in patient number on the GUI popup

-Enter in the patient information in the next popup

Start here if the patient is already in database

2. Add New Experiment to Database

Add or update session for existing patient

A screenshot of a computer screen

Description automatically generated

Enter in the patient number and experiment numbers on the next popups.

You want the Experiment Number to be the same as what the Lab Manager used/put in the Blue Folder.

Go to NAS > NLData > D567

Look for which experiment is Screening (example Experiment 3)

[For this example we use 99 so to not interfere with real screening]

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

A close-up of a document

Description automatically generated

Enter in the montage information in the next popup. Make sure to also select whether the experiment was recorded on Blackrock or Neuralynx, and the Experiment Type. For Screening it will be “Screening”, for the movie it will be “MovieParadigm”

A screenshot of a computer

Description automatically generated

3. Link to raw neural data

– If NLX, first run the command cleanUpNlxDirectories(‘/PATHTORAWNLXDATAONLTS’)

A screenshot of a computer

Description automatically generated

>> directoryToClean = ‘/Volumes/DATA/NLData/D567/EXP5_Screening_2/2023-08-22_17-16-55’

>> cleanUpNlxDirectories(directoryToClean);

– Click on “Link to raw neural data”

– If NLX, navigate to the folder on LTS with the data

– If Blackrock, navigate to the .ns5 file on LTS with the data

4. Enter Paradigm Parameters:

Screening:

click on “Paradigm Specific Parameters”, enter 1 for seconds before image onset, 2 for seconds after image offset

A screenshot of a message

Description automatically generated

click on “Stimulation Parameters”

click “None” for Stimulation Type and Submit

A screenshot of a computer

Description automatically generated

For Movie Paradigm, entering these parameters are not necessary

5.Click “Unpack Raw Data”

This will unpack the data to Hoffman. This step will take some time.

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

6.Process and Verify TTLs

Copy Screening folder from Box

A screenshot of a computer

Description automatically generated

Place in Screening folder on Hoffman

A screenshot of a computer

Description automatically generated

Click on “Process and Verify TTLs”

A screenshot of a computer

Description automatically generated

Only for Screening (and other paradigms with TTLs)

If it asks for an Events file, navigate to the directory on the LTS where the events are stored

It will ask for the ttlLog file. This is recorded on the experimental laptop while the paradigm is running. I usually download the “from laptop” folder from Box (put there by the Lab Manager) and then put it in “/home/chris/hoffman/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment2/from laptop”. The file it’s looking for is “from laptop/ttlLog.mat”.

A computer screen shot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

It will stop TTL parsing so that you can align the TTLs. The “ttlLog” variable is from the experimental laptop, while the “TTL” and “ts” variables are from the recording system. All 3 must have to have the same number of rows.

A screenshot of a computer

Description automatically generated

A screenshot of a computer

Description automatically generated

Often times there will be test TTLs sent in the ttlLog that don’t show up in the TTL/ts variables, so you can delete them from the ttlLog variable with a command like:

Need to remove 5 entries to match 1889 and 1894

ttlLog(1:5, 🙂 = [];

If it’s not there, then select the file with the highest room number, like “TTLs566-03-Aug-2023-12-45-25_room8.mat”

A white background with black and orange text

Description automatically generated

Most likely the recording will have been interrupted and there will be multiple files. The ttlLog will be off from the TS and TTLCode files by a lot:

A close-up of a computer screen

Description automatically generated

So then you have to piece together the different ttlLogs from the different recordings and remove the unwanted entries from the ts and ttlCode files.

This is the earliest recording:

‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/567_Screening/Experiment97/screening_1/567-21-Aug-2023-14-23-4/from laptop/TTLs567-21-Aug-2023-14-11-13_room2.mat’

#copy into ttlLog1

K>> ttlLog1 = ttlLog;

#load second recording

K>> load(‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/567_Screening/Experiment97/screening_1/567-21-Aug-2023-14-23-4/from laptop/TTLs567-21-Aug-2023-14-23-4_room2.mat’, ‘ttlLog’)

#copy into ttlLog2:

K>> ttlLog2 = ttlLog;

#load third recording

K>> load(‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/567_Screening/Experiment97/screening_1/567-21-Aug-2023-14-35-19/from laptop/TTLs567-21-Aug-2023-14-35-19_room2.mat’, ‘ttlLog’)

K>> ttlLog3 = ttlLog;

K>> load(‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/567_Screening/Experiment97/screening_1/567-21-Aug-2023-14-45-11/from laptop/TTLs567-21-Aug-2023-14-45-11_room2.mat’, ‘ttlLog’)

K>> ttlLog4 = ttlLog;

#combine all the ttlLog files into one file:

K>> ttlLog = [ttlLog1; ttlLog2; ttlLog3; ttlLog4];

K>> ttlLog(1:5, 🙂 = [];

A white and red line

Description automatically generated with medium confidence

#to look at the recording (1 -> 100) in a graph/plot

K>> figure; plot(ttlCode);

#find in ttlLog the ones that are wonky

K>> ttlsToRemove = 823:1027;

#remove those from ts and ttlCode

K>> ts(ttlsToRemove) = [];

K>> ttlCode(ttlsToRemove) = [];

K>> ttlsToRemove = 1650:2000;

K>> ts(ttlsToRemove) = [];

K>> ttlCode(ttlsToRemove) = [];

K>> ttlsToRemove = 2477:2564;

K>> ts(ttlsToRemove) = [];

K>> ttlCode(ttlsToRemove) = [];

Remove the last ones to make the ts, ttlCode, and ttlLog match

K>> ttlCode(3294)=[];

K>> ts(3294)=[];

Emily’s List of Commands:

A screenshot of a computer program

Description automatically generated

A screenshot of a computer

Description automatically generated

7. Run automatic spike detection/clustering

Currently I have the code at:

<path to Hoffman>PIPELINE_vc/ANALYSIS/AutomaticScreeningAnalysis/AutomaticScreeningAnalyze5

Add this folder to your path with:

addpath(genpath(‘/home/chris/hoffman/PIPELINE_vc/ANALYSIS/AutomaticScreeningAnalysis/AutomaticScreeningAnalyze5’))

To run, use the command

trialFolder = ‘/home/chris/hoffman/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment2/CSC_data’;

(Replace this with path to the CSC data folder on Hoffman)

automaticScreeningAnalyze5(trialFolder);

Note: One of my goals is to add this code to the rest of the database system and put it under version control as right now it’s somewhat independent

>> addpath(genpath(‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/AutomaticScreeningAnalysis/AutomaticScreeningAnalyze5’))

>> trialFolder = ‘/Users/emerchant/Hoffman_Mount/data/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment99/CSC_data’;

>> automaticScreeningAnalyze5(trialFolder);

OUTPUT

DETECTING SPIKES

Starting parallel pool (parpool) using the ‘local’ profile …

connected to 10 workers.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Warning: Error in SPC

> In do_clustering_single_AS (line 87)

In parallel_function>make_general_channel/channel_general (line 938)

In remoteParallelFunction (line 46)

Warning: Error in SPC

> In do_clustering_single_AS (line 87)

/bin/bash: ./cluster_maci.exe: cannot execute binary file

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Warning: Error in SPC

> In do_clustering_single_AS (line 87)

In parallel_function>make_general_channel/channel_general (line 938)

In remoteParallelFunction (line 46)

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

In parallel_function>make_general_channel/channel_general (line 938)

In remoteParallelFunction (line 46)

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

Wave_clus data selected. Raw data wasn’t loaded.

>>

8. Create channelMetaData

This step generates the channel metadata file necessary for subsequent analysis

updateChannelMetaData(patient,expNum)

This was the code I used

12:13

brs = {‘LA’, ‘LAH’, ‘LPHG’, ”, ‘RA’, ‘RAH’};
channelData = [];
for i = 1:48
channelData(i).brainRegion = brs{ floor((i-1)/8)+1};
end

12:14

And then you save this in ‘/home/chris/hoffman/PATIENT_DATABASE/Patient567/Experiment99’ as channelMetaData.mat

A screenshot of a computer

Description automatically generated

Save As

Replace channelMetaData.mat file in ‘/home/chris/hoffman/PATIENT_DATABASE/Patient567/Experiment99’ directory

Patient is the patient number, expNum is the experiment number. This will save the channel metadata file in the PATIENT_DATABASE

OUTPUT:

>> updateChannelMetaData(566,99)

Making patientInfo backup copy…saving changes to patientInfo…

Done.

Generating channel meta data

>>

9. Run calculateClusterCharacteristics

[clusterCharacteristics] = calculateClusterCharacteristics(subject, exp, screening)

Subject is the patient number in PDM, exp is the experiment number, and screening is 1 if it’s a screening experiment, 0 otherwise.

This step is only necessary for Screening

NOTE: This function uses a function called findpeaks in the Signal Processing Toolbox, however there is a conflicting function also called findpeaks that’s packaged in the PDM as part of the chronux toolkit. This may throw an error, so if it does you can remove chronux from path:

rmpath(genpath((‘/home/chris/research/PIPELINE/CODE/Utilities/chronux’));

10. Generate unit rasters

Generate the rasters by unit with the command:

rasters_by_unit(subject, exp, imageDirectory, plotResponsive)

Subject is the patient number, exp is the experiment number, imageDirectory is the path to where the raw image files are, plotResponsive is 1 for only responsive units, 0 for all units.

The raw image files are usually in a folder named “trial1” and uploaded to Box by the Lab Manager. I usually move it from Box to

A screenshot of a computer

Description automatically generated

“/home/chris/hoffman/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment2/trial1”. This path goes into the imageDirectory argument.

Hoffman: /u/project/ifried/data/PIPELINE_vc/ANALYSIS/Screening/566_Screening/Experiment2/trial1

I run both plotResponsive as 0 and 1 and upload both to Box.

The output of this function will be a series of individual pdf pages and Rasters_all.pdf within the CSC_data folder of the experiment folder in Hoffman. The individual pdf pages can be deleted once the Rasters_all is made. The Rasters_all.pdf is renamed to something like Rasters_p566_Screening1.pdf

11. Generate image rasters

Generate the rasters by image with the command:

rasters_by_image(subject, exp, imageDirectory);

This command works the same as the previous step.