Walabot for Wireless Gas Sensing

/*---------------------------------DATA PROCESSING------------------------------------------
|   File: DataProcessing                                                                   |
|   Type: .m (matlab)                                                                      |      
|   Purpose: Reads the signal files (csv) of different gas concentrations and plots the    |
|   energy of the returned signals.                                                        |
-------------------------------------------------------------------------------------------*/

/* _____________________________________________________________________________________
|                             DATA PROCESSING DESCRIPTION                                |
| _______________________________________________________________________________________|*/

/*The purpose of this MATLAB script is to produce a a bar graph that compares the energy
of the returned signals, for different gas concentrations. 

Overview of data processing steps:

    1) Specific frames from raw data files ('ftraw[num].csv') are read. The background frame
       is subtracted from the frame containing signals from the sensor. This data is written
       into seperate csv files. For example, '1.csv' would contain the data without background
       signals, for the highest gas concentraion.  

    2) The newly-written '[num].csv' files are read into MATLAB and each signal's amplitude 
       values are squared and then integrated; these values represent the energy of the signal. 

    3) The next step is to compare the signal energy values of each antenna pair, for all gas
       concentrations; for example, if there were 3 concentrations of gas and two 
       antenna pairs were used, then we would compare the signal energy at all three 
       concentrations for antenna pair #1 and then for antenna pair #2, seperately. The
       purpose of this comparison is to determine which antenna pair's data follows the
       expected trend. The expected trend is that the signal energies should decrease with 
       increasing ammonia concentration.

    4) After finding the antenna pair which follows the previously described trend, the 
       signal energies from that antenna pair are plotted in a bar graph, for all gas
       concentrations.
*/


/* _____________________________________________________________________________________
|                                        FUNCTIONS                                       |
| _______________________________________________________________________________________|*/

// Built-in trapz() function: used in the main program
/* Computes the integral of the input arguments. In the case that the
   input argument is a matrix (input arguments in this progrmam are matrices too), trapz() 
   computes the integral of each colunmn of the matrix. 

    Input: one argument, which is a matrix in this program

    Returns: a row vector containing the integral of each column (length of row vector
             in this program dependant on the number of antenna pairs used)
   */
// _____________________________________________________________________________________

// Built-in issorted() function: used in the main program
/* Determines if an array is sorted, i.e. the elements of the array are listed in 
   ascending order. 

    Input: one argument, which is a row vector in this program

    Returns: a logical scalar; returns 1 if the row vector is sorted, 0 if it is not*/

// _____________________________________________________________________________________


/* _____________________________________________________________________________________
|                                     MAIN PROGRAM                                       |
| _______________________________________________________________________________________|*/

// Choose the 'long' format to show a more accurate representation of the signal values
clc
clear 
format long 

/*Prompts the user to enter the number of gas concnetrations measured and the number
of antenna pairs used for the measurements.*/
num_of_conc = input('Please enter the number of measured gas concentrations: ');
ant_pairs = input('Please enter the number of antenna pairs used: ');
fprintf(1, '\n');


// Global variables
/* Initializing zero-filled matrices so the program doesn't slow down later*/

Integration_values = zeros(num_of_conc, ant_pairs);
x_use = zeros(8192,40);
antenna_pair = 0;
int_x = zeros(1,num_of_conc);

//  'for' loop for subtracting the background signals and writing that data into a 
//                                new csv file

/*  The user is asked to enter the frame number for the sensor's reflected amplitudes
    and the frame number for the background signals. The code then reads the data
    from 'ftraw[number].csv' files and extracts these specific frames for background
    subtraction.
     */
for i = 1:num_of_conc
    frame_gas_sensing =  input(['Please enter the frame number with recorded signals from the gas sensing film, for concentration ', num2str(i), ': ']);
    frame_bg = input(['Please enter the frame with recorded background noise, for concentration ', num2str(i), ': ']);

    row_start = (2048*(frame_gas_sensing-1));
    row_end = row_start + 2047;
    row_start_bg = (2048*(frame_bg-1));
    row_end_bg = row_start_bg + 2047;
    
    ftraw_data_gas_sensing = csvread(['ftraw', num2str(i),'.csv'], row_start, 0, [row_start 0 row_end ant_pairs-1]);
    ftraw_data_bg = csvread(['ftraw', num2str(i),'.csv'], row_start_bg, 0, [row_start_bg 0 row_end_bg ant_pairs-1]);
    subtracted_bg = ftraw_data_gas_sensing - ftraw_data_bg;
    csvwrite([num2str(i), '.csv'], subtracted_bg)

    disp(sprintf(['Background noise has been subtracted and the data is stored in file ', num2str(i), '.csv\n']))
end  




//            Nested 'for' loops for calculating signal energy for each antenna pair 

/* The outer loop goes up to the number of gas concentrations used, to read
    data from sequential csv files. The inner loop goes up to the number of antenna pairs
    used. Both these loops are used to index into the Integration_values matrix, which 
    represent a signal's energy. The signal energies are stored in the Integration_values
    matrix of size = num_of_conc by ant_pairs matrix, where each row represents the energy 
    from different antenna pairs at a certain gas concentration. 
*/
for k = 1:num_of_conc
    for i = 1:ant_pairs
        filename = [num2str(k) '.csv'];
        x_use = csvread(filename);      // file data read into 'x_use' variable

        integral_vector = trapz((x_use.^2));     /* Integral of the squared amplitude values
                                                are calculated using built-in function */

        Integration_values(k,i) = integral_vector(i);     /* The calculated integrals are stored in 
                                                            the Integration_values matrix*/
    end
end



//    'for' loop to find which antenna pair is the first one to follow the expected
//                                         trend

/* A 'for loop is used to iterate through all columns of the Integration_values matrix. Recall that 
    each column of this matrix contains signal evergies of different gas concnetrations, and 
    these values are obtained using the same antenna pair. If the values of a certain column
    in Integration_values matrix follows the expected trend (i.e. the values increase as we move down
    the column), then the program is instructed to break out of the loop. 
*/
bf = false;     // flag variable for breaking out of the loop once the antenna pair is determined
 for m = 1:ant_pairs
         antenna_pair = m;      // 'm' represents the antenna pair 

         col = Integration_values(1:num_of_conc, m);      /*'col' contains the data from an individual 
                                                           column of the Integration_values matrix */

         e = issorted(col);       /* Built-in function 'issorted' used to check if values 
                                   in 'col' are increasing in an acsending order. Function 
                                   returns 0 when false and 1, when true. */

         if e == 1              /* Condition that verifies if a certain column from Integration_values
                                   has values that are increasing */

             bf = true;         /* If condition holds, then flag variable is set to true and
                                   the program exits the 'if' block and then 'for loop */
             break

        elseif e == 0 && antenna_pair == 40     /* In case none of the data from any antenna pair 
                                                matches the expected trend */
            disp('NOTE: Data does not follow expected trend')

        end

         if bf,break,end
 end


//  Plotting the bar graphs of signal energies, at different ammonia concentrations 

/* The for loop goes up to the number of concnetrations, to read the signal energies from the 
previously determined antenna pair, which follows the trend. In the case that the expected trend 
is not followed by any antenna pair, the script plots the data from the last antenna pair. 
*/
 for f = 1:num_of_conc
     int_x(f) = Integration_values(f,antenna_pair);    
 end

bar(int_x, 0.4,'FaceColor',[0 .5 .5],'EdgeColor',[0 .9 .9],'LineWidth',1.5)
title('Signal Energy of Different Ammonia Conentrations');
xlabel('Concentration');
ylabel('Signal Energy');

 

INSTRUCTIONS FOR DATA COLLECTION AND DATA PROCESSING 

DATA COLLECTION USING WALABOT:

1) Place the gas sensing film under the Walabot (as shown in the video), and start 
   running the python script, "RawSignalCollection.py" A window with a live-
   updating graph will appear, showing the raw signals in real-time. The python 
   script also provides the option to orint the frame number or save the graph as 
   'frame[num].png' files, where 'num' represents the frame number.

2) After a few a seonds, remove the gas sensing film from under the Walabot to record
   background. 

3) Stop the script once you have recorded the background. The raw data will be saved in 
   'ftraw.csv'. 

3) Take note of the specific frame number which contains the signals from the gas sensing film,
   and the background data. You will need these two numbers later for data processing.

4) Repeat with different gas concentrations. Make sure to save the 'ftraw.csv' file and
   the frame images in another location, or else they will be replaced with the new data 
   collection.

DATA PROCESSING:

The purpose of this MATLAB script is to produce a a bar graph that compares the energy
of the returned signals, for different gas concentrations. 

Here are some things to keep in mind before using this script:

    1) All the raw data files obtained through 'RawSignalCollection.py' should be renamed 
      as 'ftraw[num].csv', where 'num' are integers (1,2,3,..). The file corresponding to 
      highest gas concentration should be named 'ftraw1.csv', the second highest
      concentration should be 'ftraw2.csv' and so on. This naming convention is necessary
      for MATLAB script to run properly.

    2) These raw data files must be in MATLAB's workspace directory.

    3) Enter the number of gas concentrations measured, the number of antenna pairs used,
       and the frame numbers for gas sensing film's signals and the background signals, 
       when prompted.

/*---------------------------------DATA PROCESSING------------------------------------------
|   File: AmmoniaDataProcessing                                                            |
|   Type: .m (matlab)                                                                      |      
|   Purpose: Reads the raw data files (csv) of different gas concentrations and gives      |
|   a plot of the Fourier Transform of the signals, showing change in amplitude values     |
|   at various gas concentrations.                                                         |
-------------------------------------------------------------------------------------------*/

/* _____________________________________________________________________________________
|                             DATA PROCESSING DESCRIPTION                                |
| _______________________________________________________________________________________|*/

/*The purpose of this MATLAB script is to produce a Fourier Transform (FT) graph for each 
gas concentration. FT graphs show the frequency composition and amplitudes of signals.
By plotting the FT graphs, we can clearly see the change in signals due to different 
gas concentrations.

Overview of data processing steps:

    1) Specific frames from raw data files ('ftraw[num].csv') are read. The background frame
       is subtracted from the frame containing signals from the sensor. This data is written
       into seperate csv files. For example, '1.csv' would contain the data without background
       signals, for the highest gas concentraion.  

    2) The newly-written '[num].csv' files are read into MATLAB and each signal's root-mean-
       square (RMS) values are computed. RMS values can be viewed as a signal's strength. Thus, 
       we obtain the signal strength of each signal, at all the measured gas concentrations.

    2) The next step is to compare the RMS values of each antenna pair, for all gas
       concentrations; for example, if there were 3 concentrations of gas and two 
       antenna pairs that were used, then we would compare the RMS values at all three 
       concentrations for antenna pair #1 and then for antenna pair #2, seperately. The
       purpose of this comparison is to determine which antenna pair's data follows the
       expected trend. The expected trend is that the RMS values should decrease with 
       increasing ammonia concentrations. 

    3) After finding the antenna pair which follows the previously described trend, a FFT
       is performed on the signal recorded from that antenna pair, at all concentrations.
       The FFT graphs for all concentrations are then plotted in the same figure.
*/


/* _____________________________________________________________________________________
|                                        FUNCTIONS                                       |
| _______________________________________________________________________________________|*/

// Built-in rms() function: used in the main program
/* Computes the root-mean-square (RMS) values of the input arguments. In the case that the
   input argument is a matrix (input arguments in this progrmam are matrices too), rms() 
   computes the RMS values along each colunmn of the matrix. 

    Input: one argument, which is a matrix in this program

    Returns: a row vector containing the rms value of each column (length of row vector
             in this program dependant on the number of antenna pairs used)
   */
// _____________________________________________________________________________________

// Built-in issorted() function: used in the main program
/* Determines if an array is sorted, i.e. the elements of the array are listed in 
   ascending order. 

    Input: one argument, which is a row vector in this program

    Returns: a logical scalar; returns 1 if the row vector is sorted, 0 if it is not*/

// _____________________________________________________________________________________

// Built-in fft() function: used in the custom 'positiveFFT()' function
/* Calculates the FFT of the input argument. 

    Input: one argument, which is 1D array in this program

    Returns: a row vector with the FFT of each indivudal data point, e.g. if the length
             of the input array was 2048, the returned row vector will have the same length.
             */
// _____________________________________________________________________________________

// Custom positiveFFT() function: used in the main program
/* Provides the normalized amplitude values for the fft and gets rid of the mirrored 
   image resulting form the symmtery by only using the first half of the dataset. It 
   also gives an x-axis that is in terms of frequency (GHz); it uses the dataset and 
   the sampling rate to determine those frequencies.  

    Input: two arguments; one is the dataset (1D array) and the the other is a sampling 
    rate (integer)

    Returns: two row vectors
        - first vector conatins the normalized FFT amplitude values
        - second vector contains the x-axis frequency values
    */

function [ X,freq ] = positiveFFT( x, Fs )

    N = length(x); // Taking the length of the input dataset to find the number of samples 
    k = 0:N-1;    
    T = N/Fs;   // Period
    freq = k/T; // Creates the frequency range 
    X = fft(x)/N;  // Normalizes the FFT
    upperbound = ceil(N/2);   
    X = X(1:upperbound);   /* Stores only the first half of the dataset values from the fft
                              to eliminate the mirrored image of the peaks */

    freq = freq(1:upperbound); /* Stored the x-axis values corresponding to the first half  
                                  of the dataset. */

end



/* _____________________________________________________________________________________
|                                     MAIN PROGRAM                                       |
| _______________________________________________________________________________________|*/

// Choose the 'long' format to show a more accurate representation of the signal values
clc
clear 
format long 

/*Prompts the user to enter the number of gas concnetrations measured and the number
of antenna pairs used for the measurements.*/
num_of_conc = input('Please enter the number of measured gas concentrations: ');
ant_pairs = input('Please enter the number of antenna pairs used: ');
fprintf(1, '\n');


// Global variables
/* Initializing zero-filled matrices so the program doesn't slow down later*/
RMS_values = zeros(num_of_conc, ant_pairs);
x_use = zeros(2048,40);
antenna_pair = 0;


//  'for' loop for subtracting the background signals and writing that data into a 
//                                new csv file

/*  The user is asked to enter the frame number for the sensor's reflected amplitudes
    and the frame number for the background signals. The code then reads the data
    from 'ftraw[number].csv' files and extracts these specific frames for background
    subtraction.
     */
for i = 1:num_of_conc
    frame_cnt =  input(['Please enter the frame number containing signals from the gas-sensing film for concentration ', num2str(i), ': ']);
    frame_bg = input('Please enter the frame with recorded background signals: ');

    row_start = (2048*(frame_cnt-1));
    row_end = row_start + 2047;
    row_start_bg = (2048*(frame_bg-1));
    row_end_bg = row_start_bg + 2047;
    
    ftraw_data_cnt = csvread(['ftraw', num2str(i),'.csv'], row_start, 0, [row_start 0 row_end ant_pairs-1]);
    ftraw_data_bg = csvread(['ftraw', num2str(i),'.csv'], row_start_bg, 0, [row_start_bg 0 row_end_bg ant_pairs-1]);
    subtracted_bg = ftraw_data_cnt - ftraw_data_bg;
    csvwrite([num2str(i), '.csv'], subtracted_bg)

    disp(sprintf(['Background signals have been subtracted and the data is stored in file ', num2str(i), '.csv\n']))
end  




//  Nested 'for' loops for calculating the root-mean-square (RMS) values for signals from 
//                              each antenna pair 

/* The outer loop goes up to the number of gas concentrations used, to read
    data from sequential csv files. The inner loop goes up to the number of antenna pairs
    used. Both these loops are used to index into thr RMS-values matrix. The RMS values 
    are stored in the RMS_values matrix of size = num_of_conc by ant_pairs matrix, where 
    each row represents the RMS values from different antenna pairs at a certain gas 
    concentration. 
*/
for k = 1:num_of_conc
    for i = 1:ant_pairs
        filename = [num2str(k) '.csv'];
        x_use = csvread(filename);      // file data read into 'x_use' variable

        RMS_rowvector = rms(x_use);     /* RMS values of each cocnentration (i.e. each file) 
                                           are calculated using built-in function */

        RMS_values(k,i) = RMS_rowvector(i);     /* The calculated RMS values are stored in 
                                                   the RMS_values matrix*/
    end
end



//    'for' loop to find which antenna pair is the first one to follow the expected
//                                         trend

/* A 'for loop is used to iterate through all column of the RMS_values matrix. Recall that 
    each column of this matrix contains RMS values at different gas concnetrations, and 
    these values are obtained using the same antenna pair. If the values of a certain column
    in RMS_values matrix follows the expected trend (i.e. the values increase as we move down
    the column), then the program is instructed to break out of the loop. 
*/
bf = false;     // flag variable for breaking out of the loop once the antenna pair is determined
 for m = 1:ant_pairs
         antenna_pair = m;      // 'm' represents the antenna pair 

         col = RMS_values(1:num_of_conc, m);      /*'col' contains the data from an individual 
                                                     column of the RMS_values */

         e = issorted(col);       /* Built-in function 'issorted' used to check if values 
                                   in 'col' are increasing in an acsending order. Function 
                                   returns 0 when false and 1, when true. */

         if e == 1              /* Condition that verifies if a certain column from RMS_values
                                   has values that are increasing */

             bf = true;         /* If condition holds, then flag variable is set to true and
                                   the program exits the 'if' block and then 'for loop */
             break

        elseif e == 0 && antenna_pair == 40     /* In case none of the data from any antenna pair 
                                                matches the expected trend */
            disp('NOTE: Data does not follow expected trend')

        end

         if bf,break,end
 end


// for loop to read through the sequential signal files, specifically the column of the file 
//              that corresponds to the antenna pair that follows the trend 

/* The for loop goes up to the number of concnetrations, to read the sequntial csv files and 
extract the column data (stored in fft_x) for the antenna pair determined previously. In the
case that the expected trend is not followed by any antenna pair, the script plots the data
from the last antenna pair. 
After the column data is retrieved from the file, it is passed on to 'positiveFFT' function, 
which returns the values for the x- and y-values of the FFT graphs. Using this data, the 
graphs of FFT for different concnetrations are plotted in the same figure.
*/
 Fs = 1e11;
 N = 2048;
 for f = 1:num_of_conc

    // reading data of single antenna pair and computing its fft
    fft_x = csvread([num2str(f),'.csv'], 0, antenna_pair-1, [0 antenna_pair-1 N-1 antenna_pair-1]);
    [YfreqD,freqRng] = positiveFFT(fft_x,Fs);

    // plotting the fft values, along with frequencies in the x-axis
    hold on
    xlim([3,10])
    plot(freqRng/(1e9),abs(YfreqD), 'DisplayName', ['Concentration ',num2str(f)]);
    title('FFT Graph of Different Gas Concentrations');
    xlabel('Frequency (GHz)');
    ylabel('Amplitude (a.u)');

 end

 legend(gca,'show') // Dynamically updating legend

INSTRUCTIONS FOR DATA COLLECTION AND DATA PROCESSING 

DATA COLLECTION USING WALABOT:

1) Place the gas sensing film under the Walabot at 5cm (as shown in the video), and start 
   running the python script, "RawSignalCollection.py" A window with a live-
   updating graph will appear, showing the raw signals in real-time. The python 
   script also saves the graph of each frame as 'frame[num].png' files, where 'num' 
   represents the frame number. Alternatively, the user can choose to simply print
   the frame number, rather than saving each frame (see python script for more details).

2) After a few a seonds, remove the gas sensing film from under the Walabot to record
   background. 

3) Stop the script once you have recorded the background. The raw data from this test run 
   will be saved in 'ftraw.csv'. 

4) Take note of the specific frame number which contains the signals from the gas sensing film,
   and the background data. You will need these two numbers later for data processing.

5) Repeat with different gas concentrations. Make sure to save the 'ftraw.csv' file and
   the frame images in another location, or else they will be re-written when the python script 
   is used again.

DATA PROCESSING:

The purpose of the MATLAB script is to produce a Fourier Transform (FT) graph for each 
gas concentration. FT graphs show the frequency composition and amplitudes of signals.
By plotting the FT graphs, we can clearly see the change in signals due to different 
gas concentrations.

Here are some things to keep in mind before using this script:

    1) All the raw data files obtained through 'RawSignalCollection.py' should be renamed 
      as 'ftraw[num].csv', where 'num' are integers (1,2,3,..). The file corresponding to 
      highest gas concentration should be named 'ftraw1.csv', the second highest
      concentration should be 'ftraw2.csv' and so on. This naming convention is necessary
      for the MATLAB script to run properly.

    2) The raw data files must be saved in MATLAB's workspace directory.

    3) Enter the number of gas concentrations measured, the number of antenna pairs used,
       and the frame numbers for the gas sensing film's signals and the background signals, 
       when prompted.