Skip to content
GitLab
Commit 516e2d17 authored by Marion Bandet's avatar Marion Bandet
Browse files

update WERA calibrations

parent 5fde9a30
Show whitespace changes
Inline Side-by-side
build/html/_sources/Comparison_WERA-ADCP_Sept2013.txt
View file @ 516e2d17
......@@ -37,7 +37,7 @@ Steps of the algorithm:
For each index, we load the corresponding HFR file (i.e. 20132440000_pao.mat). Each file contains a grid (X,Y) with radial currents at each grid point (Ur), a corresponding time vector (t) and the angle and distance from the site (theta,r).
.. figure:: ../../figures/RadialVelocities/ADCP/Radial_ADCPbin2_HFR_PAO_version5_Sept2013.png
.. figure:: figures/Radial_ADCPbin2_HFR_PAO_version5_Sept2013.png
:align: center
Hourly radial currents from ADCP bin 2 and HFR WERA at PAO - September 2013
......
build/html/_sources/WERAcalibration.txt
View file @ 516e2d17
WERA calibrations
===================
WERAcalibration.m
*******************
clear all
close all
In March 2015, we used data collected in September 2013 to compare old and new calibrations in order to know which ones we should use for the on-going analysis of currents data.
%cd('/home/bandma01/Documents/WERAs/')
cd('/home/bandma01/hfr-data-processing/calibrations/')
% site = 'PAO';
site = 'PAB';
cd(site)
PAP1
% system
for sys = 1:2
.. topic:: PAP1
if sys == 1
systm = 'Helzel';
elseif sys == 2
systm = 'Flament';
end
RadialConfigs_PAP1_20121213_WAPM_MUSIC402002_5Dsmooth
from 2012-12-13 to 2013-02-01 19:50 UTC
clear J I E Phase calib_file calib_hr calib_jd calib_min calib_time calib_value
clear calib_year d data delta_phase_obs delta_phase_th gps_data index_timecalib
clear theta_pairs
RadialConfigs_PAP1_20130201_BAPM_MUSIC402002_nosmooth
from 2013-02-01 19:50 UTC to now
if strcmp(site,'PAO') == 1
load('HelzelPAO_Track_29-OCT-14 183138.mat')
.. topic:: STF2
elseif strcmp(site,'PAB') == 1
RadialConfigs_STF2_20121220_BAPM_MUSIC402002_nosmooth
from 2012-12-20 to 2013-02-06 12:00 UTC
if strcmp(systm,'Helzel') == 1
load('HelzelPAB_Track_30-OCT-14 175446.mat')
elseif strcmp(systm,'Flament') == 1
load('FlamentPAB_Track_30-OCT-14 182008.mat')
end
RadialConfigs_PAP1_20130201_BAPM_MUSIC402002_nosmooth
from 2013-02-01 19:50 UTC to now
end
%% phases and time of calibration
if strcmp(site,'PAO') == 1
cd('2014302')
elseif strcmp(site,'PAB') == 1
cd('2014303')
end
% list files
clear d
d = dir('*.mat'); % returns a structure with name, date, bytes, isdir, datenum
Nfiles = length(d); % nombre de fichiers a analyser
% look at the names of files and extract information
for c = 1:Nfiles
calib_year(c) = str2double(d(c).name(1:4));
calib_jd(c) = str2double(d(c).name(5:7));
calib_hr(c) = str2double(d(c).name(8:9));
calib_min(c) = str2double(d(c).name(10:11));
load(d(c).name,'phase')
Phase(:,c) = phase;
end
clear phase
delta_phase_obs = modsym(diff(Phase,1),180);
To = datenum(2014,01,01,0,0,0);
% add 20 sec to center calibration time
calib_time = To + calib_jd -1 + calib_hr./24 + calib_min/24/60 + 20/24/3600;
% datevec(calib_time)
if strcmp(site,'PAO') == 1
% calib time 2min34sec late wrt GPS
calib_time = calib_time + 2/24/60 + 34/24/3600;
end
%% separate between calibrations made with Helzel's system and Pierre's system
% date of new calibration files
% Helzel 2014-10-30 16:52 to 17:54
% Flament 2014-10-30 17:55 to 18:20
T1 = datenum(2014,10,30,17,59,00);
clear J
if strcmp(systm,'Helzel') == 1
J = find(calib_time > T1);
elseif strcmp(systm,'Flament') == 1
J = find(calib_time > T1);
end
%remove unwanted data to avoid confusion!
calib_year(J)=[];
calib_jd(J)=[];
calib_hr(J)=[];
calib_min(J)=[];
Phase(:,J)=[];
delta_phase_obs(:,J)=[];
calib_time(J)=[];
d(J)=[];
clear c Nfiles
%+++++++++++++++++++++++++++++++++++++++++
% for each calibration file (series# 1)
%+++++++++++++++++++++++++++++++++++++++++
for calib_file = 1:length(calib_time)
time_calib = calib_time(calib_file);
% find corresponding file in gps data
clear I
I = find(data(:,1) > time_calib);
% take closest point for now
I = I(1);
% datevec(data(I,1))
% datevec(time_calib)
index_timecalib(calib_file) = I;
end
% check
% datevec(data(index_timecalib,1))
% isolate gps calibration data
gps_data = data(index_timecalib,:); % time lat lon
figure(1)
hold on
h1=plot(gps_data(:,2),gps_data(:,3),'.');
grid on
if strcmp(systm,'Helzel') == 1
set(h1,'marker','.','color','b')
elseif strcmp(systm,'Flament') == 1
set(h1,'marker','x','color','k')
end
% load antenna coordinates
cd ..
load([site '_Rx_coords.mat'])
hold on
plot(lon,lat,'x')
grid on
title(['site = ' site ])
%% theoretical values
% wavelength
lambda = 3E8/16.15E6;
% initialize variables
theta_pairs = nan(11,length(gps_data));
% for each gps point
for pt = 1:length(gps_data)
% compute distance between GPS and antennas
for ant = 1:12
clear E W
[E,W] = lonlat2km(gps_data(pt,2),gps_data(pt,3),lon(ant),lat(ant));
ddist = sqrt(E.^2+W.^2);
r(ant) = ddist*1000; %meters
end
%for each pair of antennas
for p = 1:11
%delta_r
delta_r(p) = r(p+1) - r(p);
% theoretical phase
delta_phase_th(p,pt) = modsym((delta_r(p)*360./lambda),180);
% position of middle of pairs
clear lon_mid lat_mid
lon_mid = (lon(p+1)+lon(p))/2;
lat_mid = (lat(p+1)+lat(p))/2;
% distance between middle of pair and one of the antenna in the pair
% clear E W
% [E,W] = lonlat2km(lon_mid,lat_mid,lon(p),lat(p));
% ddist = sqrt(E.^2+W.^2);
% a(p) = ddist*1000; %meters
% distance between middle of pair and GPS pt
clear ddsit phasAng
[ddist,phasAng] = sw_dist([gps_data(pt,3) lat_mid],[gps_data(pt,2) lon_mid],'km');
r_mid(p) = ddist*1000; %meters
zeta(p) = phasAng;
% distance between pair of antennas
clear ddsit phasAng
[ddist,phasAng] = sw_dist([lat(p) lat(p+1)],[lon(p) lon(p+1)],'km');
phi(p) = phasAng;
% azimuth
theta_pairs(p,pt) = modsym(zeta(p)-phi(p)+90,180);
%rad2deg(theta_pairs)
end
end
figure(1)
legend('gps','antennas','location','southeast')
xlabel('longitude')
ylabel('latitude')
% for each pair of antenna
for p = 1:11
figure(p+1)
h2=plot(theta_pairs(p,:),delta_phase_th(p,:),'x');
hold on
h3=plot(theta_pairs(p,:),-delta_phase_obs(p,:),'or');
grid on
xlabel('incidence (degrees)')
ylabel('phase difference (degrees)')
legend('theoretical','observed','location','best')
title([site ' - antennas ' num2str(p) '-' num2str(p+1)])
if strcmp(systm,'Helzel') == 1
set(h2,'color','b')
set(h3,'color','r','marker','o')
elseif strcmp(systm,'Flament') == 1
set(h2,'color','k','marker','*')
set(h3,'color','r','marker','sq','markerfacecolor','r')
end
% compute calibration value for antennas pairs
calib_value(p) = nanmean(modsym(-delta_phase_obs(p,:)-delta_phase_th(p,:),180),2);
% compute calibration value for each antenna
phi_ant(p+1,sys) = sum(calib_value);
end
P=ones(16,3);
P(13:16,:)=0;
P(1:12,3)=phi_ant(:,1);
% save file (we only keep results from Helzel
if strcmp(site,'PAO') == 1
dlmwrite('calibration_pao_29OCT2014.wera',P,'delimiter','\t','precision','%.7f')
elseif strcmp(site,'PAB') == 1
dlmwrite('calibration_pab_30OCT2014.wera',P,'delimiter','\t','precision','%.7f')
end
figure(13)
hold on
h4=plot(calib_value,'.-');
grid on
xlabel('pairs of antennas')
ylabel({'phase difference between','theoretical and observed values (degrees)'})
title(site)
if strcmp(systm,'Helzel') == 1
set(h4,'color','b')
elseif strcmp(systm,'Flament') == 1
set(h4,'color','k')
legend('Helzel','Flament','location','southeast')
end
figure(14)
hold on
h5=plot([1:12],phi_ant(:,sys),'r');
grid on
if strcmp(systm,'Helzel') == 1
set(h5,'marker','o')
elseif strcmp(systm,'Flament') == 1
set(h5,'marker','sq','markerfacecolor','r')
legend('Helzel','Flament','location','southeast')
end
xlabel('antenna number')
ylabel('calibration phase to be applied (degrees)')
title(site)
if strcmp(site,'PAO') == 1
break
end
end
%add current calibration phases
if strcmp(site,'PAO') == 1
load calibration_pao.wera
figure(14)
plot([1:12],calibration_pao(1:12,3),'b.-')
legend('Helzel','original calibration','location','northwest')
elseif strcmp(site,'PAB') == 1
load calibration_pab.wera
figure(14)
plot([1:12],calibration_pab(1:12,3),'b.-')
legend('Helzel','Flament','original calibration','location','southeast')
end
% save figures
save_flag = 1;
figdir = ('/home/bandma01/Dropbox/UQAR/MEOPAR_Marion/figures/WERAcalibration');
cd(figdir)
fig_hdl = figure(1);
if save_flag
fname = ['configuration_' site '_calibrations'];
fig_w = 8;%cm
fig_h = 5;%cm
set(fig_hdl,'PaperUnits','centimeters')
set(fig_hdl,'PaperPosition',[0 0 fig_w fig_h])
set(fig_hdl, 'PaperSize', [fig_w fig_h]);
print(fig_hdl,'-dpng','-r500',fname)
%crop([fname '.png'])
end
for i = 2:12
fig_hdl = figure(i);
if save_flag
fname = ['Phases_' site '_pair' num2str(i-1) '-' num2str(i)];
fig_w = 8;%cm
fig_h = 5;%cm
set(fig_hdl,'PaperUnits','centimeters')
set(fig_hdl,'PaperPosition',[0 0 fig_w fig_h])
set(fig_hdl, 'PaperSize', [fig_w fig_h]);
print(fig_hdl,'-dpng','-r500',fname)
% crop([fname '.png'])
end
end
fig_hdl = figure(13);
if save_flag
fname = ['PhaseDifference_' site '_calibrations'];
fig_w = 8;%cm
fig_h = 5;%cm
set(fig_hdl,'PaperUnits','centimeters')
set(fig_hdl,'PaperPosition',[0 0 fig_w fig_h])
set(fig_hdl, 'PaperSize', [fig_w fig_h]);
print(fig_hdl,'-dpng','-r500',fname)
% crop([fname '.png'])
end
fig_hdl = figure(14);
if save_flag
fname = ['PhaseCalibration_' site];
fig_w = 8;%cm
fig_h = 5;%cm
set(fig_hdl,'PaperUnits','centimeters')
set(fig_hdl,'PaperPosition',[0 0 fig_w fig_h])
set(fig_hdl, 'PaperSize', [fig_w fig_h]);
print(fig_hdl,'-dpng','-r500',fname)
% crop([fname '.png'])
end
build/html/_sources/calibration.txt
View file @ 516e2d17
......@@ -10,7 +10,7 @@ CODAR
**POINTE-AU-PERE:**
***********************
Please see notes from field work on `2013-02-01`_
Please see notes from field work on `2013-02-01`_.
.. _2013-02-01: http://demeter.uqar.ca/submesoscale/index.php/Pap
......@@ -67,6 +67,94 @@ To generate the new Antenna diagram:
WERA
#####
Go to the **"Submesoscale"** `wiki`_ and click on "useful links" under **Outils**, then "WERA HF Radar Software - Fortran Programs Documentation". Here we find which Fortran program we need to use to do what we want.
.. _wiki: http://demeter.uqar.ca/submesoscale/index.php/Accueil
.. topic:: Objectives
We need to "remove wrong calibration data and apply new ones", so we need to use Modify_WERA_SORT.
Steps to generate new calibration file
*****************************************
#. loggin to the wera server wera@132.215.11.14
#. cd etc/
In this folder, we find the original calibration files:
- calibration_pab.wera
- calibration_pao.wera
If we look at what the content of the calibration file:
- cat calibration_pab.wera
the phase is given in the 3rd column.
On 2014-10-29 (PAO) and 2014_10-30 (PAB), calibrations were undertaken in the field.
Calibration data are stored in:
- /hfr-data-processing/calibrations/PAB/2014303 every minute from 16:53 to 17:55
- /hfr-data-processing/calibrations/PAO/2014302 every minute from 17:46 to 18:23
While the calibrations were running, tracks were recorded with an instrument belonging to Helzel (PAO and PAB) as well as with an instrument belonging to Flament (PAB). So for PAB we are going to be able to compare the two.
The tracks can be found in :
- /hfr-data-processing/calibrations/PAO/HelzelPAO_Track_29-OCT-14 183138.gpx
- /hfr-data-processing/calibrations/PAB/HelzelPAB_Track_30-OCT-14 175446.gpx
- /hfr-data-processing/calibrations/PAB/FlamentPAB_Track_30-OCT-14 182008.gpx
We use the routine **GPS_gpx2mat.m** to convert .gpx files into .mat files.
We use the routine **WERA_AntennaCoord_txt2mat.m** to transform the .txt files with antennas'coordinates to .mat files.
We use the routine **WERAphase_txt2mat.m** to convert the .txt files containing phases into .mat files.
The routine **WERAcalibration.m** is available at this `page`_.
.. _page: WERAcalibration.html
All the Matlab routines are located in ~/hfr-data-processing/calibrations/.
Re-processing of radial currents with new phases
***************************************************
For the all month of September 2013, we are going to generate new files using the new calibration phases.
Post-processing of WERA radial currents is documented `here`_.
.. _here: WERAprocessing.html
For example, let us see what we have to do to generate radial currents at PAO using the new calibration phases for september 1st, 2013 (Julian day 244):
Data in the ~/data1/PAO/2013244/raw/ folder are converted in .SORT and .RFI in the ~/data1/PAO/2013244/results/ folder using the **do_process_SORT_pol** shell script. These files are generated using the "original" calibration phases.
First we need to change the original .SORT and .RFI files and save them to .SORT0 and .RFI0 to save them for future reference. This is done with the shell script **changeName.sh** located in /hfr-data-processing/postprocessing/wera/.
.SORT====> .SORT0 (generated with original calibration)
.RFI ====> .RFI0
as well as 2 symbolic links to point towards which file is the calibration to remove (with link calibration.wera.remove) and which file is the calibration to use (with link calibration.wera.apply).
If we look at what the content of the calibration file is:
* cat calibration_pab.wera
We want to compare the value of the phase given in the 3rd column with the one that we compute.
COMPARISON OLD/NEW WERA CALIBRATIONS
**************************************
......
build/html/calibration.html
View file @ 516e2d17
......@@ -55,7 +55,7 @@
<h2>CODAR<a class="headerlink" href="#codar" title="Permalink to this headline"></a></h2>
<div class="section" id="pointe-au-pere">
<h3><strong>POINTE-AU-PERE:</strong><a class="headerlink" href="#pointe-au-pere" title="Permalink to this headline"></a></h3>
<p>Please see notes from field work on <a class="reference external" href="http://demeter.uqar.ca/submesoscale/index.php/Pap">2013-02-01</a></p>
<p>Please see notes from field work on <a class="reference external" href="http://demeter.uqar.ca/submesoscale/index.php/Pap">2013-02-01</a>.</p>
<p>Starting 2013-02-01: <strong>Change of frequency</strong> due to a misreading in the frequency permits...</p>
<blockquote>
<div><ul class="simple">
......@@ -108,6 +108,77 @@
</div>
<div class="section" id="wera">
<h2>WERA<a class="headerlink" href="#wera" title="Permalink to this headline"></a></h2>
<p>Go to the <strong>&#8220;Submesoscale&#8221;</strong> <a class="reference external" href="http://demeter.uqar.ca/submesoscale/index.php/Accueil">wiki</a> and click on &#8220;useful links&#8221; under <strong>Outils</strong>, then &#8220;WERA HF Radar Software - Fortran Programs Documentation&#8221;. Here we find which Fortran program we need to use to do what we want.</p>
<div class="topic">
<p class="topic-title first">Objectives</p>
<p>We need to &#8220;remove wrong calibration data and apply new ones&#8221;, so we need to use Modify_WERA_SORT.</p>
</div>
<div class="section" id="steps-to-generate-new-calibration-file">
<h3>Steps to generate new calibration file<a class="headerlink" href="#steps-to-generate-new-calibration-file" title="Permalink to this headline"></a></h3>
<blockquote>
<div><ol class="arabic simple">
<li>loggin to the wera server <a class="reference external" href="mailto:wera&#37;&#52;&#48;132&#46;215&#46;11&#46;14">wera<span>&#64;</span>132<span>&#46;</span>215<span>&#46;</span>11<span>&#46;</span>14</a></li>
<li>cd etc/</li>
</ol>
</div></blockquote>
<dl class="docutils">
<dt>In this folder, we find the original calibration files:</dt>
<dd><ul class="first last simple">
<li>calibration_pab.wera</li>
<li>calibration_pao.wera</li>
</ul>
</dd>
<dt>If we look at what the content of the calibration file:</dt>
<dd><ul class="first last simple">
<li>cat calibration_pab.wera</li>
</ul>
</dd>
</dl>
<p>the phase is given in the 3rd column.</p>
<p>On 2014-10-29 (PAO) and 2014_10-30 (PAB), calibrations were undertaken in the field.</p>
<dl class="docutils">
<dt>Calibration data are stored in:</dt>
<dd><ul class="first last simple">
<li>/hfr-data-processing/calibrations/PAB/2014303 every minute from 16:53 to 17:55</li>
<li>/hfr-data-processing/calibrations/PAO/2014302 every minute from 17:46 to 18:23</li>
</ul>
</dd>
</dl>
<p>While the calibrations were running, tracks were recorded with an instrument belonging to Helzel (PAO and PAB) as well as with an instrument belonging to Flament (PAB). So for PAB we are going to be able to compare the two.</p>
<dl class="docutils">
<dt>The tracks can be found in :</dt>
<dd><ul class="first last simple">
<li>/hfr-data-processing/calibrations/PAO/HelzelPAO_Track_29-OCT-14 183138.gpx</li>
<li>/hfr-data-processing/calibrations/PAB/HelzelPAB_Track_30-OCT-14 175446.gpx</li>
<li>/hfr-data-processing/calibrations/PAB/FlamentPAB_Track_30-OCT-14 182008.gpx</li>
</ul>
</dd>
</dl>
<p>We use the routine <strong>GPS_gpx2mat.m</strong> to convert .gpx files into .mat files.
We use the routine <strong>WERA_AntennaCoord_txt2mat.m</strong> to transform the .txt files with antennas&#8217;coordinates to .mat files.
We use the routine <strong>WERAphase_txt2mat.m</strong> to convert the .txt files containing phases into .mat files.</p>
<p>The routine <strong>WERAcalibration.m</strong> is available at this <a class="reference external" href="WERAcalibration.html">page</a>.</p>
<p>All the Matlab routines are located in ~/hfr-data-processing/calibrations/.</p>
</div>
<div class="section" id="re-processing-of-radial-currents-with-new-phases">
<h3>Re-processing of radial currents with new phases<a class="headerlink" href="#re-processing-of-radial-currents-with-new-phases" title="Permalink to this headline"></a></h3>
<p>For the all month of September 2013, we are going to generate new files using the new calibration phases.</p>
<p>Post-processing of WERA radial currents is documented <a class="reference external" href="WERAprocessing.html">here</a>.</p>
<p>For example, let us see what we have to do to generate radial currents at PAO using the new calibration phases for september 1st, 2013 (Julian day 244):</p>
<p>Data in the ~/data1/PAO/2013244/raw/ folder are converted in .SORT and .RFI in the ~/data1/PAO/2013244/results/ folder using the <strong>do_process_SORT_pol</strong> shell script. These files are generated using the &#8220;original&#8221; calibration phases.</p>
<p>First we need to change the original .SORT and .RFI files and save them to .SORT0 and .RFI0 to save them for future reference. This is done with the shell script <strong>changeName.sh</strong> located in /hfr-data-processing/postprocessing/wera/.</p>
<p>.SORT====&gt; .SORT0 (generated with original calibration)
.RFI ====&gt; .RFI0</p>
<p>as well as 2 symbolic links to point towards which file is the calibration to remove (with link calibration.wera.remove) and which file is the calibration to use (with link calibration.wera.apply).</p>
<dl class="docutils">
<dt>If we look at what the content of the calibration file is:</dt>
<dd><ul class="first last simple">
<li>cat calibration_pab.wera</li>
</ul>
</dd>
</dl>
<p>We want to compare the value of the phase given in the 3rd column with the one that we compute.</p>
</div>
<div class="section" id="comparison-old-new-wera-calibrations">
<h3>COMPARISON OLD/NEW WERA CALIBRATIONS<a class="headerlink" href="#comparison-old-new-wera-calibrations" title="Permalink to this headline"></a></h3>
<p>In March 2015, we used data collected in September 2013 to compare old and new calibrations in order to know which ones we should use for the on-going analysis of currents data.</p>
......@@ -142,6 +213,8 @@ Superimposed are the 48-h averages</div>
</ul>
</li>
<li><a class="reference internal" href="#wera">WERA</a><ul>
<li><a class="reference internal" href="#steps-to-generate-new-calibration-file">Steps to generate new calibration file</a></li>
<li><a class="reference internal" href="#re-processing-of-radial-currents-with-new-phases">Re-processing of radial currents with new phases</a></li>