Blame view

source/calibration.rst 11.5 KB
5fde9a30   Marion Bandet   update documentation
1
2
3
4
**************
Calibrations
**************

7e281bfb   Marion Bandet   update
5
6
.. note:: **Calibrations made in 2014**

5fde9a30   Marion Bandet   update documentation
7
8
9
10
11
12
13
14
######
CODAR
######


**POINTE-AU-PERE:** 
***********************

516e2d17   Marion Bandet   update WERA calib...
15
Please see notes from field work on `2013-02-01`_.
5fde9a30   Marion Bandet   update documentation
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

.. _2013-02-01: http://demeter.uqar.ca/submesoscale/index.php/Pap

Starting 2013-02-01: **Change of frequency** due to a misreading in the frequency permits... 

      
   * Up to 2013-02-01 19:50 UTC use the "old" antenna diagram to do the calibration
   * From 2013-02-01 19:50 UTC use the "new" antenna diagram 


We reprocessed all data from CODAR PAP **starting 2012-12-13** date at which the site was operational.
   

.. topic:: from 2012-12-13 to 2013-02-01 19:50 UTC
   
        RadialConfigs_PAP1_20121213_WAPM_MUSIC402002_5Dsmooth
        

Steps to generate new RadialConfigs files:

   * Copy all content of the directory in **RadialConfigs**
   * run SeaSondeRadialSetUp
   * change output: metrics, processing check MUSIC 40 20 02, no smoothing, filtered


To generate the new Antenna diagram:

   * cd Data/Loops/PATT_PAP1_2012_12_15...xlp
        ==> open .patt
        -  Walk APM up to 2013-02-01
        -  Boat APM until now


.. topic:: from 2013-02-01 19:50 UTC to now

        RadialConfigs_PAP1_20130201_BAPM_MUSIC402002_nosmooth
  


**SAINTE-FLAVIE**
*****************

.. topic:: from 2012-12-20 to 2013-02-06 12:00 UTC
      
          RadialConfigs_STF2_20121220_BAPM_MUSIC402002_nosmooth
       

.. topic:: from 2013-02-01 19:50 UTC to now
      
a52e8119   Marion Bandet   update proccessin...
65
          RadialConfigs_STF2_20130206_BAPM_MUSIC402002_nosmooth
5fde9a30   Marion Bandet   update documentation
66
67
68
69
70
71
      

#####
WERA
#####

516e2d17   Marion Bandet   update WERA calib...
72
73
74
75
76
77
78
79
80
81
82
83
Go to the **"Submesoscale"** `wiki`_ and click on "useful links" under **Outils**, then "WERA HF Radar Software - Fortran Programs Documentation". Here we find which Fortran program we need to use to do what we want.

.. _wiki: http://demeter.uqar.ca/submesoscale/index.php/Accueil


.. topic:: Objectives

        We need to "remove wrong calibration data and apply new ones", so we need to use Modify_WERA_SORT.




9ce85642   Marion Bandet   update calibratio...
84
85
1. Steps to generate new calibration phases
*********************************************
516e2d17   Marion Bandet   update WERA calib...
86
87
88
89
90
91
92
93

  #. loggin to the wera server wera@132.215.11.14
  #. cd etc/

In this folder, we find the original calibration files: 
     - calibration_pab.wera
     - calibration_pao.wera

7e281bfb   Marion Bandet   update
94
If we look at what the content of the calibration file:ca
516e2d17   Marion Bandet   update WERA calib...
95
96
97
98
99
100
101
102
     - cat calibration_pab.wera

the phase is given in the 3rd column.


On 2014-10-29 (PAO) and 2014_10-30 (PAB), calibrations were undertaken in the field.

Calibration data are stored in:
ac1a3290   Marion Bandet   modification wera...
103
104
105
     - /run/media/bandma01/SeagateWireless/MEOPAR/DATA/HFRs/WERAs/PAB/2014303 every minute from 30-Oct-2014 16:53:20 to 30-Oct-2014 18:11:20
     - /run/media/bandma01/SeagateWireless/MEOPAR/DATA/HFRs/WERAs/PAO/2014302 every minute from 29-Oct-2014 17:48:54 to 29-Oct-2014 18:25:54

516e2d17   Marion Bandet   update WERA calib...
106
107
108
109
 
While the calibrations were running, tracks were recorded with an instrument belonging to Helzel (PAO and PAB) as well as with an instrument belonging to Flament (PAB). So for PAB we are going to be able to compare the two. 

The tracks can be found in :
ac1a3290   Marion Bandet   modification wera...
110
111
112
     - /run/media/bandma01/SeagateWireless/MEOPAR/DATA/HFRs/WERAs/PAO/HelzelPAO_Track_29-OCT-14 183138.gpx
     - /run/media/bandma01/SeagateWireless/MEOPAR/DATA/HFRs/WERAs/PAB/HelzelPAB_Track_30-OCT-14 175446.gpx 
     - /run/media/bandma01/SeagateWireless/MEOPAR/DATA/HFRs/WERAs/PAB/FlamentPAB_Track_30-OCT-14 182008.gpx 
516e2d17   Marion Bandet   update WERA calib...
113
114

We use the routine **GPS_gpx2mat.m** to convert .gpx files into .mat files.
9ce85642   Marion Bandet   update calibratio...
115

516e2d17   Marion Bandet   update WERA calib...
116
We use the routine **WERA_AntennaCoord_txt2mat.m** to transform the .txt files with antennas'coordinates to .mat files.
9ce85642   Marion Bandet   update calibratio...
117

516e2d17   Marion Bandet   update WERA calib...
118
119
We use the routine **WERAphase_txt2mat.m** to convert the .txt files containing phases into .mat files.

7e281bfb   Marion Bandet   update
120
The routine **WERAcalibration_2014.m** is available at this `page`_.
516e2d17   Marion Bandet   update WERA calib...
121

7e281bfb   Marion Bandet   update
122
.. _page: WERAcalibration_2014.html
516e2d17   Marion Bandet   update WERA calib...
123

f45a2133   Marion Bandet   update calibratio...
124
125
126
All the Matlab routines used for the data processing are located in **~/hfr-data-processing/calibrations/**.


7e281bfb   Marion Bandet   update
127
Outputs of **WERAcalibration_2014.m** for PAO site are available here: `calibrationPAO2014`_
f45a2133   Marion Bandet   update calibratio...
128

7e281bfb   Marion Bandet   update
129
.. _calibrationPAO2014: calibPAO2014.html
f45a2133   Marion Bandet   update calibratio...
130
131

+---------------------------------------------+
9ce85642   Marion Bandet   update calibratio...
132
|  PAO - Helzel- Phases in degrees            |
f45a2133   Marion Bandet   update calibratio...
133
134
135
136
137
+-------------+-----------------+-------------+
| Antenna nb  | original phase  |  new phase  |
+=============+=================+=============+
|  1          |      0          |     0       |
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
138
|  2          |     -0.6181     |    2.2270   |
f45a2133   Marion Bandet   update calibratio...
139
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
140
|  3          |     -19.9027    |   -20.8291  |
f45a2133   Marion Bandet   update calibratio...
141
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
142
|  4          |       1.2299    |     1.2548  |
f45a2133   Marion Bandet   update calibratio...
143
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
144
|  5          |     -21.2304    |   -15.9611  |
f45a2133   Marion Bandet   update calibratio...
145
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
146
|  6          |      10.7207    |    12.7500  |
f45a2133   Marion Bandet   update calibratio...
147
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
148
|  7          |      -5.2161    |     6.0602  |
f45a2133   Marion Bandet   update calibratio...
149
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
150
|  8          |      -14.8695   |   -14.5626  |
f45a2133   Marion Bandet   update calibratio...
151
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
152
|  9          |      -2.5658    |     1.1416  |
f45a2133   Marion Bandet   update calibratio...
153
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
154
|  10         |      -16.1023   |   -14.8303  |
f45a2133   Marion Bandet   update calibratio...
155
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
156
|  11         |      -6.7589    |   -10.0689  |
f45a2133   Marion Bandet   update calibratio...
157
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
158
|  12         |      -9.7778    |   -13.8845  |
f45a2133   Marion Bandet   update calibratio...
159
160
+-------------+-----------------+-------------+

9ce85642   Marion Bandet   update calibratio...
161
The new phase values were saved under **calibration_pao_29OCT2014.wera**.
f45a2133   Marion Bandet   update calibratio...
162

9ce85642   Marion Bandet   update calibratio...
163
 
7e281bfb   Marion Bandet   update
164
Outputs of WERAcalibration.m for PAB site are available here: `calibrationPAB2014`_
f45a2133   Marion Bandet   update calibratio...
165

7e281bfb   Marion Bandet   update
166
.. _calibrationPAB2014: calibPAB2014.html
f45a2133   Marion Bandet   update calibratio...
167
168
169


+---------------------------------------------+
9ce85642   Marion Bandet   update calibratio...
170
|  PAB - Helzel - Phases in degrees           |
f45a2133   Marion Bandet   update calibratio...
171
172
173
174
175
+-------------+-----------------+-------------+
| Antenna nb  | original phase  |  new phase  |
+=============+=================+=============+
|  1          |      0          |     0       |
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
176
|  2          |     1.0514      |   -19.6031  |
f45a2133   Marion Bandet   update calibratio...
177
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
178
|  3          |     8.3719      |    77.3093  |
f45a2133   Marion Bandet   update calibratio...
179
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
180
|  4          |       2.0496    |    -2.8669  |
f45a2133   Marion Bandet   update calibratio...
181
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
182
|  5          |      13.5527    |    29.2102  |
f45a2133   Marion Bandet   update calibratio...
183
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
184
|  6          |      -3.0394    |    -1.9353  |
f45a2133   Marion Bandet   update calibratio...
185
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
186
|  7          |       0.9866    |   -30.6625  |
f45a2133   Marion Bandet   update calibratio...
187
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
188
|  8          |      -2.9561    |    -7.2423  |
f45a2133   Marion Bandet   update calibratio...
189
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
190
|  9          |      -1.5691    |   -28.9055  |
f45a2133   Marion Bandet   update calibratio...
191
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
192
|  10         |       -2.4549   |   -24.0200  |
f45a2133   Marion Bandet   update calibratio...
193
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
194
|  11         |       1.4106    |   -12.9370  |
f45a2133   Marion Bandet   update calibratio...
195
+-------------+-----------------+-------------+
2b149458   Marion Bandet   mise a jour
196
|  12         |      -2.7529    |    -7.4966  |
f45a2133   Marion Bandet   update calibratio...
197
198
+-------------+-----------------+-------------+

9ce85642   Marion Bandet   update calibratio...
199
The new phase values were saved under **calibration_pab_30OCT2014.wera**.
516e2d17   Marion Bandet   update WERA calib...
200
201
202
203




9ce85642   Marion Bandet   update calibratio...
204
2. Re-processing of radial currents with new phases
516e2d17   Marion Bandet   update WERA calib...
205
206
***************************************************

9ce85642   Marion Bandet   update calibratio...
207
For the all month of September 2013, we are going to generate new radial currents using the new calibration phases. 
516e2d17   Marion Bandet   update WERA calib...
208

9ce85642   Marion Bandet   update calibratio...
209
Common steps for the post-processing of WERA radial currents is documented `here`_.
516e2d17   Marion Bandet   update WERA calib...
210
211
212

.. _here: WERAprocessing.html

516e2d17   Marion Bandet   update WERA calib...
213

9ce85642   Marion Bandet   update calibratio...
214
215
216
217
218
219
220
221
222
223
For example, let us see what we have to do to generate new radial currents at PAO using the new calibration phases for september 1st, 2013 (Julian day 244):

   - loggin to the wera server

   - go to the folder containing the data:

        . cd  ~/data1/PAO/2013244/
        . the raw currents are located in **2013244/raw/** 
        . the analysis takes the raw currents, apply the calibration phases and save the resulting radial currents in **/2013244/results/** under .SORT and .RFI files using the **do_process_SORT_pol** shell script. This is done using the original calibration phases.
        . the new radial currents using the updated calibration phases will be saved under **/2013244/results/new**.
516e2d17   Marion Bandet   update WERA calib...
224
225
226

 

9ce85642   Marion Bandet   update calibratio...
227
    - First we need to change the original .SORT and .RFI files and save them to .SORT0 and .RFI0 to save them for future reference. This is done with the shell script **changeName.sh** located in **/hfr-data-processing/postprocessing/wera/**.
516e2d17   Marion Bandet   update WERA calib...
228

9ce85642   Marion Bandet   update calibratio...
229
    - then we need to create a symbolic link to specify with files to use in terms of calibration phases. This file is different for each site. 
516e2d17   Marion Bandet   update WERA calib...
230

9ce85642   Marion Bandet   update calibratio...
231
     For the example taken here (site = PAO): calibration.wera.apply ---> calibration_pao_29OCT2014.wera  and calibration.wera.remove -----> calibration_pao.wera
516e2d17   Marion Bandet   update WERA calib...
232
233


9ce85642   Marion Bandet   update calibratio...
234
    - then we need to run **MODIFY_WERA_SORT** to generate new .SORT and .RFI files using the new calibration phases.
516e2d17   Marion Bandet   update WERA calib...
235

9ce85642   Marion Bandet   update calibratio...
236
237
All these steps are done in a shell script **do_postprocess_SORT_newCalib.sh** that calls **wera_new_calibration.sh**.
  
516e2d17   Marion Bandet   update WERA calib...
238

9ce85642   Marion Bandet   update calibratio...
239
240
241
Large data files transfer has been automatized in **do_filetransferssh.sh** that calls **fileTransferSSH.sh**.     
      
All the shell scripts are located on the WERA server under **/home/wera/bin/**. 
516e2d17   Marion Bandet   update WERA calib...
242

516e2d17   Marion Bandet   update WERA calib...
243

516e2d17   Marion Bandet   update WERA calib...
244
245


5fde9a30   Marion Bandet   update documentation
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270

COMPARISON OLD/NEW WERA CALIBRATIONS
**************************************

In March 2015, we used data collected in September 2013 to compare old and new calibrations in order to know which ones we should use for the on-going analysis of currents data.

.. figure:: figures/CompaNewcalib_ADCP_WERA_Sept2013_PAO.png
    :width: 500px
    :align: center
    :height: 250px
    
    Hourly radial currents from ADCP bin 2 and HFR WERA at PAO - September 2013

    Superimposed are the 48-h averages

.. figure:: figures/Compa_old-newcalib_ADCP_WERA_Sept2013_PAO.png
    :width: 500px
    :align: center
    :height: 250px

.. figure:: figures/Compa_old-newcalib_ADCP_WERA_Sept2013_PAO_corrcoef.png 
    :width: 500px
    :align: center
    :height: 250px

64977cd9   Marion Bandet   check Rx antennas...
271
272
273

:ref:`rst_checkWERAposition`

7e281bfb   Marion Bandet   update
274
275
276
277
278
279
280


.. note::  **Calibrations made in 2016**


New calibrations were made in Juy 2016 for the WERA sites.

e9cb9445   Marion Bandet   update calibratio...
281
On 2016-07-05  at the PAB site, calibrations were undertaken in the field walking and with a boat.
7e281bfb   Marion Bandet   update
282
283
284
285
286

Calibration data are stored in:
     - wera server: /home/wera/data2/PAB/2016187/cal/
     - /run/media/bandma01/SeagateWireless/MEOPAR/DATA/HFRs/WERAs/PAB/2016187 every minute from 20161870013 to 20161872011 

7e281bfb   Marion Bandet   update
287
288
289
290
291
292
293
294

 While the calibrations were running, tracks were recorded with a gps on the beach (WALK) and on a boat (BOAT). 

The tracks can be found in :
     - wera server: /home/wera/gps4cal/
     - /run/media/bandma01/SeagateWireless/MEOPAR/DATA/HFRs/WERAs/PAB/

Track_05-JUL-16 150101.gpx is the file for the WALK calibrations
e9cb9445   Marion Bandet   update calibratio...
295
296
Track_12-JUL-16 131906.gpx AT THE BEGGINNING  are the tracks for the boat calibrations from July 6, 2016

7e281bfb   Marion Bandet   update
297
298
299

We use the routine **GPS_gpx2mat.m** to convert .gpx files into .mat files.

e9cb9445   Marion Bandet   update calibratio...
300
301
302
303
304
305
306
307
308
309
310
311
The WALK calibrations were made between 13:53 and 14:58 UTC so I clipped the data so that only good data remained (+/- 10 seconds)
.
The BOAT calibrations were made between 18:20 and 19:18 (first cicle at 64 samples/run) and 19:20 and 20:11 (second circle at 32 samples/run).

The .mat files were saved under: 
		- Track_05-JUL-16 150101.mat for the walk calibrations
		- FirstCircle_BOAT_Track_05-JUL-16 for the boat calibrations (first circle)
		- SecondCircle_BOAT_Track_05-JUL-16 for the boat calibrations (second circle)


Use the routine **WERAphase_txt2mat.m** to convert the .txt files containing phases into .mat files.

6e208ef8   Marion Bandet   update
312
313
Then run the **WERAcalibrations_2016.m**. 
I added an I/Q balence check in he script to reject "bad" values (I/Q > 95 deg and I/Q < 85 deg). 
e9cb9445   Marion Bandet   update calibratio...
314

6e208ef8   Marion Bandet   update
315
Results of the new (july 2016) calibrations are here for PAB: `PABcalib2016`_.
e9cb9445   Marion Bandet   update calibratio...
316

6e208ef8   Marion Bandet   update
317
.. _PABcalib2016: calibPAB_2016.html 
e9cb9445   Marion Bandet   update calibratio...