dailylog.txt 10.7 KB
Log Work Marion 
****************


**19 OCT 2015:**

- IBM data management request (IBMassessment.odf)
- regenerate new calibration_pao_29OCT2014.wera located in /run/media/bandma01/SeagateWireless/MEOPAR/DATA/HFRs/WERAs/PAO
- regenerate new calibration_pab_30OCT2014.wera located in /run/media/bandma01/SeagateWireless/MEOPAR/DATA/HFRs/WERAs/PAB
- transfer files via ssh to WERA srv:
   * rsync -v -e ssh /run/media/bandma01/SeagateWireless/MEOPAR/DATA/HFRs/WERAs/PAB/calibration_pab_30OCT2014.wera  wera@132.215.11.14:~/etc/
   * rsync -v -e ssh /run/media/bandma01/SeagateWireless/MEOPAR/DATA/HFRs/WERAs/PAO/calibration_pao_29OCT2014.wera  wera@132.215.11.14:~/etc/


**20 OCT 2015:**

reprocess calibrations with new, corrected phases:

- log onto wera server
- cd data1/PAO/
- create a directory 2013244_bkp to run test
- rsync -aud 2013244/ 2013244_bkp/
- cd 2013244
- what to do?     
   * cd /home/wera/bin
   * vi do_postprocess_SORT_newCalib.sh
		=> add lines 89 nd 90 rm .SORT, rm .RFI
   * wera_new_calibration.sh 
		=> comment lines 28-32 (mv .SORT to SORT0 and .RFI to .RFI0)
- try on 2013244: > bash -x do_postprocess_SORT_newCalib -s PAO -y 2013244
- to process more than one day at a time, use:> bash -x Process_TimePeriod_wera.sh -s PAO -d 2013245 -f 2013273
- files transfer (PAO) using shell script in /home/wera/bin: do_filetransferssh.sh
 

**21 OCT 2015:**

- After the new .SORT and .RFI have been generated, we will generate new .crad_pol_lst files with:
    do_process_SORT_pol 2013244 2013273 /home/wera/data1/PAO

- work on a shell to find out if we have data in the "raw" directories 
    * ./checkDataSize.sh 2012347 2012365
    * ./checkDataSize.sh 2013001 2013365
    * ./checkDataSize.sh 2014001 2014365

 all logged in ~/data1/PAO/RAWfileSize.log


**27 OCT 2015:**

- write script to plot raw data size vs time to assess if there is data at a certain time period         (/home/bandma01/Dropbox/UQAR/MEOPAR_Marion/Matlab_Analysis/dataConversion/DataPresence_log2mat.m)
- run same script on wera server for PAB (same dates)
- modifier Header.txt dans RadialConfigs des CODARs (PAP1) pour mettre CurrMax a 100.
   * /Volumes/CodarData/Codar/SeaSonde/Configs/RadialConfigs_PAP1_20130201_BAPM_MUSIC402002_nosmooth
- reprocesser CODAR PAP1 pour 10 mai 2013 with:
   * cd hfr-data-processing/postprocessing/codar/
   * ./lancerTraitementSpectraCodar.sh -n ModifCurrMax -s PAP1 -a 2013 -m 05 -j 10


**28 OCT 2015:**

1- reprocess CODAR PAP1 for May 11, 2013

- ./lancerTraitementSpectraCodar.sh -n ModifCurrMax -s PAP1 -a 2013 -m 05 -j 11
- modifier Header.txt dans RadialConfigs des CODARs (STF2) pour mettre CurrMax a 100.
   * /Volumes/CodarData/Codar/SeaSonde/Configs/RadialConfigs_STF2_20130206_BAPM_MUSIC402002_nosmooth
- reprocesser CODAR STF2 pour 10 mai 2013
   * ./lancerTraitementSpectraCodar.sh -n ModifCurrMax -s STF2 -a 2013 -m 05 -j 10
   * ./lancerTraitementSpectraCodar.sh -n ModifCurrMax -s STF2 -a 2013 -m 05 -j 11

2- reprocesser WERA PAB pour september 2013 avec nouvelles phases
 => no data at this time

3- WERA PAO: convert lst files to .mat for new calibrations using radiallst2uv.m
- use RadialVeloCompa_Sept2013_HFR_ADCP_WERA_ADCPcorrected_v2.m to generate new .mat data and use PlotRadialVeloCompa_Sept2013_HFR_ADCP_WERA_v2.m to generate new plots.


**29 OCT 2015:**

- work on figures of WERA PAO generated with new phases.
- realise there is a problem with new calibration output files: they are the same as with the old calibrations...
- investigate on wera server if possible mistake in shell scripts
- transfer with ssh of old calibrations results
- work on MEOPAR HQP section for NCE renewal application


**2 NOV 2015:**

- work on pb with new phases PAO: check shell scripts on WERA serveur
- try with 1 day (2013244): seems ok
- process by hand each day: ./do_postprocess_SORT_newCalib.sh -s PAO -y 2013273
 

**3 NOV 2015:**

- finish analysis by hand: take .SORT and output .crad_pol_lst
   => do_process_SORT_pol 2013244 2013273 /home/wera/data1/PAO


**5 NOV 2015:**

- ftp transfer of .crad_pol_lst to local hard Drive
   => ./do_filetransferssh.sh 2013245 2013273
rsync -v -e ssh wera@132.215.11.14:~/data1/PAO/${JOUR}/results/new/*.crad_pol_lst /run/media/bandma01/SeagateWireless/MEOPAR/DATA/HFRs/WERAs/PAO/optimisationPAO/version5_PAO_old_nofullcal/checkNewCalib/

- use radiallst2uv.m


**the new phases are so close to the old ones that there isn't any real differences in the radial currents** 


**16 Nov 2015:**

- work on HFR WERA direction


**17 Nov 2015:**

- generate PlotRadialVeloCompa_Sept2013_HFR_ADCP_WERA_v2.m for old and new calibrations (PAO)


**23 Nov 2015:**

- in the field AM to sample DOC and mud for Alex Rao
- discussion with Cedric about HFR orientation. From PAO spet 2013 results, it seems we need to limit the angles of HFR between 143-233 deg.
- find a time when we have ADCP data, POA and PAB working at the same time.  


**24 Nov 2015:**

- move data from 
/run/media/bandma01/SeagateWireless/MEOPAR/DATA/HFRs/WERAs/PAO/optimisationPAO/version5_PAO_old_nofullcal/NewCalibrations/

to 

/run/media/bandma01/SeagateWireless/MEOPAR/DATA/HFRs/WERAs/PAO/September2013_optimV5_Newcalib

- found time period when we have PAO, PAB and ADCP working in 2013: day 129 to 171 (may 9 to June 20, 2013.

- on wera server
	* cd bin
	* comment lines 89 and 90 in do_postprocess_SORT_newCalib.sh
        * uncomment lines 28-32 in wera_new_calibration.sh to archive .SORT in .SORT0
	* run bash -x do_postprocess_SORT_newCalib.sh -s PAO -y 2013129
		+ check if ok
		+ if ok do a batch processing of the remaining days to process
	* batch processing:  bash -x Process_TimePeriod_wera.sh -s PAO -d 2013130 -f 2013171
	* After the new .SORT and .RFI have been generated, we generate new .crad_pol_lst files with: do_process_SORT_pol 2013129 2013171 /home/wera/data1/PAO (takes a few hours, do a "screen")

- do the same steps for PAB on wera server
	* cd bin
	* run bash -x do_postprocess_SORT_newCalib.sh -s PAB -y 2013129
	* batch processing:  bash -x Process_TimePeriod_wera.sh -s PAB -d 2013130 -f 2013171
	* After the new .SORT and .RFI have been generated, we generate new .crad_pol_lst files with: do_process_SORT_pol 2013129 2013171 /home/wera/data2/PAB (takes a few hours, do a "screen")

- on local machine, transfer data from wera server: ./do_filetransferssh.sh PAO 2013129 2013171


**25 Nov 2015:**

- on local machine, transfer data from wera server: ./do_filetransferssh.sh PAB 2013129 2013171
- use radiallst2uv.m to convert ascii to mat files (PAO and PAB)
- make RadialVeloCompa_MayJune2013_HFR_ADCP_WERA_ADCPcorrected_v2.m and run for PAO
- make Plot_WERA_direction_MayJune2013_ADCP.m and run for PAO
- make PlotRadialVeloCompa_MayJune2013_HFR_ADCP_WERA_v2.m and run for PAO

**there is a mistake in the clipping angles of PAB...**

**30 Nov 2015:**

- mistake found in radiallst2mat.m (was not updated)
- process radiallst2uv.m to convert ascii to mat files PAB
- RadialVeloCompa_MayJune2013_HFR_ADCP_WERA_ADCPcorrected_v2.m and run for PAB
- PlotRadialVeloCompa_MayJune2013_HFR_ADCP_WERA_v2.m and run for PAB
- Plot_WERA_direction_MayJune2013_ADCP.m and run for PAB

**results are ugly!!**

**1 Dec 2015 - 2 Dec 2015**
Work from home MC sick

**week of Dec 7**
rush in acoutics

**14 Dec 2015**
- look at data from IML4 buoy (ADCP)
- modify script load_buoy_iml4.m to add data from 2014
- save data as .mat files
- look into outting data as archive in Mingan (see James for account on Mingan)
- talked to CC about paper

**15 Dec 2015**
- gather thoughts + data for paper
- created a "dropbox paper" to work with CC

**16 Dec 2015**
- started paper brainstorming on Dropbox paper
- look at data from May 6, 2014 experiments with drifting buoys

**20 Dec 2015**
- continued work on data analysis

**6 Jan 2016**

**7 Jan 2016**

**8 Jan 2016**
- work on sending Mr. Maltais a set of HFR .cvs file

**11 Jan 2016**
No school

**14 Jan 2016**
- administrative work
- flight to MEOPeers training Workshop in Banff, AB
- data analysis of May 6 experiment

**15 Jan 2016**
-book flight for MEOPeers training Workshop in Banff, AB
- meet with CC to discuss paper

**20 jan 2016**
- work on paper

**21 jan 2016**
- work on paper: trajectory of drifters at same depth. Generated figures.

**22 jan 2016**
- work on paper: trajectory of drifters at same depth. work on script generated radial velocity
- need to re-process HFR data with new calibration and new antenna's positions
- on wera server: cd bin
- run do_postprocess_SORT_newCalib.sh -s PAO -y 2014126
		+ check if ok
		+ if ok do a batch processing of the remaining days to process


**25 jan 2016**
- on wera server:
- batch processing: Process_TimePeriod_wera.sh -s PAO -d 2014127 -f 2014130
- After the new .SORT and .RFI have been generated, we generate new .crad_pol_lst files with: do_process_SORT_pol 2014126 2014130 /home/wera/data1/PAO (takes a few hours, do a "screen")
- same for PAB: Process_TimePeriod_wera.sh -s PAB -d 2014126 -f 2014130
- do_process_SORT_pol 2014126 2014130 /home/wera/data2/PAB


**1er fevrier 2016**
- transfer data on local machine from wera server
./do_filetransferssh.sh PAO 2014126 2014130
./do_filetransferssh.sh PAB 2014126 2014130

- departure for Calgary/Banff MEOPeer training meeting Feb 1-5, 2016

**10-11-12 feb, 2016**
- work on data processing and comparisons, correlation, RMS diff with new procesing in hope it will improve the data...

**17 feb, 2016**
- wrap up of last data analyis 
- getting ready for field work

**18-19 feb, 2016**
- field work parc du Bic - BinWin 2016

**24-25-26 feb, 2016**


**2-3 Mar, 2016**
- spring break

**4 Mar, 2016**

**9-10-11 Mar 2016**
- Download data from wera server to create movie of combined currents
- work on script
- make movie: CombinedCurrents_20150421-20160203fig.avi

**16 Mar, 2016**
- modifiy script to make a monthly movie of combined currents

**17 Mar 2016**
- implemente script, generate movies for each month between April 2015 and January 2016
- update repository Matlab_scripts on gitlasso
- download .csv files from wera server
- make package for Louis Maltais with hourly data for Dec1-2 2015 (csv files + movie)

**18 Mar 2016**
- send package to Louis Maltais
- write paper of HF radars and buoy comparison

**29 Mar 2016**
- MEOPAR report for Dany
- work on paper
- comparison SPOT and GPS data with Paul (year 2014) for archive on Mingan

**30 Mar 2016**
- work on paper
- download tide data for may 2014
- download wind data for May 5, 6, 7, 8 2014


**6 & 7 jun 2016**
- work on paper: estimate coverage area of radial velocities for each site: HFR_coverage_May2014.m
- write bash script to transfer data from CODAR server to external HD: fileTransferCODAR.sh
- transfer data from PAP for May 1-10, 2014
- transfer data from STF fro May 1-10, 2014
- apply matlab routine make_ruv2matMBtest.m to convert binary data to .mat
- run HFR_coverage_May2014.m for PAO, PAB, PAP, STF