diff --git a/build/doctrees/20140506.doctree b/build/doctrees/20140506.doctree index 65b3b536220da2e3a3ab6ab63844de5e472cc8ee..0ce4be5a9b0c894c4997edd42a75a178b0deef15 100644 Binary files a/build/doctrees/20140506.doctree and b/build/doctrees/20140506.doctree differ diff --git a/build/doctrees/CODAR_processing.doctree b/build/doctrees/CODAR_processing.doctree new file mode 100644 index 0000000000000000000000000000000000000000..5836ab8fa68e05030a80fbbe35ac808ab8fa67cd Binary files /dev/null and b/build/doctrees/CODAR_processing.doctree differ diff --git a/build/doctrees/CurrentsCombination.doctree b/build/doctrees/CurrentsCombination.doctree new file mode 100644 index 0000000000000000000000000000000000000000..43fb34af117bd9e6ef90c4f0b34bd837cd861456 Binary files /dev/null and b/build/doctrees/CurrentsCombination.doctree differ diff --git a/build/doctrees/OGSL.doctree b/build/doctrees/OGSL.doctree new file mode 100644 index 0000000000000000000000000000000000000000..f0fb8fa9b1ca9f41257a5ffb51eb9358e19674f5 Binary files /dev/null and b/build/doctrees/OGSL.doctree differ diff --git a/build/doctrees/WEAR_processing.doctree b/build/doctrees/WEAR_processing.doctree new file mode 100644 index 0000000000000000000000000000000000000000..4b5320d748c7d7c82d385c28704b7ac27e9f5b56 Binary files /dev/null and b/build/doctrees/WEAR_processing.doctree differ diff --git a/build/doctrees/WERA_processing.doctree b/build/doctrees/WERA_processing.doctree new file mode 100644 index 0000000000000000000000000000000000000000..559f5d48e664fd4dcdd3bee3d1e9aede13432ea5 Binary files /dev/null and b/build/doctrees/WERA_processing.doctree differ diff --git a/build/doctrees/environment.pickle b/build/doctrees/environment.pickle index 41b575b29a32f07db64c2ab16445217ee2d713ae..83e6d8ebef50762769af640dba11ab3000bf8631 100644 Binary files a/build/doctrees/environment.pickle and b/build/doctrees/environment.pickle differ diff --git a/build/doctrees/index.doctree b/build/doctrees/index.doctree index ac72d1908b1af17f80536a17489c3162c629efaa..6e49b6f32e91ed8d088faa38d38994ee479f4222 100644 Binary files a/build/doctrees/index.doctree and b/build/doctrees/index.doctree differ diff --git a/build/html/CODAR_processing.html b/build/html/CODAR_processing.html new file mode 100644 index 0000000000000000000000000000000000000000..0f5f1f6a2f83ec2ad574272340ac36e5adfb92da --- /dev/null +++ b/build/html/CODAR_processing.html @@ -0,0 +1,284 @@ + + + + + + + + CODAR post-processing — documentation_HFR_analysis 0.0.1 documentation + + + + + + + + + + + + + + + +
+
+
+
+ +

You have to loggin to the CODAR server at UQAR.

+

Then follow the directions:

+

== Check processed data == +Go to

+

‘’‘CodarData/Codar/SeaSonde/Data/Spectra/SpectraProcessed’‘’

+
    +
  • check that all files have been processed
  • +
  • If ok, remove all file (move to trash)
  • +
+

== RadialConfig == +Go to

+

‘’‘CodarData/Codar/SeaSonde/Configs/RadialConfigs’‘’

+
    +
  • Delete everything
  • +
+

Go to (for example)

+

‘’‘CodarData/Codar/SeaSonde/Configs/RadialConfigs_PAP1_20130201_BAPM_MUSIC402002_nosmooth’‘’

+
    +
  • Copy the content of the folder into
  • +
+

‘’‘CodarData/Codar/SeaSonde/Configs/RadialConfigs’‘’

+
+

CODAR post-processing

+

== Antenna diagrams for processing ==

+

Go to

+

‘’‘CodarData/Codar/SeaSonde/Data/Loops/ PATT_PAP1_2014_01_14_1154_NoSmooth.xlp’‘’

+

Copy MeasPattern.txt and Phases.txt

+

Paste in +‘’‘CodarData/Codar/SeaSonde/Configs/RadialConfigs’‘’

+

Open Header.txt and change line 19 if needed

+

save. (MUSIC parameters 40 20 2 or 80 40 1)

+

==’‘’ADD-ON Analysis on March 14, 2014’‘’==

+

Go to

+

‘’‘CodarData/Codar/SeaSonde/Configs/RadialConfigs’‘’

+

Run “Seasonde Radial Site Setup”.

+

In “Advanced”, uncheck “filtered and filled” and check “filtered” only.

+

Save.

+

Copy the content of the folder RadialConfigs back to the original folder, for example if what is in the “/RadialConfigs” folder comes from “/RadialConfigs_PAP1_20130201_BAPM_MUSIC402002_20Dsmooth”, after running the “Seasonde Radial Site Setup”, copy the content of “/RadialConfigs” back to “/RadialConfigs_PAP1_20130201_BAPM_MUSIC402002_20Dsmooth”.

+

== Spectra ==

+

Go get spectra on disks (CODAR1 or CODAR2) (on desktop)

+

Go to (for example)

+

‘’‘Codar2/PAP1/Archives/Spectra/’‘’

+

Copy the day(s) we are interested in

+

(Attention: copy the CSS_....cs4 files in PAP1_CSS_... folder!!) +and paste in

+

‘’‘CodarData/Codar/SeaSonde/Data/Spectra/SpectraToProcess’‘’

+

==> Can also store spectra in

+

CodarData/Codar/SeaSonde/Data/Spectra/SpectraSeries

+

Before copy/paste in SpectraToProcess

+

Once the spectra are in SpectraToProcess, the analysis starts automatically!

+

ATTENTION: make sure all the files are copied in “SpectraToProcess” sometimes copy-paste does miss files….

+

== Retrieve processed data ==

+

Go to

+

‘’‘CodarData/Codar/SeaSonde/Data/Radials/’‘’

+

Processed data are in “Filtered”, “IdealPattern” and “MeasPattern” folders.

+

Save data in appropriate folders (to be created if needs be)

+

MUSIC 40-20-2 +*no smoothing +*measured no filtered +*ideal no filtered +*measured filtered +*ideal filtered

+

MUSIC 80-40-1 +*no smoothing +*measured no filtered +*ideal no filtered +*measured filtered +*ideal filtered

+

MUSIC 40-20-2 +*20D smoothing +*measured no filtered +*measured filtered

+

MUSIC 80-40-1 +*20D smoothing +*measured no filtered +*measured filtered

+

== Transfer data to local disk ==

+

== Transform .ruv to .mat ==

+

Using radial2uv.m

+

Be careful to the flag f.

+

*f=0 for not filtered data; +*f=1 for filtered data

+

== CHECK_UP ==

+

{| +| +| +| +| +|PAP +|STF +|- +|1. measured +|MUSIC 40-20-2 +|no smoothing +|no filtered +|ok +|ok +|- +|2. ideal +|MUSIC 40-20-2 +|no smoothing +|no filtered +|ok +|ok +|- +|3. measured +|MUSIC 40-20-2 +|no smoothing +|filtered +|ok +|ok +|- +|4. ideal +|MUSIC 40-20-2 +|no smoothing +|filtered +|ok +|ok +|- +|5. measured +|MUSIC 80-40-1 +|no smoothing +|no filtered +|ok +|ok +|- +|6. ideal +|MUSIC 80-40-1 +|no smoothing +|no filtered +|ok +|ok +|- +|7. measured +|MUSIC 80-40-1 +|no smoothing +|filtered +|ok +|ok +|- +|8. ideal +|MUSIC 80-40-1 +|no smoothing +|filtered +|ok +|ok +|- +|9. measured +|MUSIC 40-20-2 +|20D smoothing +|no filtered +|ok +|ok +|- +|10. measured +|MUSIC 40-20-2 +|20D smoothing +|filtered +|ok +|ok +|- +|11. measured +|MUSIC 80-40-1 +|20D smoothing +|no filtered +|ok +|ok +|- +|12. measured +|MUSIC 80-40-1 +|20D smoothing +|filtered +|ok +|ok +|}

+

Modified by Marion Bandet on 2015/07/14

+
+ + +
+
+
+
+
+

Previous topic

+

Welcome to the documentation of High Frequency Radars (CODAR and WERA)

+

Next topic

+

WERA POST=PROCESSING

+

This Page

+ + + +
+
+
+
+ + + + \ No newline at end of file diff --git a/build/html/CurrentsCombination.html b/build/html/CurrentsCombination.html new file mode 100644 index 0000000000000000000000000000000000000000..d1d35d8aff35555288476053e183e0b454735442 --- /dev/null +++ b/build/html/CurrentsCombination.html @@ -0,0 +1,137 @@ + + + + + + + + Current combination — documentation_HFR_analysis 0.0.1 documentation + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Current combination

+

Radial currents from WERA and CODAR systems are combined together on a grid to provide total currents approximately every hour. +The processing is conducted on the WERA server (ssh wera@132.215.11.14) and the resulting hourly currents are exported to .csv files to be displayed on he OGSL website (http://ogsl.ca/observations/).

+

All the matlab scripts are located in ~/projets/hfr_data_processing/hfr_processing.

+

A crontab runs the analysis (crontab).

+

The processing runs as follows:

+

‘’‘1- download the last available data at WERA sites’‘’ (radial currents - ascii format)

+

For PAO, data is available at HH+00 and HH+30

+

For PAB, data is available at HH+15 et HH+45

+

Every 10 min the crontab tries to download data via a shell script:

+

5,15,25,35,45,50 * * * * /home/wera/projets/hfr-data-processing/hfr_process/download_wera_rt.sh

+

The data is located in: /home/wera/codar_wera_ogsl/archives/data

+

‘’‘2- download the last available data at CODAR sites’‘’ (.ruv)

+

For PAP and STF, data is available at HH+00

+

Every 5 minutes, the crontab tries to download data via a shell script:

+

0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/wera/projets/hfr-data-processing/hfr_process/download_codar_rt.sh

+

The data is located in: /home/wera/codar_wera_ogsl/archives/data

+

‘’‘3-‘’’ at H+55min the crontab runs to ‘’‘combine currents for the last hour’‘’ via a shell script:

+

55 * * * * /home/wera/projets/hfr-data-processing/hfr_process/ogsl_hfr_treatment.sh

+

What does the shell do?

+

1- copy the .ruv and ascii files located in /home/wera/codar_wera_ogsl/archives/data in the working directory /home/wera/codar_wera_ogsl/travail

+

2- unzip .gz files

+

3- convert .ruv to .mat format (ruv2mat)

+

4- convert ascii to .mat format (ascii2mat)

+

5- generate (combine) total currents (rad2tot)

+

At this stage a .mat file is generated (i.e. HFR_LSLE_20150817T170000Z.mat) for each hour.

+

6- the routine currentsHfrWithAverageExport.m computes the 25-hour current average and exports the hourly currents into a .csv file to be displayed on the OGSL website.

+

7- the .csv files are moved to the webserver (/srv/www/htdocs/wera/hfrdata) and the .mat are moved in archives (/home/wera/codar_wera_ogsl/archives/matfiles and /srv/www/htdocs/wera/hfrmat) and logs are kept in /home/wera/codar_wera_ogsl/logs (download_codar_rt.log, download_wera_rt.log, ogsl_hfr_treatment.log, reprocess_ogsl_hfr_treatment.log)

+
+ + +
+
+
+
+
+

Previous topic

+

WERA POST=PROCESSING

+

Next topic

+

OGSL quasi-realtime data feed

+

This Page

+ + + +
+
+
+
+ + + + \ No newline at end of file diff --git a/build/html/OGSL.html b/build/html/OGSL.html new file mode 100644 index 0000000000000000000000000000000000000000..d7e5cc751068bc3dc814c80b98305660b87ca7eb --- /dev/null +++ b/build/html/OGSL.html @@ -0,0 +1,110 @@ + + + + + + + + OGSL quasi-realtime data feed — documentation_HFR_analysis 0.0.1 documentation + + + + + + + + + + + + + + + +
+
+
+
+ +
+

OGSL quasi-realtime data feed

+
+ + +
+
+
+
+
+

Previous topic

+

Current combination

+

Next topic

+

Experiment on May 15, 2013

+

This Page

+ + + +
+
+
+
+ + + + \ No newline at end of file diff --git a/build/html/WEAR_processing.html b/build/html/WEAR_processing.html new file mode 100644 index 0000000000000000000000000000000000000000..64769c1a940220305089ab7abb901bf67fbfeb0e --- /dev/null +++ b/build/html/WEAR_processing.html @@ -0,0 +1,87 @@ + + + + + + + + <no title> — documentation_HFR_analysis 0.0.1 documentation + + + + + + + + + + + + + +
+
+
+
+ + + +
+
+
+
+
+

This Page

+ + + +
+
+
+
+ + + + \ No newline at end of file diff --git a/build/html/WERA_processing.html b/build/html/WERA_processing.html new file mode 100644 index 0000000000000000000000000000000000000000..64521a3c3ec1cf55da7643de9149bf9a823aac20 --- /dev/null +++ b/build/html/WERA_processing.html @@ -0,0 +1,118 @@ + + + + + + + + WERA POST=PROCESSING — documentation_HFR_analysis 0.0.1 documentation + + + + + + + + + + + + + + + +
+
+
+
+ +
+

WERA POST=PROCESSING

+

Post-processing of WERA radial currents is much simpler than for the CODAR system.

+

The WERA system outputs data in folders identified by the year and Julian day (for example 2014101 for April 11, 2014).

+

In each folder we find .RFI and .SORT files for every hour (should find 24 files of each extention per day).

+

Use the shell script do_process_SORT_pol to generate .crad_pol_lst files. They will be found in the subfolder YYYYJJJ/results/.

+

**** Use radiallst2uv.m to convert lst 2 mat files (To be verified).****

+

To download radial currents from the sites:

+

rsync -v -e ssh wera@wera-lsle-pao.dyndns.org:/home/wera/data/ftp_transfer/${DATE}${HEURE}*_ascii* .

+

with i.e DATE=2015121 (yyyyJJJ) and HEURE =0100

+
+ + +
+
+
+
+
+

Previous topic

+

CODAR post-processing

+

Next topic

+

Current combination

+

This Page

+ + + +
+
+
+
+ + + + \ No newline at end of file diff --git a/build/html/_sources/CODAR_processing.txt b/build/html/_sources/CODAR_processing.txt new file mode 100644 index 0000000000000000000000000000000000000000..7160edafa663693e8fffa9d8b20a5822c2355856 --- /dev/null +++ b/build/html/_sources/CODAR_processing.txt @@ -0,0 +1,247 @@ +You have to loggin to the CODAR server at UQAR. + +Then follow the directions: + + +== Check processed data == +Go to + +'''CodarData/Codar/SeaSonde/Data/Spectra/SpectraProcessed''' + +* check that all files have been processed +* If ok, remove all file (move to trash) + + + +== RadialConfig == +Go to + +'''CodarData/Codar/SeaSonde/Configs/RadialConfigs''' + +* Delete everything + +Go to (for example) + +'''CodarData/Codar/SeaSonde/Configs/RadialConfigs_PAP1_20130201_BAPM_MUSIC402002_nosmooth''' + +* Copy the content of the folder into + +'''CodarData/Codar/SeaSonde/Configs/RadialConfigs''' + +CODAR post-processing +===================== + +== Antenna diagrams for processing == + +Go to + +'''CodarData/Codar/SeaSonde/Data/Loops/ PATT_PAP1_2014_01_14_1154_NoSmooth.xlp''' + +Copy MeasPattern.txt and Phases.txt + +Paste in +'''CodarData/Codar/SeaSonde/Configs/RadialConfigs''' + +Open Header.txt and change line 19 if needed + +save. (MUSIC parameters 40 20 2 or 80 40 1) + + +=='''ADD-ON Analysis on March 14, 2014'''== + +Go to + +'''CodarData/Codar/SeaSonde/Configs/RadialConfigs''' + +Run “Seasonde Radial Site Setup”. + +In “Advanced”, uncheck “filtered and filled” and check “filtered” only. + +Save. + +Copy the content of the folder RadialConfigs back to the original folder, for example if what is in the “/RadialConfigs” folder comes from “/RadialConfigs_PAP1_20130201_BAPM_MUSIC402002_20Dsmooth”, after running the “Seasonde Radial Site Setup”, copy the content of “/RadialConfigs” back to “/RadialConfigs_PAP1_20130201_BAPM_MUSIC402002_20Dsmooth”. + + + + + +== Spectra == + +Go get spectra on disks (CODAR1 or CODAR2) (on desktop) + +Go to (for example) + +'''Codar2/PAP1/Archives/Spectra/''' + +Copy the day(s) we are interested in + +(Attention: copy the CSS_....cs4 files in PAP1_CSS_... folder!!) +and paste in + +'''CodarData/Codar/SeaSonde/Data/Spectra/SpectraToProcess''' + +==> Can also store spectra in + +CodarData/Codar/SeaSonde/Data/Spectra/SpectraSeries + +Before copy/paste in SpectraToProcess + +Once the spectra are in SpectraToProcess, the analysis starts automatically! + +ATTENTION: make sure all the files are copied in “SpectraToProcess” sometimes copy-paste does miss files…. + + +== Retrieve processed data == + +Go to + +'''CodarData/Codar/SeaSonde/Data/Radials/''' + +Processed data are in “Filtered”, “IdealPattern” and “MeasPattern” folders. + +Save data in appropriate folders (to be created if needs be) + + +MUSIC 40-20-2 +*no smoothing +*measured no filtered +*ideal no filtered +*measured filtered +*ideal filtered + + +MUSIC 80-40-1 +*no smoothing +*measured no filtered +*ideal no filtered +*measured filtered +*ideal filtered + + +MUSIC 40-20-2 +*20D smoothing +*measured no filtered +*measured filtered + + +MUSIC 80-40-1 +*20D smoothing +*measured no filtered +*measured filtered + + + +== Transfer data to local disk == + + + +== Transform .ruv to .mat == + +Using radial2uv.m + +Be careful to the flag f. + +*f=0 for not filtered data; +*f=1 for filtered data + + + + +== CHECK_UP == + +{| +| +| +| +| +|PAP +|STF +|- +|1. measured +|MUSIC 40-20-2 +|no smoothing +|no filtered +|ok +|ok +|- +|2. ideal +|MUSIC 40-20-2 +|no smoothing +|no filtered +|ok +|ok +|- +|3. measured +|MUSIC 40-20-2 +|no smoothing +|filtered +|ok +|ok +|- +|4. ideal +|MUSIC 40-20-2 +|no smoothing +|filtered +|ok +|ok +|- +|5. measured +|MUSIC 80-40-1 +|no smoothing +|no filtered +|ok +|ok +|- +|6. ideal +|MUSIC 80-40-1 +|no smoothing +|no filtered +|ok +|ok +|- +|7. measured +|MUSIC 80-40-1 +|no smoothing +|filtered +|ok +|ok +|- +|8. ideal +|MUSIC 80-40-1 +|no smoothing +|filtered +|ok +|ok +|- +|9. measured +|MUSIC 40-20-2 +|20D smoothing +|no filtered +|ok +|ok +|- +|10. measured +|MUSIC 40-20-2 +|20D smoothing +|filtered +|ok +|ok +|- +|11. measured +|MUSIC 80-40-1 +|20D smoothing +|no filtered +|ok +|ok +|- +|12. measured +|MUSIC 80-40-1 +|20D smoothing +|filtered +|ok +|ok +|} + + +Modified by Marion Bandet on 2015/07/14 + diff --git a/build/html/_sources/CurrentsCombination.txt b/build/html/_sources/CurrentsCombination.txt new file mode 100644 index 0000000000000000000000000000000000000000..70b5bb7b60b8620d14d440b363f7dd7b06f7c112 --- /dev/null +++ b/build/html/_sources/CurrentsCombination.txt @@ -0,0 +1,58 @@ +Current combination +=================== + +Radial currents from WERA and CODAR systems are combined together on a grid to provide total currents approximately every hour. +The processing is conducted on the WERA server (ssh wera@132.215.11.14) and the resulting hourly currents are exported to .csv files to be displayed on he OGSL website (http://ogsl.ca/observations/). + +All the matlab scripts are located in ~/projets/hfr_data_processing/hfr_processing. + +A crontab runs the analysis (crontab). + +The processing runs as follows: + +'''1- download the last available data at WERA sites''' (radial currents - ascii format) + +For PAO, data is available at HH+00 and HH+30 + +For PAB, data is available at HH+15 et HH+45 + +Every 10 min the crontab tries to download data via a shell script: + +5,15,25,35,45,50 * * * * /home/wera/projets/hfr-data-processing/hfr_process/download_wera_rt.sh + +The data is located in: /home/wera/codar_wera_ogsl/archives/data + + +'''2- download the last available data at CODAR sites''' (.ruv) + +For PAP and STF, data is available at HH+00 + +Every 5 minutes, the crontab tries to download data via a shell script: + +0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/wera/projets/hfr-data-processing/hfr_process/download_codar_rt.sh + +The data is located in: /home/wera/codar_wera_ogsl/archives/data + + + +'''3-''' at H+55min the crontab runs to '''combine currents for the last hour''' via a shell script: + +55 * * * * /home/wera/projets/hfr-data-processing/hfr_process/ogsl_hfr_treatment.sh + +What does the shell do? + +1- copy the .ruv and ascii files located in /home/wera/codar_wera_ogsl/archives/data in the working directory /home/wera/codar_wera_ogsl/travail + +2- unzip .gz files + +3- convert .ruv to .mat format (ruv2mat) + +4- convert ascii to .mat format (ascii2mat) + +5- generate (combine) total currents (rad2tot) + +At this stage a .mat file is generated (i.e. HFR_LSLE_20150817T170000Z.mat) for each hour. + +6- the routine currentsHfrWithAverageExport.m computes the 25-hour current average and exports the hourly currents into a .csv file to be displayed on the OGSL website. + +7- the .csv files are moved to the webserver (/srv/www/htdocs/wera/hfrdata) and the .mat are moved in archives (/home/wera/codar_wera_ogsl/archives/matfiles and /srv/www/htdocs/wera/hfrmat) and logs are kept in /home/wera/codar_wera_ogsl/logs (download_codar_rt.log, download_wera_rt.log, ogsl_hfr_treatment.log, reprocess_ogsl_hfr_treatment.log) diff --git a/build/html/_sources/OGSL.txt b/build/html/_sources/OGSL.txt new file mode 100644 index 0000000000000000000000000000000000000000..9f9c802523a47985224d8c9c0c4b976b552a4092 --- /dev/null +++ b/build/html/_sources/OGSL.txt @@ -0,0 +1,3 @@ +OGSL quasi-realtime data feed +============================= + diff --git a/build/html/_sources/WEAR_processing.txt b/build/html/_sources/WEAR_processing.txt new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/build/html/_sources/WERA_processing.txt b/build/html/_sources/WERA_processing.txt new file mode 100644 index 0000000000000000000000000000000000000000..0833817049785fb2585d0b1357709fcb119cbc6a --- /dev/null +++ b/build/html/_sources/WERA_processing.txt @@ -0,0 +1,20 @@ +WERA POST=PROCESSING +==================== + +Post-processing of WERA radial currents is much simpler than for the CODAR system. + +The WERA system outputs data in folders identified by the year and Julian day (for example 2014101 for April 11, 2014). + +In each folder we find .RFI and .SORT files for every hour (should find 24 files of each extention per day). + +Use the shell script do_process_SORT_pol to generate .crad_pol_lst files. They will be found in the subfolder YYYYJJJ/results/. + +**** Use radiallst2uv.m to convert lst 2 mat files (To be verified).**** + + +To download radial currents from the sites: + +rsync -v -e ssh wera@wera-lsle-pao.dyndns.org:/home/wera/data/ftp_transfer/${DATE}${HEURE}*_ascii* . + +with i.e DATE=2015121 (yyyyJJJ) and HEURE =0100 + diff --git a/build/html/index.html b/build/html/index.html index 42e4b655123ac2066337bb16910c993c453e2138..1c3958949b339be82a28e74c01c38bfe0d982452 100644 --- a/build/html/index.html +++ b/build/html/index.html @@ -24,7 +24,7 @@ - +