This is an old revision of the document!
Perdigão's datasets repository using the THREDDS Data Server (TDS)
The UDA can be accessed using the TDS at https://windsptds.fe.up.pt/thredds/catalog_perdigao.html
The UDA can be explored using the WindsP App at https://windsp.fe.up.pt/experiments/3/datasets/~2Fthredds~2Fcatalog_perdigao.xml
As a WindsP user you get credentials to the Web App.
When you request access to data that is in the Data Archive (outside WindsP and embargoed during 12 months) you must provide the TDS credentials: perdigao / Bl@yer! (the same as in the UCAR ftp site).
Each institution also owns credentials to their catalogue in our Data Server, to upload and maintain their data.
UPORTO uses the UDA export dtu@windsptds.fe.up.pt::dtu
to sync data collected by DTU.
ssh nejoco@login.neweuropeanwindatlas.eu
First a complete mirror was in place, by automatically syncing every 4 hours the DTU data directory using the cron command:
rsync -az –delete /newa/WP2/PERDIGAO/ dtu@windsptds.fe.up.pt::dtu
Later the –delete
option was removed and some diretories excluded to achieve the Perdigão Data Archive at UDA.
cat ~nejoco/sync-excluded-list archive/ data/DLT_WindScanner/ >/code> <code bash crons> # DTU data sync to UDA, "At minute 31 past every 4th hour." 31 */4 * * * /usr/bin/rsync -az --exclude-from '~nejoco/sync-exclude-list' /newa/WP2/PERDIGAO/ dtu@windsptds.fe.up.pt::dtu > /dev/null 2>&1 # # UDA archive to DTU, "At midnight every day." 0 0 * * * /home/nejoco/sync-uda.sh >| sync-uda_last.log 2>&1
UCAR uses the UDA export ucar@windsptds.fe.up.pt::ucar
DLR
INEGI
WindsForS
The mirror of UPORTO Data Archive (UDA) to the NEWA storage is now complete.
To manually update the archive, you may run the shell script (see the attached log of today's execution ): ~nejoco/sync-uda.sh
We are automatically syncing every 24 hours using the cron command:
rsync -az /newa/WP2/PERDIGAO/data/ dtu@windsptds.fe.up.pt::dtu/data/
The DTU NEWA directory /newa/WP2/PERDIGAO/archive/ contains an exact copy of UDA, except for the DTU data that are links to existing NEWA directories (in order to void using more 2.9 TiB of storage).
/newa/WP2/PERDIGAO/archive ├── dlr │ ├── HATPRO_level-1 │ ├── HATPRO_level-2 │ ├── HATPRO_surface-met │ ├── mcs_data │ ├── netcdf_lidar │ └── raw_data ├── dtu │ ├── DTU_Leica_Scanning -> /newa/WP2/PERDIGAO/data/DTU_Leica_Scanning │ ├── DTU_Mast_Data -> /newa/WP2/PERDIGAO/data/DTU_Mast_Data │ └── DTU_WindScanner -> /newa/WP2/PERDIGAO/data/DTU_WindScanner ├── inegi │ ├── EnerconWindTurbine │ ├── LeosphereWindcube │ └── LidarAerialSurvey_RawData ├── ucar │ ├── isfs │ ├── iss │ └── ncas └── windfors ├── 2017 └── cross
Data | DTU | UPORTO | UCAR |
---|---|---|---|
ALR | ✘ | ✘ | ✔ |
CU | ✘ | ✘ | ✘ |
DLR | ✔ | ✔ | ✘ |
DTU | ✔ | ✔ | ✘ |
ENERCON | ✔ | ✔ | ✘ |
IPMA | ✘ | ✘ | ✘ |
ISFS | ✔ | ✔ | ✔ |
ISS | ✔ | ✔ | ✔ |
Leosphere | ✔ | ✔ | ✔ |
NCAS | ✔ | ✔ | ✔ |
ND | ✘ | ✘ | ✔ |
OU | ✘ | ✘ | ✘ |
WindForS | ✔ | ✔ | ✘ |
— Correia Lopes 2017/11/17 11:10