WindsPT@U.PORTO

windscanner.eu & NEWA

User Tools

Site Tools


uda:index

This is an old revision of the document!


UPORTO DATA ARCHIVE (UDA)

Perdigão's datasets repository using the THREDDS Data Server (TDS)

- Using the UDA

The UDA can be accessed using the TDS at https://windsptds.fe.up.pt/thredds/catalog_perdigao.html

The UDA can be explored using the WindsP App at https://windsp.fe.up.pt/experiments/3/datasets/~2Fthredds~2Fcatalog_perdigao.xml

As a WindsP user you get credentials to the Web App.

When you request access to data that is in the Data Archive (outside WindsP and embargoed during 12 months) you must provide the TDS credentials: perdigao / Bl@yer! (the same as in the UCAR ftp site).

Each institution also owns credentials to their catalogue in our Data Server, to upload and maintain their data.

- Collecting DTU data

UPORTO uses the UDA export dtu@windsptds.fe.up.pt::dtu

ssh nejoco@login.neweuropeanwindatlas.eu

We are automatically syncing every 4 hours the DTU data directory using the cron command: rsync -az /newa/WP2/PERDIGAO/data/ dtu@windsptds.fe.up.pt::dtu/data/

–delete in the beginning

crons
# DTU data sync to UDA, "At minute 31 past every 4th hour."
#31 */4 * * * /usr/bin/rsync -az --delete /newa/WP2/PERDIGAO/ dtu@193.136.38.237::dtu > /dev/null 2>&1
31 */4 * * * /usr/bin/rsync -az /newa/WP2/PERDIGAO/ dtu@193.136.38.237::dtu > /dev/null 2>&1
#
# UDA archive to DTU, "At midnight every day."
0 0 * * * /home/nejoco/sync-uda.sh >| sync-uda_last.log 2>&1

- Collecting UCAR data

UCAR uses the UDA export ucar@windsptds.fe.up.pt::ucar

DLR

INEGI

WindsForS

- Mirror UDA to DTU

The mirror of UPORTO Data Archive (UDA) to the NEWA storage is now complete.

To manually update the archive, you may run the shell script (see the attached log of today's execution ): ~nejoco/sync-uda.sh

We are automatically syncing every 24 hours using the cron command: rsync -az /newa/WP2/PERDIGAO/data/ dtu@windsptds.fe.up.pt::dtu/data/

The DTU NEWA directory /newa/WP2/PERDIGAO/archive/ contains an exact copy of UDA, except for the DTU data that are links to existing NEWA directories (in order to void using more 2.9 TiB of storage).

/newa/WP2/PERDIGAO/archive
├── dlr
│   ├── HATPRO_level-1
│   ├── HATPRO_level-2
│   ├── HATPRO_surface-met
│   ├── mcs_data
│   ├── netcdf_lidar
│   └── raw_data
├── dtu
│   ├── DTU_Leica_Scanning -> /newa/WP2/PERDIGAO/data/DTU_Leica_Scanning
│   ├── DTU_Mast_Data -> /newa/WP2/PERDIGAO/data/DTU_Mast_Data
│   └── DTU_WindScanner -> /newa/WP2/PERDIGAO/data/DTU_WindScanner
├── inegi
│   ├── EnerconWindTurbine
│   ├── LeosphereWindcube
│   └── LidarAerialSurvey_RawData
├── ucar
│   ├── isfs
│   ├── iss
│   └── ncas
└── windfors
    ├── 2017
    └── cross

Correia Lopes 2017/11/17 11:10

uda/index.1511044279.txt.gz · Last modified: 2017/11/18 23:31 by Correia Lopes