The ICDC Technical Concept
Technically, ICDC has two main components:
- An internal data server accessible from all KlimaCampus where climate data sets are being stored, made easily accessible through common formats, and continuously updated. Climate data are also being visualized and displayed in public through the CLISAP VIS expert.
- A web site and a data portal to the inside and outside world (e.g., ftp, THREDDS, OPeNDAP). The web site serves meta data to the members of the KlimaCampus and to the outside world. In collaboration with the CLISAP outreach, it provides information about climate variability such as climate indices. Web applications like THREDDS and Live Access Server provides easy data access.
1) Internal Data Server Concept
ICDC provides direct access via the file system on an internal data server accessible from all CEN/MPI Unix Workstations and Servers via /data/icdc. The data sets are stored on a storage connected to the server in an easily accessible file directory tree. In order to simplify the usage, data sets are provided through common formats like netCDF or ASCII. Meta data information to every dataset can be found on our content base, the ICDC website. Automatic scripts can provide regular data updates, weekly data backups save the collected data. There is a goal to integrate data sets from other sources of the KlimaCampus network.
2) Web Server Concept
The content base of our data center is the Content Management System TYPO3. ICDC provides data access via a FTP server and the THREDDS data server. The Live Access Server (LAS) is used for visualizing data. ICDC provides separate data access to internal (inside our network) and external users due to different needs of these user groups and to different rights to spread data.
Content base TYPO3
ICDC uses TYPO3 as a content base, where all information and links to downloads, visualizations and external references flow together. TYPO3 is a commonly used Content Management System (CMS) and supported by our IT-department CIS. ICDC decided to replace the former used CMS Plone due to a number of reasons. Although both CMS are capable for the needs of ICDC, Plone is less user-friendly than TYPO3, which makes content editing and administration less effective than in TYPO3. Plone is the better choice for individual access management, but as the data are stored on a LAS/Thredds server and not on the web-server, this feature is not useful. The programming of templates and common web-features for Plone needs expert knowledge of the system, whereas the plugin concept of TYPO3 offers several hundred solutions for many typical needs which can be installed rapidly. As TYPO3 is widely used at the University of Hamburg, CIS offers good expert knowledge and support.
Data access via THREDDS / LAS
A THREDDS data server provides data access for scientific datasets, especially for geo-referenced data and offers two important remote data access protocols: Users can download datasets via your web browser through HTTP and use subsets of data without downloading the entire dataset with the help of OPeNDAP. THREDDS also creates virtual datasets through aggregation, so you can merge a chronological sequence of datasets to one and access it via OPeNDAP. The Live Access Server provides visualizations of geo-referenced data in netCDF format and via OPeNDAP, so that it can also visualize aggregations from THREDDS. LAS can also read (insitu) data from a SQL database, so that we can visualize many geo-referenced datasets including satellite, in situ and model data.