Utilities and Guides

Overview of the GCOOS Data Portal
Service Status: Online

The Gulf of Mexico Coastal Ocean Observing System (GCOOS) Data Portal (hereafter will be referred to as Portal) was conceived to aggregate near real-time oceanographic data from geographically-distributed data systems located around the Gulf of Mexico. The data sources of the region are maintained under a variety of data standards and archival schema, and the core challenge of this Portal was to interface these data, model output and products via automated standards-based service layers and through web-based human-accessible graphical user interfaces.

The current version of the Portal is limited only to in situ observations. The system read data from Local Data Nodes (LDN and also referred to as Regional Partners) and assemble these onto a database. Data assembled will be validate and validated data posted onto databases for long/short-term storage. Open Geospatial Consortium, Inc. (OGC) compliant modules were developed to interface with other modules to present data that are easy to read and imported. A special user-interface (this web pages) was drafted to facilitate the demonstration of the facilities/services and as the media to distribute the codes developed for others to also have that capability as may be desired.


The modules and services and categorized onto components and components grouped onto composites. The developers have taken into consideration, that the standards and technology in general are still evolving and security-related issues needs to be evaluated regularly to ensure data integrity and compliance to community standards.

A list of these service and their status are provided on a separate page and a series of HOW TO pages will be drafted as this Portal is developed. For general inquiry, comments and suggestions to improve the Portal, please do not hesitate to contact us.

 

HOWTO: Install services to generate OWL files
Service Status: Online

NOTE: The Marine Metadata Interoperability Project (MMI), of which GCOOS is an active participant, has developed a number of utilities to facilitate themapping of ontologies for data providers. The Resource Description Framework (RDF), a lightweight ontology system and fully compatible to Web Ontology Language (OWL), is the common framework employed by MMI. The following is an alternative to the Voc2RDF (developed by the Marine Metadata Interoperability Project) utility that you may use. This utility is use by GCOOS to auto-generate the RDF/OWL files required by the MMI registry as changes are made in the lists. PHP and SQLite3.x are required to successfully install this utility. The following is a step-by-step procedure on how to install this facility.

It will be noted that these package does not include modules to encode or manipulate data in the database. There are a nuber of freeware that can do the data encoding and manipulation job with great efficiency. The SQLite Manager is the author's choice (this one is an add-on to FireFox).

Click here to view a demonstration of this utility.

  1. Download the package by clicking here and expand the file onto a folder accessible by the web server. It should have the following:

    Item
    Description

    (1) srv_gcoos_voc.cfg

    This is the configuration file that contains the header parameters for the OWL generation.
    (2) srv_gcoos_generateOWL.php Module to generate the OWL files from data on the srv_gcoos_voc.cfg and the data in the SQLite database, gcoos_voc.sqlite.
    (3) srv_gcoos_generateCSV.php Module to generate the CSV files from data in the SQLite database, gcoos_voc.sqlite.
    (4) gcoos_voc.sqlite This is an SQLite database template and it contains some data for sample runs if needed. The updated database may also be download from: http://gcoos.rsmas.miami.edu/dp/sqlite_voc.sqlite
    (5) gcoos_vocabulary.php The core script used in the demonstration page (http://gcoos.rsmas.miami.edu/dp/gcoos_vocabulary.php).

  2. As is, you are ready to test the utility by accessing gcoos_vocabulary.php. However, if you want to configure the system to your needs, please proceed with the remaining steps outlined in this guide.
  3. Open the srv_gcoos_voc.cfg using any text editor and make the necessary changes and save the file as a simple text file. The file has the following structure:

    Line 1: This is the base URL for the OWL file (i.e. where you intent to save the output).
    Line 2: This is the title of the listing/compilation
    Line 3: This is a short description of what this collection contains
    Line 4: This is the name of the primary author
    Line 5: This is the general category (one word) for the collection (e.g. parameters, qaqc_flags)

    Please note that the entries are enclosed on double-qoutes.

  4. Open the SQLite database and make necessary edits to the listing/compilation as you see fit.
  5. If you want to change the location of the database for whatever reason, please edit the srv_gcoos_generateOWL.php and srv_gcoos_generateCSV.php, and change the $db values to connect to the right database. Also, in srv_gcoos_generateCSV.php, you may change the "Content-type" header to read header("Content-type: application/vnd.ms-excel") if you want to activate the save to file command.
  6. You are now ready to publish the scripts and test run the facility by accessing gcoos_vocabulary_example.php.
HOWTO: Install the IOOS DIF Sensor Observation Services
Service Status: Online

NOTE: The following is an alternative to the OOSTethys option of providing OGC Sensor Observation Service (SOS) applying the most recent version of IOOS DIF (ver. 0.6.1) schema. Please note that the IOOS DIF working group (Web Services and Data Encoding; WSDE) has not made any official announcement with regards to the publication of this schema. Also, the authors of the PHP scripts and template presented here do not claim completeness on the implementation. Visit the NDBC SOS implementation site (http://sdf.ndbc.noaa.gov/sos/) for sample application of this template based on MySQL DBE or to GCOOS Data Portal advanced search page based on SQLite 3 DBE.

These services requires systems with PHP and SQLite3.x installed.

  1. Download the package by clicking here and expand the file onto a folder accessible by the web server. It should have the following:

    Item
    Description

    (1) sql_ioos_dif.pdf

    This PDF files contains the design of the database from which scripts were written.
    (2) sos_server.php
    Component developed in PHP that implements the following services: GetCapabilities, DescribeSensor and GetObservation. Please note that the capabilities of these services are not fully implemented.
    (3) ioosdif.sql A sample MySQL file with the structure are presented in sql_ioos_dif.pdf
    (4) test.shtml A sample SHTML page that implements the services.
  2. You now have the option to: (A) use the ioosdif.sql as is or (B) use what you already have and modify the scripts. If you choose (A) write a small program (not provided here) to populate the database.
  3. In either option (i.e. A or B above), you need to modify the sos_server.php using any text editor (PHP Editors are preferred). Change Lines 3 to 24 to contain appropriate values and line 36 that connects to the database (connection string; this is an include file).
  4. If you decided to use the ioosdif.sql as is, you are done and ready to publish the files and test.Otherwise, please search all '$sql =" and make necessary changes to the SQL statement. Please remember to retieve the data and designate to the pre-defined varibale names as posted in sql_ioos_dif.pdf.

The other changes that needs to be made but they are minor. For example, if you want to use your own record definition for the parameters measured, change all values for recDef to point to a user-designated XML. Also, you will notice that the URN used is the URN convention with NDBC station ID (i.e. urn:x-noaa:def:station:ndbc::<NDBC designated station ID> or urn:x-noaa:def:sensor:ndbc::<NDBC designate sensor ID>). Others will be tempted to use a different URN. GCOOS DP developers recommend that it may be wise to use similar URNsto facilitate data exchanges among data providers and users.

 

 

HOWTO: Install In Situ Monitoring Services and Regional Resource Inventory
Service Status:Online

NOTE: The following is a step-by-step procedure to install a basic monitoring system of the stations and sensors, and resource inventory using the available infrastructure already available to the regional partners, i.e. XML files for IOOS Registry. This implementation has made few changes to further standardize the presentation of the parameters. These services requires systems with PHP and SQLite3.x installed.

The Monitoring tab is a demonstration of these services and click here to see a basic page using the services. Please see notes below to filter data specific to an operator. The later presents the table values for detailed examination of the resources and the later, is a graphical presentation using Earht Google API.

  1. Download the package by clicking here and expand the file onto a folder accessible by the web server. It should have the following:

    Item
    Description
    (1) gcoos_monitor.sqlite SQLite database with several tables that are pre-initialized for the region (NOTE: The values will need to be updated using the srv_gcoos_readIOOSReg.php that is activated either manually or via a chron file - type of system).
    (2) srv_gcoos_readIOOSReg.php

    Module to read the GML files that are stored on pre-defined URLs. Another utility, srv_gcoos_readADCPReg.php is also available upon request to plot ADCP reading-stations in the region. Operator-specific request is also available by passing the variable 'operator' with their respective acronyms. For example, to read only stations from XXX operator, the URL call should read:

       srv_gcoos_readIOOReg.php?operator=XXX

    The acceptable short names of the operators of in situ platforms to date are (see also the ShortName variabes in the table, gcoos_monitor.sqlite): TABS, LUMCON, TCOONS, COMPS, WAVCIS, DISL, CenGOOS and MOTE.

    (3) srv_gcoos_generateKML.php

    Module to re-generate the KML files (structurally different from the IOOS Registry system). Operator-specific request is also available by passing the variable 'operator' with their respective acronyms. For example, to read only stations from XXX operator, the URL call should read:

    http://<path>/srv_gcoos_readIOOReg.php?operator=XXX

    The following are some of the acceptable acronyms of operators to date (see also the ShortName variabes in the table, gcoos_monitor.sqlite): TABS, LUMCON, TCOONS, COMPS, WAVCIS, DISL, CenGOOS, MOTE. Example, to generate the only the LUMCON stations, the URL call should read:

    http://gcoos.rsmas.miami.edu/dp/srv_gcoos_generateKML.php?operator=LUMCON

    Another value that can be passed is All. This will generate a KML file with all the stations in GCOOS inventory, including inactive ones: http://gcoos.rsmas.miami.edu/dp/srv_gcoos_generateKML.php?operator=All.
    (4) srv_gcoos_displayResource.php Module to present the detailed resource status grouped by operator.

    Operator-specific request is also available by passing the variable 'operator' with their respective acronyms. For example, to read only stations from XXX operator, the URL call should read:

    http://<path>/srv_gcoos_readIOOReg.php?operator=XXX

    The following are some of the acceptable acronyms of operators to date (see also the ShortName variabes in the table, gcoos_monitor.sqlite): TABS, LUMCON, TCOONS, COMPS, WAVCIS, DISL, CenGOOS, MOTE. Example, to generate the only the WAVCIS stations, the URL call should read:

    http://gcoos.rsmas.miami.edu/dp/srv_gcoos_displayResource.php?operator=WAVCIS

  2. As is, you are ready to test the utility by accessing srv_gcoos_displayResource.php from a browser. However, if you want to view the latest data and regenerate the files for display and mapping, please proceed with the remaining steps outlined in this guide.
  3. If you intent to store the dabase on another (secured) directory, change Line 9 in both scripts (srv_gcoos_generateKML.php, srv_gcoos_generateKML.php and srv_gcoos_displayResource.php) to point to the correct path.
  4. Also, you are most welcome to open the SQLite database (gcoos_monitor.sqlite) and make necessary edits to the "Operator" table as you see fit. Unlike the other databases, you may want to store this on a another folder and use the .htaccess file to control a database download. To ensure that you have the latest record, click here to download the XML file that you can use to initialize the "Operator" table of the database.
  5. You are now ready to publish the scripts and test run the services.