From ICISWiki

Meetings/VCs

January 16, 2008

Present: Jay,Weng,Jeff,Ching,Mylah

Agenda:

• To clarify issues related to setting up ICIS databases for the ICIS and Cropfinder Web Applications
  • these are separate applications.
  • both are GCP Koios Web Applications

Minutes:

Deployment of ICIS Web

• Jay went through the deployment instructions for ICIS Web (posted on the GCP Pantheon site) - including the set up of ICIS databases.
• Weng will update the ICIS Website on UQ servers and will refer to the same deployment instructions document.
• For the set up of the data layer(i.e. ICIS databases), the details are not in the said documentation
  • consider the setup of public and central/local databases

Procedure for setting up of databases for ICIS Web and Cropfinder Web Application

• Involves several steps (not just taking a mysqldump and doing a restore on target server)
• First step: involves taking a mysqldump of a database that uses the standard ICIS schema
  • How do we check that the db uses standard ICIS schema? A tool developed by Ching is available. This tool is also able to apply patch if a problem is found.
• Second step: Once data has been loaded, manually call a stored procedure to create the data warehouse tables needed by the ICIS Web datasources for the "get all dataset per study" use case
• Third step: Create the datawarehouse tables that the ICIS datasources need for the cropfinder application

Creating datawarehouse tables for the "get all dataset per study" use case

• calling stored procedure stops on certain studies, generates errors (unsuccessful for some studies)
• get_all_dataset.sql

    --AND f.studyid < 637 and e.represno >1554

    stops on studies with large datasets

• Weng: this is due to a mysql bug, but fixed on later versions
  • bug encountered on mysql installations on production server: mysql ver 5.0.32
  • try on mysql versions where this bug was fixed:
    • 5.0.52
    • 5.1.23
    • 6.0.4
  • Jay to install latest mysql (look for a server)
    • test on a dataset, the problematic dataset
    • Jeff to validate datasource on the mysql version

can an ICISWeb-ready database be released? (an "ICIS-Web-ready database" where all datawarehouse tables are included)

• may not be advisable
• how to update this db with new studies?
• there is NO same set of data for a crop? Central + local combination
  • this is where some discussion on data flow came in. Illustrations, ERD for GMS -- were asked to be made available.

Feb 5, 2009

(Meeting via video conference)

Summary by M.Senger:

Warehouse Tables

The warehouse tables are both in the core ICIS (by "core ICIS" I do not mean the regular ICIS but in ICIS used by the GCP data sources) and in the CropFinder ICIS. In both cases, their structure (number of table columns) depends on the data (some data require less columns, some more). Because this dependency causes problems (so far seen in the core ICIS but theoretically may happen also in the CropFinder ICIS), we moved to replicate the current warehouse tables (in the core ICIS) by one of the following solution:

...

...

...2) Use the Thomas's warehouse tables. This solution was accepted for trying. So far nobody sees any problem with it. Ching is going to write a script populating these tables and Jeffrey will adapt the ICIS data source to read from these tables. Hopefully both will happen before the next VC next Wed/Thu. There was a suggestion to use only one column for values (VARCHAR type), not two as in the Thomas's original proposal. Thomas was not available to comment on it. Ching and Jeffrey is going to try with the single value column.