ICIS Developers Workshop 2011
From ICISWiki
HOSTING
Hosted by The International Crops Research Institute for the Semi-arid Tropics (ICRISAT)
Venue: C Fred Bentley Conference Center (Building 212)
ICRISAT, Patancheru 502 324, Andhra Pradesh, INDIA
REGISTRATION
Registration is Closed!
Expenses for this workshop There will be NO Registration Fee and this will be covered by our project from ACIAR. The organizers will take care of local transport, food and accommodation during the workshop. The participants have to make their OWN travel arrangements to reach Hyderabad and back..
How to get to the Workshop venue: Please make your own travel arrangement to reach Rajiv Gandhi International Airport, Shamshabad (Hyderabad) by 27th March 2011. The organizers will arrange to collect you from the Airport to ICRISAT Campus. However we need your flight details to make the arrangements for your pick up from the airport to ICRISAT campus by 20th March 2011. Please send your flight details to KDV Prasad and in case of emergencies you can call +91 40 9391139934/+91 40 24053976.
Accommodations: The organizers will make the accommodation at ICRISAT Campus in Guestel/Flatlets without any cost during the workshop (27th March – 2 April). Please send your accommodation requirement by 28th February 2011 to KDV Prasad
Food: Meal card will be issued to all the resident participants to have their breakfast/ dinner in the campus canteen (204 Dining Hall) without any cost. However during the workshop Lunch will be provided to all the participants in 204 Banquet Hall.
For special dietary needs inquiry, please email KDV Prasad. The participants can also indicate their food requirements in Registration Form, and every effort will be made to accommodate those with special dietary needs.
City Information: Please visit this site for useful information about Hyderabad.
Questions? Please direct any question to KDV Prasad .
AGENDA
The Agenda below is a work in progress and will be apdated by session chairs as we approach the meeting
We have tried to color-code the sessions:
- Light Yellow for sessions of general interest
- Light Grey for sessions of special interest to users
- Light Blue for sessions of special interest to Developers.
| Date | Time | Activity | Facilitator Reporter |
Day 1- Monday |
28 Mar 2011 |
Registration & Opening | Trushar Shah |
| | 0830-0900 | Registration | Prasad/Manjula |
| 0900-0905 | Welcome Address by Director, Dryland Cereals | Oscar Riera-Lizarazu, ICRISAT | |
| 0905-0910 | Remarks by Director, Grain Legumes | CLL Gowda, ICRISAT | |
| 0910-0915 | Remarks by AVRDC | Warwick Easdown, AVRDC | |
| 0915-0920 | Remarks by DDG, Research | Dave Hoisington, ICRISAT | |
| 0920-0925 | Remarks by GCP Director | Jean-Marcel Ribaut, GCP | |
| 0925-0945 | Opening Address by Director General ICRISAT | William Dar, ICRISAT | |
| 0945-0950 | Vote of Thanks | Trushar Shah, ICRISAT | |
| 0950-1020 | Group Photograph/Coffee break | 212 Foyer | |
| 1020 - 1045 | Vision: IBP and Configurable Workflow System | Jean-Marcel Ribaut, GCP | |
| 1045-1100 | Workshop Objectives and Agenda | Graham McLaren, GCP | |
| AM & PM Sessions | Activity Reports |
Chair: Arllet Portugal Reporter: Clarissa Pimentel | |
| 1100-1200 | Existing crop implementation
- Rice (IRRI) |
| |
| 1200-1300 | Lunch | 204 Banquet Hall | |
| 1300-1530 | New Crop Implementations as case studies - IChIS- Chickpea (ICRISAT) - ISGIS- Sorghum (ICRISAT) - ICASS-Cassava (CIAT) |
- Arturo Franco | |
| 1530-1600 | Coffee break | 212 Foyer | |
| 1600-1700 |
GCP Information Management Activities Wrap-up Discussion | Arllet Portugal | |
| 1900 | Workshop Inauguration Dinner | Mary Cummings Park | |
Day 2- Tuesday |
29 Mar 2011 |
New and Improved Crop Informatics Tools |
|
| | AM Session | Standalone Tools |
Chair: Shawn Yates Reporter: Jeffrey Morales |
| 0830-1000 | Excel and Delphi-Based Tools
– ICIS Lite |
| |
| 1000-1030 | Coffee break | 212 Foyer | |
| 1030-1200 | New Java based tools – IB Fieldbook (Demo) | Hector Sanchez | |
| 1200-1300 | Lunch | 204 Banquet Hall | |
| PM Session | Web Query and Decision Support Tools |
Chair: Hector Sanchez Reporter: Jesper Norgaard | |
| 1300-1340 | IB Web – Pedigree viewer/Study Mart | Mylah Anacleto/Jeff Morales | |
| 1340-1420 | Molecular Breeding Design Tool (Demo) | Trushar Shah | |
| 1420-1500 | OPTIMAS - MB Decision support tool | Fabio Valente | |
| 1500-1530 | Coffee break | 212 Foyer | |
| 1530-1700 | Parallel Sessions | | |
| | User Testing of IB FieldBook | Hector Sanchez/Arllet Portugal | |
| | Administrator Tool | William Eusebio | |
Day 3- Wednesday |
30 Mar 2011 |
Handheld, Analytical & Decision Support Tools |
Chair: Gouyou Ye Reporter: Warren Constantino |
| | 0830-1000 |
Hand held devices and barcoding (Which to choose? What technology? Must have functiuonality?) - ICIS Mobile |
|
| Coffee break | 212 Foyer | ||
|
1030-1100
1100-1200 |
Data Access for Analytical and Decision Support Tools
Association Genetics, ICIS and Cross prediction |
-Hector Sanchez - Gouyou Ye - Rathore Abhishek -Trushar Shah
- Howard Eagles | |
| 1200-1300 | Lunch | 204 Banquet Hall | |
| 1330-1900 | City Tour | | |
| 1930-2200 | Workshop Gala Dinner | Taramati Baradari, Hyderabad | |
Day 4- Thursday |
31 Mar 2011 |
Parallel Sessions |
|
| | 0830-1000 | Genetic Resources Information Management System - SINGER (Elizabeth Arnaud) - GRIMS (Lilibeth Sison & Grace Capilit) |
Chair: Rauraidh Hamilton Reporter: Lilibeth Sison |
| | Managing and using genotyping data for breeding decisions - Germinate (Hector Sanchez) - ICRIS (Trushar Shah) - ICIS DMS (Chengzhi Liang) |
Chair: Trushar Shah Reporter: Prasad Peteti | |
| 1000-1030 | Coffee break | 212 Foyer | |
| 1030-1200 | Continuation of Parallel Sessions | | |
| 1200-1300 | Lunch | 204 Banquet Hall | |
| 1300-1400 | Report of Parallel Sessions | Chair: Chengzhi Liang | |
| PM Session | Ontology and Data Access Layer |
Chair: Chengzhi Liang Reporter: Alex Caneda | |
| 1400-1500 |
Ontologies and Trait Dictionaries Planning for A Distributed Curation Tool | Elizabeth Arnaud/Rosemary Shresta | |
| 1500-1530 | Coffee break | 212 Foyer | |
| 1530-1630 |
Schema Changes |
| |
| 1630-1700 | Wrap-up Discussion | Chengzie Liang | |
Day 5- Friday |
1 Apr 2011 |
Collaboration & Future Plan |
Chair: Fran Clark Reporter: Shawn Yates |
| | 0830-1000 |
Data Access Layer - ICIS Datasource (Java) | - Jeffrey Morales - Hector Sanchez - Warren Constantino
|
| 1000-1030 | Coffee break | 212 Foyer | |
| 1030-1200 | Rejuvenating Communication & Collaboration | Paul Fox | |
| 1200-1300 | Lunch | 204 Banquet Hall | |
| 1300-1500 | Short Meetings of Partners | | |
| 1500-1530 | Coffee break | 212 Foyer | |
| 1600-1630 | Future Plan & Closing Remarks |
Chair: Graham McLaren Reporter: Arllet Portugal | |
| | Departures | |
DISCUSSION
Day 1
Activity Reports of Existing Crop Implementation
IRRI (Rice) by Ruaraidh Sackville Hamilton
- Many other key players in rice; new members of the rice community such as:
- BGI (whole genome accessions are being done in BGI; these data will be entered in ICIS; they're composed of 1000 plus bioinfomaticians)
- African Rice
- MBPGR (one of the leading genebanks)
- Bioversity (Note: Genebank thinks that all are accessions but Bioversity is here for better linkage bet. genetic resources and other resource)
- CRIL (part of it is genebank and the other part is in plant breeding)
- CRIL has been divided into two groups but there is a unified data management team
- What can IRRI and rice community do:
- 4 levels:
- 1. Do job well - collate good data; have a correct inventory of data (must have quality softwares and data curation by curators)
- 2. Work together - GMS is center of everything; it can track samples, seed increase, etc; cross linking of data; can accomodate DNA data, links to gid and it can also provide linkage between molecular and phenotypic databases; )
- 3. Do interesting work in the data (molecular data)
- 4. Do something useful with it (select the right parents and progeny for crosses; work with IBP)
- 4 levels:
African Rice (Rice) by Marimagne Tchamba
- Africa Rice use ICIS for their breeding platfom, data management
- ICIS has been the official system in use
- Had performed breeding trainings
- Fieldbooks generation are done using ICIS as well as genebank data
- They intend to integrate breeding practices in ICIS for 8 use cases such as upland, lowland breeding programs, etc.
- Users have adopted ICIS in their work
CIMMYT (Maize) by Hector Sanchez
- Tools: IMIS db, Fieldbook system, Fieldlog is used to collect info in the field
- Move data to Maize Finder (uses Mysql)
- convert orig FBook data to make use of Fieldbook IMIS beta version → Local IMIS to be uploaded to Central IMIS via scripts.
- convert old Maize db to the ICIS schema (i.e. new IMIS)
- Workbook is used to parse pedigree info
- ZEUS is tapped to present/display info
- Some historical data which dates back to 1993-2010 have been loaded
- CIMMYT lowland tropical has the highest data loaded in the db so far
- Maize info dvd (given to users/breeders;) contains Java fieldbook prototype as well
- IMIS fieldbook runs in diff. environments (OS, MS Office, architecture like 64 and 32 bits)
- FBook stats (pedigree info by diff. Cimmyt cooperators and genotypic data)
CIMMYT (Wheat) by Hector Sanchez
- IWIS is running for more than 20+yrs in CIMMYT
- IWIS2 in SQLServer need to be synchronized with IWIS3(GMS and DMS) in Mysql.
- Tools in place are Workbook, Fieldbooks and Reports, Icis modules, web portals and GrinGlobal
- Solution:
- Configure a new production server
- Migrate from MSAccess data to MySQL in the new server
- Generate IWIS3 installer for MySQL
- Problems:
- Need synchronization tools to migrate IWIS2 data to IWIS3
- Consolidate central dBs bec. there are many versions recently
- IWIS3 cannot generate SMTA data as of the moment
- All in all, not yet fully implemented ICIS but getting there; more success recently because of more collaboration with CRIL developers and users
- Still in progress:
- IWIS3 has limited query capabilities
- CrossInfo Alpha version is available
- Additional tables or columns in DMS (for having a standard scale for measuring yield for example)
AAFC (Wheat) by Shawn Yates
- 12 yrs of collaboration with ICIS
- 7 diff. Breeding programs → moving into genetic data now
- Tools: Data mining tool, Marker assisted selection, GEMS, IWIS3, etc
- Data mining tool (available in Cropforge; migrated development from Visual Studio 2005 to Visual Studio 2010 or VB .Net and does not use DLL because of problems with .NET)
- Build queries and output data in Excel and other format
- Marker assisted
- show genes that have been reported on germplasm
- output is in Crystal Reports (output in many formats like Excel, etc)
- GemsCat
- not upgraded; SPARC needed this though (last piece of the puzzle)
- developed the GEMS 5.4 schema
- old one is in Access (developed new GUI or catalogue)
- IWIS3
- submitted 35,000 durum genotypes and 32,000 hexaploid genotypes
- more success now in migration of wheat data
- involved with new tools like IBP meetings, etc
- implemented new security protocols
- master databases in SPARC to update local databases in diff. areas
- Biotech labs now using ICIS for marker evaluation data
NRCPB (Wheat) by Manoj Singh
- 3 breeding sites (PBI in AU; DWR in India; PAU in India)
- 3 support sites (NRCPB; IRRI and CIMMYT)
- Goal: manage molecular, phenotypic and pedigree data
- NRCPB receives MS Access data from PAU, DWR and PBI centers and consolidate it into 1 Central database (in MS Access and then converted to MySQL) at NRCPB. ICIS Zeus web apps uses this updated MySQL Central database.
- uses Bullzip software to convert ms Access to MySQL
- uses JOOMLA and PHP for front end; uses tomcat web server and accesses MySQL database
- Wheat Molecular website is currently available
- provides tech support to DWR, PBI and PAU
- Uses FileZilla to connect to NRCPB servers to upload/download data
- Future plans: collect more data from DWR, PAU, PBI; migrate Central database to a new high performance server
- suggestions: ICIS to be made more compatible with other MS environments and diff. OS
- reduce steps in converting tools from local MS Access GMS,DMS into Central MS Access data
- ICIS training is needed
- markers data workbook should be more user friendly
- Q&A:
- Manoj: not sure if it'll fully work in MySQL
- Shawn uses MySQL and it's ok
- Shawn has suggestions (in the wiki) on how to reduce the steps in migration of local MS Access GMS and DMS data to MS Access Central database
- Hamilton to Hector: cannot handle international data shipments ?
- Hector: working on it right now; added more tables in the schema to accommodate international nursery data; not modify but just include more tables
- Arllet: Dr. Hamilton will mention some of the suggested tables; such as Hector's standard scale unit
- Arllet to Manoj: ICIS installer with MS Access in Local and MySQL for central
- Shawn: uses both MySQL for Local and Central
- Shawn: Migration toolkit is available and they've used it
- Manoj: Are retrievers and workbook working ok?
- Shawn: Yes. No problems so far.
- Shawn: Few extra steps for MySQL, though.
- Graham: if data is small, MS access is great. For bigger data, MySQL can be useful
- Hector: 2GB is the limitation for MS Access database
- Tchamba: Linux or windows?
- Shawn: can be run in both
- Hector: can use also MySQL in Mac
- In summary, Arllet mentioned that the Wheat group has made progress in their work in managing, collating and migrating their data.
Activity Reports of New Crop Implementation/ Use Cases
IChIS Chickpea (ICRISAT) by Peteti Prasad
- ICRISAT is the crop lead center for Chick pea
- Tools: ICIS, pedigree data uploader, Ichis installer
- Attended workshops and trainings in Zaragoza, Madrid and upcoming workshop in May 2011
- 62 traits provided in Zaragoza, 3 more in Madrid; 15 traits for tropical legume
- Pedigree Data:
- to move in ICIS: germplasm evaluation and chickpea salinity
- Data in ICIS: chickpea reference collection, 80 genotypes, mapping population, etc.
- Data in ICRIS: 32 SSR markers, GCP data for chickpea composite collection
- Data in TL1: genotyping, markets, 311 SSRs
- PMS: cross data from 1973 – 2009 (plan to move these data to ICIS; use GMSInput tool)
- Advance line (to move into ICIS via SetGen)
- In ICIS: parental lines by years
- Handheld
- uses Honeywell (efficient and reduces error; font size is small)
- Lessons
- Data curation – time consuming
- GMS
- DMS (high learning curve for users)
- Trait Dictionary (no column for meta name ?)
- Strong interaction between developer and breeders
- Next Steps
- Data mgt. Of TL and IBP Projects
- New traits, data curation, evaluation data, etc.
- Q&A
- Graham: Only breeding data
- Prasad: no relation yet between databases in ICRIS and breeding data? Plan is to integrate.
- Rosemary: Ontology.... ?
- Prasad: Ontologies have been used for chickpea
ISGIS Sorghum (ICRISAT) by Praveen Reddy
- ISGIS
- started in 2010
- At CI workshop, added 68 traits, scales, and methods
- sorghum database available for download in IBP website
- Data Curation
- Evaluation datasets, pedigree data are available
- Some problems: uses X but ICIS uses / for crossing; some names have / in names but ICIS splits it as Female and Male;
- Genotyping and Marker data are in ICRIS database
- Lessons
- Data curation – need to talk or collaborate with key people
- GMS
- DMS – some new functionalities in Excel does not work
- DMS Retriever- takes time to retrieve data
- Features Needed
- function to add/delete/edit traits to the database
- option to cancel when uploading pedigree in SetGen
- Next steps:
- add Cereal data
- Q&A
- Arllet: Bioversity will provide facility in adding traits to the database
- Graham to William: Progress in GMSInput....
- William: GMSInput can help in uploading data but not yet compatible with ICIS 5.5 schema/DLL
- Shawn: Use Workbook template when adding new traits to the local database
ISGIS Sorghum (ICRISAT-Bamako) by Ibrahima Sissoko
- genetic materials
- Database creation:
- entered parental lines of BCNAM
- F1 names are entered (but having problems in entering its parents due to format)
- Sorghum Traits
- diff. Names/abbreviation in french/english but should link to same id
- diff. Units
- Fbook
- creates a template for diff. Trials
- sent data sheets to farms
- Q&A
- Graham: Entering population data... Can enter F1 names first and go back in in entering parent names to link them up. It's a reasonable way to start from F1.... Can enter diff. Trait names but same id
- Arllet: How to handle multi-languages in crop ontology
- Rosemary: use the same id number for the synonyms
- Elizabeth: sysnonyms in ontology cannot work all the time; mapping is the best solution
- Arllet: trait dictionaries are in the IBP website
- Bob: mentions abt. Scales... ?
- Tom: hybrid parents, etc. but f1 hybrid cultivars... how is it stored?
- Ibrahima and Graham: Hybrid method is included in ICIS.
- Graham and Arllet: some challenges for hybrid … use same id or diff. Ids? And how to link the data...
IVIS Cowpea (IITA) by Sam Ofodile
- ICIS used 10 yrs. ago but due to staff movements, it was interrupted
- Bioinformatics unit revived the use of ICIS recently
- IITA is the lead center for cassava, cowpea, yam, and soybean
- Cassava is a joint effort of IITA and CIAT
- IITA has the central databases of the crops like cassava and cowpea
- crop data have been recovered from IITA servers; IVIS has been recovered with the help of GCP and UC Riverside partners
- databases are in MS Access
- trait data have duplicates; scales are wrongly defined and some are not defined; there are some empty studies
- no hand-over of data;
- use SAS to compare date so as not to lose what is available
- problems in opening DMS Retriever due to duplicates in traits
- Progress:
- all duplicate traits/studies have been identified and removed/renamed
- on-going curation of data
- Workplan:
- setup an Intranet and Internet for all to use/access
- train technicians and breeders esp. handheld
- create genotypes and pedigree lists
- upload local data to central IVIS and have a backup copy
- curator for tropical legume 1 (TL1)
- Q&A
- Graham: commendable job in cleaning the data... no short cuts but really have to do it. Have you merged the old with the new traits? ** Sam: Yes, just last week (traits from Jeff, etc.)
- Arllet: How are the data connected bet. Local in Mali and in ICRISAT
- Trushar: There's a local db in Mali, local and central dBs in ICRISAT
- Arllet: and a local db from national partners... Chickpea data is available in Zeus and will be presented in the workshop..
ICASS Cassava (IITA) by Moshood Bakare
- pedigree data was in a flat file back in 2006
- cassava group is new in the use of ICIS
- old naming convention with a / (so needed to modify how they name their data; solution is to remove the slash)
- Names represented in ICIS and what they mean.. for example:
- 1940561/? (half sibs)
- 1940561/. (selfing)
- 1940561/1940562 (full sibs)
- likes to remove some methods in ICIS that are not intended for cassava (note: Arllet mentioned that this can be done)
- gid now in local db is over 6200
- Issues:
- enter data manually; need to learn how to enter in batch mode
- need tool for tracking seed inventory
- reassign gid to edited germplasm name
- concurrent users in a network environment
- standard procedures across users groups
- Focus for 2011
- Upload evaluation data
- train technicians on the use of handheld; more training for IITA stations
- collaborate with NARS partners
- Ontology
- Sources of cassava ontology from Dixon et al 2010, Fukuda et al handbooks
- grouped the ontologies into diff. categories (agronomic, morpho, physio, etc.)
- Lessons:
- Data curator needed for the work
- Crop descriptor- should be developed in collaboration with diff. pathologists, breeders, key persons, etc.
- Next steps:
- Map cassava ontology with ICASS trait table
- use annotation tool
- submit cassava terms to Gramene cereal ontology (TO)
- Q&A
- Graham: No problem to use ICASS with multi-users in a network. Tools have been ready for editing or entering data in a multi-users environment.
- Arllet: 2 central cassava for Latin America (CIAT) and Africa (IITA) ....
ICASS Cassava (CIAT) by Arturo Franco
- About 6,000 accessions (evaluation, passport, etc.)
- Agreement:
- integrate pedigree info, crosses phenotypical of IITA and CIAT
- Present
- Load DMS or study information in Oracle and in Excel file
- 1700 trials since 1977
- 27 common traits in the trials
- Molecular data: 2000 accessions in molecular data
- Future:
- integrate IITA pedigree info in CIAT ICASS
- standardized traits with IITA and CIAT
IPhIS Beans (CIAT) by Arturo Franco
- Tools:
- Past: uses version 1(the old version) because of good graphical interface for data; and retrieves all materials for the data in one click
- Present: migration of IBYAN data
- Q&A
- Arturo: Move the data from Oracle to ICIS
- Hector: Can use Oracle for ICIS bec. of ODBC. (Suggests to use one database for central like Oracle and local)
- Arllet and Arturo and Shawn: have used 5.5 version of GMSInput in uploading the data.
- Sam: Classic example for diff. traits... traits are in Spanish for CIAT and English for IITA ?
- Arllet and Arturo: uses IPhIS with the national partners as well....
Day 2
Standalone Tools
ICIS Lite (IRRI) by Warren Constantino
- Dazzling array of buttons
- to solve lack of current programmers in ICIS Team
- to flatten the learning curve
- one application for the new users of ices
- ICISWorkbook plus other functions
- JButton to automate everything
- 9 different ICIS applications into one
- one-stop shop application
Basic Entry Points:
1.) create a Blank germplasm list
2.) extract list from old excel file
3.) form list from query result
4.) get list from seed inventory
A little bit of history:
Yates requests to be able to create a germplasm list
Demo Explanation part-by-part:
1.) has coloring schemes to tell if there are multiple selection of a specific germplasm
red for multiple, orange for single result
2.) can build and parse the pedigree tree
a.) parse manually
b.) parse automatically
3.) Customizable Naming Conventions - basic replace function to customize your names
4.) Has file conversion wizard - convert to ices workbook format using just one button
5.) GCP Phenotypic Data Input Tool
6.) Study Retriever
7.) Seed Inventory
8.) Breeding: Germplasm Management
9.) Breeding: Generate Crosses
10.) Filter the methods possible
11.) Inventory Management
12.) Support for Barcodes
13.) Fieldbook Management - simplify more the way for creating a template
14.) Administrator Tools - have search functions
15.) Integration of R (held-off but can continue developing when needed)
Added Options:
- ICISLite Menu Customization
Customizable buttons - can remove the buttons that you don't need
- User Interface Customization
- Fieldbook Customization
Q/A
1.) Beth Sison - installation not used?
Here it's already using the user ids
2.) Beth Sison - Uploading data from excel?
Warren Constantino - No taken from the database
3.) Beth Sison - Foreign key are given importance?
Warren Constantino - Yes 5.5 to 5.6 transition
4.) Beth Sison - Seeds inventory
Using scales from the genebank
How you decide what you give to the users?
Warren Constantino - they give it, can see which seed lots is the latest, you have the power to choose
5.) Beth Sison - can you set the scale e.g amount in grams
Warren Constantino - we can, there's a seed unit interface
6.) Shawn Yates - current users?
Warren Constantino - used by the transgenic but breeders not yet
7.) Hector Sanchez - platform tested
Warren Constantino - yes
8.) Fran Clarke - can those buttons be replaced by words?
Warren Constantino - yes we can customize that
9.) Fran Clarke - got confused when entering a lot of things to enter
Warren Constantino - not required, can choose what you just need
10.) Shawn Yates - how much of this is documented
Warren Constantino - starting an effort to, shows website as a course documentation
11.) Jeffrey Morales - My question are you planning to use another platform
Warren Constantino - use wizards to simplify things
- have a prototype
12.) Tschamba - how to trace movement?
Warren Constantino - i don't know yet, but you can trace that using the pedigree tree
-side answer for dr hamilton - what we do, we enter information on the seeds, your doing the job basically, when you send them seeds, you send them the gid
ICIS DataRetriever (IRRI) by Warren Constantino
1.) parsing algorithm - search traits.
2.) only the studies that has that trait is shown.
3.) can filter by trait for selection
4.) can use operations for limiting searches
5.) future plans - to use a germplasm list template
Data Mining Tool (SPARC) by Shawn Yates
- Line versus Line Comparison
- Getting Factors, have Data Filters
- Can pick specific data types : composite data, genetic data, plot data, summary data
- Filter Dataset results
- Select Quality Traits
Graham Mclaren - is it now working easier on the other databases?
Shawn Yates - yes it does
- shows the main method with a mouse-over
Graham Mclaren - please show GMS Search
- it's progressing, we'd like to put more functions there.
Graham Mclaren - please show the SetGen
*using BROWSE, deploying pedigree*
Graham Mclaren - each line has a different color
IB Web Pedigree Viewer/Study Mart by Jeffrey Morales
- Presenting Graphical reporting tools for users to help in decision making.
- Enhance of Studymart functionality will be discussed on Friday.
- Sam asks about 2 graphs, 1 discrete, 1 continuous (steeman leaves?) or box plot to check graph easier - Jasper Reports should be able to cope with those according to Jeff.
- Arllet: Web application for chickpea URL NRCPB web site
- Trushar: How to add to an existing database?
- Jeff: developing the migration tool - changes in data definitions have been quick
Molecular Breeding Design Tool ISMAB by Trushar Shah
- Comments on lots of collaborators and funding sources
- Key funtctionalities
Breeders maintaining in Excel (colors, foregound background markers) but visualized aided decisions
were difficult.
Trying to identify ideas, visualize data in integrated manner,
marker assisted flow already handled by Mosel from ICMMYT. Use of MBDT
- Plans for future developments, please see slides.
Development plans on cropforge
Graham mentions how to read in this data from ICIS
Template could be better, by choosing the study, map database would sit on top of an ICIS database.
Load and save to database. You have to pick the dataset.
Howard: how powerful are search capabilities of it, for instance to find me all the best matches of best percentage of recovery
Trushar: we can visualize, but searching is not yet implemented.
Sorting by phenotypic data is possible, but not by criteria. Multinational trials
Use it in breeding programs. Cross prediction tool.
Hector: How much data have been used more than 1000 markers. Trushar: Currently difficult to visualize
Planning for separate chromosome view - zoom specific parts are features to implement
Users should be aware which size of datasets can be handled by this tool.
OPTIMAS - MB Decision support tool by Fabio Valente
Word document describing Optimas here
Marker Assisted Selection
Working with different crops butcommon objectives
Algorithms can be really time consuming
Demonstration
Pedigree and genotyping data
All QTL together
computation of genetic probability and creating a selection list for comparison
computation of genotypic probabilities
Bob Geddens: Breeding program needs to get robust lines -
Hector: Have you tested Optimas in real life? Use crosses in the field and check how they behave? Fabio: Haven't done that
Increase the probability
PHd-student Nicolas Barder - comparing Optimas and results will be available in 3 months
Howard: Association genetics can be used
Howard: Finding dead ends working with QTLs
Data quality of QTL data coming in is a problem.
We might think for June meeting about multi-parents, bi-parental, genetic gain.
Hector: Run the tools on practical examples would be a good idea
Annotation Tool by Matteis Luca
Demonstrating the terminizer trait ontology validater
Associating Ontology IDs with units g -> grams
When values are changed with the help of the tool, the data can be exported as CSV file
Then it can be reloaded (into ICIS?)
IB Fieldbook (CIMMYT) (Demo) by Hector Sanchez
- No Domain Model used
Fieldbook Module
Coding Practices - SVN
- JavaDocs
- Unit testing
- package naming
- DEMO starts*
Java-Based Tool
- Generate Different kinds of studies selection window shown
- Shows a Wizard Function
- Follows the ICIS Template
- removable tooltips are shown for users to follow to when they are new users
- All information is read from the template
- selects a checkbox to just pick a few for excluding unwanted information, can select multiple options.
- Drag and Drop traits that they want to get
- Can export to csv file or excel file
- Has a measurement tab for view
- shows example exported template
Q/A
Sam - good tool, easy to follow, hopes that there's a good biometrician to help out with the tool
Hector - okay, that's really good feedback
Fran - restriction issues? different breeders are going to have their different needs.
- you shouldn't be restricted to any new list
Flexible Pedigree (CIMMYT) (Demo) by Jesper Norgaard
Day 3
Handheld, Analytical & Decision Support Tools
ICIS Mobile (Alex)
Mobile Desktop tool
- create the mobile study template
o select the workbook template
o select the barcode reference
o select additional data reference
o select variates to measure
- import data
Features of ICIS Mobile
- load study template
- load variate lookup values
- browse the study list
- data entry of observations
- browse and filter of observations
Menus
- connect to the database
- load the mobile study template
- variate maintenance
- file folder settings
- open study
- data entry
Graham: What database is it connecting to?
Alex: We use the SQLCE
Graham: What for?
Alex: The observation data
Graham: Where did you get the valid values?
William: It was prepared beforehand
Hector: Can it be used in the iPhone?
Alex: We haven’t tested yet
Person1: Has this been seen by real people in the field?
William: We still have to deploy it
Hector: My concern is whether this will work in different devices
Garaham: My suggestion is for Hector’s Fieldbook to be able to output a file for whatever device the user chooses
Roslen: If it's going to be used in the field, then it would be a good idea to incorporate some kind of barcoding into the whole system. Barcodes are traditionally used to represent some kind of primary key to a database table. But, new developments enable barcodes to act as static datasources representing long strings of meaningful characters. Researchers that require mobile devices to collect data from the field could leverage on this development. They could use appropriately designed barcodes to label the "entities" they collect information about (i.e., plots, plants, fields, greenhouses, nurseries, ...). They can then go on with their business of collecting data in the field using their chosen mobile devices. The data collection routine would only require 3 things: barcode of the entity they are observing, the trait, and then the score/value of the observation/measurement. This data collection activity would be very efficient and would be over before the mobile device's battery drains out (addressing the common complaint that batteries of some mobile devices used in data collection never last more than 3 hours). When the researchers' are back in their office, they could then sync the data they collected in the field with a matching desktop application and all the translations/processing happen there. The meaningful codes represented by the barcodes would then be translated into whatever was purposedly encoded by the researcher. Barcodes could incorporate a Study code (mapping it with an ICIS workbook!). The values could then be mapped into corresponding records in a database table or into cells in an ICIS workbook. Thereafter, the researcher could then proceed with doing analysis or whatever he needs to do with the data. I'm sure there is a wide arreay of requirements for data collection application using mobile devices. But simplifying these down into a simple system that reads just 3 values (entity's code, trait, value), which are the ones observed in the field anyway, then a simple but workable mobile application could be delivered in the shortest possible time.
Maize FieldLog (Hector)
- can handle 3 forms of data
- can run in these platform
How to use
1. Start fieldlog
2. Select trial/nursery
3. Select the trait
4. Measure
Graham: Could you just tell us how do you enter the trait values in the field?
Graham: You didn’t show any data entry
Arllet: Our problem is how to enter the valid values
Hector: To be open, this system can still further be developed
Tchamba: Are you not using any database?
Hector: It’s important to have database but our priority is different
Trushar: Our problem with the FieldLog is converting a workbook template to the fieldbook template, you have to specify specific columns
FiledLog (Arllet)
Steps:
1. Convert the workbook file to the fieldlog file
2. Go to the setup tab, select the IDs that define the rows
3. Go to the characteristic tab to load the csv file corresponding to the trait
4. Go to the entry tab and record the data
5. Export the csv file back to the workbook template
Roslen: We have a client who has a pedigree nursery with 8000++ breeding lines. In other ocassions he could have as much as 10000. Using a dropdown list when not properly designed would cause a serious usability issue. Regarding application configuration, I think that the configuration steps could be simplified by simply loading a configuration file. But still, looking at three different versions of mobile applications that were just presented the obvious becomes even more highlighted: there could be many different design preferences, each one satisfying a target user base but significantly underserving others. I think that any application designed for mobile applications should be properly thought through to cut across a wider user base. The barcoding system I mentioned previously I still think could be a good starting point.
Arllet: There is a cost issue in the implementation of barcodes like the machines, etc.
Shawn: In our experience, the labor cost is much higher
Ruaraidh: Designing the barcode is only one part, doing what’s next is another part
Graham: How are you going to implement the logistical issues in the mobile?
Hector: I think we need to go back to the issue of what needs to be developed
Graham: Can we choose one mobile or do we have to support three?
Jeff: I think we need to decide also what types of handheld we are going to support
Bob: I think we also need to consider the GPS support
Girl: I think we need to support the iPhone and iPad
Graham: It may take a while to redevelop the mobile applications because none of them are developed in Java so my suggestion is to have Hector’s Fieldbook to create a file input for at least two mobiles, the one in IRRI and another to merge the two FiledLogs
Data Access for Analytical and Decision Support Tools
(Hector)
(Rathore)
ICIS Analysis Pipeline
Select Data Source
- chickpea
- groundnut
- peigeonpea
- perlmillet
- sorghum
Other Formats
- text file
- Excel workbook
R DOE Analysis Pipeline with MET Analysis
Sorghum Studies
- Single Location Trials
o Single Environment RCBD Trial
o Single Env. Alpha Trial
o S.E. Strip Trial
- Multi-loc Trials
o Multi-Env. RCBD Trial
o Multi-Loc. Alpha Trial
- Others
o Lab Experiment
Graham: I didn’t see you specified a study
Graham: How do you know which column are the factors?
Abhi: We are creating views from the ICIS database
Person: Are you also considering the incomplete design blocks?
Abhi: Yes, it will be considered
William: Is it accessible outside ICRISAT?
Abhi: Right now it’s local access
Graham: I think we should start accessing the information from the database and returning back the processed output into a study dataset
Ruaraidh: I think we should put the raw data and processed data into two different databases
Cross Prediction – the next decision support tool (Howard Eagles)
Objective: Estimate genotyupic values of known genes from plant breeding datasets
Genotypic values
- expected value of a phenotype in the target population of environments
Extension to genes
P = G + R + E + GE
Operations
- identify genes of importance
- determine if alleles can be identified
An example from current research
- diagnostic markers became available for Vrn-A1, one of genes which determine winter-spring growth habit
Identification of likely variations
Emphasis
Cross predictor
- owned by MPBCRC
- using Glu, Pin Srp allele data (10,368 combinations) predict grain quality outcomes for specific crosses
- written by Guoyou Ye in C#
Cross Prediction
Hector: Do you have any method for missing values?
Howard: Never had missing values for the factors
Trushar: Can we have access to the codes?
Graham: But the methodology is public, right?
Howard: My advice is to rewrite it, use the ideas and rewrite it again
Day 4
Parallel Sessions
Genetic Resources Information Management System
Genetic Resources Information Management - Introduction (Ruaraidh Sackville Hamilton, IRRI)
About SINGER (Elizabeth Arnaud, SINGER)
Q/A
Bob Redden: In the pgr treaty compliance tracking distribution of germplasm, does the treaty provide the info to the Secretariat?
Ruaraidh Sackville Hamilton: Yes. anything sent to anyone directly, for the purpose the knowledge of germplasm. whether or not the country is party to the treaty
BR: This is info not clear to the national genebank ...
RSH: changes were made to the meeting...just go the SMTA...<for further editing>
BR: we sent materials to china...
RH: any materials sent with smta...
BR: requestors were received from china, and these mat were sent to another location
RH: SMTA still applicable to other locations as long as the materials SMTA...
RSH: if you are providing the material with the SMTA... <for further editing>
BR: you mention lim of request 1000, is it possiible to make the request more efficieint to meet ht e criteria to filter the material.. and restricting the search.
Elizabeth Arnaud: not available but this can be implemented
get excerpt from memo audio***
GPG2 Crop Registries and the new web interface (Ruaraidh Sackville Hamilton, IRRI)
Germplasm Information System (Sunil Archak, NPBGR)
Q/A
IRIS-GRIMS Updates (Lilibeth Sison, IRRI)
Genebank Information Management System (GIMS) and Chickpea Crop Register (HD Upadhyahya, M Thimma Reddy and K Vineela , ICRISAT)
Managing and using genotyping data for breeding decisions
ICIS MDMS (Jeffrey Morales for Chengzhi Liang)
- How do you bring genomic data to breeders?
- From Genomics to Breeding Informatics: the Missing Link
- Genotyping data
- SNP data
- NGS sequencing
- What’s to Be New in 2011
- ICIS MDMS schema
- Improved web interface
- Molgenis connected to New ICIS – it can show SNP Matrix and Heatmap
- Matrix Data Management System
- File Types supported by MDMS
- Excel SNP File (IRRI Format)
- Alchemy SNP File
- Excel DART File (IRRI Format)
- Flapjack DART File
- MDMS Data Model
- MDMS Schema - Key Tables, Optional Additions in the Schema
- Miscellaneous Optimizations and Enforcing Safe Integrity Checks
- Future Plans
- Integration of Ontology
- Integrate and enhance genotyping data web interface
- Load other file types: mapQTL, R/QTL, other SNP and flapjack files.
- Others planned or needed
- Q&A
- Shawn Yates – Are these tables are document in wiki for current schema?
- Jeffrey Morales – No, We haven’t document it yet. We will work on it.
- Shawn Yates – Do you have tools to migrate from current DMS System?
- Jeffrey Morales – Migration script will be developed after the workshop and will be update people interested in the schema.
- Graham Mclaren - Matrix schema proposed to have old data, is it proposed to work with existing schema in some way or replace it, For genotypig data how do you achieve consistancy across study or what the role of the old GEMS in this new schema.
- Jeffrey Morales – sample table which has a point in to Germplasm Id that can be linked to the Germplasm database
- Arllet – Allele may be linked to GEMS but markers may not, Marker name is treated as variable name.
- Graham Mclaren - Is there any concept of Allele dictionary
- Jeffrey Morales - May be Chengzhi can answer this.
- Graham Mclaren - What is the relation between matrix schema and original DMS schema
- Jeffrey Morales - Dr, Chengzhi told me that we are going to use this schema for the DMS.
- Graham Mclaren - if that is the case we have to reengineer the dll for one.
- Jeffrey Morales - If this happens we are going to do lot of recording from the tools.
- Hector – when is it going to be on wiki?
- Jeffrey Morales – It is the immediate concern so we will put it on priorities.
- Arllet - Tools to load and upload data?
- Jeffrey Morales – presently supporting 4 or 5 types depending on priority we can develop more.
- Graham Mclaren - schema and loading tools are available now what missing now is in-depth documentation.
- Hector – Main concern we are modifying the schema, trying to generating more tables and change so many things. We should be more careful about what are our goals.
- Jean-Marcel– there is need of genotype product. Store your data for experiment, retrieve during the analysis.
Germinate (Hector Sanchez)
- Sources of Information at CIMMYT – public private partnership
- Germinate-Schema Genetic Information DTMA
- Queries Response Time
- Genetic Data Schema
- Data Handling approach…
- Q&A
- Graham Mclaren - Where is the bulk of data stored?
- Hector – there is the experiment id which is going to connect to the study table, GId to the Pedigree, Marker ID will connect to the Marker Table and another ID will link to the Matrix
- Trushar- Have you looked germinate front end?
- Hector – no, I have just seen the schema.
- Arllet – Do you have loading tool?
- Hector – They are assuming that the information is being managed by database admistrator.
- Arllet - Now it is in PostgreSQL can it be in MySQL
- Hector – yes ofcourse
- Arllet – Germinate and FlapJack was developed by john marshal is there chance to connect them
- Trushar – germinate can generate flapjack file, he has a version that connect directly.
- Hector - I have no Idea, I have not checked that.
GEMS Cat (Shawn Yates)
- GEMSCat holds the information about the markers and alleles.
- We can add, edit or delete a marker using GEMSCat
- Marker assisted selected summary.
ICRIS (Trushar Shah)
- Data repository for Genotype, Phenotype and map data for ICRISAT crops
- Data retrieval based on crops
- Data upload through templates
- Different templates for different kinds of data - markers, rflp, snp and pheotyping
- Quick search
- Query across multiple datasets
- Marker search for particular dataset or for entire species.
- Different export formats.
- Raw or Called Genotyping data
- Details of Marker - search for marker, motifs, cmap id, etc
- Links to cmap and shows map details
- Link from cmap back to marker database.
- Database schema
- Q&A
- Hector - how big is the dataset?
- Trushar - No of columns if more than 256 planning for multiple sheets upload, for use the .xlsx file or use tab delimited file
- Schafleitner - Do you have database of the other center or only ICRISAT
- Trushar - It is presently only for ICRISAT.
- Graham Mclaren - Where do you store the actual data?
- Trushar – It is store in the IntegerArray and CharacterArray
- Graham Mclaren – How is it linked
- Trushar - We have AccessionId like the GID.
- Hector – can you have an experiment with duplication?
- Trushar - Yes, we can retrieve the duplicates from one or more datasets.
- Valente - where is data stored, is it ICIS?
- Trushar - No, ICRIS is a parallel development going on at ICRISAT.
- Graham Mclaren - How are you storing the phenotyping data.
- Trushar – for TL1 we are using ICIS for storing , but ICRISAT phenotyping data is in ICRIS.
- Jean-Marcel - Do you plan to move the data available in to ICIS
- Trushar - yes we could we have already started moving like traits names, but what do we do with genotyping data that’s the question.
- Graham Mclaren - are you very happy with this system performance, fast loads
- Trushar – yes, it still needs some development in terms of loading protocol, as you get larger marker data sets, we need to work on it, we did do some bench marking. The performance was good enough.
- Arllet - Is MBDT connected to ICRIS
- Trushar – no
- Jean-Marcel - very impressed, focus on the small scale for now. Very valuable software.
- Hector - We can include a GID that connects the genotype and phenotype data
- Graham Mclaren - Hector, do you start if you have a protocol
- Hector - Yes, but only if I have genotype not for genome wide selection.
- Graham Mclaren - if we have setgen list which I want to see all the loci
- Trushar - Yes, there is option to upload the list of accession form the text file.
- Graham Mclaren - Can we build a standalone application with java interface in RCP eclipse that that all the functionalities searching for Germplasm. So that it can be installed in different computers.
- Hector - yes, it can be done because it was done in java, so it is not operating system dependent, but it takes man power.
- Yates - 3 years back we have commented on the requirements for GEMS May be we can find it on wiki.
Schema Changes
Scale Conversion (Hector Sanchez,CIMMYT)
ICIS 5.6 Schema changes required to handle events (Ruaraidh Sackville Hamilton,IRRI)
Schema 6 Integration of Ontology IDs (Arllet Portugal,GCP)
DAY 5
ICIS Datasource (Jeffrey Morales):
Discussion:
- ARLLET: Those who are implementing ICISDataSource, can anyone else follow the guidelines?
- JEFF:Yes one group which are learning about it, but still takes some time
- TRUSHAR: They attending IRRI to understand the data source, but due to various issues, they have not continued to learn it
- JEFF: the problem is change
- ARLLET: did it work for CMap
- TRUSHAR: Yes
- JEFF: It is not that hard, you can do whatever you want. It is up to the developer.
- GRAHAM: 2 issues – how far have we got into a multi-database data source, how far with a write database datasource
- JEFF: we can use it, but no interface. Lack of programmers to work on it.
- Graham: cannot continue with Fieldbook until we have a good datasource
- HECTOR: need to deliver the Fieldbook by end of June. After that, when the pressure is less, we will work together to allow work with Jeff to create a common datasource. We need to get this for the breeders to use. We are separating the GUI from the middleware and just making the calls to the database directly.
- GRAHAM: Trouble is Hector will have to code data access routines, etc.
- HECTOR: already had to do this for the fieldbook, so not concerned.
- GRAHAM: what language?
- HECTOR: Java, so it is not a concern.
- GRAHAM: We do need to plan how we will have a single data access layer. This problem rests with the team at IRRI, so we need to hear from IRRI how they plan to get to a point where we can meet the needs for the groups, such as writable data access layer regardless if it is a priority for IRRI or not.
- ARLLET: Chengzhi has to answer this, not Jeff. How did you do the writable datasource?
- JEFF: Guy did this
- GRAHAM: Do you have all the code?
- JEFF: Yes
- ARLLET: you have the code for the searching?
- JEFF: Yes, we are using Pojos for the inserts
- GRAHAM: I noticed that Zeus interface that really IRIS DMS has to be overhauled. Too much confusion and it comes out very clearly when using this interface. Ruariadh, what resources do you have to do something like this? It needs to be stripped out, and reloaded with more consistency.
- JEFF: they could list their priorities with me on what needs to be done. You don’t need to look at this as a whole, just bit by bit.
- GRAHAM: but somebody needs to be responding, that’s the point.
- TRUSHAR: iPlant exchange of the middleware to see where they fit in
- GRAHAM: they wanted this to see what they wanted to use, not help with. Idea is to use standalone tools as prototypes to design what the web interface should look like. The iPlant is developing a middleware layer to integrate tools and we just showed what we had. Their infrastructure is not as mature as we hoped, so we can’t build the CWS yet. We will move much slower with iPlant maybe a year before we see tools from that. This year we need to show tools.
- TRUSHAR: middleware for both standalone and web should be the same
- GRAHAM: they will use the same data access layer, I am sure, but not sure where it will fit into their infrastructure
IBFB Data Layer (Java) (Hector Sanchez)
Discussion:
- GRAHAM: Some feedback from the group was that everyone wants to wait until they see the fieldbook working with their own databases. Need to separate the fieldbook from the database. How easy is this to change? Is it just in the XML?
- HECTOR: We have the XML file and the connectivity to the database is done in here. MySql can handle different databases. I am considering moving this set up to the port 3311 instead of port 3306, so we don’t disrupt a user’s normal installation. We need to think about this and decide what is easiest for users.
- GRAHAM: I think the most common request will be to use their existing database. The install now works, but the next option should be to use the existing database where people are putting things in. I think we will find when we meet with the tropical legumes people, that they already have their own databases.
- HECTOR: Not a big problem.
- GRAHAM: Another problem. Many people are most comfortable using access database and small databases…
- HECTOR: Simple. We can use the odbc/Jdbc connect. We need to check this, but I recommend moving to mySql. However we can look for a solution.
- GRAHAM: Any comments from anyone?
- ??: Yes, I agree with Hector. It needs to move to something better than access.
- JESPER: we have this XML that seems to be fixed on port number and database, we should have a utility that can read the ICIS .ini file and then set up the XML file according to that.
- GRAHAM: That would be the ideal solution. Users can make their lists with SetGen or whatever and make their fieldbook.
- JESPER: Ideally, there should be some layer before the middle layer to check what this .ini is and connect to something different.
- HECTOR: I call this a patch. I like the .ini file, however, I have the feeling that there are smarter solutions but this is the cheapest.
- ARLLET: The reality is that these databases are already installed on users’ machines, and we need to accommodate them.
- HECTOR: I am aware, and this first prototype was just to get some feedback.
- HOWARD: I have all this data in MS Access; it must be trivial to convert these databases. The work is creating the access database from hundreds of Excel spreadsheets.
- GRAHAM: Yes.
ICIS32.DLL: Dynamic Link Library (Warren Constantino)
- All stand-alone tools use the ICIS32.dll
- Tools read settings from the ICIS.ini file and the their behaviour conforms to those settings
- ICIS32.dll is used to access the databases - makes use of the ODBC datasource
- In order to improve the performance of the tools, optimizations were made to the code in the tools, and in the ICIS32.dll
- -Made use of sorting statements in the .dll
- -Combined functions
- SQL statements are standard so that they work across all database formats, however some functions had to be set up specifically to handle the differences (MySQL, Oracle, etc)
- The functions return only one record at a time, which slows things down. This was done to support the local and central
- Warren was looking for a better option, but has been restricted to work on the user interface.
Discussion:
- ROSLEN: This has 10+ years on thinking of behind it and for new developers, it helps others who are attempting to create software for ICIS. At Perth last year, it helped me to have an opportunity to talk to the developers of it. By having it published properly, it helps developers. Minor changes, such as connection pooling can improve performance. We were discussing this among the IRRI group – that there are things that can be done to improve the efficiency and help others to come up with better tools and use cases.
- GRAHAM: Thanks for those comments. That is the spirit of the .dll – to take the hard work out of the data integration and data access. There is a complexity and some of the complexity is the cause of the slowness. Why are we talking about the .dll? 1) We will still be using it for a long time, for example, Hector is building a big list of tools that will not be ready for some time. We need somebody to actively be maintaining it and improving it. 2) When you implement it to deal with a new db system, there are slight inconsistencies between SQL statements. Rauraidh has decided to implement ICIS in PostGres to stay consistent, but will cause some problems in the .Dll and someone needs to handle this. We have discussed how to find the manpower to do this – it is not a difficult task, but somebody needs to be coordinating it, checking that it is consistent with the rest of the functions and making sure it gets into the CVS so it is not lost. Jesper, you needed some .Dll enhancements, and Warren said no one could do it.
- JESPER: Problems just retrieving one record at a time, so we were looking at ways around this. The querying is not making inner joins, which makes things complicated. We make one rule, and then it speeds up the performance. In terms of handling local and central in java, we invent something new so we can represent that going into the consistency layer for Hector.
- GRAHAM: Okay, we have the system available to improve the .dll provided we have somebody to watch that it is documented and consistent. I don’t know how this has been maintained. There is also a lot of lessons to be learned from the .dll for the java team to build from for the jar file. We need to see how this is being done, and not on an adhoc basis.
- HECTOR: ICIS came from IWIS years ago, and I started with Graham and Arllet on this, so many discussions so I know how to handle all the functions needed. There were so many groups involved with this file. We are not throwing anything away but we are trying to apply the new technology to make life easier for the breeders.
- ROSLEN: I agree with all points. My position comes from a point of efficiency. Small choices can make a difference on the performance. Users will not care about the interface if they have to wait long times for data. I would like to look at the codes and suggest improvements or functions. Not rejecting functions or databases, but rather how the codes interact with each other. My initial response to ICIS was not good. I tried to avoid it as much as possible, but now I need to look at it to ensure proper germplasm data entry into ICIS. But we should be able to make codes run more efficiently.
- JESPER: You can set up some kind of layer that accesses the central database on a stable environment that you know and then speed up the queries by storing procedures and functions. This might optimize the searching. If the function is not stored, then go to the .dll.
Using ICIS for Chickpea and Lentil (Bob Redden)
- Installation developed for genebank by Ranjan Balachandra
- I would like to see extension of ICIS to link with CIMMYT and IRRI across our other crops
- Set up as a 4 year project but only finished the chickpea
- Linking data from ICARDA, ICRISAT, IRRI, USDA, CGN so it would be nice to upload all this to a world genebank online.
- ~40 000 Chickpea accessions and ~14 000 lentil accessions into the database
- Have multiple locals from each organization and would like to see a central GMS created from these to create a World-wide system. Would like to see the CGIAR do this, as we do not have the resources.
- Lentil data primarily came from ICARDA, chickpea came from 4 genebanks
- Initially had the idea that the same accessions could be at USDA, ICARDA and ICRISAT, so to have a database for Australian lines, they need to import data from the other genebanks. Found it was easier to download the database than to pull out the pieces one by one. Decided if that work was needed, then why not create a global database.
- ICARDA worked with us, but ICRISAT was not using ICIS so we were not able to get much information from them, hopefully that has changed with ICRISAT starting to use ICIS.
- Set up a web-based search engine (January 2005).
- Allowed for searching of data and pedigree information
- Allowed for searching and graphical representation of data
- Idea was to allow the user to go to a shopping cart and request the accessions online
- Requests for accessions would be directed to the proper genebank (USDA, ICARDA, etc).
- This function was not developed, but should be able to implement without too much trouble.
- Envisioned that ICRISAT or ICARDA would host the Central databases.
- Web-based UI –fits with ATFCC’s vision to be key part of an international group
- Using Apache, MySQL
- Where GPS data is available, would like to search for material that might be cross tolerant/heat tolerant in terms of where the line was collected. We were hoping to implement this.
Discussion:
- GRAHAM: So, if I understand, you have lots of information in your database that is not in your genebank.
- BOB: Yes.
- GRAHAM: Who is using this most of the time?
- BOB: We are. We gave presentations to breeders and others and received feedback. We just got the database on the web this year on a new server. I’ve tested it around Australia with different breeders so we are still just firing up. The chickpea and lentil breeders know it is available.
- GRAHAM: This was the first to try and implement accessions from different genebanks. There have been debates about how to do this and Rauraidh has spent much time doing this in the GMS. You were ahead of the curve here.
- BOB: If you are looking to select materials, you can set the filter to be as stringent as you want, then brings up a list of accessions with 4 synonyms at each genebank. Also, on any one of those, click on it and access the passport data and the observations for any of the chosen traits across studies and genebanks.
- GRAHAM: By name this was integrated.
- BOB: Yes. Now something will be listed as a separate accession but find out later that it is the same as another one. If we had the full distribution information, we could have figured that out. We have some duplication in our own database where they have been given a different GID and Accession number.
- GRAHAM: The other approach would be to link to neighbourhoods of germplasm but it is a lot of work.
- BOB: Yes, and we have been busy trying to meet deadlines but there is more to do.
- ??: I was asked by JS unit about whether we can provide GPS information to map material into specific regions to do GxE analysis. Is there a way to enter this into ICIS?
- GRAHAM: There is a GEOREF table that contains Lat and Long points. Rauraidh has spent some time improving this table. The hard part is getting this data.
- RAURAIDH: I looked into this table with William but it looks like the GPS points are pointing to the LOCID. If I state that a certain point was at this particular LOCID, does this mean that this was where it was created, collected, etc.? This is where you need correct GID. You need a different Method and GID and probably a different LOCID depending on the situation.
- BOB: We might have specific GPS to the spot where the line was grown, or as vague as the Province, village, etc. So there would be a level of accuracy attached to the GPS coordinates. Sometimes the sample can be collected at a market in a village, so you don’t know where it originally came from. We had questionnaires to ask the farmers questions to establish whether the sample is a traditional land race or brought from somewhere else to establish the accuracy.
- RAURAIDH: You talk about levels of precision. We use a system that will convert a text explanation of location to a longitude and latitude.
- HOWARD: Can you have a tool to check the genetics when you start to do typing? Something that goes through and actually tries to match them.
- BOB: With whole genome sequencing, we should be able to do this, but it should be put to the bioinformatics community.
- HOWARD: There is a big computing problem here.
- BOB: Yes, but with costs coming down it should be able to be done.
- ??: how do you use Maintenance ID to manage the Accessions in a genebank.
- RAURAIDH: It is documented on the wiki. No one seems to be using this concept; instead people are using GIDs to represent accessions.
- BOB: We have grown all samples first in quarantine, and then into the long term storage. We seed increase another sample, which goes into the active genebank and is regenerated after requests. After 5 recycles or more, we would sample again from the base collection to avoid errors in genetics. It is impossible not to have genetic drift over time.
- GRAHAM: Every sample in the genebank gets a different GID and is linked by the Management GID. The all link back to the original sample. That is the short answer. It is a bit more taxing to implement.
Annotation Tool for Ontology (Luca Matteis)
- At the workshop to get information about how the Annontation Tool will work with ICIS for ontology
- Used to help search for ontologies
- Not dynamic – hard to maintain
- Adding new ontologies is very difficult
- Scripts are run to maintain the database
- Provides limited information on the terms (no images, etc)
- Idea is to create a new system in parallel with the old system
- More user friendly and community driven
- Web interface to add information
- Need to have a way for ICIS to make use of this system
- Need a tool to map the TRAITID to the proper Ontology through the Annotation Tool
- Demonstration that shows how an Excel file can be searched for the correct ontology for each term. Eventually you would have images and more information that would be loaded from the API over web services. You could even submit a list of terms to be loaded into the Ontology database.
Discussions:
- ELIZABETH: What was not shown is that you can download the terms that are found. Bold terms were found, non bold terms are not in the ontology.
- GRAHAM: So you could take a template of trait dictionaries and run the whole thing down, and see the ones that are not matched.
- ELIZABETH: Yes. You do need to validate the suggestion made by the tool, however. It is building on the work that has been done in the past.
- GRAHAM: Are you comfortable to expanding from an Ontology concept to a trait dictionary concept.
- MILKO: Absolutely.
- GRAHAM: We have assumed that every trait has different scales and methods.
- MILKO: That is something we need to think about. Ontology can carry many scales, but how does ICIS support it?
- GRAHAM: ICIS does this, which is okay. Hector is working on a component to convert the scales.
- HECTOR: I am going to suggest that we work closely with Rosemary to better develop this.
- ELIZABETH: This search is from the OLS which is currently in 3 different servers. We need to figure out where this can be stored now. Even if it goes on the cloud, we still need to find some organization to host the main database.
Rejuvenating Communication/Collaboration (Graham):
- Developer Meetings are so important so we can put a human face to the names and help each other out.
- ICIS Wiki is where all the technical documentation is placed
- Curated by many people from the community
- Need to have an account – Veritas Salazar
- Will not be decommissioned, despite the email that was sent.
- Works just like WikiPedia
- Workshop minutes and presentations
- Technical Documentation – tool manuals, schemas, support information, tutorials.
- The system works: Ex) an Israel company used the wiki to re-engineer ICIS in a web-based system, which is now, oddly, proprietary.
- In 2010, IRRI developed a more user-friendly webpage for users that is less technical.
- Users cannot contribute to it directly, need to contact Warren.
- Needs to have a link to the ICIS Wiki
- CropForge is a software site to provide tools, and provide support to developers
- Has source control for developer’s code on the tools
- Divided by projects with ICIS Communication being the hub for discussion between developers.
- New users can contact Warren to become a member
- Trackers to deal with bugs, feature requests and support requests
- User can also upload images and videos to go with the reports
- All tools can be downloaded from this site
- Crop communities (Rice, Wheat) have also created projects in order to discuss issues related to the ICIS Installation for those crops
Discussion:
- ??:How to start working with a new species in ICIS and how to name the new implementation?
- GRAHAM: I have set up some notes on the ICIS Wiki to help new users, so that is a good place to start. The ICIS community is another lead. But you need to decide on many conventions such as how to properly curate your germplasm in the GMS. Also decide how much historical data is needed vs. starting today. That decision depends on how managed the historical data is. Data in databases are very easy to migrate, but data in excel sheets is a problem.
- ??: How we communicate depends how where we want to go. In the last 6 months, ICRISAT has decided to standardize on AgroBase instead of ICIS. How has the commercial side impacted ICIS?
- GRAHAM: Agrobase predates ICIS, which is still the principle contender, there is Prism and Katmandu.
- HOWARD: Katmandu hasn’t died but there is not much active development. One person was looking to use the system but decided to go to Agrobase instead.
- GRAHAM: We invited the Katmandu group to collaborate but it didn't happen. Agrobase has improved and is quite a good product, but the license fees are expensive so you can’t share it with your partners. The database is closed enough that you cannot share data very easily. We are looking at deriving a tool to export to ICIS, but there are still issues. It still has no molecular data capabilities.
- TRUSHAR: None. It handles it as a trait.
- GRAHAM: Maybe we could do more to make management see us more as a contender. Nunhems (a part of Bayer Crop Science) looked closely at a number of private and open source systems and decided ICIS was the best choice for them. They have now built a new system based on ICIS schemas. If you buy a tool, you have to work around their constraints.
- ??:Any thoughts to Google Apps?
- GRAHAM: Yes, there are some possibilities there, if you could identify some apps we might be able to use.
- TRUSHAR: It is important to identify a person for the new users to send questions, bugs, etc. to.
- WARREN: I need to clarify this with Goyou.
- TRUSHAR: We are also supposed to be support for South Asia but we are not at the stage where we can do this without help.
- GRAHAM: Yes, and Sam is also in the same position for West Africa.
- TRUSHAR: We are not directly involved with the development, so we cannot answer the questions when they come in. It would be nice to streamline this and find out who to contact.
Future Plan - Graham
Discussion
PARTICIPANTS
| No | Name | Affiliation |
| 1 | Eagles Howard | School of Agriculture Food and Wine, Australia |
| 2 | Redden Bob | Australian Temperate Field Crops Collection, Australia |
| 3 | Akintunde Akinnola | Africa Rice Center, Benin Republic |
| 4 | Marimagne Tchamba | Africa Rice Center, Benin Republic |
| 5 | Clarke Fran | Semi Arid Prairie Agricultural Research Centre, Agriculture and Agri-Food Canada, Canada |
| 6 | Yates Shawn | Semi Arid Prairie Agricultural Research Centre, Agriculture and Agri-Food Canada, Canada |
| 7 | Hongsheng L | China |
| 8 | Franco Duran Arturo | CIAT, Columbia |
| 9 | Valente Fabio | INRA, France |
| 10 | McLaren Graham | CIMMYT, Mexico |
| 11 | Norgaard Jesper | CIMMYT, Mexico |
| 12 | Portugal Arllet | CIMMYT, Mexico |
| 13 | Ribaut Jean-Marcel | CIMMYT, Mexico |
| 14 | Rosemary Shresta | CIMMYT, Mexico |
| 15 | Sanchez Hector | CIMMYT, Mexico |
| 16 | Edwin RP | CIP, Peru |
| 17 | Bansal Kailash Chander | NBPGR, India |
| 18 | Singh Manoj Kumar | NRCPB, India |
| 19 | Elizabeth Arnaud | Bioversity International, Italy |
| 20 | Matteis Luca | Bioversity International, Italy |
| 21 | Milko Skofic | Bioversity International, Italy |
| 22 | Bakare Moshood | IITA, Nigeria |
| 23 | Ofodile Sam | IITA, Nigeria |
| 24 | Caneda Alexander | IRRI, Philippines |
| 25 | Chengzi Liang | IRRI, Philippines |
| 26 | Constantino Warren | IRRI, Philippines |
| 27 | Hamilton Ruaraidh | IRRI, Philippines |
| 28 | Lilibeth Sison | IRRI, Philippines |
| 29 | Morales Jeffrey | IRRI, Philippines |
| 30 | Pimentel Clarissa | IRRI, Philippines |
| 31 | Resurreccion N | IRRI, Philippines |
| 32 | Anacleto Roslen | IRRI, Philippines |
| 33 | Victor Alcantara | IRRI, Philippines |
| 34 | William Eusebio | IRRI, Philippines |
| 35 | Ye Guoyou | IRRI, Philippines |
| 36 | Khaled El-Shamaa | ICARDA, Syria |
| 37 | Easdown Warwick | AVRDC, India |
| 38 | Nair Ramakrishnan M | AVRDC, India |
| 39 | Rattunde H Fred W | ICRISAT-Bamako, Mali |
| 40 | Sissoko Ibrahima | ICRISAT-Bamako, Mali |
| 41 | Dar William | ICRISAT-Patancheru |
| 42 | Deshpande Santhosh P | ICRISAT-Patancheru |
| 43 | Gaur Pooran M | ICRISAT-Patancheru |
| 44 | Gowda CLL | ICRISAT-Patancheru |
| 45 | Hoisington David | ICRISAT-Patancheru |
| 46 | Hash C Tom | ICRISAT-Patancheru |
| 47 | Mallikarjun Goli | ICRISAT-Patancheru |
| 48 | Prasad PVNS | ICRISAT-Patancheru |
| 49 | Rathore Abhishek | ICRISAT-Patancheru |
| 50 | Ramu Punna | ICRISAT-Patancheru |
| 51 | Reddy T Praveen | ICRISAT-Patancheru |
| 52 | Riera-Lizarazu Oscar | ICRISAT-Patancheru |
| 53 | Shah Trushar | ICRISAT-Patancheru |
| 54 | Sravani Mana | ICRISAT-Patancheru |
| 55 | Srivastava Rakesh | ICRISAT-Patancheru |
| 56 | Thimma Reddy | ICRISAT-Patancheru |
| 57 | Tuteja Reetu | ICRISAT-Patancheru |
| 58 | Upadhyaya Hari | ICRISAT-Patancheru |
| 59 | Vales Isabel | ICRISAT-Patancheru |
| 60 | Vadez Vincent | ICRISAT-Patancheru |
| 61 | Varshney Rajeev | ICRISAT-Patancheru |
