Differences between revisions 18 and 135 (spanning 117 versions)

USER MANUAL FOR THE IDEA EBV INTERFACE

Preface

The following is a manual to guide the user through the features of the new IDEA. IDEA stands for Interbull Data Exchange Area. IDEA is a restricted area accessible only to member countries through the Interbull website.

Contents

USER MANUAL FOR THE IDEA EBV INTERFACE

Software menu

The Software menu gives you access to the Interbull checking programs. By clicking on Software a drop down menu will open and you will be able to choose the type of checking program you are interested in, i.e. Pedigree or Proofs. Under Software you will also find information on the programs and instructions on how to run them.

Proof's Checking Program

A Python program called CheckProofsPara.py will check the 300/700 proof file format and the associated 301/701 parameter file for format correctness against IDEA (Appendix I+Appendix II.) The program prepares a zip file, IB-ORGCODE-IG-yyymmddThhmmss.zip for conventional MACE and IB-ORGCODE-GG-yyymmddThhmmss.zip for GMACE,if no errors are found in the file. The zip file represents your checked data file to upload to the Interbull Centre IDEA database (https://idea.interbull.org/).

The user instructions (https://wiki.interbull.org/public/CheckProofPara#preview) for the CheckProofsPara.py program and file formats (see Appendix I and II) give details on how to run the program and on the checks performed.

Proofs menu

The Proofs menu gives you access to the main proofs functions which are:Upload, Review, Messages.

Upload and Verify program

By uploading data in IDEA users will no longer need to run the Verify program prior sending data to Interbull Centre. The Verify program will, in fact, be run automatically in IDEA during uploading.

Uploading functionality for parameter and proof files are available under 'Proofs/Upload'. The only file accepted by IDEA is the zip file IB-ORGCODE-XG-yyymmddThhmmss.zip produced by the CheckProofsPara.py, no other files are accepted. Once in 'Proofs/Upload' users will be able to browse the appropriate file and upload it by click on "Submit query".

The uploading of data will not happen on real time but data will be set on a queue and processed on a later time. Right after clicking "Submit query" a message on the screen will display the amount of parameter and proofs records submitted. A confirmation email will be sent to the email address associated to the user that has uploaded data.
The following are the steps the data will go through during uploading:

CheckProofsPara.py will run once again inside IDEA to assure correctness of format and contents
The Verify program will check your data against the previous one available.

Confirmation email

Either an "EBV upload success" or an "EBV upload error" email will be sent to you upon completion of the uploading process.

UPLOAD STATUS

IDEA will soon remove the ability to receive feedback through email, and instead switch to a dedicated page where you (and other members of your organization) can view all feedback relevant to your organization. This will be in the form of a table, where you can look back at current and past feedbacks, download the relevant files and so on. There will also be an RSS feed available, so you can set up notifications of new feedback as it becomes available, in a way similar to previous email feedback.

Communications from IDEA regarding the status (success or failure) of all type of uploads performed by your organization (pedigree, EBV, GEBV, performance, animinfo) are now listed in here and divided between “Successful” or “Failed” uploads. The information displayed are from a RSS feed system. Here the steps you shall do in order to display such feeds correctly:

Go to Upload Status
Select “Successful” (or Failed)
Click on “Create subscription feed link”
Click on “Copy subscription feed link”

If you think the link has been leaked/spread (remember that the link is supposed to be personal), you can remove it at any time by clicking on “Remove subscription feed link”. If that happens you shall create and copy a new subscription feed link.

With this new feedback system, information on all type of uploads will be available to each and every members of the same organization and not only to the person making the upload. Moving away from an email based feedback methodology and to an RSS based one, will make the feedback more stable and reliable, without risk of spam or mail server problems. Also, it makes it easier to programmatically handle feedback from the IDEA system.

You should copy your subscription feed link inside your email provider. Here is how you do it if you are using Outlook:

Right click on File
Right click on “Account Settings” from the Account Settings drop down menu
In the new window that opens click on “RSS feeds”
Click on “New” and paste the RSS feed link you have copied from IDEA.

Using Thunderbird:

Right click on the News Feed folder and choose Subscription (you can also add a sub-folder and add the subscription there)
Add your Subscription link into the Feed URL text box in the Subscription popup box and click Add

For the time being, IDEA emails and RSS feeds will co-exists to give time to all IDEA users to get acquainted with the RSS feed. From 1st January 2023 IDEA will provide solely feedback based on the RSS feeds.

Upload successful

If the uploading is successful the "EBV upload success" email will list some basic information on your data such as:

Data connected to runid : 1311r
Number of records read from the parameter file : x
Number of records read from the proof file : xxxx
Number of rows in parameterfile associated
with proofs : x
Number of flagged rows in proof file : x
Number of animals found by real AID : xxxxx
Number of animals found by alias : 0
Number of animals not present in the pedigree : 0
Number of records loaded from the file per combination :
RDC-DEU-cc1: xxx
RDC-DEU-cc2: xxx
RDC-DEU-crc: xxx
RDC-DEU-hco: xxx
RDC-DEU-int: xxx

The email can contain up to three different WARNINGS. The three first warnings below are associated to attachments and requires actions which are described in the actions for attachment table below:

Warning: [n] animal(s) were referenced in your file, but not present in the pedigree database. These animals were discarded! See a detailed list in the attached 'missing_animals.csv' file.
Warning: [n] animal(s) needing updated pedigree records detected! See 'ped_needed.csv' for a complete list of the animals.
Warning: Use of aliases detected! See 'ped_alias.csv' for a complete list of the animals.
This warning is generated if the uploaded file contains re-uploadings for a given Breed-Pop-Trait combination which have not been withdrawed first (see Withdraw data):
- WARNING: This dataset re-uploads already existing data, of which some has not been withdrawn first. These combinations have been skipped; the proper way to re-upload combinations is to withdraw them first (or ask IBC staff to reset them). These combinations have been skipped:[......]

An important source of information comes from Number of flagged rows in proof file: this number represents the amount of discrepancies found by the Verify program. If it is 0 (zero) it means that no discrepancies have been found in your data. Please note that the data IS NOT AUTOMATICALLY submitted. After uploading to IDEA the data has to be submitted in the Proofs/Review page. If it is not 0 (zero) it means that the Verify program has found some discrepancies in your data compared to the previous one available or the system has detected animals with missing pedigree. To double check the data you need to log in IDEA and go to Proof/Review.

Actions for attachments

Animals not present in IDEA pedigree or lacking pedigree information (i.e. present in the pedigree database but with sire and dam unknown) will be excluded from the international evaluation.

Table 1 summarizes the action needed upon reception of a confirmation email with warnings and/or attachments:

Attachments	Meaning	Action Required	Consequences
missing_animals.csv	The animals listed in this attachment are not present in IDEA pedigree	Log in IDEA, go to Proofs/review, withdraw the data file, prepare a file200 for these animals, upload the file in IDEA pedigree, after reception of the pedigree confirmation email upload again the proof file	If pedigree is not provided, animals are excluded from the international evaluation
ped_needed.csv	The animals listed in this attachment have sire/dam set to unknown	If you have pedigree information for these animals: log in IDEA, go to Proof/review, withdraw the data file, prepare a file200 for these animals, upload the file in IDEA pedigree, after reception of the pedigree confirmation email upload again the proof file. If you don't have such information you can submit the file: you will be asked to write an explanation	If pedigree is not provided, animals are excluded from the international evaluation
ped_alias.csv	The animal IDs listed in this attachment are alias IDs	You are requested to update your own database with the correct animals' IDs.	Alias IDs are automatically switched to their corresponding official IDs.
Refer to the section 'Submit/Withdraw data' for more information

Upload not successful

There can be three main reasons for an uploading to fail, the different reasons are descripted in the specific "EBV upload error" email you could receive. The table below presents and explaine each of the possible cases:

Email content	Meaning	Action Required
'System internal error. This should usually not happen. Contact staff for clarification'	Technical problems with the database server	contact Interbull Centre
'The data already exists in the database and no combinations were updated due to not having the right status for re-uploading; the proper way to re-upload combinations is to withdraw them first (or ask IBC staff to reset them).'	The content of the file was not uploaded	Log in IDEA, go to Proofs/review, withdraw the specific breed-trait combinations present in the file re-upload the file
'Formal correctness errors were encountered in the uploaded file'	The file was not uploaded due to format errors	Review the file with the CheckProofPara re-upload the file

Review your data

General

Uploading represents only the very first step for submitting your data for an IGE. Here is a description of the actions you need to follow to submit your data for an IGE. The Proof/Review page contains all the information you need to review and submit your data for a IGE.

The page contains several important information grouped into three different tables: Filters table, Central (Include) table and Actions table

Filter table

The Filters table allows you to filter and display the information you have uploaded in a way that better suits you. By default the Central (Include) table displays all the information associated to your account but you can costumize the displaying of information choosing among:

A list of the different breeds you have uploaded data for
A list of the population for which you have uploaded data for
A list of the traits uploaded
A list of the different status of the data uploaded
A list of the data set codes you have uploaded

Clicking "Reset all" will erase all your previous filters and display again all the information associated to your account.

Central (Include) table

The Central (Include) table displays as many rows as the breed-pop-trait combinations you have uploaded. Each row shows the following information:

Status: refers to the status of your data. There are a total of 5 different status: Pending,Submitted, Withdrawed, Accepted, Rejected
Flagged: refers to the outcome of the Verify program. Value for this column are either YES or NO. Breed-Pop-trait combination flagged YES are also highlighted in yellow.
C/G: refers to the nature of your data, Conventional/Genomic.
Datasets: refers to the dataset used to upload a given Breed-Pop-trait combination
Reports: for each Breed-Pop-trait combination you get access to a "Brief" and a "Full" output of the Verify program. The "Bulls" report lists all bulls highlighted by the Verify program.

The Breed-Pop-Trait combinations not highlighted and with Flagged=NO are combinations for which the Verify program did not find any discrepancies, nevertheless they will still get a status= Pending as your final approval on the data is still required.

The Breed-Pop-Trait combinations highlighted in yellow require your attention. For each of them you need to check the Verify output. For your convenience, selecting such combinations and clicking on "View Summary for Selected", located at the end of the Central table, will display in a new page the main key checking points of the Verify output. You can select the combination either manually, by clicking in the box in the "include" column, or by clicking on "Reverse selection".

If you wish to save the full verify output as pdf, here are the steps to follow:

Open the verify output
Open the application menu from your browser
Click on "Print"
Under the 'Destination' section, click Change... > Save as PDF

Actions table

The Actions table displays the options you have for your data with status=PENDING. You can decide to submit or withdraw such data by selecting the desired action and clicking on "Submit".

Submit data

If, after checking the Verify output ,you consider your data to be ok and want to include it in the IGE you can do so by selecting the box beside each Breed-Pop-Trait combination you want to include, select the action "Submit" and click on the "Submit" button.

Every time you submit a pending data you are required to explain the reasons for the discrepancies found by the Verify program. If the reason are breed-trait dependent you need to processed these cases one by one and provide the full explanation in the designed space "Change comment". If, on the other hand, the same explanation applies to several Breed-Pop-Traits combinations you can select them together in the Review page so that you will be required to write only once the explanation that all these data share.By clicking "Send message" your message will be recorded under IDEA Proofs/Messages and will be visible by you and the Interbull Centre Staff.

In the Review page, the Breed-Pop-Trait combinations you have submitted will be now displayed with status=Submitted.

On the day of the data submission deadline for a given IGE, routine or test run, all your data in the Review page should have status=SUBMITTED. Your aim is therefore to check all pending data and either provide explanations or withdraw and upload new datasets before the data submission deadline.

Withdraw data

If you realize the data you have uploaded for some Breed-Pop-trait combinations is wrong or you want to upload pedigree information for the animals reported in the confirmation email attachments you need to withdraw your data before doing anything else. You do so by simply selecting the affected Breed-Pop-Trait combinations in the Review page, select the action Withdraw and then press the button "Submit".In the Review page that given Bredd-Pop-trait combination will now have status=WITHDRAWED.

When you withdraw a given Breed-Pop-Trait all records present in IDEA for that combination are deleted thus you have to re upload the file before the data deadline. It is not acceptable to submit a partial dataset in order to correct the evaluations of some subset of bulls. It is essential that proof records for all bulls be included in the same file for any one brd-pop-trt combination whether the dataset is a first submission for the IGE run in question, or a re-submission with some problem corrected.

If you fail to withdraw all the traits you are going to re-upload in a new file, the confirmation email will contain the following warning:

WARNING: This dataset re-uploads already existing data, of which some has not been withdrawn first. These combinations have been skipped; the proper way to re-upload combinations is to withdraw them first (or ask IBC staff to reset them). These combinations have been skipped:[......]

Example:
You notice some problem with the temperament data for RDC. You withdraw only the RDC-Tem from your Review table and in the new file to upload you also include data for RDC msp. As you did not withdraw the RDC-msp combination before uploading the new file, all the combinations referring to RDC-msp are skipped as data already exist in IDEA for that given combination.

Accepted/Rejected

Once you have submitted your data for a given IGE it will be up to the ITBC staff to finally accept or reject it. In general all data with FLAGGED=NO will also get accepted as there are no issues pending on these data. For all the data with FLAGGED=YES, ITBC staff will go through the explanations you have reported and if found sufficients will mark that given Breed-Pop-Trait combination as accepted. If more clarifications are needed ITBC staff will email you via IDEA, anytime a new message will be posted in your IDEA account by the staff you will receive an email informing about the presence of unreplied messages in IDEA.

In case the discrepancies found are considered too big then ITBC staff can reject your data and use the data from the previous run. You will be able to follow what happens to your data by looking at the Review page as the status will change according to what decision has been made.

Messages

Proofs/Messages represents the place where all your communication with the ITBC for a given IGE are displayed. All the explanations you provide for a given Breed-Pop-Trait combination are automatically listed in here. Unreplied messages will be marked in red. You can use this page to send further messages to ITBC staff. When ITBC staff replies, their messages will also be listed here and you will be notified by email about the presence of unreplied messages in IDEA.

Message Overview

In the message overview you can review all the communications you had with the staff for any given evaluation.

By selecting the evaluation you are interested in and pressing "Set Run", all messages exchanged between you and the ITBC staff during the evaluation you have selected will be listed in a table.

Proof Query

Proofs/Proof Query gives you the possibility to look up proof records associated to a given animal ID; you can filter the search to a specific runid or retrieve the information available for that ID in all the run id available in the database as well as filter it by population, traits or trait group. Once you have selected your filters and press "Search" two tables will be displayed: an animal information table showing the pedigree for that animal and a second table with information on: run id, breed, population, trait, type of file (Conventional or Genomic), type of proof, publication, status of bull, number of daughters, number of herds, EDC, reliability and proof. The heading of the second table is an active heading, this means that by clicking on it you can sort the content of the table.

Through the Proof Query you will be able to look up only the animals that have been submitted by your organization.

Parameter Query

Proofs/Parameter Query gives you the possibility to look up parameter records associated to a given run ID or retrieve the cases where the parameter records differ; you can filter the search to a specific run id or by breed, population, traits or trait group. Once you have selected your filters and press "Search" a table will be displayed with information on: run id, breed, population, trait, type of file (Conventional or Genomic), evaluation date, base definition, genetic merit, publication rule, heritability. The heading of the table is an active heading, this means that by clicking on it you can sort the content of the table.

Through the Parameter Query you will be able to look up only records that have been submitted by your organization.

Handling of cross-reference and alias' proofs

When a cross-reference link is created in the pedigree via uploading of a file202, the new official ID will retain its own proofs as well as acquire the proofs of the alias ID for those combinations it didn't have previously.

APPENDIX I - Format File300-EBV and File700-GEBV

Col	Name	Start	Format	Description	Example
1	rec type	1	a3	Record type ¹	300
2	brd_eval	5	a3	Breed of evaluation ²	HOL
3	pop	9	a3	Population code ³	USA
4	trt	13	a3	Trait of evaluation ⁴	mil
5	brd_anim	17	a3	Breed of animal	HOL
6	cou_orig	20	a3	Country of first registration	USA
7	sex	23	a1	Sex of animal	M
8	id_no	24	a12	Animal identification number	003000336289
9	typ_prf	37	i2	Type of proof ⁵	11
10	off_pub	40	a1	Official publicationof proof ⁶	Y
11	status	42	i2	Animal status ⁷	10
12	ndau	44	i8	Number of daughters ⁸	115
13	nhrd	52	i8	Number of herds ⁹	75
14	edc	60	i8	Number of effective daughter contributions ¹⁰	133
15	rel	69	f7.4	Repeatability/Reliability ¹¹	82
16	ebv	76	f10.	National predicted genetic merit ¹²	2.780

  IMPORTANT NOTE !!!!!

  In the old fileformat 01x-020 and 115, the national proofs were multiplied by a factor: (prod=100; conf=100;udder=1000;long=1000;calv=1000;fert=1000; fert=1000;work=1000).  This multiplication will no longer be needed.

¹Valid record types:
- 300 for EBV
- 700 for GEBV
²Breed codes accepted:
- BSW=Brown Swiss type; GUE=Guernsey type; HOL=Holstein-Friesian (Black & White) type; JER=Jersey type; RDC=Red Dairy Cattle type ; SIM=Simmental type.
³Valid population codes: ARG AUS BEL CAN CHE CZE ^aDEA DEU ^bDFS ESP EST FIN FRA ^cFRM GBR HUN IRL ISR ITA JPN LTU LVA NLD NZL POL PRT SVN SVK SWE USA URY ZAF
- where: ^a Austria+Germany; ^bDenmark + Finland + Sweden; ^c France Montbeliarde;
⁴Accepted traits abbreviations:
- Production ==> mil = milk;fat =fat; pro = protein;
- Conformation ==> sta = stature;cwi = chest width;bde = body depth;ang = angularity;ran = rump angle;rwi = rump width; rls = rear-leg set;rlr = rear-leg rear view;fan = foot angle;hde = heel depth/hoof height; fua = fore udder attachment; ruh = rear udder height; ruw = rear udder width; usu = udder support;ude = udder depth;ftp = front teat placement;ftl = (front) teat length;rtp = rear teat placement;ous = overall udder score; ofl = overall feet&legs score; ocs = overall conformation score; bcs = body condition score; loc = locomotion;
- Udder ==>scs = somatic cell; mas = mastitis
- Longevity ==> dlo = direct longevity;
- Calving ==> dce = direct calving ease;mce = maternal calving ease;dsb= direct stillbirth;msb = maternal stillbirth
- Female fertility ==>hco = heifer conception;crc = cow recycling;cc1 = lactating cow's ability to conceive (1);cc2 = lactating cow's ability to conceive (2);int= internval traits
- Workability ==> msp = milking speed;tem = temperament
- SNP Training ==> cma = clinical mastitis
⁵Accepted codes:
- 00 (unknown);
- 11 (based on first crop sampling daughters);
- 12 (based on first and second crop daughters);
- 13 (based on parent average and genomic information only);
- 21(based on imported semen of proven bull, second crop daughters only);
- 22 (based on mostly, more than 50%, imported daughters or daughters born from imported embryos.)
- 23 (GEBV with foreign PA)
- 24 (GEBV with foreign proof)
⁶Accepted abbreviations:
- Y (if bull proof meets national standards for official publication in the country sending information.);
- P (if bull is part of a simultaneous progeny-testing program, but the proof does not yet meet national standards for official publication);
- N (otherwise).
⁷Valid codes for status of bulls:
- 00(unknown);
- 10(bull randomly sampled through an official AI scheme);
- 15 (young bull, genomically tested, not yet selected for AI);
- 20(other bull. Records with “20” in this file will be excluded from the international evaluation, unless type of proof is “21”).
⁸Field for number of daughters should be positive. For missing value put 0.
⁹Field for number of herds should be positive. For missing value put 0.
¹⁰ Production, conformation, udder health, fertility, workability, and SNP training traits: Weighting factor used for these traits is “the effective daughter contribution (EDC)”, which is described In the Interbull document Code of practice, Appendix IV, “Weighting factor for international genetic evaluation”, updated April 27, 2004. EDC values should be rounded to the nearest integer value.
- Calving: The weighting factors used for calving traits it the total number of calvings for the direct effects and number of daughters with calving for maternal effect
- Longevity: The weighting factor used for longevity traits depends on the national genetic evaluation model. For linear models the weighting factor is the same as described above for conformation, fertility, production, udder health and workability traits. For survival models number of culled daughters is used as the weighting factor.
¹¹Reliability values are nationally calculated reliability values expressed in percents with 4 decimials. For missing value put 0.
¹²National predicted genetic merit values published domestically. For threshold models the submitted values are from the underlying scale. For missing values put 9999999999. Please note! In the old fileformat 01x-020 and 115, the national proofs were multiplied by a factor: (prod=100; conf=100;udder=1000;long=1000;calv=1000;fert=1000; fert=1000;work=1000). This multiplication will no longer be needed.

APPENDIX II - Format File301 - Parameters EBV and File701- Parameters GEBV

Col	Name	Start	Format	Description	Example
1	rec_type	1	a3	Record type ¹	301
2	brd_eval	5	a3	Breed of evaluation ²	HOL
3	pop	9	a3	Population code ³	USA
4	trt	13	a3	Trait of evaluation ⁴	scs
5	evdate	17	i8	National evaluation date ⁵	20121201
6	herit	26	f8.6	Heritability ⁶	0.12
7	refbase	35	a7	Reference base definition ⁷	H10CB05
8	pgmdef	43	a2	Genetic merit definition ⁸	T-
9	pub_rule	46	a1	Official publication rules ⁹	Y

¹Valid record types:
- 301 for EBV
- 701 for GEBV
²Breed codes accepted:
- BSW=Brown Swiss type; GUE=Guernsey type; HOL=Holstein-Friesian (Black & White) type; JER=Jersey type; RDC=Red Dairy Cattle type ; SIM=Simmental type.
³Valid population codes: ARG AUS BEL CAN CHE CZE ^aDEA DEU ^bDFS ESP EST FIN FRA ^cFRM GBR HUN IRL ISR ITA JPN LTU LVA NLD NZL POL PRT SVN SVK SWE USA URY ZAF
- where: ^a Austria+Germany; ^bDenmark + Finland + Sweden; ^c France Montbeliarde;
⁴Accepted traits abbreviations:
- Production ==> mil = milk;fat =fat; pro = protein
- Conformation ==> sta = stature;cwi = chest width;bde = body depth;ang = angularity;ran = rump angle;rwi = rump width; rls = rear-leg set;rlr = rear-leg rear view;fan = foot angle;hde = heel depth/hoof height; fua = fore udder attachment; ruh = rear udder height; ruw = rear udder width; usu = udder support;ude = udder depth;ftp = front teat placement;ftl = (front) teat length;rtp = rear teat placement;ous = overall udder score; ofl = overall feet&legs score; ocs = overall conformation score; bcs = body condition score; loc = locomotion
- Udder ==> scs = somatic cell; mas = mastitis
- Longevity ==> dlo = direct longevity
- Calving ==> dce = direct calving ease;mce = maternal calving ease;dsb= direct stillbirth;msb = maternal stillbirth
- Female fertility ==>hco = heifer conception;crc = cow recycling;cc1 = lactating cow's ability to conceive (1);cc2 = lactating cow's ability to conceive (2);int= internval traits)
- Workability ==> msp = milking speed;tem = temperament.
- SNP Training ==> cma = clinical mastitis
⁵ National evaluation dates expressed using the format YYYYMMDD
⁶Heritability for a specific trait in format f8.6. Should be larger than 0 and smaller than 0,999999
⁷ Reference (genetic) base definition in the country sending information: breed initial (1 char), year established (YY), bull(b) or cow(C) (1 char); birth(B), calving (C) or evaluation (E) (1 char); year of event (use middle year if base on multiple year (YY); for breed initial see breed code in footnote2 (use X If based on multiple breeds):e.g. H00BB95 means a base defined 2000 base on Holstein bulls born in 1995.
⁸Genetic merit definition consists of a letter and a sign:
- Sign: ‘+’ = Higher values are desirable, ‘-‘= Lower values are desirable
- B = Breeding value
- T = Transmitting ability;
⁹ Allowed characters :
- Y=Yes
- N=No

APPENDIX III - Frequently Asked Questions

My data has been rejected, what can i do?

National data can sometimes be rejected from ITBC staff if changes in sire standard deviations between previous and current data are larger than 5% (in case of routine run), or if problems in the data are found. If your data get rejected it means that the previous available data will be used for the IGE. In case you are able to fix the problem(s), and the data deadline has not yet passed, you can try to upload a new corrected dataset. You will need to check the confirmation email, the verify output and submit the data in the Review page again.

I discovered a problem with a dataset with FLAG=NO, what do I do?

As long as status=PENDING you can always upload a new corrected dataset, the new upload will overwrite the previou one. If you discover the error after submitting the data, and if the data deadline has not passed yet, you need to inform the ITBC staff about the problem and the breed-pop-trait combination affected by it. ITBC staff will reset your data so that you will be able to upload a new file again. You will need to check the confirmation email, the verify output and submit the data in the Review page again.

-  ⇤ ← Revision 18 as of 2013-08-15 13:13:01 → 
  Size: 13171
  Editor: Valentina
  Comment:
+   ← Revision 135 as of 2022-01-11 12:02:56 → ⇥
  Size: 24598
  Editor: Valentina
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 10:
-=== File Encoding ===
For the checking programs to run successfully the encoding of your files must be set to UTF-8.

If this is not the case a message saying

''The file seems not to be in UTF-8 encoding and an illegal character has been detected. Please, convert the file to UTF-8 and rerun the check program''

will appear on the screen. This means the program DID NOT run successfully.

To change the encoding of a file you have at least two options:

 * in Unix/Linux: use the command <<BR>>

'' iconv  –f “ISO-8859-1” –t “UTF-8” –o outputfile inputfile'' <<BR>>

 . where ''ISO-8859-1'' stands for the current encoding of your file, here we assume is the standard ISO encoding. But if the encoding used is a different one you have to change it accordingly.

<<BR>>

 * you can use Notepad++: open the file, click on ''Encoding'' and choose the option ''Convert to UTF8 without BOM'' and save the file.

== Software ==
The Software menu gives you access to the Interbull checking programs. By clicking on Software a drop down menu will open and you will be able to choose the type of checking program you are interested in, i.e. Pedigree, Proofs or GenoList. Under Software you will also find information on the programs and instructions on how to run them.
+== Software menu ==
The Software menu gives you access to the Interbull checking programs. By clicking on Software a drop down menu will open and you will be able to choose the type of checking program you are interested in, i.e. Pedigree or Proofs. Under Software you will also find information on the programs and instructions on how to run them.
-Line 35:
+Line 14:
-Member organizations will upload national evaluation datasets through IDEA in much the same way as they upload pedigree datasets. There is a downloadable Python program called ''CheckProofsPara.py'' available for checking proofs and parameter files. The program will check a format 300 (proof file) and an associated format 301 (parameter file) for formal correctness and prepare a zip file for upload if no errors are found in the files.

The proof file (format 300) is designed to allow considerable flexibility for the user. Each record contains an evaluation (i.e. proof) and associated information for a single brd-pop-trt combination. One file may contain records for as many or few brd-pop-trt combinations as desired. It may be natural for organizations to include all traits in a trait group in the same file, as they have in the past, but this is not essential. They may also choose to upload evaluations for all breeds, populations and traits in a single file if they find that more convenient. If problems are found with some specific combinations, only those combinations need to be re-loaded.

It is essential that proof records for all bulls be included in the same file for any one brd-pop-trt combination whether the dataset is a first submission for the IGE run in question, or a re-submission with some problem corrected. When new data are uploaded for any combination, all records present in IDEA for that combination are deleted prior to loading the new records. Thus, it is not acceptable to submit a partial dataset in order to correct the evaluations of some subset of bulls.

'''Every proof file must be accompanied by a parameter file!'''.  The records in the parameter file give information about the national evaluations that is common to all records in the proof file for a given brd-pop-trt combination. There must be a record in the parameter file for each brd-pop-trt combination present in the proof file. Extra records present in the parameter file will simply be ignored. This allows users to maintain a single parameter file to be submitted with several proof files, if desired.

The user instructions and file formats (see Appendix I and II) give details on how to run the program and on the checks performed. If no errors are found, a zip file is created with the name IB-ORGCODE-IG-yyymmddThhmmss.zip containing the input proof and parameter files.

The program requires access to the internet and specifically to a few functions/pages in the IDEA web application. Firstly, in order to ensure that the user has the most recent version of the software, the internal version tag is compared to the version listed on the download page and if the user’s copy is older an appropriate message is printed and the program exits. Lists of valid orgcode’s and breed-pop-trait combinations for which the user’s organization has EBV upload authority are also obtained from the IDEA web application.  If errors occur they are listed to the screen and no zip file is created. When checking the proof file only the first 10 errors of each type are listed.

== Proofs ==
The Proofs menu gives you access to the main proofs functions which are: Upload, Review, Messages.
+A Python program called CheckProofsPara.py will check the 300/700 proof file format and the associated 301/701 parameter file for format correctness against IDEA ([[https://wiki.interbull.org/public/IDEA_EBV_UserManual#APPENDIX_I_-_Format_File300-EBV_and_File700-GEBV|Appendix I]]+[[https://wiki.interbull.org/public/IDEA_EBV_UserManual#APPENDIX_II_-_Format_File301_-_Parameters_EBV_and_File701-_Parameters_GEBV|Appendix II]].)  The program prepares a zip file, IB-ORGCODE-<<span(style="color: #F518BA;")>>IG<<span>>-yyymmddThhmmss.zip for conventional<<span(style="color: #F518BA;")>> MACE <<span>> and IB-ORGCODE-<<span(style="color: #3D18F5;")>>GG<<span>>-yyymmddThhmmss.zip for <<span(style="color: #3D18F5;")>>GMACE<<span>>,if no errors are found in the file. The zip file represents your checked data file to upload to the Interbull Centre IDEA database (https://idea.interbull.org/).

The [[https://wiki.interbull.org/public/CheckProofPara#preview|user instructions]] (https://wiki.interbull.org/public/CheckProofPara#preview) for the !CheckProofsPara.py program and file formats (see [[https://wiki.interbull.org/public/IDEA_EBV_UserManual#APPENDIX_I_-_Format_File300_-_Proof_file|Appendix I]] and [[https://wiki.interbull.org/public/IDEA_EBV_UserManual#APPENDIX_II_-_Format_File301_-_Parameter_file|II]]) give details on how to run the program and on the checks performed.

== Proofs menu ==
The Proofs menu gives you access to the main proofs functions which are:[[https://wiki.interbull.org/public/IDEA_EBV_UserManual#Upload_and_Verify_program|Upload]], [[https://wiki.interbull.org/public/IDEA_EBV_UserManual#Review_your_data|Review,]] [[https://wiki.interbull.org/public/IDEA_EBV_UserManual#Messages|Messages.]]
-Line 53:
+Line 24:
-To uploading functionality for parameter and proof files is available under 'Proofs/Upload'. The only file accepted by IDEA is the zip file IB-ORGCODE-IG-yyymmddThhmmss.zip produced by the CheckProofsPara.py, no other files are accepted. Once in 'Proofs/Upload' users will be able to browse the appropriate file and upload it by click on "Submit query".
+Uploading functionality for parameter and proof files are available under 'Proofs/Upload'. The only file accepted by IDEA is the zip file IB-ORGCODE-XG-yyymmddThhmmss.zip produced by the !CheckProofsPara.py, no other files are accepted. Once in 'Proofs/Upload' users will be able to browse the appropriate file and upload it by click on "Submit query".
-Line 57:
+Line 28:
- a. CheckProofsPara.py will run once again inside IDEA to assure correctness of format and contents
+ a. !CheckProofsPara.py will run once again inside IDEA to assure correctness of format and contents
-Line 60:
+Line 31:
-=== Confirmation email ===
Either an "EBV upload success" or an "EBV upload failure" email will be sent to you upon completion of the uploading process. If the uploading is successful the "EBV upload success" email will list some basic information on your data such as:  <<BR>>

 a. Number of rows in parameter file associated with proofs
 a. Number of records read from the proof file
 a. Number of animals found by real AID
 a. Number of animals found by alias ID
 a. Number of flagged rows in proof file

The email can contain up to three different WARNINGs associated to three attachments: <<BR>>

 a. Warning: [n] animal(s) were referenced in your file, but not present in the pedigree database. These animals were discarded! See a detailed list in the attached 'missing_animals.csv' file.
 a. Warning: [n] animal(s) needing updated pedigree records detected! See 'ped_needed.csv' for a complete list of the animals.
 a. Warning: Use of aliases detected! See 'ped_alias.csv' for a complete list of the animals.

An important source of information comes from ''Number of flagged rows in proof file'':  this number represents the amount of discrepancies found by the Verify  program.  If it is 0 (zero) it means that no discrepancies have been  found and your data is automatically submitted for the IGE. If it is not  0 (zero) it means that the Verify program has found some discrepancies  in your data compared to the previous one available or the system has  detected animals with missing pedigree.  To double check the data you  need to log in IDEA and go to Proof/Review.

''' Animals not present in IDEA pedigree or lacking pedigree information (i.e. present in the pedigree database but with sire and dam unknown) will be excluded from the international evaluation '''.

<<BR>>

Table 1 summarizes the action needed upon reception of a confirmation email with such warnings and attachments:
+==== Confirmation email ====
Either an "EBV upload success" or an "EBV upload error" email will be sent to you upon completion of the uploading process.

==== UPLOAD STATUS ====
IDEA will  soon remove the ability to receive feedback through email, and instead  switch to a dedicated page where you (and other members of your  organization) can view all feedback relevant to your organization. This  will be in the form of a table, where you can look back at current and  past feedbacks, download the relevant files and so on. There will also  be an RSS feed available, so you can set up notifications of new  feedback as it becomes available, in a way similar to previous email  feedback.

Communications  from IDEA regarding the status (success or failure) of all type of  uploads performed by your organization (pedigree, EBV, GEBV,  performance, animinfo) are now listed in here and divided between  “Successful” or “Failed” uploads. The information displayed are from a  RSS feed system. Here the steps you shall do in order to display such  feeds correctly:

 * Go to Upload Status
 * Select “Successful” (or Failed)
 * Click on “Create subscription feed link”
 * Click on “Copy subscription feed link”

If  you think the link has been leaked/spread (remember that the link is  supposed to be personal), you can remove it at any time by clicking on  “Remove subscription feed link”. If that happens you shall create and  copy a new subscription feed link.

With  this new feedback system, information on all type of uploads will be  available to each and every members of the same organization and not  only to the person making the upload. Moving away from an email based  feedback methodology and to an RSS based one, will make the feedback  more stable and reliable, without risk of spam or mail server problems.  Also, it makes it easier to programmatically handle feedback from the  IDEA system.

You should copy your subscription feed link inside your email provider. Here is how you do it if you are using Outlook:

 * Right click on File
 * Right click on “Account Settings” from the Account      Settings drop down menu
 * In the new window that opens click on “RSS feeds”
 * Click on “New” and paste the RSS feed link you have      copied from IDEA.

Using Thunderbird:

 * Right  click on the News Feed folder and choose      Subscription (you can  also add a sub-folder and add the subscription      there)
 * Add your Subscription link into the Feed URL text box      in the Subscription popup box and click Add

For  the time being, IDEA emails and RSS feeds will co-exists to give time  to all IDEA users to get acquainted with the RSS feed. From 1st January  2023 IDEA will provide solely feedback based on the RSS feeds.

==== Upload successful ====
If the uploading is successful the "EBV upload success" email will list some basic information on your data such as:  <<BR>>

 . Data connected to runid                        : 1311r
 . Number of records read from the parameter file : x
 . Number of records read from the proof file     : xxxx
 . Number of rows in parameterfile associated
 . with proofs                                    : x
 . Number of flagged rows in proof file           : x
 . Number of animals found by real AID            : xxxxx
 . Number of animals found by alias               : 0
 . Number of animals not present in the pedigree  : 0

 . Number of records loaded from the file per combination :

 . RDC-DEU-cc1: xxx
 . RDC-DEU-cc2: xxx
 . RDC-DEU-crc: xxx
 . RDC-DEU-hco: xxx
 . RDC-DEU-int: xxx

The email can contain up to three different WARNINGS. The three first warnings below are associated to attachments and requires actions which are described in the actions for attachment table below: <<BR>>

 1. Warning: [n] animal(s) were referenced in your file, but not present in the pedigree database. These animals were discarded! See a detailed list in the attached 'missing_animals.csv' file.
 1. Warning: [n] animal(s) needing updated pedigree records detected! See 'ped_needed.csv' for a complete list of the animals.
 1. Warning: Use of aliases detected! See 'ped_alias.csv' for a complete list of the animals.
 1. This warning is generated if the uploaded file contains re-uploadings for a given Breed-Pop-Trait combination which have not been withdrawed first (see Withdraw data):
  . WARNING: This dataset re-uploads already existing data, of which some has not been withdrawn first. These combinations have been skipped; the proper way to re-upload combinations is to withdraw them first (or ask IBC staff to reset them). These combinations have been skipped:[......]

An important source of information comes from ''Number of flagged rows in proof file'':  this number represents the amount of discrepancies found by the Verify  program.  If it is 0 (zero) it means that no discrepancies have been  found in your data. <<span(style="color: #FF0000;")>>Please note that the data IS NOT AUTOMATICALLY submitted. After uploading to IDEA the data has to be submitted in the Proofs/Review page<<span>>. If it is not  0 (zero) it means that the Verify program has found some discrepancies  in your data compared to the previous one available or the system has  detected animals with missing pedigree.  To double check the data you  need to log in IDEA and go to Proof/Review.

==== Actions for attachments ====
<<span(style="color: #FF0000;")>> Animals not present in IDEA pedigree or lacking pedigree information (i.e. present in the pedigree database but with sire and dam unknown) will be excluded from the international evaluation.<<span>>

Table 1 summarizes the action needed upon reception of a confirmation email with warnings and/or attachments:
-Line 91:
+Line 106:
-=== After uploading: What's next? ===
Uploading represents only the very first step for submitting your data for an IGE. Here is a description of the actions you need to follow to submit your data for an IGE.

=== IDEA Proof/Review ===
The Proof/Review page contains all the information you need to review and submit your data for a IGE.

The page contains several important information grouped into three different tables: Filters table, Central table and Actions table

The __Filters__ table allows you to filter and display the information you have uploaded in a way that better suits you. By default the Central table displays all the information associated to your account but you can costumize the displaying of information choosing among:
+==== Upload not successful ====
There can be three main reasons for an uploading to fail, the different reasons are descripted in the specific "EBV upload error" email you could receive. The table below presents and explaine each of the possible cases:
||<tablewidth="1281px" tableheight="308px">'''Email content''' ||'''Meaning''' ||'''Action Required''' ||
||'System internal error. This should usually not happen. Contact staff for clarification' ||Technical problems with the database server ||contact Interbull Centre ||
||'The data already exists in the database and no combinations were updated due to not having the right status for re-uploading; the proper way to re-upload combinations is to withdraw them first (or ask IBC staff to reset them).' ||The content of the file was not uploaded ||Log in IDEA, <<BR>>go to Proofs/review,  <<BR>>withdraw the specific breed-trait combinations present in the file <<BR>> re-upload the file ||
||'Formal correctness errors were encountered in the uploaded file' ||The file was not uploaded due to format errors ||Review the file with the CheckProofPara <<BR>> re-upload the file ||




<<BR>>

=== Review your data ===
==== General ====
Uploading represents only the very first step for submitting your data for an IGE. Here is a description of the actions you need to follow to submit your data for an IGE. The Proof/Review page contains all the information you need to review and submit your data for a IGE.

The page contains several important information grouped into three different tables: [[https://wiki.interbull.org/public/IDEA_EBV_UserManual#Filter_table|Filters table]], [[https://wiki.interbull.org/public/IDEA_EBV_UserManual#Central_.28Include.29_table|Central (Include) table]] and [[https://wiki.interbull.org/public/IDEA_EBV_UserManual#Actions_table|Actions table]]

==== Filter table ====
The __Filters__ table allows you to filter and display the information you have uploaded in a way that better suits you. By default the Central (Include) table displays all the information associated to your account but you can costumize the displaying of information choosing among:
-Line 109:
+Line 135:
-{{attachment:Review.png}}

The __Central__ table displays as many rows as the breed-pop-trait combinations you have uploaded. Each row shows the following information:

 * Status: refers to the status of your data. There are a total of 5 different status: Pending, Submitted, Withdrawed, Accepted, Rejected
+{{attachment:review1.png}}

==== Central (Include) table ====
The __Central__ (Include) table displays as many rows as the breed-pop-trait combinations you have uploaded. Each row shows the following information:

 * Status: refers to the status of your data. There are a total of 5 different status: Pending,[[https://wiki.interbull.org/public/IDEA_EBV_UserManual#Submit_data|Submitted]], [[https://wiki.interbull.org/public/IDEA_EBV_UserManual#Withdraw_data|Withdrawed]], [[https://wiki.interbull.org/public/IDEA_EBV_UserManual#Accepted.2BAC8-Rejected|Accepted, Rejected]]
-Line 115:
+Line 142:
- * C/G: refers to the nature of your data, Conventional/Genomic. At the moment only Conventional data are accepted.
 * Datasets: refers to the dataset used to upload that given Breed-Pop-trait combination
+ * C/G: refers to the nature of your data, Conventional/Genomic.
 * Datasets: refers to the dataset used to upload a given Breed-Pop-trait combination
-Line 119:
+Line 146:
-The Breed-Pop-Trait combinations not highlighted and with Flagged=NO are combinations for which the Verify program did not find any discrepancies therefore they get automatically a status=Submitted. No more actions is required from you for these combinations.

The Breed-Pop-Trait combinations highlighted in yellow require your attention. For each of them you need to check the Verify output. For your convenience a "View Summary for Selected" at the end of the Central table will display in a new page the main key checking points of the Verify output. In order for this option to work you need to select some combinations. You can do that either manually of by clicking on "Reverse selection" and then click on "View Summary for Selected".

The __Actions__ table displays the options you have for your data with status=PENDING. You can decide to submit or withdraw such data by selecting the desired action and clicking on "Submit".

=== Submit data ===
If after checking the Verify output you consider your data to be ok and want to include it in the IGE you can do so by selecting the box beside each Breed-Pop-Trait combination you want to include, select the action "Submit" and click on the "Submit" button.

Every time you submit a pending data you are required to explain the reasons for the discrepancies found by the Verify program. If the same explanation applies to several Breed-Pop-Traits combinations you can select them in the Review page so that you will be required to write only once the explanation all these data share.
+The Breed-Pop-Trait combinations not highlighted and with Flagged=NO are combinations for which the Verify program did not find any discrepancies, nevertheless they will still get a status= Pending as your final approval on the data is still required.

The Breed-Pop-Trait combinations highlighted in yellow require your attention. For each of them you need to check the Verify output. For your convenience, selecting such combinations and clicking on "View Summary for Selected", located at the end of the Central table, will display in a new page the main key checking points of the Verify output. You can select the combination either manually, by clicking in the box in the "include" column,  or by clicking on "Reverse selection".

If you wish to save the full verify output as pdf, here are the steps to follow:

 1. Open the verify output
 1. Open the application menu from your browser
 1. Click on "Print"
 1. Under the 'Destination' section, click Change... > Save as '''PDF'''

==== Actions table ====
The __Actions__ table displays the options you have for your data with status=PENDING. You can decide to [[https://wiki.interbull.org/public/IDEA_EBV_UserManual#Submit_data|submit]] or [[https://wiki.interbull.org/public/IDEA_EBV_UserManual#Withdraw_data|withdraw]] such data by selecting the desired action and clicking on "Submit".

==== Submit data ====
If, after checking the Verify output ,you consider your data to be ok and want to include it in the IGE you can do so by selecting the box beside each Breed-Pop-Trait combination you want to include, select the action "Submit" and click on the "Submit" button.

Every time you submit a pending data you are required to explain the reasons for the discrepancies found by the Verify program. If the reason are breed-trait dependent you need to processed these cases one by one and provide the full explanation in the designed space "Change comment". If, on the other hand, the same explanation applies to several Breed-Pop-Traits combinations you can select them together in the Review page so that you will be required to write only once the explanation that all these data share.By clicking "Send message" your message will be recorded under IDEA Proofs/Messages and will be visible by you and the Interbull Centre Staff.

In the Review page, the Breed-Pop-Trait combinations you have submitted will be now displayed with status=Submitted.

On the day of the data submission deadline for a given IGE, routine or test run, all your data in the Review page should have status=SUBMITTED. Your aim is therefore to check all pending data and either provide explanations or withdraw and upload new datasets before the data submission deadline.

{{attachment:Submit_pending.png}}

==== Withdraw data ====
If you realize the data you have uploaded for some Breed-Pop-trait combinations is wrong or you want to upload pedigree information for the animals reported in the confirmation email attachments you need to withdraw your data before doing anything else. You do so by simply selecting the affected Breed-Pop-Trait combinations in the Review page, select the action Withdraw and then press the button "Submit".In the Review page that given Bredd-Pop-trait combination will now have status=WITHDRAWED.

When you withdraw a given Breed-Pop-Trait '''all''' records present in IDEA for that combination are deleted thus you have to re upload the file before the data deadline. It is not acceptable to submit a partial  dataset in order to correct the evaluations of some subset of bulls.  It is essential that proof records for all bulls be included in the  same file for any one brd-pop-trt combination whether the dataset is a  first submission for the IGE run in question, or a re-submission with  some problem corrected.

If you fail to withdraw all the traits you are going to re-upload in a new file, the confirmation email will contain the following warning:

 . WARNING: This dataset re-uploads already existing data, of which some has not been withdrawn first. These combinations have been skipped; the proper way to re-upload combinations is to withdraw them first (or ask IBC staff to reset them). These combinations have been skipped:[......]

Example: <<BR>>You notice some problem with the temperament data for RDC. You withdraw only the RDC-Tem from your Review table and in the new file to upload you also include data for RDC msp. As you did not withdraw the RDC-msp combination before uploading the new file, all the combinations referring to RDC-msp are skipped as data already exist in IDEA for that given combination.

==== Accepted/Rejected ====
Once you have submitted your data for a given IGE it will be up to the ITBC staff to finally accept or reject it. In general all data with FLAGGED=NO will also get accepted as there are no issues pending on these data. For all the data with FLAGGED=YES, ITBC staff will go through the explanations you have reported and if found sufficients will mark that given Breed-Pop-Trait combination as accepted. If more clarifications are needed ITBC staff will email you via IDEA, anytime a new message will be posted in your IDEA account by the staff you will receive an email informing about the presence of unreplied messages in IDEA.

In case the discrepancies found are considered too big then ITBC staff can reject your data and use the data from the previous run. You will be able to follow what happens to your data by looking at the Review page as the status will change according to what decision has been made.

=== Messages ===
Proofs/Messages represents the place where all your communication with the ITBC for a given IGE are displayed. All the explanations you provide for a given Breed-Pop-Trait combination are automatically listed in here. Unreplied messages will be marked in red. You can use this page to send further messages to ITBC staff. When ITBC staff replies, their messages will also be listed here and you will be notified by email about the presence of unreplied messages in IDEA.

{{attachment:messages.png}}

=== Message Overview ===
In the message overview you can review all the communications you had with the staff for any given evaluation.

By selecting the evaluation you are interested in and pressing "Set Run", all messages exchanged between you and the ITBC staff during the evaluation you have selected will be listed in a table.

=== Proof Query ===
Proofs/Proof Query gives you the possibility to look up proof records associated to a given animal ID; you can filter the search to a specific runid or retrieve the information available for that ID in all the run id available in the database as well as filter it by population, traits or trait group. Once you have selected your filters and press "Search" two tables will be displayed: an ''animal information'' table showing the pedigree for that animal and a second table with information on: run id, breed, population, trait, type of file ('''C'''onventional or '''G'''enomic), type of proof, publication, status of bull, number of daughters, number of herds, EDC, reliability and proof. The heading of the second table is an active heading, this means that by clicking on it you can sort the content of the table.

Through the Proof Query you will be able to look up only the animals that have been submitted by your organization.

{{attachment:proof_query1.png}}

=== Parameter Query ===
Proofs/Parameter Query gives you the possibility to look up parameter records associated to a given run ID or retrieve the cases where the parameter records differ; you can filter the search to a specific run id or by breed, population, traits or trait group. Once you have selected your filters and press "Search" a table will be displayed with information on: run id, breed, population, trait, type of file ('''C'''onventional or '''G'''enomic), evaluation date, base definition, genetic merit, publication rule, heritability. The heading of the table is an active heading, this means that by clicking on it you can sort the content of the table.

Through the Parameter Query you will be able to look up only records that have been submitted by your organization.

{{attachment:query_param.png}}

=== Handling of cross-reference and alias' proofs ===
When a cross-reference link is created in the pedigree via uploading of a file202, the new official ID will retain its own proofs as well as acquire the proofs of the alias ID for those combinations it didn't have previously.

----
<<Include(public/File300)>>

----
<<Include(public/File301)>>

----
== APPENDIX III - Frequently Asked Questions ==
=== My data has been rejected, what can i do? ===
National  data can sometimes be rejected from ITBC staff if changes in sire  standard deviations between previous and current data are larger than 5%  (in case of routine run), or if problems in the data are found. If your  data get rejected it means that the previous available data will be  used for the IGE. In case you are able to fix the problem(s), and the  data deadline has not yet passed, you can try to upload a new corrected  dataset. You will need to check the confirmation email, the verify  output and submit the data in the Review page again.

=== I discovered a problem with a dataset with FLAG=NO, what do I do? ===
As long as status=PENDING you can always upload a new corrected dataset, the new upload will overwrite the previou one. If you discover the error after submitting the data, and if the data deadline has not passed yet, you need to inform the ITBC staff about the problem and the breed-pop-trait combination  affected by it. ITBC  staff will reset your data so that you will be able to upload a new  file again. You will need to check the confirmation email, the verify  output and submit the data in the Review page again.