IDEA AnimInfo User Manual

Introduction

The AnimInfo is a module in the Interbull Centre Data Exchange Area(IDEA) website which allows member organizations to upload additional information connected to existing animals in the pedigree module. Examples on additional information are coat color, crossbreed information, herdbook number, eartag number, genetic defects etc.

The purpose of the AnimInfo module is to collect reported information from member organizations and to use the module as an exchange area for information, not to verify or authorize information. The system allows different security levels for the information which means that for some AnimInfo information only the authorized organization may view and upload, for other information it is possible for some or all organizations to view and/or upload.

The type of additional information that will be registered in IDEA Animinfo is decided by member organizations in collaboration with Interbull Centre. Interbull Centre will have to register the type of additional information (coat color, herdbook number, etc) in IDEA before member organizations can be able to upload the information via the IDEA Animinfo module. Therefore, member organizations are encouraged to send requests on new additional information types to Interbull Centre.

The AnimInfo file format is an XML file format. For basic information on XML, see https://en.wikipedia.org/wiki/XML or XMLdigest. XML is a flexible system for complex data files and was choosen for AnimInfo in order to ensure easy future development and extension of the module's file format and capabilities, as well as a fitting format for the current data model.

The following is a description on how to, as an end-user, prepare and upload additional animal information to IDEA.

Quick workflow

The workflow to upload additional information for animals existing in the pedigree module:

After uploading , member organizations will be able to query an animal and get the additional information on the screen or in a datafile in csv or XML fileformat as an attachment to an email.

Additional information available for uploading

An overview of current available type of information to upload in IDEA/Animinfo can be found in the AnimInfo -> Types page in IDEA (figure 1)

figure_1_border.png
Figure 1

Each type of information (Types) have different attributes and values where:

Figure 2 shows the specification of the Crossbreed AnimInfo data structure where:

figure_2_border.png
Figure 2

Detailed workflow

Create an AnimInfo file

The structure of the AnimInfo XML file format is as following:

Examples of AnimInfo files can be found in section EXAMPLES.

Run Checking program

The correctness of the AnimInfo file is checked by a Python2 checking program called CheckAniminfo.py. The program with instructions are available from IDEA/Software https://idea.interbull.org/software/index. When no errors are found in the !Animininfo file, an AnimInfo zip file is created ready to be uploaded through the IDEA web interface. The AnimInfo zip file is called IB-ANIMINFO-{org code}-{YEAR-MONTH-DAY}T{HOUR-MINUTE-SECOND}.zip.

AnimInfo Upload

The Animinfo zip file can be uploaded by clicking on the AnimInfo -> Upload menu item (see figure 3) and then using the appropriate upload buttons.

figure_3_border.png
Figure 3

After upload, the file will be checked by the server-side CheckAnimInfo script. If no errors are found, the data will be passed onto the import functions in IDEA. After the data has been processed an email with feedback information will be sent to the uploading organization. The email contains general statistics about the upload; how many AnimInfo types and attributes processed, discarded and so on. Also included is an XML AnimInfo feedback file with more detailed information about the upload. The structure of the feedback XML file is:

The file is sent in a compact XML format without unnecessary whitespace. All decent XML-capable editors and tools can pretty print XML, which will structure the data in a good manner for easy reading. One good XML editor, XML Copy Editor (for Windows and Linux), can pretty print XML by pressing F11 or selecting the Pretty print option in the Tools menu. From the command line in Linux the xmllint command can be used to pretty print:

xmllint --format <filename.xml>

Other tools, including many text editors, have similar functionality. For programmatical, easy extraction of information out of an XML document like the AnimInfo feedback file, a good starting point is our short XML tutorial.

UPLOAD STATUS

IDEA will soon remove the ability to receive feedback through email, and instead switch to a dedicated page where you (and other members of your organization) can view all feedback relevant to your organization. This will be in the form of a table, where you can look back at current and past feedbacks, download the relevant files and so on. There will also be an RSS feed available, so you can set up notifications of new feedback as it becomes available, in a way similar to previous email feedback.

Communications from IDEA regarding the status (success or failure) of all type of uploads performed by your organization (pedigree, EBV, GEBV, performance, animinfo) are now listed in here and divided between “Successful” or “Failed” uploads. The information displayed are from a RSS feed system. Here the steps you shall do in order to display such feeds correctly:

If you think the link has been leaked/spread (remember that the link is supposed to be personal), you can remove it at any time by clicking on “Remove subscription feed link”. If that happens you shall create and copy a new subscription feed link.

With this new feedback system, information on all type of uploads will be available to each and every members of the same organization and not only to the person making the upload. Moving away from an email based feedback methodology and to an RSS based one, will make the feedback more stable and reliable, without risk of spam or mail server problems. Also, it makes it easier to programmatically handle feedback from the IDEA system.

You should copy your subscription feed link inside your email provider. Here is how you do it if you are using Outlook:

Using Thunderbird:

For the time being, IDEA emails and RSS feeds will co-exists to give time to all IDEA users to get acquainted with the RSS feed. From 1st January 2023 IDEA will provide solely feedback based on the RSS feeds.

Querying AnimInfo data

The AnimInfo data is accessible from 1) AnimInfo/Query/Report and from 2) Pedigree/Query.

From the AnimInfo Query/Report it is possible to get the information in AnimInfo in datafiles.

1) AnimInfo/Query/Report

figure_4_border.png
Figure 4

Paste in any text containing animal international ID's. Here there are two options 1) get the output on the screen 2) get the result as attachment in XML or CSV fileformat in an email. If the result is preferred as an attachment, click in the box beside “Send query result in a file by email” and choose the fileformat (figure 5).

figure_5_border.png
Figure 5

In the AnimInfo query/Report menu it is also possible to get a report per Animal Information Type / Breed(s). The available types /breeds will be shown in the scroll down menu. Here it is also possible to get the result on the screen or as an attachment in an email (figure 6).

figure_6_border.png
Figure 6

2) Pedigree/Query

figure_7_border.png
Figure 7

Result from Query

The result from the query will be presented in a table. The columns in the table are: AID, AnimInfo Type, AnimInfo Attribute, AnimInfo Value and Submitting Organization. The information can be filtered by using filter list boxes. Additionally, when doing a pedigree query there will be an indication on the animal presentation page with a link to the additional information associated with the animal (figure 8).

figure_8_border.png
Figure 8

Permissions

AnimInfo has an elaborate permissions system which can be used to allow or disallow uploading and viewing of AnimInfo information depending on AnimInfo type, Organization and AID. The current permission settings can be found by chose AnimInfo -> Permissions in IDEA (see figure 9).

figure_9_border.png
Figure 9

The permissions are presented in a table with an overview of all permissions pertaining to that organization's AnimInfo data. The columns are Organization, AnimInfo Type and Permission. Organizations have full access to all animal information types for the animals where they are the authorized organization, ie. they can always upload information to animals they own, and also read all uploaded information to these animals. For other organizations' animals the permission system applies.

Permissions can be of five types:

When the Organization column reads "--DEFAULT" the row indicates the default permission for that AnimInfo type, as set by the Interbull Centre. Currently, only default permissions are allowed, and permissions set by the Interbull Centre. In the future organizations will be able to upload AnimInfo files with permission rules included, to further extend and modify the default permissions.

Examples

Crossbreed %RH genes example

Here is an example of an %RH genes / Crossbreed XML file that sets the CROSSBREED AnimInfo type for three animals (HOLUSAM000000000X11, HOLDEUF000000000Y22 and HOLUSAM000000000X45):

   1 <interbull type="animinfo" version="1.0">
   2   <animals>
   3     <a id="HOLUSAM000000000X11">
   4       <CROSSBREED percent="RHOL:50;" />
   5     </a>
   6     <a id="HOLDEUF000000000Y22">
   7       <CROSSBREED percent="RHOL:25;SIM:25;" />
   8     </a>
   9     <a id="HOLUSAM000000000X45">
  10       <CROSSBREED percent="RHOL:75;" />
  11     </a>
  12   </animals>
  13 </interbull>

Genetic trait

Below is an example of an XML file with information about WHFF genetic traits where:

The code whff in the attribute indicates that the organization creating the file has used World Holstein Friesian (WHFF) business rules (http://www.whff.info/documentation/genetictraits.php#go1) , xx(in the attibute)=the gene abbreviation for the trait and the value shows the gene abbreviation and expression code(i.g BL=gene abbreviation and C= expression code ).

Other types available are at the moment: BSW_GENETIC_TRAIT

Allowed values and attributes to use for all available types can be found on https://idea.interbull.org/pedigree/animinfotypes

   1 <interbull type="animinfo" version="1.0">
   2   <animals>
   3      <a id="HOLNLDF000000123456">
   4         <GENETIC_TRAIT whff_bl="BLF" />
   5      </a>
   6       <a id="HOLNLDM000000-1234">
   7         <GENETIC_TRAIT whff_mf="MFC"  />
   8      </a>
   9       <a id="HOLNLDF000000478102">
  10        <GENETIC_TRAIT whff_po="POS" />
  11      </a>
  12      <a id="HOLCANM000000311569">
  13        <GENETIC_TRAIT whff_bl="BLF" whff_mf="MFC" />
  14      <a/>
  15    </animals>
  16 </interbull>

Genotyped animals

This service has been developed for beef organizations. The information gives information on if an animal has been genotyped or not, if the genotypes is sharable and the tissue used for genotyping. The actual genotypes are not reported. The code for the snp chip used for genotyping will be added as soon as standardized codes have been agreed on. Allowed values for each type of information can be found on https://idea.interbull.org/pedigree/animinfotypes.

Below is an exampel of an XML file with information about genotyped animals where:

   1 <interbull type="animinfo" version="1.0">
   2   <animals>
   3     <a id="LIMIRLM12314567890">
   4       <GENO_BEEF pop="IRL" genotyped="Y"  share="Y" tissue="E"  />
   5     </a>
   6     <a id="LIMIRLM1234567891">
   7       <GENO_BEEF pop="IRL" genotyped="Y" share="N" tissue="N" />
   8     </a>
   9     <a id="LIMIRLM1234567892">
  10       <GENO_BEEF pop="IRL" genotyped="Y" share="N" tissue="H" />
  11     </a>
  12     <a id="LIMIRLM1234567893">
  13       <GENO_BEEF pop="IRL" genotyped="Y" share="Y" tissue="M" />
  14     </a>
  15     <a id="LIMIRLM1234567894">
  16       <GENO_BEEF pop="IRL" genotyped="Y" share="Y" tissue="U" />
  17     </a>
  18     <a id="LIMIRLM1234567895">
  19       <GENO_BEEF pop="IRL" genotyped="Y" share="Y" tissue="S" />
  20     </a>
  21     <a id="LIMIRLM1234567896">
  22       <GENO_BEEF pop="IRL" genotyped="Y" share="Y" tissue="B" />
  23     </a>
  24   </animals>
  25 </interbull>

InterGenomics-Holstein publishable young bulls

This Animinfo type has been developed for InterGenomics-Holstein Contributors. InterGenomics-Holstein Contributors should use the "IGHOL_YOUNG_BULL" type to provide information on their genotyped young bulls in A.I. evaluated in the InterGenomcis-Holstein Service, specifying for which of them InterGenomics-Holstein Organisations have to publish gEBVs. For each genotyped young bull in A.I. identified as publishable, the Contributor should also specify in which country or countries its gEBVs have to be published.

Below is an example of an XML file with information about genotyped young bulls in A.I. evaluated in the InterGenomics-Holstein Service where:

   1 <interbull type="animinfo" version="1.0">
   2   <animals>
   3     <a id="HOLITAM123456789000">
   4       <IGHOL_YOUNG_BULL publish="N" />
   5     </a>
   6     <a id="HOLITAM000123456789">
   7       <IGHOL_YOUNG_BULL publish="Y" countries="KOR;SVN" />
   8     </a>
   9   </animals>
  10 </interbull>

Combine types

A more complex example shows how to combine CROSSBREED information and GENO_BEEF information in one file.

   1 <interbull type="animinfo" version="1.0">
   2   <animals>
   3     <a id="CHAIRLF00000GW12345">
   4       <GENO_BEEF pop="IRL" genotyped="Y"  share="Y" tissue="E"  />
   5       <CROSSBREED percent="LIM:25;CHA:50;"/>
   6     </a>
   7     <a id="CHAIRLM231139123456">
   8       <GENO_BEEF pop="IRL" genotyped="Y" share="N" tissue="H" />
   9       <CROSSBREED percent="LIM:25;CHA:75;" />
  10     </a>
  11   </animals>
  12 </interbull>

public/IDEA_animinfo_manual (last edited 2024-01-17 15:55:20 by Valentina)