IDEA AnimInfo User Manual

Introduction

The AnimInfo is a module in the Interbull Centre Data Exchange Area(IDEA) website which allows member organizations to upload additional information connected to existing animals in the pedigree module. Examples on additional data are information about coat color, crossbred information, herdbook number, eartag number, genetic defects etc.

The purpose of the AnimInfo module is to collect reported data from member organizations and to use the module as an exchange are for information, not to verify or authorize information. It is possible to make different security levels for the information which means that for some AnimInfo data only the authorized organization may view and upload data, for other data it is possible for some or all organizations to view and/or upload it.

The type of additional information that will be registered in IDEA Animinfo is decided by member organizations in collaboration with Interbull Centre. Interbull Centre has to update IDEA with the type of additional information to upload before member organizations can upload additional information in IDEA. Member organizations are encourage to send requests on additional data to upload in IDEA to Interbull Centre. The current type of data registered for uploading in IDEA can be found by clicking on the AnimInfo -> Types menu item in the bar menu in IDEA (see figure 1).

The following is a description on how to, as an end-user, prepare and upload additional animal information to IDEA.

AnimInfo usage

The workflow to upload additional information for animals existing in the pedigree modul:

Create a XML file with the relevant information according to ..."link"

Run a checking program to check the correctness of the file ..... "link"".

Upload the file to IDEA ... link....

• If more than one organization uploads the same type of data for the same animal all the uploaded information will be shown when querying that animal, with indications from which organization the different pieces of data come from.

Preparation

The type of additional information that will be registered in IDEA Animinfo is decided by member organizations in collaboration with Interbull Centre.

Interbull Centre has to update IDEA with the type of data to upload before member organizations can upload additional information in IDEA. Member organizations are encourage to send requests on additional data to upload in IDEA to Interbull Centre. The current type of data registered for uploading in IDEA can be found by clicking on the AnimInfo -> Types menu item in the bar menu in IDEA (see figure 1).

Figure 1

In the AnimInfo -> Types page in IDEA you will find an overview of all the available types, attributes and values in the AnimInfo module. There one finds a bullet list of the AnimInfo types, in bold, uppercase letters, and with a short description in italic. Below each type, in a sub-listlevel, the type's attributes are listed in lowercase. After each attribute, in parantheses, the attribute value type is specified. Also, there is a short description in italics for each attribute. In the case of attributes with a predefined set of allowed values, these are listed in yet another level of the list, below the relevant attribute. Below the AnimInfo types and attribute overview one finds a specification of the value types available, with a description on which pattern the value must conform to.

Here, in Figure 2, one can see the specification of the %RH-genes AnimInfo data structure, where the type is specified as RH_GENES, and it has a single attribute, percent. The percent has the value type percent with not available, which is defined in the list of value types as A numerical, integer value between 0 and 100, with the string N/A if not available. This means that one can upload %RH-genes percent values (which is either a percent value or N/A for missing values) for every animal, using the AnimInfo file format.

Figure 2

Create an AnimInfo file

The AnimInfo file format is an XML file format. For basic information on XML, see https://en.wikipedia.org/wiki/XML or public/XMLdigest. XML is a flexible system for complex data files and was choosen for AnimInfo in order to ensure easy future development and extension of the module's file format and capabilities, as well as a fitting format for the current data model.

The structure of the AnimInfo XML file format is as following:

• interbull: The root element of the Interbull XML file formats. It requires the XML attributes type and version, where the values should be animinfo and 1.0 respectively.

  • animals: The animals element defines the section which lists all animals and their AnimInfo data.

    • a: the animals section contains several a-elements which each represents a single animal. Every a-element should have an id-attribute which is the animal's international id (AID).

      • ANIMINFO TYPE: Every animal specified by the a-element may have one or more unique AnimInfo types specified, with each's respective attributes defined.

Example 1: Take a file which will upload %RH-genes information about two animals. Then the correctly formatted AnimInfo XML data file would look like this:

   1 <interbull type="animinfo" version="1.0">
   2   <animals>
   3     <a id="HOLUSAM000000000X11">
   4       <RH_GENES percent="50" />
   5     </a>
   6     <a id="HOLDEUF000000000Y22">
   7       <RH_GENES percent="N/A" />
   8     </a>
   9   </animals>
  10 </interbull>

Run Checking program

Once the AnimInfo XML file has been prepared, it has to be run through the CheckAnimInfo program, just like with Pedigree, Links and EBV/GEBV files has to be checked before uploading to IDEA. The CheckAnimInfo program is downloadable from the IDEA website, under Software, and is a Python 2 script https://ideatest.hgen.slu.se/idea_animinfo/software/index

This will check that the AnimInfo file format is correct and that the file includes the right AnimInfo types, attributes and values. It will warn and give hints about what kind of errors it encounters, and how to fix them. If all goes well it will produce an AnimInfo zip file, ready to upload through the IDEA web interface.

AnimInfo Upload

Figure 3

The AnimInfo zip file is called IB-ANIMINFO-{org code}-{YEAR-MONTH-DAY}T{HOUR-MINUTE-SECOND}.zip, and can be uploaded by clicking on the AnimInfo -> Upload menu item (see figure 3) and then using the appropriate upload buttons. After upload the file will be checked by the server-side CheckAnimInfo script, and if everything checks out OK the data will be passed onto the import functions in IDEA. After the data has been processed an email with feedback information will be sent. The email contains general statistics about the upload; how many AnimInfo types and attributes that were processed, discarded and so on. Also included is an XML AnimInfo feedback file with more detailed information about the upload. Here is the structure of the feedback XML file:

• interbull: Root element with type="animinfo_upload_feedback" and dscode equal to the data set code for the upload (similar to the file name, minus the initial IB- and the file ending).

  • processed: Containing element for processed (ie. imported/updated) data. The attribute type describes what kind of information that was processed, usually "animal information".

    • action: Containing element for a certain type of processed data according to the action taken. The type attribute determines the type, usually "new" or "updated" for AnimInfo.

      • item: Describes a single item that was processed, and its attributes, which may include: aid for an animal international id, type for an AnimInfo type, attribute for an AnimInfo attribute.

  • discarded: Containing element for discarded data. The attribute type describes what kind of information that was processed, usually "animal information".

    • action: Containing element for a certain type of discarded data according to the action taken. The type attribute describes the reason for discarding, for example "infotype discarded due to animal missing".

      • item: Describes a single item that was discarded, and its attributes, which may include: aid for an animal international id, type for an AnimInfo type, attribute for an AnimInfo attribute.

Example:

   1 <interbull type="animinfo_upload_feedback" dscode="ANIMINFO-VIT-20151222T104450">
   2   <processed type="animal information">
   3     <action type="updated">
   4       <item aid="HOLDEUM000000050208" type="RH_GENES" attribute="percent"/>
   5       <item aid="HOLDEUM000000050210" type="GENOLIST" attribute="genotyped"/>
   6     </action>
   7   </processed>
   8   <discarded type="animal information">
   9     <action type="animal info discarded due to existing identical data">
  10       <item aid="HOLDEUM000000050208" type="RH_GENES" attribute="percent"/>
  11       <item aid="HOLDEUM000000050210" type="GENOLIST" attribute="genotyped"/>
  12     </action>
  13     <action type="infotype discarded due to animal missing">
  14       <item aid="HOLDEUM99930030030X" type="RH_GENES"/>
  15     </action>
  16   </discarded>
  17 </interbull>

Querying AnimInfo data

Figure 4

To query AnimInfo click on the AnimInfo -> Query menu item (see figure 4), and paste in any text containing animal international IDs. If any of the animals have AnimInfo data that you are allowed to view, then the data will be presented to you in a dynamic data table. The columns are: AID, AnimInfo Type, AnimInfo Attribute, AnimInfo Value and Submitting Organization. It is possible to filter the results dynamically by using the filter list boxes. Additionally, when doing a pedigree query there will be an indication on the animal presentation page if the animal has AnimInfo data associated with it, and a link to said data.

Permissions

Figure 5

AnimInfo has a quite elaborate permissions system which can be used to allow or disallow uploading and viewing of AnimInfo data depending on AnimInfo type, Organization and AID. To see the current permissions one can click on the AnimInfo -> Permissions menu item (see figure 5). There is presented a dynamic data table with an overview of all permissions pertaining to that organization's AnimInfo data. The columns are Organization, AnimInfo Type and Permission. Permissions can be of four types: (r) means read access to all of the data, (rw) means read and write access, ie. one can both read all and upload own data, (d) means denied access, ie. can't read the value of the AnimInfo data, (x) means the permission is inherited from the default permissions for that AnimInfo type. When the Organization column reads "--DEFAULT" the row indicates the default permission for that AnimInfo type, as set by the Interbull Centre. Currently, only default permissions are allowed. In the future organizations will be able to upload AnimInfo files with permission rules included, so they can have more control over who can view and upload what.

*****************************

Here you will be shown a list of the allowed AnimInfo types, attributes and values:

• AnimInfo Types groups related AnimInfo attributes (see below) together into a coherent logical unit in the AnimInfo file format. They are indicated by UPPERCASE names and only alphanumeric characters are allowed, plus underscore(_) and hyphen(-).

• AnimInfo Attributes holds the actual information of the relevant AnimInfo type in a list of name/value-pairs, for example genotyped="Y". The first part, before the equal(=) sign is the name of the attribute, and is in lowercase, and the part after the equal sign and within quotation marks (") is the value of that attribute (see below). As with the types the attribute name can only contain alphanumeric characters, underscore and hyphen. An AnimInfo type can contain from one to several related attributes — some of which are required, others are optional. An attribute can only be specified once for each AnimInfo type, organization and animal.

• AnimInfo Values sets the value of each attribute for each animal. The value must conform to the specification of the attribute, which can be different from attribute to attribute; ranging from a free-form text string, to a set of predefined values, to a defined pattern the value must match.

Example 2: A more complex example would like to upload %RH-genes and Genolist data (which specifies whether the animal has been genotyped or not, and if it this animal's genotype is public or not) for four animals:

   1 <interbull type="animinfo" version="1.0">
   2   <animals>
   3     <a id="HOLUSAM000000000X11">
   4       <RH_GENES percent="50" />
   5       <GENOLIST genotyped="Y" public="Y" />
   6     </a>
   7     <a id="HOLDEUF000000000Y22">
   8       <RH_GENES percent="N/A" />
   9       <GENOLIST genotyped="Y" public="N" />
  10     </a>
  11     <a id="HOLUSAM000000000X45">
  12       <RH_GENES percent="75" />
  13       <GENOLIST genotyped="N" />
  14     </a>
  15     <a id="HOLDEUF000000000Y67">
  16       <GENOLIST genotyped="Y" public="Y" />
  17       <RH_GENES percent="25" />
  18     </a>
  19   </animals>
  20 </interbull>