GenoEx-GDE User’s manual v.1.0

part 2 – Database Manual

GDE stands for Genomic Data Exchange and GenoEx-GDE is the database that allows users to exchange whole SNP sets, where users define how and with whom the data is to be shared.

The whole system consists of the database itself, where data files can be uploaded and downloaded from, and the support program – gxprep.py – for input files’ preparation.

The instructions for preparing the input files from raw laboratory files and for assigning sharing permissions can be found in the first part of this manual at https://wiki.interbull.org/public/GDE_gxprep_manual.

This manual describes step by step how to:

• upload the data to the database
• define data selection choice and download the data from the database

1. Upload file formats

GenoEx-GDE allows upload of two types of files

• data file (format 706)
• sharing permissions file (format 711)

Detailed file formats can be found in the manual describing the usage of the support program gxprep.py (https://wiki.interbull.org/public/GDE_gxprep_manual). Here we only remind the examples of these files.

706 example

706;BSW;ITA;M;000000A12345;ANARB;09c98b1e-6af8-4254-9768-58d7cd1ddafd;Weatherbys Ireland;S1234WI2001;;54609_a;021010…

711 example

711;BSW;ITA;M;000000A12345;09c98b1e-6af8-4254-9768-58d7cd1ddafd;IBC,BFRO

2. Organizations, Users and Logins

Every organization which signs the contact for becoming GenoEx-GDE Service User is from the service’s point of view considered as single User. On the other hand, it is possible to create several separate personal logins for different persons within the organization. As one Service User, they share the History of uploads and downloads as well as all rights and permissions assigned to the User. New logins can be added to the Service User’s account at any point of time. To do so, the user should send the request to genoex@slu.se

At the login page, every person should log in with their personal credentials. In case of lost password, the Reset Password functionality is available under the login tab.

3. GDE data upload

As part of GenoEx services, GDE is sharing the entry point with PSE, therefore please make sure that you choose the right tab for the upload in the drop-down menu.

GenoEx-GDE allows for simultaneous or separate upload of 706 and 711 files. Choosing different alternatives results in one of the following scenarios:

• Uploading both 706 and 711 files, containing the same animals results in adding both genotype data and sharing permissions to the database

• Uploading only 706 file results in adding genotype data to the database with no sharing permissions attached i.e. these data is accessible only to the owner organization, until relevant sharing information is provided.

• Uploading only 711 file results in updating the sharing permissions for all the genotypes listed in it only if these genotypes, identified by combination of animal ID and UUid are already present in the database. Note: Newly uploaded sharing permissions will overwrite the existing ones. That means that uploading new 711 containing empty sharing list will remove all the existing sharing permissions for given genotypes. In order to update the existing sharing list, the User should re upload the relevant 711 file updated with the desired changes

In order to upload the files to the GenoEx-GDE database, both 706 and 711 files have to be zipped in two separate zip files. Zip format is the only one accepted by the GenoEx-GDE database for the upload.
Zip files can be created using the support program gxprep.py as described here: https://wiki.interbull.org/public/GDE_gxprep_manual#zip_example
If the User creates the zip files without using the support program gxprep.py, please make sure that the file containing genotypes (in 706 format) and compressed in the zip archive has name "genotypes.csv" and that the file containing genotypes (in 711 format) and compressed in the zip archive has name "711_file.csv"
To upload the zip files please choose the correct file from your computer and press UPLOAD

When upload starts, you will be redirected to the status page.

Note: this page does not update automatically, thus you must refresh it in order to see the progress.

The first status you will see is INITIATED, then PROCESSING and finally either FINISHED or FAILED, dependently whether the upload was successful or not.

Like in the example above, FAILED is always accompanied with the informative error message, which allows the User to find and correct the problem.

In any case, the User will receive an e-mail when the data processing is finalized.

4. GDE data download

In order to download data from GenoEx-GDE the User has to select the Download options from GDE drop-down menu.

The genotypes or the sharing permissions to be downloaded can be chosen by breed, country of origin, submitting Organisation (only for extraction of genotypes), gender and chip array used for genotyping. The User is also able to choose if to download all the genotypes available for the animals within chosen criteria (option "All") or only one per animal, with the highest call rate (option "Best Only").

Here as well, the status page should be reloaded to see the current status of the process. When the extraction is ready, a link to download the genotype file is provided:

The link to download the genotype file is also saved in the "Job history" table, after selecting the specific "Job ID". The downloaded genotypes file is formatted according to 706 file format specifications, while the sharing permissions file according to the 711 file format specifications. This means that corresponding SNP order file is needed to decode the 706 genotype string. In order to download both the genotypes and the sharing permissions, two different extraction requests should be created. The page has to be refreshed between the two requests.

5. User’s History

All the User’s actions related to genotypes upload and download are saved and can be later reviewed. As mentioned in section 2, GenoEx considers an Organisation that has a User’s Contract with Interbull Centre signed, is considered as single User, which however, can have several logins. The History is User, not Login specific, thus actions from all logins within one Organisation are listen in the history table together.

6. System Data

If the User has already prepared the upload data with the same SNP order file than the downloaded data using the gxprep.py support program, the relevant SNP order file has been saved in the defined directory (see GenoEx-GDE User’s manual part 1, section 2.1. parse). However, all the available SNP orders can be downloaded from the SYSTEM DATA page. The page allows to display also breeds, countries and allowed laboratories.

If you have any further questions and/or suggestions for improving this manual, please contact us at genoex@slu.se