Differences between revisions 5 and 7 (spanning 2 versions)

GEBVtest Merged Files

The gebvtest.py program offers an option (-m, --mergefiles) to create a file of merged Cf/Df/Cr/Gr records as a convenience to the user. These files can make it easier to check for the correctness of the input datasets and they can be used to perform additional checks and/or statistical analyses, using R or SAS,for example.

Two sets of files are created for each of the trait analysed:

One file containing information on the base adjustment performed by the software (trait_bEtDGbase.csv)
One file to perform additional tests (trait_bEtDG_test.csv).

Please see the file format below.

By default, the files are created in directory DATADIR/merged. The merged files do not have a _POPBRD extension, so if you would like to create files for more than one population or breed, you should also supply the -M, --mergedir option with a different destination directory for each population/breed. The destination directory can be an absolute path, or it can be relative to the programs directory (eg. ../sample_data/my_merges). The directory will be created automatically if it does not exist.

File format trait_bEtDGbase.csv

The file is in comma-separated-variable (csv) format, using commas as the separator.

Column	Variable	Type	Description
1	aid	char(19)	animal ID
2	byear	int	Birth year
3	top	char(2)	Type of proof
4	off	char(1)	Official status (Y/N)
5	status	char(2)	Status of bull
6	nd	int	Number of daughters
7	nh	int	number of herds
8	Cf	char(2)	Fixed separator for Cf info
9	edc	int	EDC from Cf file
10	rel	real	Reliability from Cf file (x100)
11	ebv	real	Predicted genetic merit ("proof") from Cf file
12	Df	char(2)	Fixed separator for Df info
13	edcd	int	EDC from Df file
14	reld	real	Reliability from Df file (x100)
15	dpgm	real	DD or deregressed proof from Df file
16	Gf	char(2)	Fixed separator for Gf info
17	edcgf	int	EDC from Gf file
18	relgf	real	reliability from Gf file
19	gebvf	real	Predicted genomic merit ("proof") from Gf file
20	DGf	Char (3)	Fixed separator for DGf info
21	edcvf	int	EDC from vf file
22	relvf	real	reliability from vf file
23	dgebvf	real	deregressed genomic merit ("proof")
24	Cr	char(2)	Fixed separator for Cr info
25	edcr	int	EDC from Cr file
26	relr	real	Reliability from Cr file (x100)
27	ebvr	real	Predicted genetic merit ("proof") from Cr file
28	Gr	char(2)	Fixed separator for Gr info
29	edcg	int	EDC from Gr file
30	relg	real	Reliability from Gr file (x100)
31	gebv	real	GEBV from Gr file

File format trait_bEtDG_test.csv

The file is in comma-separated-variable (csv) format, using commas as the separator.

Column	Variable	Type	Description
1	aid	char(19)	animal ID
2	byear	int	Birth year
3	flag	char(1)	flag
4	candidate	char(1)	Candidate bull (Y/N)
5	test bull	char(1)	test bulls (Y/N)
6	top	char(2)	Type of proof
7	off	char(1)	Official status (Y/N)
8	status	char(2)	Status of bull
9	nd	int	Number of daughters
10	nh	int	number of herds
11	Cf	char(2)	Fixed separator for Cf info
12	edc	int	EDC from Cf file
13	rel	real	Reliability from Cf file (x100)
14	ebv	real	Predicted genetic merit ("proof") from Cf file
15	Df	char(2)	Fixed separator for Df info
16	edcd	int	EDC from Df file
17	reld	real	Reliability from Df file (x100)
18	dpgm	real	DD or deregressed proof from Df file
19	Gf	char(2)	Fixed separator for Gf info
20	edcgf	int	EDC from Gf file
21	relgf	real	reliability from Gf file
22	gebvf	real	Predicted genomic merit ("proof") from Gf file
23	DGf	Char (3)	Fixed separator for DGf info
24	edcvf	int	EDC from vf file
25	relvf	real	reliability from vf file
26	dgebvf	real	deregressed genomic merit ("proof")
27	Cr	char(2)	Fixed separator for Cr info
28	edcr	int	EDC from Cr file
29	relr	real	Reliability from Cr file (x100)
30	ebvr	real	Predicted genetic merit ("proof") from Cr file
31	Gr	char(2)	Fixed separator for Gr info
32	edcg	int	EDC from Gr file
33	relg	real	Reliability from Gr file (x100)
34	gebv	real	GEBV from Gr file

-  ⇤ ← Revision 5 as of 2013-04-02 18:36:13 → 
  Size: 2780
  Editor: 88-149-167-82
  Comment:
+   ← Revision 7 as of 2024-08-12 15:40:51 → ⇥
  Size: 4880
  Editor: Valentina
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 2:
+The [[https://wiki.interbull.org/public/gebvtest_py?action=print|gebvtest.py]] program offers an option (''-m, --mergefiles'') to create a file of merged Cf/Df/Cr/Gr records as a convenience to the user. These files can make it easier to check for the correctness of the input datasets and they can be used to perform additional checks and/or statistical analyses, using R or SAS,for example.
-Line 3:
+Line 4:
-The [[https://wiki.interbull.org/public/gebvtest_py?action=print|gebvtest.py]] program offers an option (''-m, --mergefiles'') to create a file of merged Cf/Df/Cr/Gr records as a convenience to the user. These files can make it easier to check for the correctness of the input datasets and they can be used perform additional checks and/or statistical analyses, using R for example as in this small [[attachment:test.R]].
+Two sets of files are created for each of the trait analysed:
-Line 5:
+Line 6:
-One file is created for each trait present in the traits_POPBRD file and the file300Xy_POPBRD files. There is a record for each bull present in the Cf file born in/after the cutoff year specified in the traits_POPBRD file.
Flags are supplied to indicate whether the bull qualifies as a candidate or a test bull. Please see the file format below.
+ * One file containing information on the base adjustment performed by the software (''trait''_bEtDG''base''.csv)
 * One file to perform additional tests (''trait''_bEtDG_''test''.csv).

Please see the file format below.
-Line 10:
+Line 13:
-== File format ==
+== File format trait_bEtDGbase.csv ==
The file is in comma-separated-variable (csv) format, using commas as the separator.
||'''Column''' ||'''Variable''' ||'''Type''' ||'''Description''' ||
||1 ||aid ||char(19) ||animal ID ||
||2 ||byear ||int ||Birth year ||
||3 ||top ||char(2) ||Type of proof ||
||4 ||off ||char(1) ||Official status (Y/N) ||
||5 ||status ||char(2) ||Status of bull ||
||6 ||nd ||int ||Number of daughters ||
||7 ||nh ||int ||number of herds ||
||8 ||Cf ||char(2) ||Fixed separator for Cf info ||
||9 ||edc ||int ||EDC from Cf file ||
||10 ||rel ||real ||Reliability from Cf file (x100) ||
||11 ||ebv ||real ||Predicted genetic merit ("proof") from Cf file ||
||12 ||Df ||char(2) ||Fixed separator for Df info ||
||13 ||edcd ||int ||EDC from Df file ||
||14 ||reld ||real ||Reliability from Df file (x100) ||
||15 ||dpgm ||real ||DD or deregressed proof from Df file ||
||16 ||Gf ||char(2) ||Fixed separator for Gf info ||
||17 ||edcgf ||int ||EDC from Gf file ||
||18 ||relgf ||real ||reliability from Gf file ||
||19 ||gebvf ||real ||Predicted genomic merit ("proof") from Gf file ||
||20 ||DGf ||Char (3) ||Fixed separator for DGf info ||
||21 ||edcvf ||int ||EDC from vf file ||
||22 ||relvf ||real ||reliability from vf file ||
||23 ||dgebvf ||real ||deregressed genomic merit ("proof") ||
||24 ||Cr ||char(2) ||Fixed separator for Cr info ||
||25 ||edcr ||int ||EDC from Cr file ||
||26 ||relr ||real ||Reliability from Cr file (x100) ||
||27 ||ebvr ||real ||Predicted genetic merit ("proof") from Cr file ||
||28 ||Gr ||char(2) ||Fixed separator for Gr info ||
||29 ||edcg ||int ||EDC from Gr file ||
||30 ||relg ||real ||Reliability from Gr file (x100) ||
||31 ||gebv ||real ||GEBV from Gr file ||
-Line 12:
+Line 48:
-The file is in comma-separated-variable (csv) format, using commas as the separator.
-Line 14:
+Line 49:
-||'''Column'''||'''Variable'''||'''Type'''||'''Description'''||
||1||aid||char(19)||animal ID||
||2||byear||int||Birth year||
||3||geno||char(1)||Bull has a GEBV (Gr) record (Y/N)||
||4||cand||char(1)||Bull qualifies as a candidate bull (Y/N)||
||5||test||char(1)||Bull qualifies as a test bull (Y/N)||
||6||top||char(2)||Type of proof (from Cf file)||
||7||off||char(1)||Official proof (Y/N; from Cf file)||
||8||sta||char(2)||Bull status (from Cf file)||
||9||nd||int||Number of daughters (from Cf file)||
||10||nh||int||Number of herds (from Cf file)||
||11||Cf||char(2)||Fixed separator for Cf info||
||12||edc||int||EDC from Cf file||
||13||rel||real||Reliability from Cf file (x100)||
||14||ebv||real||Predicted genetic merit ("proof") from Cf file||
||15||Df||char(2)||Fixed separator for Df info||
||16||edcd||int||EDC from Df file||
||17||reld||real||Reliability from Df file (x100)||
||18||dpgm||real||DD or deregressed proof from Df file||
||19||Cr||char(2)||Fixed separator for Cr info||
||20||edcr||int||EDC from Cr file||
||21||relr||real||Reliability from Cr file (x100)||
||22||ebvr||real||Predicted genetic merit ("proof") from Cr file||
||23||Gr||char(2)||Fixed separator for Gr info||
||24||edcg||int||EDC from Gr file||
||25||relg||real||Reliability from Gr file (x100)||
||26||gebv||real||GEBV from Gr file||
+== File format trait_bEtDG_test.csv ==
The file is in comma-separated-variable (csv) format, using commas as the separator.
||'''Column''' ||'''Variable''' ||'''Type''' ||'''Description''' ||
||1 ||aid ||char(19) ||animal ID ||
||2 ||byear ||int ||Birth year ||
||3||flag||char(1)|| flag||
||4||candidate||char(1)|| Candidate bull (Y/N)||
||5||test bull||char(1)|| test bulls (Y/N)||
||6 ||top ||char(2) ||Type of proof ||
||7 ||off ||char(1) ||Official status (Y/N) ||
||8 ||status ||char(2) ||Status of bull ||
||9 ||nd ||int ||Number of daughters ||
||10 ||nh ||int ||number of herds ||
||11||Cf ||char(2) ||Fixed separator for Cf info ||
||12||edc ||int ||EDC from Cf file ||
||13 ||rel ||real ||Reliability from Cf file (x100) ||
||14 ||ebv ||real ||Predicted genetic merit ("proof") from Cf file ||
||15 ||Df ||char(2) ||Fixed separator for Df info ||
||16 ||edcd ||int ||EDC from Df file ||
||17 ||reld ||real ||Reliability from Df file (x100) ||
||18 ||dpgm ||real ||DD or deregressed proof from Df file ||
||19 ||Gf ||char(2) ||Fixed separator for Gf info ||
||20 ||edcgf ||int ||EDC from Gf file ||
||21 ||relgf ||real ||reliability from Gf file ||
||22 ||gebvf ||real ||Predicted genomic merit ("proof") from Gf file ||
||23 ||DGf ||Char (3) ||Fixed separator for DGf info ||
||24 ||edcvf ||int ||EDC from vf file ||
||25 ||relvf ||real ||reliability from vf file ||
||26 ||dgebvf ||real ||deregressed genomic merit ("proof") ||
||27 ||Cr ||char(2) ||Fixed separator for Cr info ||
||28 ||edcr ||int ||EDC from Cr file ||
||29 ||relr ||real ||Reliability from Cr file (x100) ||
||30 ||ebvr ||real ||Predicted genetic merit ("proof") from Cr file ||
||31 ||Gr ||char(2) ||Fixed separator for Gr info ||
||32 ||edcg ||int ||EDC from Gr file ||
||33 ||relg ||real ||Reliability from Gr file (x100) ||
||34 ||gebv ||real ||GEBV from Gr file ||