Gramps DNA Data Model

[iand]

This is a proposal of a new data model for maintaining DNA test and match data with Gramps.

Note: I maintain a working prototype of this design on the dna-core branch in my fork of gramps (comparison with current gramps)

Background

Gramps has no native DNA data model. The existing workaround uses Person Attributes, Events, Associations, and Notes. This has three consequences: no standard schema, no purpose-built UI, and no reliable interoperability between addons.

Existing Addons

DNA Segment Map Gramplet

• Visualises shared autosomal segments as coloured bands across chromosomes.
• Data model: Associations of type "DNA" between people, with a Note containing segment data (Chromosome, Start, End, cM, SNPs) as structured text.
• Reciprocal associations required; SyncAssociations addon automates creation.

DNA Matches Gramplet

• Tabular view of all DNA matches for a person.
• Columns: person, Gramps ID, relationship, shared DNA, segments, largest segment, sources.
• Uses the same Association + Note + Citation model as Segment Map.

FamilyTreeDNA Gramplet (2024)

• Imports and displays FTDNA-specific data for use with the DNA Segment Map gramplet.

Current De Facto Data Model

No official schema exists. Addons have converged on the following conventions.

Kit and identity data - Person Attributes

Attribute key	Value
AncestryID	username
DNAkit	GEDmatch kit number
FTDNA kit	kit number
Y-DNA Hg	haplogroup (e.g. "I-L22*, confirmed")
mtDNA Hg	haplogroup

Y-DNA and mtDNA test results - Event

• Event type: "DNA Test" on the Person.
• One Attribute per STR marker (DYS393 = 13, etc.).
• HVR1, HVR2, Coding attributes for mtDNA mutations (comma-separated values).
• Raw data files attached as Media.

Autosomal match data - Association

• PersonRef between two people, relation type "DNA" (or "cM").
• Attached Note: one segment per line as Chromosome, Start, End, cM, SNPs.
• Attached Citations: one per testing provider.
• Reciprocal records required (A -> B and B -> A).

Limitations of current de facto model

• No support for common ancestor research: this is a three-way relationship between two test takers and an ancestor which associations can't support.
• No way to record an unidentified match. Associations link two Person records. If a match has not been identified in the tree, either a placeholder Person must be created (polluting the People database with non-people) or the match is simply not recorded. The vast majority of matches fall into this category.
• Individual match records have no independent privacy or tagging. Privacy is inherited from the Person record. A sensitive match such as an unknown biological relative, non-paternity event, cannot be independently marked private or tagged without affecting the Person.
• A Person Attribute has no provider context. The kit number is stored on the Person, but the format, the genome build, and the test type must all be inferred from naming conventions.
• Segment coordinates have no associated genome build. GRCh37 and GRCh38 positions are not interchangeable.
• Segment data cannot be queried without parsing. "Find all segments on chromosome 7 over 10 cM" requires re-parsing free text each time. No addon can filter or sort without reimplementing the parser.
• Phase (maternal/paternal) is ephemeral. The Segment Map gramplet computes it from the tree at display time and discards it. It cannot be stored alongside the segment, corrected by the user, or used as a filter criterion.
• Generally, writing filters and searches against DNA data can't use any of the rich facilities that Gramps provides for navigating the database.

Proposed Data Model

DNA test data divides into two distinct things:

1. A kit / test - one person, one provider, one result set. It anchors many matches and carries structured result data.
2. A match - a pairwise relationship between two kits. Not owned by either person. A bilateral relationship that genealogists actively research, accumulate evidence for, and track to a common ancestor.

DNATest (primary object)

Represents a single test kit for a single person at a single provider.

Field	Type	Notes
handle	str	Internal DB key
gramps_id	str	Prefix "T%05d" -> T00001
change	timestamp
private	bool
tag_list	list
person_handle	Person ref	Nullable; set when the kit owner is identified
account_name	str	Name as shown on the provider platform
provider	DNAProviderType	AncestryDNA, 23andMe, MyHeritage, FTDNA, GEDmatch, LivingDNA, ...
kit_id	str	Provider-assigned kit identifier (e.g. kit number on FTDNA, alphanumeric code on GEDmatch)
test_type	DNATestType	Autosomal, Y-DNA 12, Y-DNA 37, Y-DNA 67, Y-DNA 111, Big Y, mtDNA HVR1, mtDNA Full
genome_build	DNAGenomeBuildType	Coordinate system used by this kit's provider: GRCh37, GRCh38, or unknown
date	DateBase	When the test was taken or results received
haplogroup	str	e.g. "R-L21", "H1b"; populated for Y-DNA and mtDNA
attribute_list	AttrBase	Y-STR markers, mtDNA mutations, and other key-value metadata
citation_list	CitationBase
note_list	NoteBase
media_list	MediaBase	Raw data files (CSV, BAM, etc.)

Marker and mutation storage

Y-STR markers and mtDNA mutations are stored as CUSTOM attributes in attribute_list. The marker or locus name becomes the type string and the allele or mutation value becomes the attribute value:

Attribute type (CUSTOM string)	Attribute value
DYS393	13
DYS390	24
HVR1	16069T, 16126C
Coding	315.1C

Multi-copy Y-STR markers store both allele values as a comma-separated string, following the convention used by the existing DNA Gramplet addon.

DNAMatch (primary object)

Represents a pairwise DNA match between two kits.

One side is the subject's DNATest (the kit whose match list this came from). The other side always has a DNATest record. The genealogical research task is assigning Person records to DNATest records; once done, the match is fully identified without any changes to DNAMatch itself.

Field	Type	Notes
handle	str	Internal DB key
gramps_id	str	Prefix "M%05d" -> M00001
change	timestamp
private	bool
tag_list	list
subject_test_handle	DNATest ref	The subject's DNA Test that generated this match
match_test_handle	DNATest ref	The matching DNA Test
shared_cm	float	Total shared centimorgans
percent_shared	float	Percentage of genome shared
segment_count	int	Number of shared segments
largest_segment_cm	float	Largest segment in cM
predicted_relationship	str	Platform's predicted relationship label
predicted_generations	float	Platform's estimated generations to MRCA
shared_ancestor_list	list of SharedAncestor	See below
segment_list	list of DNASegment	See below
attribute_list	AttrBase
citation_list	CitationBase
note_list	NoteBase
media_list	MediaBase

DNASegment (secondary object of DNAMatch)

Field	Type	Notes
chromosome	str	"1"-"22", "X", "Y", "MT"
start_bp	int	Start position in base pairs
end_bp	int	End position in base pairs
start_rsid	str	RSID of the first SNP in the segment; nullable
end_rsid	str	RSID of the last SNP in the segment; nullable
shared_cm	float	cM for this segment
snp_count	int	Number of SNPs compared in this segment
phase	int	Class constants: PHASE_UNASSIGNED=0, PHASE_UNKNOWN=1, PHASE_MATERNAL=2, PHASE_PATERNAL=3

SharedAncestor (secondary object of DNAMatch)

The core research task on a match is identifying the individual or individuals through whom the shared DNA passes. Each SharedAncestor entry names one person. Multiple entries represent parallel working hypotheses, independently confirmed connections through separate lines, or both members of an ancestral couple - in which case one entry is added per person.

Field	Type	Notes
person_handle	Person ref	Nullable; the specific individual who is the MRCA
description	str	Free text for ancestors not yet in the tree
confidence	int	Class constants: CONF_POSSIBLE=0, CONF_PROBABLE=1, CONF_CONFIRMED=2, CONF_REJECTED=3
citation_list	CitationBase	Documentary evidence for the connection
note_list	NoteBase

person_handle is nullable to allow recording a hypothesis before the person has been added to the tree. description carries the free-text description in that case and can supplement a linked person with additional context.

Provider export examples

GEDmatch match list export. Kit codes and names are anonymised. Email addresses are PII and are discarded on import. LargestXSeg and TotalXCM are X-DNA summaries not stored in the model; X segments appear in segment_list when chromosome browser data is fetched separately. Overlap is a GEDmatch-specific SNP overlap count with no equivalent in the schema and is discarded. TestCompany maps to the match DNATest's provider. Gen maps to predicted_generations.

PrimaryKit,PrimaryName,PrimaryEmail,MatchedKit,MatchedName,MatchedEmail,LargestSeg,TotalCM,Gen,LargestXSeg,TotalXCM,Overlap,CreatedDate,TestCompany
AA001,*jbrown,primary@example.com,AA002,Jane Brown,match1@example.com,49.634,153.980,3.300,0.000,0.000,454009,2018-09-18,Migration - F2 - A
AA001,*jbrown,primary@example.com,AA003,Mary Jones,match2@example.com,26.602,147.002,3.300,0.000,0.000,485414,2023-08-09,Ancestry
AA001,*jbrown,primary@example.com,AA004,*JC,match3@example.com,29.354,76.608,3.800,0.000,0.000,544298,2021-01-18,Ancestry
AA001,*jbrown,primary@example.com,AA005,T Smith,match4@example.com,36.771,66.480,3.900,0.000,0.000,544169,2022-06-05,Ancestry

MyHeritage chromosome browser export. Start RSID and End RSID map to
start_rsid and end_rsid.

Name        Match Name  Chromosome  Start Location  End Location  Start RSID  End RSID  Centimorgans  SNPs
Jane Brown  John Smith         12       106436777     114410428   rs61940195  rs999444        7.2     3712
Jane Brown  John Smith         15       29582647       33301508   rs11071875  rs1375933       7.2     1664
Jane Brown  John Smith         16       55880825       59662867   rs11076120  rs62059121      6.3     2432
Jane Brown  John Smith         18       10703200       25002778   rs16975229  rs2879447      14.6     5632

GEDmatch segment data. The "B37" prefix in the column headers declares the genome build explicitly; genome_build can be set to GRCh37 without user input. The Segment threshold, Bunch limit, and SNP Density Ratio columns are GEDmatch
algorithm parameters with no equivalent in the schema and are discarded on import.

Chr  B37 Start Pos'n  B37 End Pos'n   Centimorgans (cM)   SNPs  Segment threshold  Bunch limit  SNP Density Ratio
1           752,721      3,512,800            8.9        421             201            120          0.25
3        59,409,524     68,684,050           15.2       2,185            190            114          0.29
7         4,503,136     20,156,313           26.3       3,388            206            123          0.29
11       83,566,115    107,580,985           18.4       3,629            209            125          0.28
15       35,364,792     38,990,803            7           699            221            133          0.34
18          479,891      4,092,001           12           893            204            122          0.35
20       56,098,733     60,294,651           16.6       1,214            236            141          0.36
22       20,876,509     46,879,687           49.6       5,031            220            132          0.31

Design Notes

Kit anchoring. Both sides of a match are represented by DNATest records. Importing a match list creates one DNATest per match (with account_name from the platform and person_handle null). Identifying a match means setting DNATest.person_handle on that record; the DNAMatch itself does not change. A person with kits at multiple providers has independent match lists per kit. A match from AncestryDNA and a match from 23andMe are separate DNAMatch records
even if the other side is the same person.

Haplogroup propagation. Haplogroup is stored on DNATest as the raw result reported by the provider. Confirmation, triangulation, and inference from tested relatives are derived analyses outside the scope of the data model; a future plugin could compute and display these without writing to the database.

Relationship display. No confirmed relationship field is stored. Once both DNATest records have person_handle set, the relationship is computable from the tree using Gramps' existing relationship calculator. A single stored string cannot represent multiple lines of descent (endogamy, pedigree collapse). The UI displays computed relationship(s) on demand. predicted_relationship is retained as a raw record of what the provider platform reported.

Genome build. Stored on DNATest.genome_build. The build is a property of the kit's provider platform (e.g. MyHeritage uses GRCh37, FTDNA uses GRCh38). Segment coordinates in any match are implicitly in the build of subject_test_handle.genome_build. GEDmatch issues separate kit codes per build; the kit code determines which build applies.

Gramps XML Format

New top-level sections

<dnatests> and <dnamatches> are added inside <database>, after <notes> and
before <bookmarks>, following the same order convention as all other primary object
types.

DNATest element

<dnatest handle="_abc123" change="1234567890" id="T00001" priv="0">
  <person hlink="_person123"/>
  <account_name>JohnSmith1972</account_name>
  <provider>FTDNA</provider>
  <kit_id>123456</kit_id>
  <test_type>Y-DNA 111</test_type>
  <genome_build>GRCh37</genome_build>
  <dateval val="2018-11-04" quality="Regular" .../>
  <haplogroup>R-L21</haplogroup>
  <attribute type="DYS393" value="13"/>
  <objref hlink="..."/>
  <noteref hlink="..."/>
  <citationref hlink="..."/>
  <tagref hlink="..."/>
</dnatest>

<person> is omitted when person_handle is null. <provider>, <test_type>, and <genome_build> use xml_str() and set_from_xml_str(), the same pattern as EventType. An unrecognised string in any of these fields round-trips as CUSTOM.
<kit_id>, <haplogroup>, and <account_name> are omitted when empty.

DNAMatch element

<dnamatch handle="_def456" change="1234567890" id="M00001" priv="0">
  <subject_test hlink="_abc123"/>
  <match_test hlink="_def789"/>
  <shared_cm val="153.98"/>
  <percent_shared val="2.17"/>
  <segment_count val="8"/>
  <largest_segment_cm val="49.63"/>
  <predicted_relationship>3rd-4th cousin</predicted_relationship>
  <predicted_generations val="3.3"/>
  <shared_ancestor confidence="2">
    <person hlink="_person456"/>
    <description>via Smith line</description>
    <noteref hlink="..."/>
    <citationref hlink="..."/>
  </shared_ancestor>
  <dna_segment chromosome="1" start_bp="83566115" end_bp="107580985"
               shared_cm="18.4" snp_count="3629" phase="0"
               start_rsid="rs12345678" end_rsid="rs98765432"/>
  <objref hlink="..."/>
  <noteref hlink="..."/>
  <citationref hlink="..."/>
  <tagref hlink="..."/>
</dnamatch>

<person> within <shared_ancestor> is omitted when person_handle is null.
confidence and phase are stored as their integer class constant values.
start_rsid and end_rsid on <dna_segment> are omitted when null.
The numeric stat fields (shared_cm, percent_shared, etc.) are omitted when zero.

Bookmarks

The existing <bookmarks> section gains two new target values:

<bookmark target="dnatest" hlink="_abc123"/>
<bookmark target="dnamatch" hlink="_def456"/>

GEDCOM

GEDCOM export and import are out of scope. The GEDCOM community is actively discussing DNA extensions as of 2024 but no ratified standard exists. GEDCOM support should be revisited once a stable extension proposal is available.

User Interface Mockups

Screenshots are taken from my working prototype dna-core branch

DNA Tests Navigator View

DNA Test Edit Form

DNA Matches Navigator View

DNA Match Edit Form

Shared Ancestor Edit Form

DNA Segment Edit Form