B ‰°b„ã@s„dZddlZddlmZddlmZddlmZddlm Z ddl m Z dd l m Z d ZGd d „d e ƒZdd d„ZGdd„dƒZdS)aSBio.AlignIO support for the "maf" multiple alignment format. The Multiple Alignment Format, described by UCSC, stores a series of multiple alignments in a single file. It is suitable for whole-genome to whole-genome alignments, metadata such as source chromosome, start position, size, and strand can be stored. See http://genome.ucsc.edu/FAQ/FAQformat.html#format5 You are expected to use this module via the Bio.AlignIO functions(or the Bio.SeqIO functions if you want to work directly with the gapped sequences). Coordinates in the MAF format are defined in terms of zero-based start positions (like Python) and aligning region sizes. A minimal aligned region of length one and starting at first position in the source sequence would have ``start == 0`` and ``size == 1``. As we can see on this example, ``start + size`` will give one more than the zero-based end position. We can therefore manipulate ``start`` and ``start + size`` as python list slice boundaries. For an inclusive end coordinate, we need to use ``end = start + size - 1``. A 1-column wide alignment would have ``start == end``. éN)Úislice)Údbapi2)ÚMultipleSeqAlignment)ÚSeq)Ú SeqRecordé)ÚSequentialAlignmentWriteréc@s(eZdZdZdd„Zdd„Zdd„ZdS) Ú MafWriterz9Accepts a MultipleSeqAlignment object, writes a MAF file.cCs|j d¡|j d¡dS)zWrite the MAF header.z##maf version=1 scoring=none z# generated by Biopython N)ÚhandleÚwrite)Úself©rú0lib/python3.7/site-packages/Bio/AlignIO/MafIO.pyÚ write_header4s zMafWriter.write_headerc Cs¬|j d¡dkrd}n|j d¡dkr,d}nd}dd|j dd ¡d |j d d ¡d |j dtt|jƒ dd¡ƒ¡|d |j dd ¡t|jƒg}|j dd  |¡¡dS)zHWrite a single SeqRecord object to an 's' line in a MAF block (PRIVATE).Ústrandrú+éÿÿÿÿú-Úsz%-40sú Ú_z%15sÚstartrz%5sÚsizeÚÚsrcSizez%s N) Ú annotationsÚgetÚidÚreplaceÚlenÚstrÚseqr r Újoin)r ÚrecordrZfieldsrrrÚ _write_record9s  zMafWriter._write_recordcCs®t|tƒstdƒ‚tdd„|Dƒƒdkr0tdƒ‚yd dd„|j ¡Dƒ¡}Wntk rfd }YnX|j   d |f¡d }x|D]}|  |¡|d7}q„W|j   d ¡|S) zÅWrite a complete alignment to a MAF block. Writes every SeqRecord in a MultipleSeqAlignment object to its own MAF block (beginning with an 'a' line, containing 's' lines). zExpected an alignment objectcSsh|] }t|ƒ’qSr)r )Ú.0Úxrrrú Zsz,MafWriter.write_alignment..rz%Sequences must all be the same lengthrcSs$g|]\}}|dkrd||f‘qS))ZscoreÚpassz%s=%sr)r&r'Úyrrrú esz-MafWriter.write_alignment..z score=0.00za %s rÚ ) Ú isinstancerÚ TypeErrorr Ú ValueErrorr#Ú _annotationsÚitemsÚAttributeErrorr r r%)r Ú alignmentÚannoZrecs_outr$rrrÚwrite_alignmentQs"      zMafWriter.write_alignmentN)Ú__name__Ú __module__Ú __qualname__Ú__doc__rr%r5rrrrr 1sr c csBd}g}g}x.y t|ƒ}Wntk r4d}YnX|rÒ| d¡rB| ¡ ¡}t|ƒdkrhtdƒ‚|ddkrzd}n|dd krŒd }nd}t|d ƒt|d ƒ|t|d ƒdœ}|d} d| kr|sÔtdƒ‚|dj} g} x.t | | ƒD] \} } |   | dkr| n| ¡qîWd  | ¡} |  t t | ƒ|d|dd|d¡nŽ| d¡rPn€| d¡r^nr| d¡rlnd| d¡rznV| ¡sÂ|dk r t|ƒ|ks t‚t|ƒ}||_|Vd}g}g}ntd|fƒ‚q| d¡r&d}| ¡ ¡dd…}t|ƒ| d¡krtdƒ‚tdd„|Dƒƒ}q| d¡r4q|sPqWdS) zåIterate over a MAF file handle as MultipleSeqAlignment objects. Iterates over lines in a MAF file-like object (handle), yielding MultipleSeqAlignment objects. SeqRecord IDs generally correspond to species names. Frréz5Error parsing alignment - 's' line must have 7 fieldsérrrrr éé)rrrréÚ.z/Found dot/period in first sequence of alignmentr)rÚnameÚ descriptionrÚiÚeÚqú#Nz-Error parsing alignment - unexpected line: %sÚaTú=z1Error parsing alignment - invalid key in 'a' linecss|]}| d¡VqdS)rGN)Úsplit)r&Za_stringrrrú ószMafIterator..)ÚnextÚ StopIterationÚ startswithÚstriprHr r/Úintr"ÚzipÚappendr#rrÚAssertionErrorrr0ÚcountÚdict)r Z seq_countZ in_a_bundlerZrecordsÚlineÚ line_splitrr4ÚsequenceÚrefÚnewZletterZ ref_letterr3Z annot_stringsrrrÚ MafIterator}sŠ                     rYc@sreZdZdZdd„Zdd„Zdd„Zdd „Zed d „ƒZ ed d „ƒZ dd„Z dd„Z ddd„Z dd„Zdd„ZdS)ÚMafIndexz¼Index for a MAF file. The index is a sqlite3 database that is built upon creation of the object if necessary, and queried when methods *search* or *get_spliced* are used. cCs~||_||_tj tj |¡¡|_||_t|jƒ|_ tj  |¡rXt   |¡|_ | ¡|_nt   |¡|_ | ¡|_t|j ƒ|_dS)z)Indexes or loads the index of a MAF file.N)Ú_target_seqnameÚ_index_filenameÚosÚpathÚabspathÚdirnameÚ_relative_pathÚ _maf_fileÚopenÚ_maf_fpÚisfilerZconnectÚ_conÚ_MafIndex__check_existing_dbÚ _record_countÚ_MafIndex__make_new_indexrYÚ_mafiter)r Z sqlite_fileZmaf_fileZtarget_seqnamerrrÚ__init__s      zMafIndex.__init__c Csxy6t|j d¡ ¡dƒ}|tkrDd d|d|jg¡}t|ƒ‚|j d¡ ¡d}tj   |¡rj|}ntj  |j |  dtj j ¡¡}|tj  |j¡krªtd||jfƒ‚|j d ¡ ¡d}||jkrÚtd ||jfƒ‚t|j d ¡ ¡dƒ}|d krtd ƒ‚t|j d¡ ¡dƒ}||kr6td||fƒ‚|Stjtjfk rr}ztd|ƒd‚Wdd}~XYnXdS)zEPerform basic sanity checks upon loading an existing index (PRIVATE).z1SELECT value FROM meta_data WHERE key = 'version'rr,z=Index version (%s) incompatible with this version of MafIndexz;You might erase the existing index %s for it to be rebuilt.z2SELECT value FROM meta_data WHERE key = 'filename'ú/z&Index uses a different file (%s != %s)z8SELECT value FROM meta_data WHERE key = 'target_seqname'z-Provided database indexed for %s, expected %sz6SELECT value FROM meta_data WHERE key = 'record_count'rz$Unfinished/partial database providedz SELECT COUNT(*) FROM offset_dataz.Expected %s records, found %s. Corrupt index?z Problem with SQLite database: %sN)rNrfÚexecuteZfetchoneÚMAFINDEX_VERSIONr#r\r/r]r^ÚisabsrarÚsepr_rbr[rZOperationalErrorZ DatabaseError) r Z idx_versionÚmsgÚfilenameZ tmp_mafpathZ db_targetZ record_countZ records_foundÚerrrrrZ__check_existing_dbsZ      zMafIndex.__check_existing_dbcCs„|j d¡|j dt¡|j d¡|j d|jf¡tj |j¡sxtj |j¡sxtj  |j|j ¡  tjj d¡}n\tj  tj |j¡¡tjj  |j tjj ¡rÆtj  |j|j ¡  tjj d¡}ntj |j¡}|j d|f¡|j d¡d}| ¡}x@tt|d ƒƒ}|sP|j d |¡|j ¡|t|ƒ7}qW|j d ¡|j d ¡|j d ¡|j d|f¡|j ¡|S)z2Read MAF file and generate SQLite index (PRIVATE).z.CREATE TABLE meta_data (key TEXT, value TEXT);z:INSERT INTO meta_data (key, value) VALUES ('version', %s);z?INSERT INTO meta_data (key, value) VALUES ('record_count', -1);zCINSERT INTO meta_data (key, value) VALUES ('target_seqname', '%s');rlz=INSERT INTO meta_data (key, value) VALUES ('filename', '%s');zSCREATE TABLE offset_data (bin INTEGER, start INTEGER, end INTEGER, offset INTEGER);rédzCINSERT INTO offset_data (bin, start, end, offset) VALUES (?,?,?,?);z9CREATE INDEX IF NOT EXISTS bin_index ON offset_data(bin);z=CREATE INDEX IF NOT EXISTS start_index ON offset_data(start);z9CREATE INDEX IF NOT EXISTS end_index ON offset_data(end);zrrz=Invalid length for target coordinates (expected %s, found %s)N) rdÚreadlinerLÚtellr rMr/r[rHrNrÚ_ucscbin)r rTÚoffsetrUrrÚendrrrZ __maf_indexer³s.         zMafIndex.__maf_indexercCs ddg}| td|d?d|dd?ƒ¡| td|d?d|dd?ƒ¡| td|d ?d |dd ?ƒ¡| td |d ?d |dd ?ƒ¡t|ƒS)zªFind bins that a region may belong to (PRIVATE). Converts a region to a list of bins that it may belong to, including largest and smallest bins. rrér é éé éIééJiIéiJ)ÚextendÚrangeÚset)rr|ZbinsrrrÚ _region2binäs $$$$zMafIndex._region2bincCsddddddg}d}d}|}|d}||L}||L}x,|D]$}||krL||S||L}||L}q8WdS)z—Return the smallest bin a given region will fit into (PRIVATE). Adapted from http://genomewiki.ucsc.edu/index.php/Bin_indexing_system iIrr~rrr„r<r)rr|Z bin_offsetsZ_bin_first_shiftZ_bin_next_shiftZ start_binZend_binZ bin_offsetrrrrzôs  zMafIndex._ucscbincCs|j |¡t|jƒS)zFRetrieve a single MAF record located at the offset provided (PRIVATE).)rdÚseekrJrj)r r{rrrÚ _get_record s zMafIndex._get_recordc cs”t|ƒt|ƒkrtdƒ‚x8t||ƒD]*\}}||}|dkr$td|||fƒ‚q$W|j}tƒ}x.t||ƒD]\}}yd tt| ||¡ƒ¡}Wn&t k r¸t d||fƒd‚YnX|  d|||d|df¡} |   ¡} xª| D]¢\} } } | | f|krqæn|  | | f¡|  t| ƒ¡}xb|D]Z}|j|jkr$|jd}||jd d}|| krf|| ks$td | | | ||fƒ‚q$W|VqæWqlWdS) aäSearch index database for MAF records overlapping ranges provided. Returns *MultipleSeqAlignment* results in order by start, then end, then internal offset field. *starts* should be a list of 0-based start coordinates of segments in the reference. *ends* should be the list of the corresponding segment ends (in the half-open UCSC convention: http://genome.ucsc.edu/blog/the-ucsc-genome-browser-coordinate-counting-systems/). z2Every position in starts must have a match in endsrz7Exon coordinates (%d, %d) invalid: exon length (%d) < 1z, z4Exon coordinates must be integers (start=%d, end=%d)NzžSELECT DISTINCT start, end, offset FROM offset_data WHERE bin IN (%s) AND (end BETWEEN %s AND %s OR %s BETWEEN start AND end) ORDER BY start, end, offset ASC;rrz'Expected %s-%s @ offset %s, found %s-%s)r r/rOrfr‡r#Úmapr!rˆr.rmZfetchallÚaddrŠrNrr[r)r ÚstartsÚendsÚ exonstartÚexonendZexonlenZconZyielded_rec_coordsZ possible_binsÚresultZrowsÚ rec_startÚrec_endr{Úfetchedr$rr|rrrÚsearchsJ     zMafIndex.searchrc Cs|dkrtd|ƒ‚t| ||¡ƒ}tdd„t||ƒDƒƒ}t|ƒdkrdtttd|ƒ|j dgƒSdd „|Dƒ}d d „|Dƒ}d}d } x’|D]ˆ} x| D]î} | j |j kržyL| d krÔ| j d } | dkrøtdƒ‚n$| | j d krøtd| j d | fƒ‚Wn&t k r td|j ƒd ‚YnXt| ƒ} | j d} | j d}| |d}||7}x6| D].} x&t | |dƒD]}d|| j |<qlWqXWPqžWtd|j fƒ‚| }xtt d| ƒD]f}xB| D]:} | j |j krÖ| j|}|| j || j|7<qºW|dkr°||kr°|d7}q°WqWt||j ƒ|krPtd|j t||j ƒ|fƒ‚dd „||j  ¡Dƒ}i}x®|D]¦}||}g}||j kr’dnd}|j}xjt||ƒD]\\}}xPt ||ƒD]B}||krØ|||ƒn$||krô||||ƒn||ƒq¼Wq¨Wd |¡||<qrWt||j  dd¡ƒ|kr^tdt||j  dd¡ƒ|j |fƒ‚t||j ƒ}x:| ¡D].\}}t|ƒ|krvtdt|ƒ||fƒ‚qvWg}xJ| ¡D]>\}}t|ƒ}|| krÔ|n| ¡}| t|||dd¡q¶Wt|ƒS)aéReturn a multiple alignment of the exact sequence range provided. Accepts two lists of start and end positions on target_seqname, representing exons to be spliced in silico. Returns a *MultipleSeqAlignment* of the desired sequences spliced together. *starts* should be a list of 0-based start coordinates of segments in the reference. *ends* should be the list of the corresponding segment ends (in the half-open UCSC convention: http://genome.ucsc.edu/blog/the-ucsc-genome-browser-coordinate-counting-systems/). To ask for the alignment portion corresponding to the first 100 nucleotides of the reference sequence, you would use ``search([0], [100])`` )rrzStrand must be 1 or -1, got %scss|]\}}||VqdS)Nr)r&rr|rrrrIsz'MafIndex.get_spliced..rÚN)rcSsh|]}|D] }|j’q qSr)r)r&ÚmultiseqrVrrrr(˜sz'MafIndex.get_spliced..cSsi|] }i|“qSrr)r&Zseq_namerrrú ¡sz(MafIndex.get_spliced..NrzStrand must be 1 or -1z8Encountered strand='%s' on target seqname, expected '%s'z-No strand information for target seqname (%s)rrrrz#Did not find %s in alignment bundlerz/Target seqname (%s) has %s records, expected %scSs&i|]\}}t|ƒdkrt|ƒ|“qS)r)r )r&ÚposZgapped_fragmentrrrr˜õsz9Returning %s letters for target seqname (%s), expected %sz'Returning length %s for %s, expected %s)rr@rA)r/rwr•ÚsumrOr rrrr[rrÚKeyErrorr†r"r1rPr#rZreverse_complement) r rrŽrr”Zexpected_lettersZ all_seqnamesZsplit_by_positionZtotal_rec_lengthZref_first_strandr—ZseqrecZ rec_lengthr’Zungapped_lengthr“r™Zreal_posZ gapped_posZ track_valZrealpos_to_lenZsubseqZseqidZ seq_splitZ seq_spliceZ filler_charrPrrZref_subseq_lenr"Zresult_multiseqrrrÚ get_splicedus®              "      zMafIndex.get_splicedcCsd|jj|jfS)z,Return a string representation of the index.z%MafIO.MafIndex(%r, target_seqname=%r))rdr@r[)r rrrÚ__repr__8szMafIndex.__repr__cCs|jS)z*Return the number of records in the index.)rh)r rrrÚ__len__?szMafIndex.__len__N)r)r6r7r8r9rkrgrirvÚ staticmethodrˆrzrŠr•rœrržrrrrrZûsGR1  c DrZ)N)r9r]Ú itertoolsrZsqlite3rZ Bio.AlignrZBio.SeqrZ Bio.SeqRecordrZ Interfacesrrnr rYrZrrrrÚ"s      L ~