/spaln

Genome mapping and spliced alignment of cDNA or amino acid sequences

Primary LanguageC++GNU General Public License v2.0GPL-2.0

SPALN information

Map and align a set of cDNA/EST or protein sequences onto a genome

Present Version 2.4.4

Last updated: 2021-04-26

Overview

Spaln (space-efficient spliced alignment) is a stand-alone program that maps and aligns a set of cDNA or protein sequences onto a whole genomic sequence in a single job. Spaln also performs spliced or ordinary alignment after rapid similarity search against a protein sequence database, if a genomic segment or an amino acid sequence is given as a query. From Version 1.4, spaln supports a combination of protein sequence database and a given genomic segment. From Version 2.2, spaln also performs rapid similarity search and (semi-)global alignment of a set of protein sequence queries against a protein sequence database. Spaln adopts multi-phase heuristics that makes it possible to perform the job on a conventional personal computer running under Unix/Linux and MacOS with limited memory. The program is written in C++ and distributed as source codes and also as executables for a few platforms. Unless binaries are not provided, users must compile the program on their own system. Although the program has been tested only on a Linux operating system, it is likely to be portable to most Unix systems with little or no modifications. The accessory program sortgrcd sorts the gene loci found by spaln in the order of chromosomal position and orientation. From version 2.3.2, spaln and sortgrcd can handle gzipped genome/database files and 'block' files without prior expansion if USE_ZLIB mode is activated upon compilation. From version 2.3.2a, compressed query sequence file(s) may also be accepted. From version 2.4.0, multiple files corresponding to different output forms can be generated at a single run.

Install

To compile the source codes in the default settings, follow the instructions below. When you download the source file in the directory download, five directories will be generated under download/spalnXX/ after installation, where XX is a version code. We assume work is your workspace, which may or may not be identical to download

  • bin : binaries
  • doc : documents
  • seqdb : sample sequences. In this directory you should format genomic or database files
  • src : source codes
  • table : parameter files used by spaln

To modify the location of executables and/or other settings, run 'configure --help' at step 6 below. (Warning: Full path name rather than relative path name must be given for executables or other directories as the arguments of the configure command.) These locations are hard coded in spaln. The locations of the 'seqdb' and 'table' directories will be respectively denoted by seqdb and table below. Hence, seqdb=download/spalnXX/seqdb, and table=download/spalnXX/table in the default settings.

  1. % mkdir download
  2. % cd download
  3. Download spalnXX.tar.gz
  4. % tar xfz spalnXX.tar.gz
  5. % cd ./spalnXX/src
  6. For compilation
    % ./configure [--help]
    Please manually edit Makefile if $(CXX) does not indicate a C++ compiler or
    % CXX=g++ ./configure [other options]
    To use zlib facilities, confirm that libz.* are installed in the load library path. Then,
    % ./configure [other options] --use_zlib=1
  7. % make
  8. % make install Executables are copied to ../bin
    makmdm program makes mutation data matrices of various PAM levels in the ../table directory
  9. % make clearall
  10. Add download/spalnXX/bin to your PATH
    % setenv PATH $PATH:download/spalnXX/bin (csh/tsh)
    $ export PATH=$PATH:download/spalnXX/bin (sh/bsh)
    Preferably, you may add the above line in your start up rc file (e.g. ~/.bashrc)
    Alternatively, move or copy download/spalnXX/bin/* to a directory on your PATH, if you have not specified the location of executables at step 6 above.
  11. If you have changed the location of table and/or seqdb directory after installation, set the env variables ALN_TAB and/or ALN_DBS as follws:
  • % setenv ALN_TAB New_Aln_Tab (csh/tsh)
  • $ export ALN_TAB=New_Aln_Tab (sh/bsh)
  • % setenv ALN_DBS New_Aln_Dbs (csh/tsh)
  • $ export ALN_DBS=New_Aln_Dbs (sh/bsh)
    Add the above lines to your rc file, so that you don't have to repeat the commands at every login time.
  1. Proceed to Format.

Format

If you do not need genome mapping or database search, you may skip this section. All sequence files should be in (multi-)fasta format.

To perform genome mapping, the genomic sequence must be formatted before use. Formatting is optional for amino acid sequence database search.

  1. % cd seqdb
  2. Download or copy genomic sequences or protein database sequence in multi-fasta format. If spaln is compiled accordingly, gzipped file need not be uncompressed (the file name must be X.gz).
  3. To use 'makeidx.pl' command, chromosomal sequences must be concatenated into a single file. The extension of the genomic sequence file must be '.mfa' or '.gf', and protein database sequence must be '.faa', to render 'make' command effective. With 'spaln -W' command, these restrictions are not obligatory. Hereafter, the file name is assumed to be xxxgnm.gf or prosdb.faa.
  4. To format xxxgnm.gf(.gz), run either of the following two commands, which are equivalent to each other except that the former is faster, accepts multiple input files, and does not need Makefile.
    % spaln -W -K[D|P] [-XGMAX_GENE] [spaln options] xxxgnm.gf(.gz) ...
    % ./makeidx.pl -i[n|p|np] [-XGMAX_GENE] [spaln options] xxxgnm.gf(.gz)
    To format prosdb.faa(.gz), run either of the following two commands, which are equivalent to each other except that the former accepts multiple input files.
    % spaln -W -KA [spaln options] prosdb.faa(.gz) ...
    % ./makeidx.pl -ia [spaln options] prosdb.faa(.gz)
  • -KX (or corresponding -ix) option specifies the "block file" xxxgnm.bkx to be constructed, where X is 'A', 'D' or 'P' and x is 'a', 'n' or 'p'. The -inp option will construct both xxxgnm.bkn (for cDNA queries) and xxxgnm.bkp (for protein queries) files together with the xxxgnm.idx and associated files. -KX option is mandatory. If -ix is omitted or x is empty, xxxgnm.idx and associated files are created but no block file is constructed.
  • The block size and k-mer size are estimated from the genome size unless explicitly specified (see below).
  • If MAX_GENE (the length of the plausibly longest gene on the genome) is not specified, MAX_GENE is also estimated from the genome size. Don't forget to specify MAX_GENE if xxxgnm.gf represents only a part of the genome!! Otherwise, MAX_GENE may be seriously underestimated.
  • Options : (default value)
    • -g: The outputs are gzipped.
    • -tN: Number of threads. (1)
    • -E: Generate local lookup table.
    • -XAN: Alphabet size of the reduced amino acids: 6 < N <= 20 (20)
    • -XBS: Bit patterns of the spaced seeds concatenated with commas. The pattern should be asymmetric when the number of patterns > 2.
    • -XCN: Number of seed patterns: 0 <= N <= 5 (0: contiguous seed)
    • -XGN: Maximum gene length (inferred from genome size)
    • -XaN: A parameter used to filter excessively abundant words (10)
    • -XbN: Block size (inferred from genome size)
    • -XkN: Word size (inferred from block size)
    • -XsN: Distance between neighboring seeds (= k)

Execution

  1. Prepare protein, cDNA, or genomic segment sequence(s) in (multi-)fasta format (denoted by query below). From 2.3.2a, gzipped fasta file(s) may be used as the query without prior expansion.
  2. Store query to work.
  3. % cd work
  4. Run spaln in one of the following four modes. Spaln does not support comparison between two genomic segments.
    (A) % spaln -Q[0|1|2|3] [-ON] [other options] genome_segment query
    (B) % spaln -Q[4|5|6|7] [-ON] [other options] -[d|D] xxxgnm query
    (C) % spaln -Q[4|5|6|7] [-ON] [other options] -[a|A] prosdb query
    (D) % spaln -Q[4|5|6|7] [-ON] [other options] prosdb.faa query
  • In the last case, prosdb.faa will be internally formatted, and the formatted results will be discarded after the end of execution.
  • Only a subset of queries may be examined if query is replaced with 'query (from to)' (quotations are necessary), where 'from' and 'to' are the first and last entry numbers in query to be examined.
  • Options: (default value)

    • -C N: Use the genetic code specified by the "transl_table number" defined in NCBI transl_table (1).

    • -E: Use local lookup table.

    • -H N: Output is suppressed if the alignment score is less than N. See also -pw. (35)

    • -K[D|P|A]: Format either genomic DNA for sequence search with DNA (D) or Protein (P) queries or protein database sequences for search with a genomic segment or protein queries (A). Use in combination with -W option below.

    • -LS: Smith-Waterman-type local alignment. This option may prune out weakly matched terminal regions.

    • -M[N[.M]]: Single or multiple output for each query

      • No option (default): single locus
      • No argument: Multiple loci up to the maximum number specified by the program (4 in the present implementation)
      • N=1: Re-search the query region not aligned in the first trial. May be useful to detect chimera or fragmented genomic region
      • N>1: Output multiple loci maximally up to N
      • M: Maximal number of candidate loci to be subjected to spliced alignment (4). If M $&lt;$ N, M is reset to N.

    • -O N[,N2,N3...]: Select output format for genome vs cDNA or aa (4)
      It is possible to output multiple files with extensions of .ON at a run by multiply applying this option. Or by concatenating the format numbers with commas or colons, ex. -O0,1,4. See also -o option.

      • N=0: Gff3 gene format
      • N=1: Alignment
      • N=2: Gff3 match format
      • N=3: Bed format
      • N=4: Exon-oriented format similar to the output of megablast -D 3
      • N=5: Intron-oriented output
      • N=6: Concatenated exon sequence in extended (multi-)fasta format, in which the exon-intron structure of the parental gene is supplied by one or more comment lines starting with ';C', such as
        ;C complement(join(1232555..1232760,1233786..1233849,1233947..1234119,
        ;C 1234206..1234392))
      • N=7: Translated amino-acid sequence. Presently not very useful for cDNA queries because the entire exon rather than an ORF is translated
      • N=8: Mapping (block) information only. Use with -Q4
      • N=10: SAM format
      • N=12: Output the same information as -O4 in the binary format. If -oOutput is set, three files named Output.grd, Output.erd, and Output.qrd will be created. Otherwise, query.grd, query.erd, and query.qrd will be created. If the -g option is set, gzip-compressed outputs will be generated.
      • N=15: Copy of the query sequence supplemented with the inferred gene structural information.

    • -O N[,N2,N3...]: Select output format for aa vs aa (4)
      It is possible to output multiple files with extensions of .ON at a run by multiply applying this option. Or by concatenating the format numbers with commas or colons, ex. -O0,1,4. See also -o option.

      • N=0: statistics (%divergence alignment_score #match #mismatch #gap #unpaired)
      • N=1: Alignment
      • N=2: Sugar format
      • N=3: Psl format
      • N=4: XYL = Coordinate + match length
      • N=5: statistics + XYL
      • N=8: Cigar format
      • N=9: Vulgar format
      • N=10: SAM format

    • -Q N: Select algorithm (3)

      • 0<=N<=3: Genomic segment in the fasta format given by the first argument vs. query given by the following arguments. See also -i option below. One may skip the formatting step described above if only this mode of operation is used.
      • 4<=N<=7: Genome mapping and alignment. The genomic sequence must be formatted beforehand.
      • N=0,4: DP procedure without HSP search. Considerably slow
      • N=1,2,3,5,6,7: Recursive HSP search up to the level of (N % 4)

    • -R S: Read block index table from file S. If omitted, the xxxgnm.bkn, xxxgnm.bkp, or prosdb.bka file will be read depending on the type of query. The appropriate file is searched for in the current directory, the directory specified by the env variable 'ALN_DBS', and the 'seqdb' directory specified at the compile time in this order.

    • -S N: Orientation of the DNA query sequence (3)

      • N=0: The orientation is inferred from the phrases (e.g. 5' end) in the header line of each entry within a fasta file. If no information is available, both orientations are examined, and the result with the better score is reported. Terminal polyA or polyT sequence is not trimmed.
      • N=1: Forward orientation only. PolyA tail may be trimmed off.
      • N=2: Reverse-complement orientation only. Leading polyT sequence may be trimmed off.
      • N=3: Examine both orientations. Terminal polyA or polyT sequence may be trimmed off.

    • -T S: Specify the species-specific parameter set. S corresponds to the subdirectory in the table directory. Alternatively, S may be the 1st or the 3rd term in table/gnm2tab file, where the 2nd term on the line indicates the subdirectory.

    • -V N: Minimum space to induce Hirschberg's algorithm (16M)

    • -W S: Write block index table to files S.bkx. if S is omitted, the file name (without directory and extension) of the first argument is used as S.

    • -g: gzipped output used in combination with -O12 option.

    • -i[a|p]: Input mode with -Q[0<=N<=3].

      • -ia: Alternative mode; a genomic segment of an odd numbered entry in the input file is aligned with the query of the following entry.
      • -ip: Parallel mode; the i-th entry in the file specified by the first argument is aligned with the i-th entry in the file specified by the second argument.
      • default: The genomic segment specified by the first argument is aligned with each entry in the file specified by the second argument.

    • -o S: Destination of output file name (stdout). If multiple output formats are specified by -O option(s), S specifies the directory or prefix to which the file names with .ON extensions are concatenated.

    • -paN: Terminal polyA or polyT sequence longer than N (12) is trimmed off and the orientation is fixed accordingly. If N = 0 or empty, these functionalities are disabled.

    • -pi: Mark exon-intron junctions by color in the alignment (-O1).

    • -pq: Suppress warning messages sent to stderr.

    • -pw: Report result even if alignment score is below threshold value.

    • -px: Suppress self-comparisons in the execution mode (C) or (D).

    • -u N: Gap-extension penalty (3, 2, 2)

    • -v N: Gap-opening penalty (8, 6, 9)

    • -xB S: Bit pattern of seeds used for HSP search at level 1

    • -xb S: Bit pattern of seeds used for HSP search at level 3

    • -ya N: Dinucleotide pairs at the ends of an intron (0)

      • N=0: Accept only the canonical pairs (GT..AG,GC..AG,AT..AC)
      • N=1: accept also AT..AN
      • N=2: allow up to one mismatch from GT..AG
      • N=3: accept any pairs. An omission of N implies N = 3

    • -yi N: Intron penalty (11, 8, 11)

    • -yj N: Incline of long gap penalty (0.6)

    • -yk N: Flex point where the incline of gap penalty changes (7)

    • -yl N: Double affine gap penalty if N=3; otherwise affine gap penalty

    • -ym N: Score for a nucleotide match (2, 2)

    • -yn N: Penalty for nucleotide mismatch (6, 2)

    • -yo N: Penalty for an in-frame termination codon (100)

    • -yp N: PAM level used in the alignment (third) phase (150)

    • -yq N: PAM level used in the second phase (50)

    • -yx N: Penalty for a frame shift (100)

    • -yy N: Relative contribution of splicing signal (8)

    • -yz N: Relative contribution of coding potential (2)

    • -yA N: Relative contribution of the translational initiation or termination signal (8)

    • -yB N: Relative contribution of branch point signal (0)

    • -yE N: Minimum exon length (2)

    • -yI S: Intron distribution parameters

    • -yJ N: Relative contribution of the bonus given to a conserved intron position (10)

    • -yL N: Minimum intron length (30, 30)

    • -yS N: N specifies the percentile contribution of the species-specific splice signal. The other part is derived from the universal signal given to the dinucleotides at the ends of an intron. An omission of N implies N = 100.

    • -yX N: N = 0: set parameter values for intra-species comparison. N = 1: set parameter values for cross-species comparison. The default value for N is 0 or 1 for DNA or protein query, respectively.

    • -yY N: Relative contribution of length-dependent part of intron penalty (8)

    • -yZ N: Relative contribution of oligomer composition within an intron (0)

    • -XS: Activete salvage mode. Considerably slow.

  1. Sortgrcd
  • Sortgrcd is used to recover the output of spaln with -O12 option, to apply some filtering, and also to rearrange the output of multiple spaln runs.
  • Run sortgrcd as follows:
    % sortgrcd [options] xxx.grd(.gz)
  • Options:
    • -C N: Minimum cover rate = % nucleotides in predicted exons / length of query (x 3 if query is protein) (0-100)
    • -E N: Report only the best (N=1) or all (N=2) results per gene locus (1)
    • -F N: Filter level (N=0: no; N=1: mild; N=2: medium; N=3: stringent)
    • -I N: Minimum sequence identity (0-100)
    • -H N: Minimum alignment score (35)
    • -O N: Output mode. Same as that of spaln except that N=1, 2, and >=8 are not supported. -O15 reports -O5 format information for only unique introns.
    • -S C: Sort chromosomes/contigs in the order of C=a: alphabetical, b: abundance, c: appearance in genome database, r: reverse order for minus strand
    • -V N: Internal memory size used for core sort. If the data size is greater than N, the sorting procedure will be done in pieces.
    • -m N: Maximum number of mismatches within 20 bp from the nearest exon-intron boundary
    • -n N: Maximum number of non-canonical (other than GT..AG, GC..AG, AT..AC) intron ends
    • -u N: Maximum number of unpaired (gap) sites within 20 bp from the nearest exon-intron boundary
  • By default, no filter listed above is applied.
  • When the output of spaln is separated into several files, the combined results are subjected to the sorting. Although xxx.grd (or xxx.grd.gz) files are assigned as the argument, there must be corresponding xxx.erd and xxx.qrd (or xxx.efd.gz and xxx.qrd.gz) files in the same directory.
  • In the default output format, the gene structure corresponding to each transcript is delimited by a line starting with '@', whereas each gene locus is delimited by a line starting with '!' [4]. Two transcripts belong to the same locus if their corresponding genomic regions overlap by at least one nucleotide on the same strand.
  • With -O0 option, the outputs follow the instruction of Gff3 where a gene locus is defined as described above.

Example

  • To experience the flow of procedures with the samples in seqdb, type in the following series of commands after moving to seqdb.
    % make dictdisc.cf
    % make dictdisc.faa
    % make dictdisc_g.gf
    % perl makeidx.pl -inp dictdisc_g.gf
    % make dictdisc.srd
    % make dictdisc.spn
  • Alternatively, you may try below if USE_ZLIB is activated..
    % perl makeidx.pl -inp [-g] dictdisc_g.gf.gz
    % spaln -Q7 -d dictdisc_g -T dictdisc [-t10] dictdisc.faa.gz
    % spaln -Q7 -d dictdisc_g -yS -T dictdisc -O12 -g [-t10] dictdisc.cf.gz
    % sortgrcd -O15 -F2 dictdisc.grd.gz

Changes from previous version

  1. Fix a bug in preventing orphan exons in DP for DNA query.
  2. Revival of the salvage mode with -XS option (examine all positively scored blocks).

Changes in version 2.4.3

  1. Small improvement in memory usage.
  2. A few bugs related to the unidirectional Hirschberg algorithm have been fixed.
  3. Illegal memory access in generating splice-junction sequence has been fixed.

Changes in version 2.4.2

  1. Minor improvements in the DP-based spliced alignment engines.
  2. A few bugs related to the unidirectional Hirschberg algorithm have been fixed.
  3. A bug in the Boyer-Moore algorithm has been fixed. This is relevant to protein queries with -yX0 option, and when the search goes beyond a block boundary.

Changes in version 2.4.1

  1. The algorithm for delimiting a genic region has been modified to find remote terminal coding exon(s) separated by long (up to 99.6% quantile) intron(s) from the main body of the gene.
  2. The -yx0 option now tries to search for missing internal micro exons and terminal very short coding exons.
  3. Selenocysteine (denoted by U) is now regarded as the 21th amino acid which favorably matches an in-frame TGA termination codon (U in the Tron code) upon DNA vs amino acid sequence alignment.
  4. Gene candidates are now sorted according to the final alignment score rather than the intermediate chained HSP score. This modification has improved the chance of true orthologous hits rather than paralog hits at an expense of a slight increase in computational load.
  5. Compared with the previous versions, a larger number of species-specific parameter sets (247 <- 102) are provided to support more species (1495 <- 688). Note that some parameter-set identifiers are changed. Please use eight-digit species identifies (e.g. zea_mays) rather than former parameter-set identifiers (e.g. Magnolio) as the argument of -T option.

Changes in version 2.4.0

  1. Spaln can now directly format genomic sequences without relying on 'make' command. See Format.
  2. The internal format of index files is slightly modified. Although previously-formatted files can be used by the new version, the opposite is not true. Note that use of older files with the new version can lead to a slight loss in sensitivity.
  3. The above change has been done to facilitate multi-thread operation at the format time, although the acceleration rate by multi-threading is only marginal.
  4. Multiple output forms can be produced at a single run. See -O and -o options.
  5. The traditional bidirectional Hirschberg algorithm is changed to the unidirectional variant.
  6. Also, the bidirectional 'sandwich' or 'attack by both sides' spliced alignment algorithm has been changed to unidirectional 'skipped' spliced alignment algorithm. This and the above changes have considerably reduced code complexity.
  7. Local lookup table (xxxgnm.lun or xxxgnm.lup) is generated and used with -E option. Be cautious to use this option, as a large disk space is required to store the generated file, and a large memory is required at the runtime.
  8. Paired-ends mode has been removed.
  9. Many small bugs have been fixed.

Changes in version 2.3.3

  1. The heuristic alignment engine has been updated, resulting in marginal but significant improvement in speed and accuracy, especially with -Q3/7 option.
  2. chachr.pl has been extended to accept Ganbank/DDBJ and EMBL-formatted files in addition to FASTA files. Maybe used as a format convertor.
  3. Update help and error messages of Spaln and Sortgrcd.
  4. Prevent segmentation fault invoked with -ia or -ip option.
  5. The maximal path size has been extended from 255 to 2047 characters.
  6. The 'NEVSEL' constant value has been changed to avoid underflow of 2 * NEVSEL.
  7. In utilseq.c and utilseq.h, a member variable in class PatMat has been moved to a local variable to recover thread safety.
  8. When Spaln is run: % spaln protein genome, the order of the 1st and 2nd arguments is exchanged with a warning message.

Changes in version 2.3.2

  1. From this version, query fasta file(s) may be compressed.
  2. The new option of spaln '-g' directly generates compressed output(s) when used in combination with -W or -O12 option.
  3. makeidx.pl and makblk.pl have been modified to accord with gzipped genome/database fasta files.
  4. A small bug in makdbs.c has been fixed.
  5. From this version, input genome/database fasta files (X.mfa, X.gf, or X.faa), formatted data files (X.seq,X.bka, X.bkn, and X.bkp) for spaln, and X.grd, X.erd, and X.qrd for sortgrcd may be gzipped if USE_ZLIB mode is activated upon compilation. Note: other data files (X.ent, X.grp, X.idx, and X.odr) must not be compressed.
  6. A serious bug concerning with multiple queries has been fixed. This has considerably improved mapping sensitivity particularly when -M option is set under single thread operation mode.
  7. Fixation of several small bugs and fine tuning of codes further enhanced mapping sensitivity and specificity particularly for short protein queries.
  8. -O N option of sortgrcd has been extended. -O0: Gff3: -O3: BED; -O4: exon-oriented; -O5: intron-oriented; -O6: concatenated exons; -O7: translated amino acid sequence; -O15: unique introns.

References

[1] Gotoh, O. A space-efficient and accurate method for mapping and aligning cDNA sequences onto genomic sequence. Nucleic Acids Research 36 (8) 2630-2638 (2008).

[2] Gotoh, O. Direct mapping and alignment of protein sequences onto genomic sequence. Bioinformatics 24 (21) 2438-2444 (2008).

[3] Iwata, H. and Gotoh, O. Benchmarking spliced alignment programs including Spaln2, an extended version of Spaln that incorporates additional species-specific features. Nucleic Acids Research 40 (20) e161 (2012)

[4] Gotoh, O. Homology-based gene structure prediction: simplified matching algorithm using a translated codon (tron) and improved accuracy by allowing for long gaps. Bioinformatics 16 (3) 190-202 (2000)

[5] Nagasaki, H., Arita, M., Nishizawa, T., Suwa, M., Gotoh, O. Automated classification of alternative splicing and transcriptional initiation and construction of a visual database of the classified patterns. Bioinformatics 22 (10) 1211-1216 (2006).

[6] Gotoh, O. Cooperation of Spaln and Prrn5 for construction of gene-structure-aware multiple sequence alignment. In: Katoh K. (eds) Multiple Sequence Alignment. Methods in Molecular Biology 2231, Humana, New York, NY. (2021).

Copyright (c) 1997-2021 Osamu Gotoh (o.gotoh@aist.go.jp) All Rights Reserved.