Process was killed when trying to merge large Fasta files
Opened this issue · 9 comments
ypoh1 commented
Hi,
The script worked really well when I merged up to 255 loci from 196 species. However, I was having issue when the taxa number increased to 996, and the error message showed the process was killed due to out of memory. I've run this process on cluster and assigned 256G of memory. Is there any way to fix it? Thanks!
PatrickKueck commented
Hi ypoh1,
Usually, FcC should not run into a memory problem. However, I got a couple of years the same report from another user. It turned out that this user used different sequence names for the same taxon in each infile.
Have you checked the correctness of your sequence names? Sequences which have to be concatenated MUST have the same name in each infile (be aware of upper and lower cases, underscores etc). Otherwise, these sequence-names are treated as different characters and thus, inflate your supermatrix file.
For example:
-------------------------------------------------------------
- infile_1.fas: >tax1
- infile_2.fas: >Tax1
are treated as different taxa, resulting in:
tax1: ACCCTTTGGGNNNNNNNNN
Tax1: NNNNNNNNNACCCTTTGGG
instead of the same taxon (tax1):
…tax1: ACCCTTTGGGACCCTTTGGG
----------------------------------------------------------
Best
Patrick
----------------
Dr. Patrick Kück
Algorithmic Development
Leibniz Institute for the Analysis of Biodiversity Change
Adenauerallee 160, 53113 Bonn, Germany
email: ***@***.***
www.bonn.leibniz-lib.de
Gesendet: Mittwoch, 23. März 2022 um 00:01 Uhr
Von: "ypoh1" ***@***.***>
An: "PatrickKueck/FASconCAT-G" ***@***.***>
Cc: "Subscribed" ***@***.***>
Betreff: [PatrickKueck/FASconCAT-G] Process was killed when trying to merge large Fasta files (Issue #10)
Hi,
The script worked really well when I merged up to 255 loci from 196 species. However, I was having issue when the taxa number increased to 996, and the error message showed the process was killed due to out of memory. I've run this process on cluster and assigned 256G of memory. Is there any way to fix it? Thanks!
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
ypoh1 commented
Thanks for pointing me to the right direction. Indeed, I had inconsistent names in my input files. The problem is solved now!
kroeve commented
Hi @PatrickKueck,
I run into the same error with my data set. I'm trying to concatenate 1400 gene alignments of a few hundred bp and ca. 150 samples. My sample names are consistent over all files. Any idea what might cause the issue?
Cheers,
Evelin
PatrickKueck commented
Hi Evelin,Sorry to read that you run into trouble with FasConCat. I am currently on holiday. Please double check if all sequence names are identical in each data (case sensitive). Another idea is that your data is too big for the script, causing RAM memories problems. I am on holiday until june 4th and will have an important audit on june 13th. So I can check fcc-g afterwards. Best Patrick Am 26.05.24 um 21:17 schrieb Evelin Krol
Von: "Evelin Krol" ***@***.***>Datum: 26. Mai 2024An: "PatrickKueck/FASconCAT-G" ***@***.***>Cc: "Mention" ***@***.***>,"Patrick Kueck" ***@***.***>Betreff: Re: [PatrickKueck/FASconCAT-G] Process was killed when trying to merge large Fasta files (Issue #10)
Hi @PatrickKueck,
I run into the same error with my data set. I'm trying to concatenate 1400 gene alignments of a few hundred bp and ca. 150 samples. My sample names are consistent over all files. Any idea what might cause the issue?
Cheers,
Evelin
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
ypoh1 commented
Hi,
In my previous case, the issue was solved after I increased the RAM usage.
I had ~1000 species * 255 loci in my dataset, and I assigned 300 GB for the
RAM usage. FYI.
…On Mon, May 27, 2024 at 9:34 PM Patrick Kueck ***@***.***> wrote:
Hi Evelin,Sorry to read that you run into trouble with FasConCat. I am
currently on holiday. Please double check if all sequence names are
identical in each data (case sensitive). Another idea is that your data is
too big for the script, causing RAM memories problems. I am on holiday
until june 4th and will have an important audit on june 13th. So I
can check fcc-g afterwards. Best Patrick Am 26.05.24 um 21:17 schrieb
Evelin Krol
Von: "Evelin Krol" ***@***.***>Datum: 26. Mai 2024An:
"PatrickKueck/FASconCAT-G" ***@***.***>Cc: "Mention" ***@***.***>,"Patrick
Kueck" ***@***.***>Betreff: Re: [PatrickKueck/FASconCAT-G] Process was
killed when trying to merge large Fasta files (Issue #10)
Hi @PatrickKueck,
I run into the same error with my data set. I'm trying to concatenate 1400
gene alignments of a few hundred bp and ca. 150 samples. My sample names
are consistent over all files. Any idea what might cause the issue?
Cheers,
Evelin
—Reply to this email directly, view it on GitHub, or unsubscribe.You are
receiving this because you were mentioned.Message ID: ***@***.***>
—
Reply to this email directly, view it on GitHub
<https://urldefense.com/v3/__https://github.com/PatrickKueck/FASconCAT-G/issues/10*issuecomment-2133498851__;Iw!!IKRxdwAv5BmarQ!f2nfBIDELAuC6A_PZQ4-wCrBz76WROajcJy21-JxssqgmbtqZ9t8psGTAZ1rlkwPCUuC9kc3UY7JkbYXUL8_YoDYFTE$>,
or unsubscribe
<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AV6IY6ZFGBJ3DDT4T3XE24TZEMY5HAVCNFSM5RMJWB7KU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJTGM2DSOBYGUYQ__;!!IKRxdwAv5BmarQ!f2nfBIDELAuC6A_PZQ4-wCrBz76WROajcJy21-JxssqgmbtqZ9t8psGTAZ1rlkwPCUuC9kc3UY7JkbYXUL8_6TeOKxM$>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
PatrickKueck commented
Hi,thank you for the info. An improvement of the RAM usage in FcC is mandatory required. I hope I can working on it soon. Am 27.05.24 um 23:08 schrieb ypoh1
Von: "ypoh1" ***@***.***>Datum: 27. Mai 2024An: "PatrickKueck/FASconCAT-G" ***@***.***>Cc: "Patrick Kueck" ***@***.***>,"Mention" ***@***.***>Betreff: Re: [PatrickKueck/FASconCAT-G] Process was killed when trying to merge large Fasta files (Issue #10)
Hi,
In my previous case, the issue was solved after I increased the RAM usage.
I had ~1000 species * 255 loci in my dataset, and I assigned 300 GB for the
RAM usage. FYI.
On Mon, May 27, 2024 at 9:34 PM Patrick Kueck ***@***.***> wrote:
Hi Evelin,Sorry to read that you run into trouble with FasConCat. I am
currently on holiday. Please double check if all sequence names are
identical in each data (case sensitive). Another idea is that your data is
too big for the script, causing RAM memories problems. I am on holiday
until june 4th and will have an important audit on june 13th. So I
can check fcc-g afterwards. Best Patrick Am 26.05.24 um 21:17 schrieb
Evelin Krol
Von: "Evelin Krol" ***@***.***>Datum: 26. Mai 2024An:
"PatrickKueck/FASconCAT-G" ***@***.***>Cc: "Mention" ***@***.***>,"Patrick
Kueck" ***@***.***>Betreff: Re: [PatrickKueck/FASconCAT-G] Process was
killed when trying to merge large Fasta files (Issue #10)
Hi @PatrickKueck,
I run into the same error with my data set. I'm trying to concatenate 1400
gene alignments of a few hundred bp and ca. 150 samples. My sample names
are consistent over all files. Any idea what might cause the issue?
Cheers,
Evelin
—Reply to this email directly, view it on GitHub, or unsubscribe.You are
receiving this because you were mentioned.Message ID: ***@***.***>
—
Reply to this email directly, view it on GitHub
<https://urldefense.com/v3/__https://github.com/PatrickKueck/FASconCAT-G/issues/10*issuecomment-2133498851__;Iw!!IKRxdwAv5BmarQ!f2nfBIDELAuC6A_PZQ4-wCrBz76WROajcJy21-JxssqgmbtqZ9t8psGTAZ1rlkwPCUuC9kc3UY7JkbYXUL8_YoDYFTE$>,
or unsubscribe
<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AV6IY6ZFGBJ3DDT4T3XE24TZEMY5HAVCNFSM5RMJWB7KU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJTGM2DSOBYGUYQ__;!!IKRxdwAv5BmarQ!f2nfBIDELAuC6A_PZQ4-wCrBz76WROajcJy21-JxssqgmbtqZ9t8psGTAZ1rlkwPCUuC9kc3UY7JkbYXUL8_6TeOKxM$>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
kroeve commented
Hi Patrick,
I tried again a few times increasing the computing resources but stopped trying at 352 GB RAM and 64 cores since it's still not working. I will think of something else for now but looking forward to an update once you have time to work on FCC!
Cheers,
Evelin
PatrickKueck commented
Hi Evelin,thanks for the update. From what you report I think it isn‘t a RAM problem. Please can you send the error message to my email and maybe send me two of your infiles. I‘ll check your problem next week when I am back at work. Sorry for running into trouble with FcC-G. You can also try the original FasConCat script (without „-G“ ending).Anyway, I‘ll let you know what I have figured out next week as soon as possible.BestPatrickAm 30.05.24 um 10:06 schrieb Evelin Krol
Von: "Evelin Krol" ***@***.***>Datum: 30. Mai 2024An: "PatrickKueck/FASconCAT-G" ***@***.***>Cc: "Mention" ***@***.***>,"Patrick Kueck" ***@***.***>Betreff: Re: [PatrickKueck/FASconCAT-G] Process was killed when trying to merge large Fasta files (Issue #10)
Hi Patrick,
I tried again a few times increasing the computing resources but stopped trying at 352 GB RAM and 64 cores since it's still not working. I will think of something else for now but looking forward to an update once you have time to work on FCC!
Cheers,
Evelin
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
PatrickKueck commented
#!/usr/bin/perl
use strict;
use File::Copy;
use Tie::File;
use Term::ANSIColor qw(:constants);
use Getopt::Std;
######################################### ENTER THE PROTTEST SOFTWARE NAME OF YOUR RUNNING SYSTEM IN SINGLE QUOTATION MARKS ''
# DEFINEMENT OF ACTUAL PROTTEST SOFTWARE NAME
my $prottest = 'prottest-3.3.jar' ;
##############################
# written by patrick k�ck, zentrales forschungsmuseum alexander koenig, bonn, germany
# email: ***@***.***
# updated on 12th,january , 2009 by patrick kueck
# updated on 25th may , 2009 by patrick kueck
# updated on 28th may , 2009 by patrick kueck
# updated on 7th june , 2009 by patrick kueck
# updated on 8th june , 2009 by patrick kueck
# updated on 28th september , 2009 by patrick kueck
# updated on 16th october , 2009 by patrick kueck
# updated on 17th november , 2009 by patrick kueck
# updated on 7th january , 2010 by patrick kueck
# updated on 7th april , 2010 by patrick kueck
# updated on 8th april , 2010 by patrick kueck -> FcC v1.0
# updated on april , 2014 by patrick kueck -> FcC-G v1.0
# updated on 30th september , 2014 by patrick kueck -> FcC-G v1.01 Implementation of additional Screen Output (%Missing Data and Single Infile Sequence Ranges in the concatenated supermatrix) as well as some output result file format changes
# updated on 16th november , 2014 by patrick kueck ->input file endings line 1529-1531->FcC v1.03
# updated on 22nd november , 2016 by patrick kueck -> concatenation and print out of supermatrix ranges due to sorted infile names and not randomly. Debuged read in error of .FASTA or .fasta named files
# updated on 22nd march , 2021 by patrick kueck -> FcC-G v1.05 Including an additional character coding option for absent sequence data (-g). With -g, absent sequence data is filled by '-' rather than 'N/X' during the concatenation process
# updated on 22nd february , 2022 by jgl -> FcC-G v1.05.1 Resolved command conflict: 'Replace missing sequences with' set to (-g); 'Preface' set to (-x); some minor character corrections to comments
# updated on 19th april , 2022 by patrick kueck -> FcC-G_v1.05.2 Resolved empty line conflict at the end of phylip files (last sequence followed by multiple newlines) in &phy2fas
# updated on 22nd april , 2022 by patrick kueck -> FcC-G_v1.05.3 Enabled help-menu entree via command line option '-h' additionally to '-help' option, allowing "perl FASconCAT_v1.05.3.pl -h <enter>"
# updated on 12th june , 2024 by patrick kueck -> FcC-G_v1.06 Reconstruction of single infile handling and data storage across the whole source code for better RAM usage
####################################################### START #######################################################################################
my %outfile_name = (
'supermatrixFAS' => 'FcC_smatrix.fas', # Supermatrix outfile in FASTA format
'supermatrixPHY' => 'FcC_smatrix.phy', # Supermatrix outfile in PHYLIP format
'supermatrixNEX' => 'FcC_smatrix.nex', # Supermatrix outfile in NEXUS format
'structure' => 'FcC_structure.txt', # Structure sequence info file
'info' => 'FcC_info.xls', # Sequence states & concatenation info file
'prottest' => $prottest, # Name of defined prottestversion
) ;
my @parameter_all = ( 'YES', 'NO' ) ; # Parameter option of data concatenation -> 'YES' -> concatenate all files; 'NO' -> concatenate defined files
my @parameter_info = ( 'YES', 'NO' ) ; # Parameter option of additional sequence state analysis -> 'YES' -> perform and print ; 'NO' -> print only basal sequence info
my @parameter_tra = ( 'NO', 'NUC to AA','AA to NUC' ) ; # Parameter option of sequence transaltion -> 'NUC to AA' -> translate nucleotide data to amino acid data; 'AA to NUC' -> translate amino acid data to nucleotide data
my @parameter_phy = ( 'NO', 'STRICT', 'RELAXED' ) ; # Print supermatrix in PHYLIP format
my @parameter_nex = ( 'NO', 'BLOCK', 'MrBAYES' ) ; # Print supermatrix in NEXUS format
my @parameter_fas = ( 'YES', 'NO' ) ; # Print supermatrix in FASTA format
my @parameter_con = ( 'NO', 'Freq', 'Maj', 'Strict' ) ; # Perform consensus sequence of defined sequence blocks
my @parameter_file = ( 'Supermatrix', 'Convert', 'Supermatrix/Convert' ) ; # Concatenate, Convert, or both define by associated option -> 'Supermatrix': concatenation; Convert: Converting single files without adding missing sequences; 'Supermatrix/Convert': Concatenated single files and convert single files with included missing sequence taxa
my @parameter_3rd = ( 'Remain', 'Reject' ) ; # Parameter option of 3rd sequence position exclusion -> 'Remain' -> remove third position; 'Reject' -> keep third position
my @parameter_ryc = ( 'NO', 'All', '3rd' ) ; # Parameter option of RY coding -> 'NO' -> no RY coding; 'All' -> RY coding of complete sequence; '3rd' -> RY coding of 3rd sequence positions
my @parameter_par = ( 'NO', 'YES' ) ; # Print parsimonious sites as extra msa file -> 'NO' -> no print; 'YES' -> print
my @parameter_ren = ( 'NO', 'YES' ) ; # Rename taxon names of given infiles -> 'NO' ; 'YES'
my @parameter_prt = ( 'NO', 'Supermatrix' ) ; # Print OUT additional partition files if concatenated -> 'NO' ; 'Supermatrix' means yes for concatenatde files
my @parameter_pro = ( 'NO', 'Supermatrix' ) ; # Start prottest analyses for aa data -> 'NO' ; 'YES'
my @parameter_mis = ( 'Missing', 'Indel' ) ; # Replace missing sequences with -> 'Missing': 'X' or 'N' ; 'Indel': '-'
##############################
## READ IN ARGV and assign user defined parameter options
&argv_handling (
***@***.***_all, # List of parameter options of data concatenation -> IN (defined) / OUT (changed)
***@***.***_info, # List of parameter options of info print out -> IN (defined) / OUT (changed)
***@***.***_phy, # List of supermatrix parameter options of PHYLIP print OUT -> IN (defined) / OUT (changed)
***@***.***_nex, # List of supermatrix parameter options of NEXUS print OUT -> IN (defined) / OUT (changed)
***@***.***_con, # List of consensus options -> IN (defined) / OUT (changed)
***@***.***_file, # List of filehandling options -> IN (defined) / OUT (changed)
***@***.***_fas, # List of supermatrix parameter options of FASTA print OUT -> IN (defined) / OUT (changed)
***@***.***_3rd, # List of third position handling -> IN (defined) / OUT (changed)
***@***.***_ryc, # List of RY coding -> IN (defined) / OUT (changed)
***@***.***_tra, # List of Sequence transaltion options -> IN (defined) / OUT (changed)
***@***.***_par, # Print parsimonious sites as extra msa file -> IN (defined) / OUT (changed)
***@***.***_ren, # Rename taxon names of given ifiles -> IN (defined) / OUT (changed)
***@***.***_prt, # Print partition files for concatenated data -> IN (defined) / OUT (changed)
***@***.***_pro, # Start prottest analyses for aa data -> IN (defined) / OUT (changed)
***@***.***_mis, # Replace missing gene sequences by '-' instead using 'X' or 'N' -> IN (defined) / OUT (changed)
\%outfile_name, # Outfilename of output option -> IN (defined) / OUT (unchanged)
) ;
##############################
##############################
## Open MENU
&menu ;
##############################
##############################
## Open Parameter setup
¶meter(
***@***.***_all, # List of parameter options of data concatenation -> IN (defined) / OUT (changed)
***@***.***_info, # List of parameter options of info print out -> IN (defined) / OUT (changed)
***@***.***_phy, # List of supermatrix parameter options of PHYLIP print OUT -> IN (defined) / OUT (changed)
***@***.***_nex, # List of supermatrix parameter options of NEXUS print OUT -> IN (defined) / OUT (changed)
***@***.***_con, # List of consensus options -> IN (defined) / OUT (changed)
***@***.***_file, # List of filehandling options -> IN (defined) / OUT (changed)
***@***.***_fas, # List of supermatrix parameter options of FASTA print OUT -> IN (defined) / OUT (changed)
***@***.***_3rd, # List of third position handling -> IN (defined) / OUT (changed)
***@***.***_ryc, # List of RY coding -> IN (defined) / OUT (changed)
***@***.***_tra, # List of Sequence transaltion options -> IN (defined) / OUT (changed)
***@***.***_par, # Print parsimonious sites as extra msa file -> IN (defined) / OUT (changed)
***@***.***_ren, # Rename taxon names of given ifiles -> IN (defined) / OUT (changed)
***@***.***_prt, # Print partition files for concatenated data -> IN (defined) / OUT (changed)
***@***.***_pro, # Start prottest analyses for aa data -> IN (defined) / OUT (changed)
***@***.***_mis, # Replace missing gene sequences by '-' instead using 'X' or 'N' -> IN (defined) / OUT (changed)
\%outfile_name, # Outfilename of output option -> IN (defined) / OUT (unchanged)
);
##############################
################################################################## END ###############################################################################
sub argv_handling{
my $aref_parameter_all = $_[0] ; # List of parameter options of data concatenation -> IN (defined) / OUT (changed)
my $aref_parameter_inf = $_[1] ; # List of parameter options of info print out -> IN (defined) / OUT (changed)
my $aref_parameter_phy = $_[2] ; # List of supermatrix parameter options of PHYLIP print OUT -> IN (defined) / OUT (changed)
my $aref_parameter_nex = $_[3] ; # List of supermatrix parameter options of NEXUS print OUT -> IN (defined) / OUT (changed)
my $aref_parameter_con = $_[4] ; # List of consensus options -> IN (defined) / OUT (changed)
my $aref_parameter_fil = $_[5] ; # List of of file handling -> IN (defined) / OUT (changed)
my $aref_parameter_fas = $_[6] ; # List of supermatrix parameter options of FASTA print OUT -> IN (defined) / OUT (changed)
my $aref_parameter_3rd = $_[7] ; # List of third position handling -> IN (defined) / OUT (changed)
my $aref_parameter_ryc = $_[8] ; # List of RY coding -> IN (defined) / OUT (changed)
my $aref_parameter_tra = $_[9] ; # List of sequence translation options -> IN (defined) / OUT (changed)
my $aref_parameter_par = $_[10]; # List of sequence translation options -> IN (defined) / OUT (changed)
my $aref_parameter_ren = $_[11]; # Rename taxon names of given ifiles -> IN (defined) / OUT (changed)
my $aref_parameter_prt = $_[12]; # Print partition files for concatenated data -> IN (defined) / OUT (changed)
my $aref_parameter_pro = $_[13]; # Start prottest analyses for aa data -> IN (defined) / OUT (changed)
my $aref_parameter_mis = $_[14]; # Replace missing gene sequences by '-' instead using 'X' or 'N' -> IN (defined) / OUT (changed)
my $href_outfile_name = $_[15]; # Outfilename of output option -> IN (defined) / OUT (unchanged)
############################## START ARGV READ IN
## READ IN and assign terminal command options
## Set chosen parameter option one step further if associated command option is given
my ( $commandline ) = join "", @argv ;
if ( $commandline ){
##############################
## substitute blanks to '' & split up commandline using '-' signs
## delete first (empty) entry in @commands
$commandline =~ s/ |\s+// ;
my @commands = split '-', $commandline ;
shift @commands;
##############################
##############################
## Change parameter of given command
## Skip to main menu if a command option is unknown
## or '-s' command missing
for my $single_command ( sort @commands ){
if ( $single_command =~ /^help$|^h$/i ) { &help } # '-help' -> open help menu
elsif ( $single_command =~ /^i$/i ) { @$aref_parameter_inf = reverse @$aref_parameter_inf } # '-i' -> set additional info print to NO; '-i -i' -> to NO
elsif ( $single_command =~ /^f$/i ) { @$aref_parameter_all = reverse @$aref_parameter_all } # '-f' -> set infile READ IN to defined; '-f -f' -> to all possible infiles
elsif ( $single_command =~ /^a$/i ) { @$aref_parameter_fas = reverse @$aref_parameter_fas } # '-a' -> set FASTA output to NO; '-a -a' -> to YES
elsif ( $single_command =~ /^d$/i ) { @$aref_parameter_3rd = reverse @$aref_parameter_3rd } # '-d' -> Reject 3rd position; '-d -d' -> Remain 3rd position
elsif ( $single_command =~ /^j$/i ) { @$aref_parameter_par = reverse @$aref_parameter_par } # '-j' -> set PARSIMONY output to YES; '-j -j' -> to NO
elsif ( $single_command =~ /^k$/i ) { @$aref_parameter_ren = reverse @$aref_parameter_ren } # '-k' -> set rename of sequence names to YES; '-k -k' -> to NO
elsif ( $single_command =~ /^l$/i ) { @$aref_parameter_prt = reverse @$aref_parameter_prt } # '-l' -> set part. file of conc matrix to Supermatrix; '-l -l' -> to NO
elsif ( $single_command =~ /^m$/i ) { @$aref_parameter_pro = reverse @$aref_parameter_pro } # '-m' -> start prottest under linux; '-m -m' -> don't start prottest (default)
elsif ( $single_command =~ /^g$/i ) { @$aref_parameter_mis = reverse @$aref_parameter_mis } # '-g' -> replace missing sequences by '-'; '-g -g' -> replace missing sequences by 'X' or 'N' (default) #jgl#'-x -> -g'#jgl#
elsif ( $single_command =~ /^e$/i ) { my $tl = shift @$aref_parameter_tra ; push @$aref_parameter_tra, $tl } # '-e' -> translate nuc data to aa data; '-e -e' -> translate aa data to nuc data '-e -e -e' -> no translation
elsif ( $single_command =~ /^c$/i ) { my $tl = shift @$aref_parameter_con ; push @$aref_parameter_con, $tl } # '-c' -> Generate frequency consensus sequence; '-c -c' -> majority consensus; '-c -c -c' -> strict consensus; '-c -c -c -c' -> to default (no consensus)
elsif ( $single_command =~ /^p$/i ) { my $tl = shift @$aref_parameter_phy ; push @$aref_parameter_phy, $tl } # '-p' -> set supermatrix format to strict phylip; '-p -p' -> to relaxed phylip; '-p -p -p' -> to default (no phylip)
elsif ( $single_command =~ /^n$/i ) { my $tl = shift @$aref_parameter_nex ; push @$aref_parameter_nex, $tl } # '-n' -> set supermatrix format to nexus block; '-n -n' -> to MrBayes nexus block; '-n -n -n' -> to default (no nexus)
elsif ( $single_command =~ /^o$/i ) { my $tl = shift @$aref_parameter_fil ; push @$aref_parameter_fil, $tl } # '-o' -> convert only infiles; '-o -o' -> Supermatrix & infiles; '-o -o -o' -> print out only supermatrix file
elsif ( $single_command =~ /^b$/i ) { my $tl = shift @$aref_parameter_ryc ; push @$aref_parameter_ryc, $tl } # '-b' -> RY coding complete sequences '-b -b' -> RY coding 3rd positions; '-b -b -b' -> no RY coding
elsif ( $single_command =~ /^s$/i ) { # '-s' -> Start FASconCAT
&start(
\@$aref_parameter_all, # List of parameter options of data concatenation -> IN (defined) / OUT (changed)
\@$aref_parameter_inf, # List of parameter options of info print out -> IN (defined) / OUT (changed)
\@$aref_parameter_phy, # List of supermatrix parameter options of PHYLIP print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_nex, # List of supermatrix parameter options of NEXUS print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_con, # List of consensus options -> IN (defined) / OUT (changed)
\@$aref_parameter_fil, # List of of file handling -> IN (defined) / OUT (changed)
\@$aref_parameter_fas, # List of supermatrix parameter options of FASTA print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_3rd, # List of third position handling -> IN (defined) / OUT (changed)
\@$aref_parameter_ryc, # List of RY coding -> IN (defined) / OUT (changed)
\@$aref_parameter_tra, # List of sequence translation options -> IN (defined) / OUT (changed)
\@$aref_parameter_par, # Print parsimonious sites as extra msa file -> IN (defined) / OUT (changed)
\@$aref_parameter_ren, # Rename taxon names of given ifiles -> IN (defined) / OUT (changed)
\@$aref_parameter_prt, # Print partition files for concatenated data -> IN (defined) / OUT (changed)
\@$aref_parameter_pro, # Start Prottest -> IN (defined) / OUT (changed)
\@$aref_parameter_mis, # Replacement code of missing sequences -> IN (defined) / OUT (changed)
\%$href_outfile_name, # Outfilename of output option -> IN (defined) / OUT (unchanged)
)
}
else {
print "\n\tCOMMAND-ERROR!: Unknown command ", $single_command, "!\n" ; # 'unknown command' -> print ERROR prompt
##############################
## Open Menu if unknown command is given
&menu ;
¶meter (
\@$aref_parameter_all, # List of parameter options of data concatenation -> IN (defined) / OUT (changed)
\@$aref_parameter_inf, # List of parameter options of info print out -> IN (defined) / OUT (changed)
\@$aref_parameter_phy, # List of supermatrix parameter options of PHYLIP print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_nex, # List of supermatrix parameter options of NEXUS print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_con, # List of consensus options -> IN (defined) / OUT (changed)
\@$aref_parameter_fil, # List of of file handling -> IN (defined) / OUT (changed)
\@$aref_parameter_fas, # List of supermatrix parameter options of FASTA print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_3rd, # List of third position handling -> IN (defined) / OUT (changed)
\@$aref_parameter_ryc, # List of RY coding -> IN (defined) / OUT (changed)
\@$aref_parameter_tra, # List of sequence translation options -> IN (defined) / OUT (changed)
\@$aref_parameter_par, # Print parsimonious sites as extra msa file -> IN (defined) / OUT (changed)
\@$aref_parameter_ren, # Rename taxon names of given ifiles -> IN (defined) / OUT (changed)
\@$aref_parameter_prt, # Print partition files for concatenated data -> IN (defined) / OUT (changed)
\@$aref_parameter_pro, # Start Prottest -> IN (defined) / OUT (changed)
\@$aref_parameter_mis, # Replacement code of missing sequences -> IN (defined) / OUT (changed)
\%$href_outfile_name, # Outfilename of output option -> IN (defined) / OUT (unchanged)
)
##############################
}
}
##############################
## Open Menu if '-s' command is missing
&menu ;
¶meter (
\@$aref_parameter_all, # List of parameter options of data concatenation -> IN (defined) / OUT (changed)
\@$aref_parameter_inf, # List of parameter options of info print out -> IN (defined) / OUT (changed)
\@$aref_parameter_phy, # List of supermatrix parameter options of PHYLIP print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_nex, # List of supermatrix parameter options of NEXUS print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_con, # List of consensus options -> IN (defined) / OUT (changed)
\@$aref_parameter_fil, # List of of file handling -> IN (defined) / OUT (changed)
\@$aref_parameter_fas, # List of supermatrix parameter options of FASTA print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_3rd, # List of third position handling -> IN (defined) / OUT (changed)
\@$aref_parameter_ryc, # List of RY coding -> IN (defined) / OUT (changed)
\@$aref_parameter_tra, # List of sequence translation options -> IN (defined) / OUT (changed)
\@$aref_parameter_par, # Print parsimonious sites as extra msa file -> IN (defined) / OUT (changed)
\@$aref_parameter_ren, # Rename taxon names of given ifiles -> IN (defined) / OUT (changed)
\@$aref_parameter_prt, # Print partition files for concatenated data -> IN (defined) / OUT (changed)
\@$aref_parameter_pro, # Start Prottest -> IN (defined) / OUT (changed)
\@$aref_parameter_mis, # Replacement code of missing sequences -> IN (defined) / OUT (changed)
\%$href_outfile_name, # Outfilename of output option -> IN (defined) / OUT (unchanged)
)
##############################
}
############################## END ARGV READ IN
}
sub menu{ system('cls');
print "";
printf "\n%68s\n","------------------------------------------------------------" ;
printf "%53s\n" , "Welcome to FASconCAT-G v1.0 !" ;
printf "%58s\n" , "A perlscript for sequence concatenation" ;
printf "%59s\n" , "written by Patrick Kueck (ZFMK Bonn, 2010/14)" ;
printf "%68s\n\n", "------------------------------------------------------------" ;
}
sub parameter{
my $aref_parameter_all = $_[0] ; # List of parameter options of data concatenation -> IN (defined) / OUT (changed)
my $aref_parameter_inf = $_[1] ; # List of parameter options of info print out -> IN (defined) / OUT (changed)
my $aref_parameter_phy = $_[2] ; # List of supermatrix parameter options of PHYLIP print OUT -> IN (defined) / OUT (changed)
my $aref_parameter_nex = $_[3] ; # List of supermatrix parameter options of NEXUS print OUT -> IN (defined) / OUT (changed)
my $aref_parameter_con = $_[4] ; # List of consensus options -> IN (defined) / OUT (changed)
my $aref_parameter_fil = $_[5] ; # List of of file handling -> IN (defined) / OUT (changed)
my $aref_parameter_fas = $_[6] ; # List of supermatrix parameter options of FASTA print OUT -> IN (defined) / OUT (changed)
my $aref_parameter_3rd = $_[7] ; # List of third position handling -> IN (defined) / OUT (changed)
my $aref_parameter_ryc = $_[8] ; # List of RY coding -> IN (defined) / OUT (changed)
my $aref_parameter_tra = $_[9] ; # List of sequence translation options -> IN (defined) / OUT (changed)
my $aref_parameter_par = $_[10]; # List of sequence translation options -> IN (defined) / OUT (changed)
my $aref_parameter_ren = $_[11]; # Rename taxon names of given ifiles -> IN (defined) / OUT (changed)
my $aref_parameter_prt = $_[12]; # Print partition files for concatenated data -> IN (defined) / OUT (changed)
my $aref_parameter_pro = $_[13]; # Start prottest analyses f�r aa data -> IN (defined) / OUT (changed)
my $aref_parameter_mis = $_[14]; # Replacement code of missing gene sequences -> IN (defined) / OUT (changed)
my $href_outfile_name = $_[15]; # Outfilename of output option -> IN (defined) / OUT (unchanged)
##############################
## Print parameter setting and available command options
print "\tSTART\t FASconCAT :\t\t type <s> <enter>",
"\n",
"\n\tINFILES\t ALL/SINGLE :\t\t type <f> <enter>",
"\n\tINFO\t ALL/BASIC :\t\t type <i> <enter>",
"\n",
"\n\tPROCESSING SMATRIX/CONV. :\t\t type <o> <enter>",
"\n\tSEQ.TRANS. NO/NUC/AA :\t\t type <e> <enter>",
"\n\t3rd POS. REMAIN/REJECT :\t\t type <d> <enter>",
"\n\tRY CODING NO/ALL/3rd :\t\t type <b> <enter>",
"\n\tCONSENSUS N0/YES :\t\t type <c> <enter>",
"\n\tRENAME SEQ NO/YES :\t\t type <k> <enter>",
"\n\tABSENT SEQ MISSING/INDEL :\t\t type <g> <enter>",
"\n",
"\n\tNEXUS BLOCK/MrBAYES :\t\t type <n> <enter>",
"\n\tPHYLIP NO/YES :\t\t type <p> <enter>",
"\n\tFASTA YES/NO :\t\t type <a> <enter>",
"\n\tPARTITION NO/Supermatrix:\t\t type <l> <enter>",
"\n\tPARSIMONY NO/YES :\t\t type <j> <enter>",
"\n\tPROTTEST NO/YES :\t\t type <m> <enter>",
"\n",
"\n\tHELP FASconCAT :\t\t type <h> <enter>",
"\n\tQUIT FASconCAT :\t\t type <q> <enter>",
"\n\tPREFACE FASconCAT :\t\t type <x> <enter>", #jgl#'<g> -> <x>'#jgl#
"\n",
"\n\t------------------------------------------------------------",
"\n",
"\n\tFILE INPUT",
"\n\t-----------------",
"\n\tPROCESSING ALL FILES :\t$aref_parameter_all->[0]",
"\n\tPROCESSING SINGLE FILES :\t$aref_parameter_all->[1]",
"\n\tPROCESSING SEQ. TRANSL. :\t$aref_parameter_tra->[0]",
"\n\tPROCESSING 3rd POSITION\t :\t$aref_parameter_3rd->[0]",
"\n\tPROCESSING RY CODING\t :\t$aref_parameter_ryc->[0]",
"\n\tPROCESSING CONSENSUS\t :\t$aref_parameter_con->[0]",
"\n\tRENAME SEQUENCE NAMES\t :\t$aref_parameter_ren->[0]",
"\n\tREPLACE ABSENT SEQUENCE\t :\t$aref_parameter_mis->[0]",
"\n",
"\n\tFILE OUTPUT",
"\n\t-----------------",
"\n\tALL INFO :\t$aref_parameter_inf->[0]",
"\n\tFILE PROCESSING :\t$aref_parameter_fil->[0]",
"\n\n\tNEXUS :\t$aref_parameter_nex->[0]",
"\n\tPHYLIP :\t$aref_parameter_phy->[0]",
"\n\tFASTA :\t$aref_parameter_fas->[0]\n",
"\n\tPARTITION FILE :\t$aref_parameter_prt->[0]",
"\n\tPARSIMONY :\t$aref_parameter_par->[0]",
"\n\tPROTTEST :\t$aref_parameter_pro->[0]\n",
"\n\t------------------------------------------------------------"
;
##############################
##############################
## (1) Read IN user defined command #jgl#'=> d'#jgl#
## (2) If command option allowed, change parameter setting
## (3) unless user defined command allowed print error prompt
my $start_string ;
my $single_command = &commandline ( \$start_string ) ; # (1)
# (2)
unless ( $single_command =~ /^s$|^i$|^f$|^q$|^h$|^n$|^p$|^a$|^c$|^o$|^a$|^b$|^q$|^d$|^e$|^g$|^j$|^k$|^l$|^g$|^m$|^x$/i ){ print "\n\tCOMMAND-ERROR!: Unknown command ".$single_command."!\n" } #jgl#'=> $|^x'#jgl#
# (3)
if ( $single_command =~ /^h$/i ) { &help } # '-h' -> open help menu
elsif ( $single_command =~ /^x$/i ) { &preface} # '-x' -> open preface menu #jgl#'/^g> -> /^x' ; '-e -> -x'#jgl#
elsif ( $single_command =~ /^q$/i ) { exit } # '-q' -> exit FASconCAT
elsif ( $single_command =~ /^i$/i ) { @$aref_parameter_inf = reverse @$aref_parameter_inf } # '-i' -> set additional info print to YES; '-i -i' -> to NO
elsif ( $single_command =~ /^f$/i ) { @$aref_parameter_all = reverse @$aref_parameter_all } # '-f' -> set infile READ IN to defined; '-f -f' -> to all possible infiles
elsif ( $single_command =~ /^a$/i ) { @$aref_parameter_fas = reverse @$aref_parameter_fas } # '-a' -> set FASTA output to NO; '-a -a' -> to YES
elsif ( $single_command =~ /^j$/i ) { @$aref_parameter_par = reverse @$aref_parameter_par } # '-j' -> set PARSIMONY output to YES; '-j -j' -> to NO
elsif ( $single_command =~ /^d$/i ) { @$aref_parameter_3rd = reverse @$aref_parameter_3rd } # '-d' -> Reject 3rd position; '-d -d' -> Remain 3rd position
elsif ( $single_command =~ /^k$/i ) { @$aref_parameter_ren = reverse @$aref_parameter_ren } # '-k' -> set rename of sequence names to YES; '-k -k' -> to NO
elsif ( $single_command =~ /^l$/i ) { @$aref_parameter_prt = reverse @$aref_parameter_prt } # '-l' -> set part. file of conc matrix to Supermatrix; '-l -l' -> to NO
elsif ( $single_command =~ /^m$/i ) { @$aref_parameter_pro = reverse @$aref_parameter_pro } # '-m' -> start prottest under linux; '-m -m' -> don't start prottest (default)
elsif ( $single_command =~ /^g$/i ) { @$aref_parameter_mis = reverse @$aref_parameter_mis } # '-g' -> replace missing sequences by '-'; '-g -g' -> replace missing sequences by 'X' or 'N' (default) #jgl#'-x -> -g'#jgl#
elsif ( $single_command =~ /^e$/i ) { my $tl = shift @$aref_parameter_tra ; push @$aref_parameter_tra, $tl } # '-e' -> translate nuc data to aa data; '-e -e' -> translate aa data to nuc data '-e -e -e' -> no translation
elsif ( $single_command =~ /^c$/i ) { my $tl = shift @$aref_parameter_con ; push @$aref_parameter_con, $tl } # '-c' -> Generate frequency consensus sequence; '-c -c' -> majority consensus; '-c -c -c' -> strict consensus; '-c -c -c -c' -> to default (no consensus)
elsif ( $single_command =~ /^p$/i ) { my $tl = shift @$aref_parameter_phy ; push @$aref_parameter_phy, $tl } # '-p' -> set supermatrix format to strict phylip; '-p -p' -> to relaxed phylip; '-p -p -p' -> to default (no phylip)
elsif ( $single_command =~ /^n$/i ) { my $tl = shift @$aref_parameter_nex ; push @$aref_parameter_nex, $tl } # '-n' -> set supermatrix format to nexus block; '-n -n' -> to MrBayes nexus block; '-n -n -n' -> to default (no nexus)
elsif ( $single_command =~ /^b$/i ) { my $tl = shift @$aref_parameter_ryc ; push @$aref_parameter_ryc, $tl } # '-b' -> RY coding complete sequences '-b -b' -> RY coding 3rd positions; '-b -b -b' -> no RY coding
elsif ( $single_command =~ /^o$/i ) { my $tl = shift @$aref_parameter_fil ; push @$aref_parameter_fil, $tl } # '-o' -> convert only infiles; '-o -o' -> Supermatrix & infiles; '-o -o -o' -> print out only supermatrix file
elsif ( $single_command =~ /^s$/i ) { # '-s' -> Start FASconCAT
&start(
\@$aref_parameter_all, # List of parameter options of data concatenation -> IN (defined) / OUT (changed)
\@$aref_parameter_inf, # List of parameter options of info print out -> IN (defined) / OUT (changed)
\@$aref_parameter_phy, # List of supermatrix parameter options of PHYLIP print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_nex, # List of supermatrix parameter options of NEXUS print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_con, # List of consensus options -> IN (defined) / OUT (changed)
\@$aref_parameter_fil, # List of of file handling -> IN (defined) / OUT (changed)
\@$aref_parameter_fas, # List of supermatrix parameter options of FASTA print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_3rd, # List of third position handling -> IN (defined) / OUT (changed)
\@$aref_parameter_ryc, # List of RY coding -> IN (defined) / OUT (changed)
\@$aref_parameter_tra, # List of sequence translation options -> IN (defined) / OUT (changed)
\@$aref_parameter_par, # Print parsimonious sites as extra msa file -> IN (defined) / OUT (changed)
\@$aref_parameter_ren, # Rename taxon names of given ifiles -> IN (defined) / OUT (changed)
\@$aref_parameter_prt, # Print partition files for concatenated data -> IN (defined) / OUT (changed)
\@$aref_parameter_pro, # Start Prottest -> IN (defined) / OUT (changed)
\@$aref_parameter_mis, # Replacement code of missing gene sequences -> IN (defined) / OUT (changed)
\%$href_outfile_name, # Outfilename of output option -> IN (defined) / OUT (unchanged)
)
}
&menu;
¶meter(
\@$aref_parameter_all, # List of parameter options of data concatenation -> IN (defined) / OUT (changed)
\@$aref_parameter_inf, # List of parameter options of info print out -> IN (defined) / OUT (changed)
\@$aref_parameter_phy, # List of supermatrix parameter options of PHYLIP print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_nex, # List of supermatrix parameter options of NEXUS print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_con, # List of consensus options -> IN (defined) / OUT (changed)
\@$aref_parameter_fil, # List of of file handling -> IN (defined) / OUT (changed)
\@$aref_parameter_fas, # List of supermatrix parameter options of FASTA print OUT -> IN (defined) / OUT (changed)
\@$aref_parameter_3rd, # List of third position handling -> IN (defined) / OUT (changed)
\@$aref_parameter_ryc, # List of RY coding -> IN (defined) / OUT (changed)
\@$aref_parameter_tra, # List of sequence translation options -> IN (defined) / OUT (changed)
\@$aref_parameter_par, # Print parsimonious sites as extra msa file -> IN (defined) / OUT (changed)
\@$aref_parameter_ren, # Rename taxon names of given ifiles -> IN (defined) / OUT (changed)
\@$aref_parameter_prt, # Print partition files for concatenated data -> IN (defined) / OUT (changed)
\@$aref_parameter_pro, # Start Prottest -> IN (defined) / OUT (changed)
\@$aref_parameter_mis, # Replacement code of missing gene sequences -> IN (defined) / OUT (changed)
\%$href_outfile_name, # Outfilename of output option -> IN (defined) / OUT (unchanged)
) ;
##############################
}
sub help{
system('cls');
print
<<help
…--------------------FASconCAT-G HELP-MENU---------------------------
'Features'
--------------------------
- Sequence concatenation of single infiles of different formats:
- FASTA (.fas or .FASTA)
- PHYLIP (.phy -> Relaxed or Strict)
- CLUSTAL (.aln)
- Individual character fill of absent gene sequences
(either by missing chars 'X'|'N' or indels '-')
- Multiple supermatrix output formats in one process run:
- FASTA (.fas or .FASTA)
- PHYLIP (.phy -> Relaxed or Strict)
- NEXUS (.nex -> BLOCK with or without MrBayes commands)
- Multiple file conversion of infiles in one process run:
- FASTA (.fas or .FASTA)
- PHYLIP (.phy -> Relaxed or Strict)
- NEXUS (.nex -> BLOCK with or without MrBayes commands)
- Processing of specific consensus sequences using either...
- Strict consensus rules
- Majority consensus rules or
- Frequency consensus rules
- RY-Coding of complete sequences (nucleotide data only)
- RY-Coding of 3rd codon positions (nucleotide data only)
- Removing of 3rd codon positions (nucleotide data only)
- Sequence translation (nucleotide data to amino acid data
and vice versa)
- Print OUT of parsimony informative sites within infiles
and concatenated supermatrix
- Handling & identification of specific secondary structure positions
- Additional output information includes...
- Infile ranges in supermatrix sequences
- Character state distributions within individual infiles and the supermatrix
- Sequence type of individual infiles and the supermatrix
- Sequence lengths of individual infiles
- Number of gaps and ambiguous sites within infiles and the supermatrix
- Number of missing sequences for each individual from the supermatrix
- Number of GC character states within individual infiles and the supermatrix
- Number of parsimony informative sites in individual infiles and the supermatrix
- Proportion of character states in supermatrix sequences
- Proportion of gaps and ambiguous sites in supermatrix sequences
- Proportion of GC in supermatrix sequences
- Proportion of parsimony informative sites in supermatrix sequences
- Number of structure characters seperated in loops and stems
- Number and percent of loop and stem positions per fragment
- Separated list (FcC_structure.txt) for loop positions and
stem pairings whitin the supermatrix and the infiles
- All features can be optionally combined or modified via:
- Terminal command line
- The FASconCAT (FcC) terminal menu
--------------------------
'Start FASconCAT'
--------------------------
To start FASconCAT, open the script by via the terminal.
Via FASconCAT terminal menu:
- Open the menu by entering command:
perl FASconCAT-G_v1.0.pl <enter> (Linux/Mac)
FASconCAT-G_v1.0.pl <enter> (Windows)
- To start FASconCAT under default, enter command:
s <enter>
Via terminal command line, enter command:
- perl FASconCAT_v1.0.pl [command options] <enter> (Linux/Mac)
- FASconCAT_v1.0.pl [command options] <enter> (Windows)
- To start FASconCAT under default, enter command:
perl FASconCAT_v1.0.pl -s <enter>
--------------------------
Command options:
--------------------------
The following command options can be used as stand alone commands
or in combination with other commands. Command options can be invoked
in a single line via the terminal command window or individually via
the FASconCAT menu (without the minus'-' specification in the menu).
Command options can be input in any order.
- Output processing (-o option)
- FcC can be used for sequence concatenation and/or file
conversion.
- Under default, FcC prints OUT a supermatrix with concatenated
sequences in FASTA format in a file called
"FcC_supermatrix.fas".
- For infile format conversions WITHOUT printing
a supermatrix, enter command:
-o
- For format conversions of given infiles AND
concatenated supermatrix output, enter command:
-o -o
- Sequence translation (-e option)
- FcC can translate nucleotide data to amino acid data and
vice versa. Sequence translation happens before exclusion of
third nucleotide codon positions (if -d option is defined).
- FcC does not recognize stop codons in amino acid data or
reading frames in nucleotide data. Stop codons have to be
excluded in amino acid data before using FcC!
- Nucleotide triplets are translated to their corresponding
amino acid. Ambiguity states in triplets are recognized if
their coding nucleotide states can be definitively assigned
to a specific amino acid state (e.g., 'YTR' -> Leucine/L).
Undefined triplets, such as 'A-G' are translated to '?'. To
select for translation of nucleotide data, enter command:
-e
- Amino acid states are translated to nucleotide triplets using
the compressed IUPAC triplet code for the corresponding amino
acid (e.g. Phenylalanin/F -> 'TTY'). Unrecognized states, such
as '-' or 'X', are translated to '???'.
To define reverse translation of amino acid data, enter command:
-e -e
- Character fill of lacking gene sequences (-g option)
- Under default, lacking sequences in single genes are filled
by 'X' (amino-acid) or 'N' (nucleotide): 'MISSING'. With the
-g command, lacking sequences are filled by indel characters
'-' instead of missing sequence code: 'INDEL'
- Renaming sequence names (-k option)
- To rename defined sequence names prior to file
file processing, enter command:
-k
- Note: the user must provide an extra info file named
"new_seq_names.txt" in the FcC home folder where each row
has the old name delimited from the new name by a tabstop.
- Sequences not specified in the "new_seq_names.txt" file are
left unchanged. FcC prints additional information of the
renaming process to "FcC_rename_control.txt".
- Rejecting 3rd codon position (-d option)
- To exclude 3rd codon positions in given nucleotide infiles and/or
supermatrices, enter command:
-d
- RY coding (-b option)
- To translate third codon positions of nucleotide data into R/Y
code (nucleotide data only), enter command:
-b
e.g., File_1 (nucleotide data):
tax1_allel_0 ACGTTTTGGTTT -> ACRTTYTGRTTY...
- To translate complete sequences into R/Y code, enter command:
-b -b
e.g., File_1 (nucleotide data):
tax1_allel_0 ACGTTTTGGTTT -> RYRYYYYRRYYY...
- Processing consensus sequences (-c option)
- FASconCAT can generate consensus sequences for given infile and/or
supermatrix sequences by building consensus states for sequences
with identical sequence names (identical alphanumeric characters
before the first underscore). Consensus outfile sequences are
named by their sequence names, followed by the suffix '_consensus'.
If no underscore is found in sequence names, FcC will use the
complete sequence name for identification of unique sequences.
e.g., MSA infile: -> Consensus outfile:
Taxon_1 AAAACCC
Taxon_2 AAAACCC
Taxon AAAACCC -> Taxon_consensus AAAACCC
taxon1 AAAACCC -> taxon_1_consensus AAAACCC
Tax_1 AAAGCCT
Tax_2 AAAGCCT -> Tax_consensus AAAGCCT
- There are three different options for building a consensus sequence:
- Most frequent, enter command: -c
- Majority rule, enter command: -c -c
- Strict, enter command: -c -c -c
- Most Frequent ('Freq')
- Builds consensus by taking most frequent character state
of each site. If two or more character states are equally frequent,
FASconCAT uses the corresponding IUPAC ambiguity code
as the consensus state for nucleotide data or '?' (amino acid data).
e.g., File_1 (nucleotide data):
tax1_allel_0 ACGTTTTCGTTT...
tax1_allel_1 ACGTTTTGTTTT...
tax1_allel_2 ACGTTTTGGTTT...
tax1 ACGTTTTTTTTT...
--------------
tax1_consensus ACGTTTTGKTTT...
e.g., File_2 (amino acid data):
tax1_allel_0 NYGKRDEDPWFP...
tax1_allel_1 NYGKRDEDCWFP...
tax1_allel_2 NYGKRDEQCWFP...
tax1 NYGKRDEQCWFP...
--------------
tax1_consensus NYGKRDE?CWFP...
- Majority Frequent ('Maj'):
- Builds consensus by only considering character states that
occur in more than 50% of sequences on a specific site as
consensus state, otherwise assigns '?' (nucleotide & amino
acid data).
e.g., File_1 (nucleotide or amino acid data):
tax1_allel_0 ACGTTTTGGTTT...
tax1_allel_1 ACGTTTTGTTTT...
tax1_allel_2 ACGTTTTGGTTT...
tax1 ACGTTTTTTTTT...
--------------
tax1_consensus ACGTTTTG?TTT...
- Strict Consensus ('Strict'):
- Builds consensus by taking either sequence states
which are fixed in all sequences at a given site or by
using the corresponding IUPAC ambiguity code (nucleotide
data) or 'X' (amino acid data)
e.g., File_1 (nucleotide data):
tax1_allel_0 ACGTTTTGGTTT...
tax1_allel_1 ACGTTTTGTTTT...
tax1_allel_2 ACGTTTTGGTTT...
tax1 ACGTTTTTTTTT...
--------------
tax1_consensus ACGTTTTKKTTT...
e.g., File_2 (amino acid data):
tax1_allel_0 NYGKRDEDPWFP...
tax1_allel_1 NYGKRDEDCWFP...
tax1_allel_2 NYGKRDEQCWFP...
tax1 NYGKRDEQCWFP...
--------------
tax1_consensus NYGKRDEXXWFP...
- READ IN specific sequence infiles (-f option)
- Under default settings, FASconCAT reads in all
files in the working directory that are in FASTA
(.fas or .FASTA), PHYLIP (.phy) or CLUSTAL (.aln)
format. If the '-f' option is chosen, a new list-window
opens after starting FcC. Here input files can be
defined by typing each files assigned numbers (separated
by commas and without line spaces) in a single row.
e.g. -f -s <enter> :
0 infile_1.fas
1 infile_2.phy
2 infile_3.aln
3 infile_4.FASTA
Type: 1,2 <enter> to choose infile_2.phy & infile_3.aln
- Reduction of FASconCAT sequence information (-i option)
- To increase computation speed and reduce the amount of
information FcC prints OUT regarding the infiles and
supermatrix, enter command:
-i
- Print OUT of parsimony informative sites (-j option)
- FASconCAT can print OUT parsimony informative sites
for given infiles and the supermatrix (regarded to
the selected -o option). To print OUT parsimony
informative sites, enter command:
-j
- Definition of output file formats:
- Under default options, FASconCAT prints all output files in
FASTA format. To switch off output of FASTA formatted files,
enter command:
-a
- FcC can also output NEXUS and PHYLIP formatted files:
- To output files in PHYLIP (STRICT) format (sequence
names are restricted to 10 characters), enter command:
-p
- To output files in PHYLIP (RELAXED) format (sequence names
can be up to 250 characters), enter command:
-p -p
- To output files in NEXUS (BLOCK) format (NEXUS blocks),
enter command:
-n
- To output files in NEXUS (MrBAYES) format (NEXUS blocks
with additional MrBayes commands), enter command:
-n -n
- Print OUT partition file(s) (-l option)
- When the sequence concatenation option has been defined, the
user can print additional partition files by entering this
command:
-l
- If the user selects to output a supermatrix in FASTA and/or
PHYLIP format, FcC will print an associated gene partition file
"FcC_supermatrix_partition.txt", which can be directly used for
maximum likelihood analyses in RAxML.
- If the user has selected to output a supermatrix in NEXUS (MrBayes)
format, FcC prints an additional NEXUS file
"FcC_supermatrix_partition.nex" in which gene partitions are
defined.
- Note: the parameters in the partitioned NEXUS (MrBayes) file
differ from those in the non-partitioned version.
--------------------------
For further detailed information please consult
the manual or write an email to ***@***.***
------------------------------------------------------------
help
;
print "\tBACK to FASconCAT MAIN-Menu:\t\t type <return>\n" ;
print "\n\t------------------------------------------------------------\n\t" ;
chomp ( my $answer_xy = <STDIN> );
&menu; ¶meter ;
}
sub preface{
system('cls');
print
<<preface
--------------------FASconCAT PREFACE---------------------
Version : G-1.0
Language : PERL
Last Update : March, 2021
Author : Patrick Kueck, ZFMK Bonn, GERMANY
e-mail : ***@***.***
Homepage : https://github.com/PatrickKueck/FASconCAT-G
This program is free software; you can distribute it
and/or modify it under the terms of the GNU General Public
License as published by the Free Software Foundation ;
either version 2 of the License, or (at your option) any
later version.
This program is distributed in the hope that it will be
useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE. See the GNU General Public License for
more details.
You should have received a copy of the GNU General Public
License along with this program; if not, write to the Free
Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139,
USA.
For further free downloadable programs visit:
http://software.zfmk.de
------------------------------------------------------------
preface
;
print "\tBACK to FASconCAT MAIN-Menu:\t\t type <return>\n" ;
print "\n\t------------------------------------------------------------\n\t" ;
chomp ( my $answer_xy = <STDIN> );
&menu; ¶meter ;
}
sub start{
my $aref_parameter_all = $_[0] ; # List of parameter options of data concatenation -> IN (defined) / OUT (unchanged)
my $aref_parameter_inf = $_[1] ; # List of parameter options of info print out -> IN (defined) / OUT (unchanged)
my $aref_parameter_phy = $_[2] ; # List of supermatrix parameter options of PHYLIP print OUT -> IN (defined) / OUT (unchanged)
my $aref_parameter_nex = $_[3] ; # List of supermatrix parameter options of NEXUS print OUT -> IN (defined) / OUT (unchanged)
my $aref_parameter_con = $_[4] ; # List of consensus options -> IN (defined) / OUT (unchanged)
my $aref_parameter_fil = $_[5] ; # List of of file handling -> IN (defined) / OUT (unchanged)
my $aref_parameter_fas = $_[6] ; # List of supermatrix parameter options of FASTA print OUT -> IN (defined) / OUT (unchanged)
my $aref_parameter_3rd = $_[7] ; # List of third position handling -> IN (defined) / OUT (unchanged)
my $aref_parameter_ryc = $_[8] ; # List of RY coding -> IN (defined) / OUT (unchanged)
my $aref_parameter_tra = $_[9] ; # List of sequence translation options -> IN (defined) / OUT (unchanged)
my $aref_parameter_par = $_[10]; # Print parsimonious sites as extra msa file -> IN (defined) / OUT (unchanged)
my $aref_parameter_ren = $_[11]; # Rename taxon names of given ifiles -> IN (defined) / OUT (unchanged)
my $aref_parameter_prt = $_[12]; # Print partition files for concatenated data -> IN (defined) / OUT (unchanged)
my $aref_parameter_pro = $_[13]; # Start prottest analyses for aa data -> IN (defined) / OUT (unchanged)
my $aref_parameter_mis = $_[14]; # Replacement code of missing gene sequences -> IN (defined) / OUT (unchanged)
my $href_outfile_name = $_[15]; # Outfilename of output option -> IN (defined) / OUT (unchanged)
##############################
## Print Menu Header and start info
&menu ;
print "\n\n\t#### FASconCAT-G: START ! ####" ;
print "\n\t------------------------------------------------------------\n\n" ;
##############################
##############################
## Check if any outputformat has been chosen
if ( ( $aref_parameter_fas->[0] eq 'NO' ) && ( $aref_parameter_phy->[0] eq 'NO' ) && ( $aref_parameter_nex->[0] eq 'NO' ) ){ die "\n\t!COMMAND-ERROR!: No output format selected!\n" }
##############################
##############################
## Definement of global variabels used in these subroutine and further sub-subroutines
my (
%taxa_all_files, # key: taxon name; value : number of occurence in infiles
%hoh_info_of_infile_of_type, # key1: infilename, key2: type of info (e.g. 'seqlength'); info type associated value
@input_files, # list of input files found in fas, FASTA, phy, or aln format
$structure_seq, # defined if structure sequence present in infiles
%taxa_presampled # sampled taxon names for sequence concatenation
) ;
##############################
##############################
## READ IN of possible infile names (CLUSTAL, FASTA, PHYLIP formatted files)
## Store file names in @input_files
for my $format ( qw/aln phy FASTA fas fasta/ ){
for my $file_input ( <*.$format> ){ push @input_files, $file_input }
}
############################## checked !
##############################
## If no infile has been found die with an error prompt
unless ( @input_files > 0 ){ die "\n\t!FILE-ERROR!: Cannot READ IN infile(s)!\n" }
##############################
##############################
## If only single, user specified infiles should be concatenated
## open file request via FASconCAT menu and store only user defined infiles in @input_files
if ( $aref_parameter_all->[0] eq 'NO' ){
&single_define (
\@$aref_parameter_all, # List of parameter options of data concatenation -> IN (defined) / OUT (unchanged)
\@$aref_parameter_inf, # List of parameter options of info print out -> IN (defined) / OUT (unchanged)
\@$aref_parameter_phy, # List of supermatrix parameter options of PHYLIP print OUT -> IN (defined) / OUT (unchanged)
\@$aref_parameter_nex, # List of supermatrix parameter options of NEXUS print OUT -> IN (defined) / OUT (unchanged)
\@$aref_parameter_con, # List of consensus options -> IN (defined) / OUT (unchanged)
\@$aref_parameter_fil, # List of of file handling -> IN (defined) / OUT (unchanged)
\@$aref_parameter_fas, # List of supermatrix parameter options of FASTA print OUT -> IN (defined) / OUT (unchanged)
\@$aref_parameter_3rd, # List of third position handling -> IN (defined) / OUT (unchanged)
\@$aref_parameter_ryc, # List of RY coding -> IN (defined) / OUT (unchanged)
\@$aref_parameter_tra, # List of sequence translation options -> IN (defined) / OUT (unchanged)
\@$aref_parameter_par, # Print parsimonious sites as extra msa file -> IN (defined) / OUT (unchanged)
\@$aref_parameter_ren, # Rename taxon names of given ifiles -> IN (defined) / OUT (unchanged)
\@$aref_parameter_prt, # Print partition files for concatenated data -> IN (defined) / OUT (unchanged)
\@$aref_parameter_pro, # Start prottest analyses for aa data -> IN (defined) / OUT (unchanged)
\@$aref_parameter_mis, # Replacement code of missing gene sequences -> IN (defined) / OUT (unchanged)
\%$href_outfile_name, # Outfilename of output option -> IN (defined) / OUT (unchanged)
***@***.***_files # list of input files found in fas, FASTA, phy, or aln format -> IN (defined) / OUT (unchanged)
);
}
##############################
########################################################################################################################
# START TAXON SAMPLING FROM SINGLE INFILES FOR CONCATENATION
########################################################################################################################
############################################################ START Single infile taxon sampling
# If concatenation specified read IN all single infiles for taxon extraction
# necessary to know all infile taxon names in advance
unless ( $aref_parameter_fil->[0] eq 'Convert' ){
&tax_sampling (
\%taxa_presampled, # sampled taxon names for sequence concatenation -> IN (undefined) / OUT (defined)
***@***.***_files, # list of input files found in fas, FASTA, phy, or aln format -> IN (defined) / OUT (unchanged)
) ;
}
############################################################ END Single infile taxon sampling
########################################################################################################################
########################################################################################################################
# START SINGLE INFILE PROCESSINGS
########################################################################################################################
my %concstring_of_taxon ;
for my $infile ( sort @input_files ){
my %seq_of_tax; # key: taxon; value: corresponding sequences (e.g. taxon1 = sequence_of_taxon1, taxon2 = sequence_of_taxon2....)
############################################################ START FILE READ IN & FILE CHECK
## READ IN file content and check for correct format of infiles -> &input_check
&input_check (
\%seq_of_tax, # key: taxon; value: corresponding sequences -> IN (undefined) / OUT (defined)
\%taxa_all_files, # key: taxon name; value : number of occurence in infiles -> IN (undefined) / OUT (defined)
\$infile, # input file found in fas, FASTA, phy, or aln format -> IN (defined) / OUT (unchanged)
\%hoh_info_of_infile_of_type, # key1: infilename, key2: type of info (e.g. 'seqlength'); info type associated value -> IN (undefined) / OUT (defined)
\$structure_seq # defined if structure sequence present in infiles -> IN (undefined) / OUT (defined)
) ;
############################################################ END FILE READ IN & FILE CHECK
############################################################ START SEQ RENAMING
unless ( $aref_parameter_ren->[0] eq 'NO' ){
&seq_renaming(
\%seq_of_tax, # key: taxon; value: corresponding sequences -> IN (defined) / OUT (changed)
\%hoh_info_of_infile_of_type, # key1: infilename, key2: type of info (e.g. 'seqlength'); info type associated value -> IN (defined) / OUT (changed)
\%taxa_all_files, # key: taxon name; value : number of occurence in infiles -> IN (defined) / OUT (changed)
\$infile, # input file found in fas, FASTA, phy, or aln format -> IN (defined) / OUT (unchanged)
\$structure_seq # defined if structure sequence present in infiles -> IN (defined) / OUT (changed)
);
}
############################################################ END SEQ RENAMING
############################################################ START SEQ TRANSLATION
unless ( $aref_parameter_tra->[0] eq 'NO' ){
&seq_translation(
\%seq_of_tax, # key: taxon; value: corresponding sequences -> IN (defined) / OUT (changed)
\%hoh_info_of_infile_of_type, # key1: infilename, key2: type of info (e.g. 'seqlength'); info type associated value -> IN (defined) / OUT (changed)
\$infile, # input file found in fas, FASTA, phy, or aln format -> IN (defined) / OUT (unchanged)
\$aref_parameter_tra->[0] # defined ry coding parameter option -> IN (defined) / OUT (unchanged)
);
}
############################################################ END SEQ TRANSLATION
############################################################ START CONSENSUS PROCESS OF SEQUENCE BLOCKS IF DEFINED
## If consensus setup is set to yes ('Freq', 'Maj', or 'Strict') built consensus sequence of defined sequence blocks (sequences with identic taxon names before the first underscore)
## regarded to chosen consensus option and store consensus sequence of defined sequence blocks in...
## ...%consensus_seq_of_taxon -> key consensus taxon name (original taxon prefix before first underscore with additional '_consensus' suffix); value: consensus sequence
## -----------------
## e.g.:
## tax1_allel_0 -> ACGTTTTCGTTT...
## tax1_allel_1 -> ACGTTTTAATTT...
## tax1_allel_2 -> ACGTTTTGCTTT...
## tax1 -> ACGTTTTTTTTT...
## $consensus_seq_of_taxon{tax1_consensus} = ACGTTTTNNTTT...
## -----------------
## Consensus sequence built in subroutine -> &make_consensus
unless ( $aref_parameter_con->[0] eq 'NO' ){
##############################
## Create consensus sequences of defined sequence blocks in given infile
&make_consensus (
\%seq_of_tax, # key: taxon; value: corresponding sequences -> IN (defined) / OUT (changed)
\$aref_parameter_con->[0], # defined consensus parameter (Freq, Maj, or Strict) -> IN (defined) / OUT (unchanged)
\%hoh_info_of_infile_of_type, # key1: infilename, key2: type of info (e.g. 'seqlength'); info type associated value -> IN (defined) / OUT (unchanged)
\$infile, # input file found in fas, FASTA, phy, or aln format -> IN (defined) / OUT (unchanged)
\%taxa_all_files, # key: taxon name; value : number of occurence in infiles -> IN (defined) / OUT (changed)
\%taxa_presampled, # sampled taxon names for sequence concatenation -> IN (defined) / OUT (changed)
)
##############################
}
############################################################ END CONSENSUS PROCESS OF SEQUENCE BLOCKS IF DEFINED
############################################################ START RY CODING
## If defined, RY coding of either complete nucleotide sequences or 3rd positions of nuc sequences
unless ( $aref_parameter_ryc->[0] eq 'NO' ){
&ry_coding(
\%seq_of_tax, # key: taxon; value: corresponding sequences -> IN (defined) / OUT (changed)
\%hoh_info_of_infile_of_type, # key1: infilename, key2: type of info (e.g. 'seqlength'); info type associated value -> IN (defined) / OUT (unchanged)
\$infile, # input file found in fas, FASTA, phy, or aln format -> IN (defined) / OUT (unchanged)
\$aref_parameter_ryc->[0] # defined ry coding parameter option -> IN (defined) / OUT (unchanged)
) ;
}
############################################################ END RY CODING
############################################################ START REMOVING 3RD SEQUENCE POSITION
## If defined, remove third sequence position (only in nucleotide data)
unless ( $aref_parameter_3rd->[0] eq 'Remain' ){
&reject_third_nuc_position(
\%seq_of_tax, # key: taxon; value: corresponding sequences -> IN (defined) / OUT (changed)
\$infile, # input file found in fas, FASTA, phy, or aln format -> IN (defined) / OUT (unchanged)
\%hoh_info_of_infile_of_type, # key1: infilename, key2: type of info (e.g. 'seqlength'); info type associated value -> IN (defined) / OUT (changed)
) ;
}
############################################################ START REMOVING 3RD SEQUENCE POSITION
############################################################ START PROTTEST
## If defined, RY coding of either complete nucleotide sequences or 3rd positions of nuc sequences
unless ( $aref_parameter_pro->[0] eq 'NO' ){
unless ( ( $aref_parameter_fas->[0] eq 'NO' ) && ( $aref_parameter_phy->[0] eq 'NO' ) ){
&start_prottest(
\%seq_of_tax, # key: taxon; value: corresponding sequences -> IN (defined) / OUT (unchanged)
\%hoh_info_of_infile_of_type, # key1: infilename, key2: type of info (e.g. 'seqlength'); info type associated value -> IN (defined) / OUT (unchanged)
\$aref_parameter_fil->[0], # List of of file handling -> IN (defined) / OUT (unchanged)
\$aref_parameter_prt->[0], # Print partition files for concatenated data -> IN (defined) / OUT (unchanged)
\$infile, # input file found in fas, FASTA, phy, or aln format -> IN (defined) / OUT (unchanged)
\$href_outfile_name->{prottest}, # PROTTEST SCRIPT NAME -> IN (defined) / OUT (unchanged)
) ;
}
else{ print "\n\n\t!COMMAND-ERROR!: ProtTest not started!\n\tFasta or phylip outfile format must be defined for ProtTest analyses!\n" }
}
############################################################ END PROTTEST
############################################################ START STRUCTURE SEQUENCE ANALYSIS
## If additional parameter info option set to yes and defined structure sequence found in infiles
## extract structure sequence info like loop and stem positions; N of states...
if ( ( $aref_parameter_inf->[0] eq 'YES' ) && ( $structure_seq ) ){
&structure_handling (
\%seq_of_tax, # key: taxon; value: corresponding sequences -> IN (defined) / OUT (unchanged)
\$infile, # input file found in fas, FASTA, phy, or aln format -> IN (defined) / OUT (unchanged)
\%hoh_info_of_infile_of_type, # key1: infilename, key2: type of info (e.g. 'seqlength'); info type associated value -> IN (defined) / OUT (changed)
) ;
}
############################################################ END STRUCTURE SEQUENCE ANALYSIS
############################################################ START EXTRACTION OF ADDITIONAL SEQUENCE STATE INFO
## Count additional info about number of character states for each file sorted by missing data, ambiguity, indel event, and informative state
## if additional info options is set to 'YES'
if ( $aref_parameter_inf->[0] eq "YES" ){
&get_info (
\%seq_of_tax, # key: taxon; value: corresponding sequences -> IN (defined) / OUT (unchanged)
\$infile, # input file found in fas, FASTA, phy, or aln format -> IN (defined) / OUT (unchanged)
\%hoh_info_of_infile_of_type, # key1: infilename, key2: type of info (e.g. 'seqlength'); info type associated value -> IN (defined) / OUT (changed)
) ;
}
############################################################ END EXTRACTION OF ADDITIONAL SEQUENCE STATE INFO
############################################################ START PRINT OUT OF SUPERMATRIX & EXTRACTED INFO
## Print OUT outfiles
unless ( $aref_parameter_fil->[0] eq 'Supermatrix' ){
&print_out (
\%seq_of_tax, # key: taxon; value: corresponding sequences -> IN (defined) / OUT (changed)
\%hoh_info_of_infile_of_type, # key1: infilename, key2: type of info (e.g. 'seqlength'); info type associated value -> IN (defined) / OUT (unchanged)
\%$href_outfile_name, # Outfilename of output option -> IN (defined) / OUT (unchanged)
\@$aref_parameter_all, # List of parameter options of data concatenation -> IN (defined) / OUT (unchanged)
\@$aref_parameter_inf, # List of parameter options of info print out -> IN (defined) / OUT (unchanged)
\@$aref_parameter_phy, # List of parameter options of PHYLIP print OUT -> IN (defined) / OUT (unchanged)
\@$aref_parameter_nex, # List of supermatrix parameter options of NEXUS print OUT -> IN (defined) / OUT (unchanged)
\@$aref_parameter_con, # List of consensus options -> IN (defined) / OUT (unchanged)