Negative values in tps files converted to NA
Closed this issue · 8 comments
kmiddleton commented
My tps file has a block:
LM=23
1.784773 -6.711417 11.20273
2.813098 -6.843384 9.80391
2.927272 -5.375836 9.157258
1.902069 -5.258978 8.542417
1.985342 -6.27904 8.50721
1.685644 -5.387742 9.193977
2.792423 -7.149733 8.359506
2.598825 -7.740273 7.641532
4.846469 -8.898491 7.700092
5.83001 -10.3386 8.802572
13.33278 -12.14554 0.7303419
7.867355 -9.058348 5.730173
8.522495 -9.230215 6.731133
8.195727 -10.58067 7.066747
7.566654 -10.18632 6.090911
9.866902 -9.652846 3.924262
10.61931 -9.899772 5.087601
10.21485 -11.39175 5.733529
9.486016 -10.9182 4.521498
12.90555 -10.68164 3.426916
13.23247 -12.50423 1.514346
11.85793 -13.19174 2.430471
11.53669 -11.74882 1.191167
ID = C26
which is converted by readland.tps() to
, , NA
[,1] [,2] [,3]
[1,] 1.784773 NA 11.2027300
[2,] 2.813098 NA 9.8039100
[3,] 2.927272 NA 9.1572580
[4,] 1.902069 NA 8.5424170
[5,] 1.985342 NA 8.5072100
[6,] 1.685644 NA 9.1939770
[7,] 2.792423 NA 8.3595060
[8,] 2.598825 NA 7.6415320
[9,] 4.846469 NA 7.7000920
[10,] 5.830010 NA 8.8025720
[11,] 13.332780 NA 0.7303419
[12,] 7.867355 NA 5.7301730
[13,] 8.522495 NA 6.7311330
[14,] 8.195727 NA 7.0667470
[15,] 7.566654 NA 6.0909110
[16,] 9.866902 NA 3.9242620
[17,] 10.619310 NA 5.0876010
[18,] 10.214850 NA 5.7335290
[19,] 9.486016 NA 4.5214980
[20,] 12.905550 NA 3.4269160
[21,] 13.232470 NA 1.5143460
[22,] 11.857930 NA 2.4304710
[23,] 11.536690 NA 1.1911670
I'm using geomorph 3.0.6.
Here's my full sessionInfo().
R version 3.5.0 (2018-04-23)
Platform: x86_64-apple-darwin17.5.0 (64-bit)
Running under: macOS High Sierra 10.13.4
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] geomorph_3.0.6 ape_5.1 rgl_0.99.16 wesanderson_0.3.6 cowplot_0.9.2 forcats_0.3.0
[7] stringr_1.3.0 dplyr_0.7.4 purrr_0.2.4 readr_1.1.1 tidyr_0.8.0 tibble_1.4.2
[13] ggplot2_2.2.1 tidyverse_1.2.1 readxl_1.1.0 fortunes_1.5-4 devtools_1.13.5
loaded via a namespace (and not attached):
[1] subplex_1.5-4 Rcpp_0.12.16 mvtnorm_1.0-7 lubridate_1.7.4 lattice_0.20-35
[6] assertthat_0.2.0 digest_0.6.15 psych_1.8.4 mime_0.5 R6_2.2.2
[11] cellranger_1.1.0 plyr_1.8.4 coda_0.19-1 httr_1.3.1 pillar_1.2.2
[16] rlang_0.2.0 lazyeval_0.2.1 curl_3.2 rstudioapi_0.7 miniUI_0.1.1
[21] Matrix_1.2-14 foreign_0.8-70 htmlwidgets_1.2 munsell_0.4.3 shiny_1.0.5
[26] broom_0.4.4 geiger_2.0.6 compiler_3.5.0 httpuv_1.4.2 modelr_0.1.1
[31] pkgconfig_2.0.1 mnormt_1.5-5 htmltools_0.3.6 crayon_1.3.4 withr_2.1.2
[36] later_0.7.2 MASS_7.3-50 grid_3.5.0 nlme_3.1-137 jsonlite_1.5
[41] xtable_1.8-2 gtable_0.2.0 git2r_0.21.0 magrittr_1.5 scales_0.5.0
[46] cli_1.0.0 stringi_1.2.2 reshape2_1.4.3 promises_1.0.1 bindrcpp_0.2.2
[51] xml2_1.2.0 deSolve_1.21 tools_3.5.0 manipulateWidget_0.9.0 glue_1.2.0
[56] jpeg_0.1-8 hms_0.4.2 crosstalk_1.0.0 parallel_3.5.0 colorspace_1.3-2
[61] rvest_0.3.2 memoise_1.1.0 knitr_1.20 bindr_0.1.1 haven_1.1.1
deanadams commented
Kevin,
This occurs because you are using the tps-style file format for data not originally digitized in tpsDig, and your file contains negative coordinate values. In tpsDig, negative values are reserved for missing landmarks, because when digitizing from images, all pixel units are by definition positive. When tpsDig recently expanded its utility to allow for ‘digitizing’ missing data (via coding their landmarks as negative), we updated ‘readland.tps’ to accommodate this change. However, because some users create tps-style files on their own from non TPSdig software, there is now a conflict.
We are considering coding solutions in ‘readland.tps’ to account for this. In the meantime, there are several work-arounds.
One is to add an XYZ constant to all landmarks for any specimen with negative values prior to reading it in via ‘readland.tps’. This will not affect downstream size or shape estimates so long as GPA is performed subsequent to reading in the data. Another option is to use another file format, or read the data into R via some other means (e.g., file by file using a wrapper) and subsequently use ‘arrayspecs’ to generate the 3D array within geomorph.
These sorts of file format conflicts are inevitable while a file format designed for one purpose gets used in other contexts (as is the case here). The geomorph team has been devising alternative and more flexible file format options to alleviate such issues, but users do need to be aware that use of one file format in other contexts can cause such issues to arise.
Best,
Dean
Dr. Dean C. Adams
Director of Graduate Education, EEB Program
Professor
Department of Ecology, Evolution, and Organismal Biology
Iowa State University
www.public.iastate.edu/~dcadams/<http://www.public.iastate.edu/~dcadams/>
phone: 515-294-3834
From: Kevin Middleton <notifications@github.com>
Sent: Monday, May 14, 2018 12:12 PM
To: geomorphR/geomorph <geomorph@noreply.github.com>
Cc: Subscribed <subscribed@noreply.github.com>
Subject: [geomorphR/geomorph] Negative values in tps files converted to NA (#15)
My tps file has a block:
LM=23
1.784773 -6.711417 11.20273
2.813098 -6.843384 9.80391
2.927272 -5.375836 9.157258
1.902069 -5.258978 8.542417
1.985342 -6.27904 8.50721
1.685644 -5.387742 9.193977
2.792423 -7.149733 8.359506
2.598825 -7.740273 7.641532
4.846469 -8.898491 7.700092
5.83001 -10.3386 8.802572
13.33278 -12.14554 0.7303419
7.867355 -9.058348 5.730173
8.522495 -9.230215 6.731133
8.195727 -10.58067 7.066747
7.566654 -10.18632 6.090911
9.866902 -9.652846 3.924262
10.61931 -9.899772 5.087601
10.21485 -11.39175 5.733529
9.486016 -10.9182 4.521498
12.90555 -10.68164 3.426916
13.23247 -12.50423 1.514346
11.85793 -13.19174 2.430471
11.53669 -11.74882 1.191167
ID = C26
which is converted by readland.tps() to
, , NA
[,1] [,2] [,3]
[1,] 1.784773 NA 11.2027300
[2,] 2.813098 NA 9.8039100
[3,] 2.927272 NA 9.1572580
[4,] 1.902069 NA 8.5424170
[5,] 1.985342 NA 8.5072100
[6,] 1.685644 NA 9.1939770
[7,] 2.792423 NA 8.3595060
[8,] 2.598825 NA 7.6415320
[9,] 4.846469 NA 7.7000920
[10,] 5.830010 NA 8.8025720
[11,] 13.332780 NA 0.7303419
[12,] 7.867355 NA 5.7301730
[13,] 8.522495 NA 6.7311330
[14,] 8.195727 NA 7.0667470
[15,] 7.566654 NA 6.0909110
[16,] 9.866902 NA 3.9242620
[17,] 10.619310 NA 5.0876010
[18,] 10.214850 NA 5.7335290
[19,] 9.486016 NA 4.5214980
[20,] 12.905550 NA 3.4269160
[21,] 13.232470 NA 1.5143460
[22,] 11.857930 NA 2.4304710
[23,] 11.536690 NA 1.1911670
I'm using geomorph 3.0.6.
Here's my full sessionInfo().
R version 3.5.0 (2018-04-23)
Platform: x86_64-apple-darwin17.5.0 (64-bit)
Running under: macOS High Sierra 10.13.4
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] geomorph_3.0.6 ape_5.1 rgl_0.99.16 wesanderson_0.3.6 cowplot_0.9.2 forcats_0.3.0
[7] stringr_1.3.0 dplyr_0.7.4 purrr_0.2.4 readr_1.1.1 tidyr_0.8.0 tibble_1.4.2
[13] ggplot2_2.2.1 tidyverse_1.2.1 readxl_1.1.0 fortunes_1.5-4 devtools_1.13.5
loaded via a namespace (and not attached):
[1] subplex_1.5-4 Rcpp_0.12.16 mvtnorm_1.0-7 lubridate_1.7.4 lattice_0.20-35
[6] assertthat_0.2.0 digest_0.6.15 psych_1.8.4 mime_0.5 R6_2.2.2
[11] cellranger_1.1.0 plyr_1.8.4 coda_0.19-1 httr_1.3.1 pillar_1.2.2
[16] rlang_0.2.0 lazyeval_0.2.1 curl_3.2 rstudioapi_0.7 miniUI_0.1.1
[21] Matrix_1.2-14 foreign_0.8-70 htmlwidgets_1.2 munsell_0.4.3 shiny_1.0.5
[26] broom_0.4.4 geiger_2.0.6 compiler_3.5.0 httpuv_1.4.2 modelr_0.1.1
[31] pkgconfig_2.0.1 mnormt_1.5-5 htmltools_0.3.6 crayon_1.3.4 withr_2.1.2
[36] later_0.7.2 MASS_7.3-50 grid_3.5.0 nlme_3.1-137 jsonlite_1.5
[41] xtable_1.8-2 gtable_0.2.0 git2r_0.21.0 magrittr_1.5 scales_0.5.0
[46] cli_1.0.0 stringi_1.2.2 reshape2_1.4.3 promises_1.0.1 bindrcpp_0.2.2
[51] xml2_1.2.0 deSolve_1.21 tools_3.5.0 manipulateWidget_0.9.0 glue_1.2.0
[56] jpeg_0.1-8 hms_0.4.2 crosstalk_1.0.0 parallel_3.5.0 colorspace_1.3-2
[61] rvest_0.3.2 memoise_1.1.0 knitr_1.20 bindr_0.1.1 haven_1.1.1
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<#15>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AGSvqR43bVWm62xu5bO6IsbOxQYWFQCuks5tybrXgaJpZM4T-LZ7>.
kmiddleton commented
Thanks. I had been using the "add a constant" workaround, which works fine.
deanadams commented
Great. Glad this hasn’t held you up, and thanks for the alert.
Best,
Dean
Dr. Dean C. Adams
Director of Graduate Education, EEB Program
Professor
Department of Ecology, Evolution, and Organismal Biology
Iowa State University
www.public.iastate.edu/~dcadams/<http://www.public.iastate.edu/~dcadams/>
phone: 515-294-3834
From: Kevin Middleton <notifications@github.com>
Sent: Monday, May 14, 2018 1:44 PM
To: geomorphR/geomorph <geomorph@noreply.github.com>
Cc: Adams, Dean [EEOBS] <dcadams@iastate.edu>; Comment <comment@noreply.github.com>
Subject: Re: [geomorphR/geomorph] Negative values in tps files converted to NA (#15)
Thanks. I had been using the "add a constant" workaround, which works fine.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<#15 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AGSvqUDhfOEIt4ogwrc7t4wcB_nm_9DGks5tydCMgaJpZM4T-LZ7>.
TP610212 commented
Apologies for my ignorance, but how do you edit the TPS file prior to readland.tps to add a constant?
I've added a constant into that part of the code, but it still reads as NA values. Sorry - am new to this kind of data.
Thanks.
antkal commented
A tps is just a formatted text file. Manually, you can open it in any
text editor, but adding a constant to that would be a headache. If you
are fluent in R, you can program reading in the tps from scratch with
some settings, which is what readland.tps does.
But before that, can you please try running readland.tps from the
Develop version of geomorph on github? I have modified how the function
behaves with negative numbers recently, and would be grateful if you
could give it a try and let the list know if it seems easy to deal with
the negative coordinates issue this way.
Thanks!
Antigoni
…On 30/05/2018 13:58, TP610212 wrote:
Apologies for my ignorance, but how do you edit the TPS file prior to
readland.tps to add a constant?
I've added a constant into that part of the code, but it still reads
as NA values. Sorry - am new to this kind of data.
Thanks.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#15 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ASqw_SbSG4v1yacrFzU2ZSRjoCvg1_kmks5t3peGgaJpZM4T-LZ7>.
--
Antigoni Kaliontzopoulou
Assistant Researcher
CIBIO/InBIO, Centro de Investigação em Biodiversidade e Recursos Genéticos
University of Porto
Campus Agrário de Vairão, 4485-661 Vairão
PORTUGAL
https://cibio.up.pt/research-groups-1/details/phenevol
http://cibio.up.pt/people/details/akaliont
http://antigonik.wix.com/akaliontzopoulou
TP610212 commented
@antkal Hi,
Just updated and worked fine! As there were a lot of negative values I had to respond to the Negative landmark coordinates have been identified. Should they be considered as NAs? (y/n) lots of times, but wasn't too much of a faff. Thanks so much!
antkal commented
Hi,
you are correct, I had the code ask for every single specimen. I have
fixed that to ask only once for the whole file. Could you please
re-install from Develop and confirm?
Thanks a lot for your help!
Antigoni
…On 30/05/2018 15:41, TP610212 wrote:
@antkal <https://github.com/antkal> Hi,
Just updated and worked fine! As there were a lot of negative values I
had to respond to the Negative landmark coordinates have been
identified. Should they be considered as NAs? (y/n) lots of times, but
wasn't too much of a faff. Thanks so much!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#15 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ASqw_ZjnShMJ7f2ikmBYqyJqZ-rXgCfCks5t3q-ngaJpZM4T-LZ7>.
--
Antigoni Kaliontzopoulou
Assistant Researcher
CIBIO/InBIO, Centro de Investigação em Biodiversidade e Recursos Genéticos
University of Porto
Campus Agrário de Vairão, 4485-661 Vairão
PORTUGAL
https://cibio.up.pt/research-groups-1/details/phenevol
http://cibio.up.pt/people/details/akaliont
http://antigonik.wix.com/akaliontzopoulou