Join data from different localities
Closed this issue · 1 comments
I am using the myClim package to analyze the output of the 50 TOMST sensors I have installed in my thesis trial. With the documentation you provide I have been able to preliminary explore and analyze some data but I am stucked tryig to resolve some issues with the function join_mc().
The problem arises when I try to merge using this function the three time periods I have for each TOMST sensor. I have 50 files named 1A, 2A, ... 50A in three different directories, that I read using the mc_read_files into three different sets I name XXX, one for each time period. Then, when I try to join all of them, the output of the mc_join reads the same sensor (with different time periods) as three different sensors ( I attach a screenshot of a test with the sensor that I called 1A).
I am contacting you to see how I could solve this problem with the code, because I don't know how to solve it. It is an important part of my thesis to have a continuity of the time periods in the same set, with the later aim of plotting the same sensor in two periods at the same time.
I attach the sensor files for the 1A sensor as an example in the three time periods and the R code lines.
# prepare working environment
round_1 <- setwd("..../19 En-01 Jun 2023")
round_2 <- setwd("....../16 Sept2022-01 Dic 2022")
# I try with an example called "prueba" that I made only with the sensor 1A in a new folder
prueba <- setwd(".../Prueba")
# read all Tomst files from current directory without metadata
test <- mc_read_files(prueba, dataformat_name = "TOMST",
recursive = F,
silent = T,
clean = T)
# cleaning time series
test_tms <- mc_prep_clean(test)
join <- mc_join(data = test_tms, comp_sensors = c("TMS_T1", "TMS_T3"))
Answer
The mc_join
function joins loggers from the same locality together. The problem with your code is that you are using the mc_read_files
function to read data, and each file is in its own locality. If you want to read multiple files into one locality, you must use the mc_read_data
function. This function has a parameter called files_table
, which specifies which file belongs to which locality. For example, you could have the files 1A.csv
, 1A_2.csv
, and 1A_3.csv
in one locality called 1A
. The files_table
parameter can be a data.frame or a path to a CSV file with metadata.
library(stringr)
library(myClim)
path <- list.files(path = ".", pattern = "*.csv")
# path <- c("1A_2.csv", "1A_3.csv", "1A.csv")
files_table <- data.frame(path=path)
# extract locality_id with regular expression
files_table$locality_id <- str_extract(path, "([^_]+)_?.*\\.csv", group=1)
files_table$data_format <- "TOMST"
test <- mc_read_data(files_table)
join <- mc_join(test)
There is a small problem in your code with cleaning. If you have clean = TRUE
in mc_read_*()
, do not call mc_prep_clean()
, because it removes information about duplicate and missing records.