ARCCSS-extremes/climpact2

dims [product xxxxx] do not match the length of object [xxxxx]

Closed this issue 2 years ago · 12 comments

akssinghe commented 4 years ago

I am working with CMIP5 climate projection data to generate climate indices. I was able to successfully generated required indices for the baseline period (1976-2005) using ncdf.wrapper. However, when try to create the thresholds file for the baseline period in order to use for the future period using ncdf.threshold.wrapper, it gives below error.

dims [product 8230750] do not match the length of object [8237064]
Calls: create.thresholds.from.file ... parLapplyLBFiltered -> local.filter.func -> lapply -> FUN
Execution halted

Could you pls help me to solve this issue...

heroldn commented 4 years ago

Hi there, it sounds like perhaps your variables are of different length. Are you specifying the same files that you used to generate the indices for the base period? And can you send or paste your two scripts here. Cheers

On 11/16/2020 4:54 AM Susantha Jayasinghe ***@***.***> wrote: I am working with CMIP5 climate projection data to generate climate indices. I was able to successfully generated required indices for the baseline period (1976-2005) using ncdf.wrapper. However, when try to create the thresholds file for the baseline period in order to use for the future period using ncdf.threshold.wrapper, it gives below error. dims [product 8230750] do not match the length of object [8237064] Calls: create.thresholds.from.file ... parLapplyLBFiltered -> local.filter.func -> lapply -> FUN Execution halted Could you pls help me to solve this issue... — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub #73 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACW3FAM36CLNAEKSVGLYQ3TSQAIWXANCNFSM4TWKRJFA .

akssinghe commented 4 years ago

Thank you. I am using the same file for both indices and thresholds calculations. Attached are the scripts and provided the link to my input file.

climpact2.ncdf.thresholds.wrapper_nex_bl.r.txt
climpact2.ncdf.wrapper_nex_bl.r.txt
Input NC file (1.3GB): https://1drv.ms/u/s!AuqzBPgEu8sXhbtcLF0cEqBFX9as8g?e=f10MLs

heroldn commented 4 years ago

Hi, I did have a try and got a similar error to you. It's not obvious to me why this is happening, I've even tried splitting your data into fewer years. I don't recall ever having a problem like this. I'll need some time to figure this one out. What years is your future time period? If you're in a rush perhaps you can combine your present and future files (since I notice your present file is rather small anyway) and process them together.

On 11/16/2020 2:32 PM Susantha Jayasinghe ***@***.***> wrote: Thank you. I am using the same file for both indices and thresholds calculations. Attached are the scripts and provided the link to my input file. climpact2.ncdf.thresholds.wrapper_nex_bl.r.txt https://github.com/ARCCSS-extremes/climpact2/files/5543933/climpact2.ncdf.thresholds.wrapper_nex_bl.r.txt climpact2.ncdf.wrapper_nex_bl.r.txt https://github.com/ARCCSS-extremes/climpact2/files/5543934/climpact2.ncdf.wrapper_nex_bl.r.txt Input NC file (1.3GB): https://1drv.ms/u/s!AuqzBPgEu8sXhbtcLF0cEqBFX9as8g?e=f10MLs — You are receiving this because you commented. Reply to this email directly, view it on GitHub #73 (comment) , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACW3FALINFKEWXVWJXETO6LSQCMLJANCNFSM4TWKRJFA .

akssinghe commented 4 years ago

Thank you for your effort. I have several future time periods such as 2030s (2016-2045), 2050s (2036-2065) and 2080s (2066-2095). All these files have 30 years data. As you have suggested, I may try to run combining them.

heroldn commented 4 years ago

Yes "cdo mergetime" will do it.

On 11/19/2020 6:17 AM Susantha Jayasinghe ***@***.***> wrote: Thank you for your effort. I have several future time periods such as 2030s (2016-2045), 2050s (2036-2065) and 2080s (2066-2095). All these files have 30 years data. As you have suggested, I may try to run combining them. — You are receiving this because you commented. Reply to this email directly, view it on GitHub #73 (comment) , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACW3FANZFAVZYSAJJLBE5JTSQQMUDANCNFSM4TWKRJFA .

Mark-MengZhang commented 2 years ago

Hello akssinghe and heroldn,

I have faced the same problem.
dims [product xx] do not match the length of object [xx]
Calls: create.thresholds.from.file ... parLapplyLBFiltered -> local.filter.func -> lapply -> FUN
I am wondering if this problem has already been solved.

Cheers,
Mark

heroldn commented 2 years ago

Hi Mark, can you upload ncdumps of your files or even better the files themselves? Are you able to process the sample netcdf? ⁣Sent from my Android (excuse the brevity).

On 15 June 2022, 12:44, at 12:44, Mark-MengZhang ***@***.***> wrote: Hello akssinghe and heroldn, I have faced the same problem. dims [product xx] do not match the length of object [xx] Calls: create.thresholds.from.file ... parLapplyLBFiltered -> local.filter.func -> lapply -> FUN I am wondering if this problem has already been solved. Cheers, Mark -- Reply to this email directly or view it on GitHub: #73 (comment) You are receiving this because you commented. Message ID: ***@***.***>

Mark-MengZhang commented 2 years ago

HI Heroldn, sorry for the delay reply.

I think I have already solved this problem. The code is as follow using cdo:
cdo setcalendar,365_day input.nc output.nc

It can solve the bug
dims [product 8230750] do not match the length of object [8237064]
Calls: create.thresholds.from.file ... parLapplyLBFiltered -> local.filter.func -> lapply -> FUN

I hope this code can also help other researchers who have experienced the same problem.

Cheers,
Mark

👍1

heroldn commented 2 years ago

But I take it your data is actually on a 360_day calendar? I'm not sure what side effects this would have. Possibly it will start counting the 5 days of the start of the next year as a part of the previous year.

Mark-MengZhang commented 2 years ago

I have exampled my original model data and the calculate files using climpact, and all of them are on 365_day calendar. The preprocesses (e.g. delete month=2,day=29) are needed if the data calendar is gregorian, noleap, etc. Moreover, I have already removed all 360_day calendar model data to avoid potential problem. Then we can change the time calendar to 365_day.

If my method have any problems, please do not hesitate to contact with me.

Mark

heroldn commented 2 years ago

Thanks. So you are simply not using any models that have a 360_day calendar?

On 15/08/2022 13:23 ACST Mark-MengZhang ***@***.***> wrote: I have exampled my original model data and the calculate files using climpact, and all of them are on 365_day calendar. The preprocesses (e.g. delete month=2,day=29) are needed if the data calendar is gregorian, noleap, etc. Moreover, I have already removed all 360_day calendar model data to avoid potential problem. Then we can change the time calendar to 365_day. If my method have any problems, please do not hesitate to contact with me. Mark — Reply to this email directly, view it on GitHub #73 (comment), or unsubscribe https://github.com/notifications/unsubscribe-auth/ACW3FAPZRCAU6SNTG3XG5EDVZG5LHANCNFSM4TWKRJFA. You are receiving this because you commented.Message ID: ***@***.***>

Mark-MengZhang commented 2 years ago

Yes, I have removed any models that have a 360_day calendar.