Region Fixer not correctly detecting problems?
Closed this issue · 6 comments
Recently, I used the Minecraft Overviewer to generate a map of my 1.15 server (system specs here), which worked out fairly well, except that it complained about corrupted chunks in the world, leaving me with "holes" in my map (see this image).
So I ran the Region Fixer, but it claimed there were no problems with the chunk data, though in the output I did notice that some of the .mca files mentioned were the same ones Overviewer was complaining about.
They have entries like Scanned r.20.21.mca (c:0, w:0, tme:0, so:0, mt:0, t:0) where t: is always 0. Every single one of them listed as t:0 were "corrupted" chunks according to Overviewer.
Now, I don't know if it's a server world problem, a Region Fixer problem or an Overviewer problem, and nobody playing on the server world has complained about unexpected behaviour or game crashes, so I'm completely lost here.
If it helps determine weather or not this is a Region Fixer issue, here's the (shortened) MCRF output from the terminal. The full output is available here.
seann on black-beast Thu Apr 09 08:10 AM
[~/Servers/PaperMC/1.15] $ python3 regionfixer.py --dc -p 8 -v --log=REDRED.log ./REDRED
Welcome to Region Fixer!
(version: 0.3.0)
############################################################
################# Scanning world: REDRED #################
############################################################
World info:
There are 1846 region files, 4 player files and 64 data files in the world directory.
-------------------- Checking level.dat --------------------
'level.dat' is readable
---------------- Scanning UUID player files ----------------
Scanned c1b97e85-b2a6-4619-b39c-8ea94f19fb4d.dat (File: "c1b97e85-b2a6-4619-b39c-8ea94f19fb4d.dat"; status: OK) 1/4
Scanned f00ea379-05db-4276-ab74-b070dd687993.dat (File: "f00ea379-05db-4276-ab74-b070dd687993.dat"; status: OK) 2/4
Scanned 7b7e64ad-ce10-450c-b4d5-0205bb87f842.dat (File: "7b7e64ad-ce10-450c-b4d5-0205bb87f842.dat"; status: OK) 3/4
Scanned f0100ada-778e-4c93-afcc-ba82b425a65e.dat (File: "f0100ada-778e-4c93-afcc-ba82b425a65e.dat"; status: OK) 4/4
------------- Scanning old format player files -------------
Info: No files to scan.
---------- Scanning structures and map data files ----------
Scanned map_36.dat (File: "map_36.dat"; status: OK)........... 1/64
Scanned map_12.dat (File: "map_12.dat"; status: OK)........... 2/64
Scanned map_11.dat (File: "map_11.dat"; status: OK)........... 3/64
Scanned map_7.dat (File: "map_7.dat"; status: OK)............ 4/64
Scanned map_60.dat (File: "map_60.dat"; status: OK)........... 5/64
Scanned map_1.dat (File: "map_1.dat"; status: OK)............ 6/64
Scanned map_21.dat (File: "map_21.dat"; status: OK)........... 7/64
Scanned map_30.dat (File: "map_30.dat"; status: OK)........... 8/64
Scanned map_19.dat (File: "map_19.dat"; status: OK)........... 9/64
Scanned map_39.dat (File: "map_39.dat"; status: OK)........... 10/64
Scanned map_44.dat (File: "map_44.dat"; status: OK)........... 11/64
Scanned map_26.dat (File: "map_26.dat"; status: OK)........... 12/64
Scanned raids.dat (File: "raids.dat"; status: OK)............ 13/64
Scanned map_52.dat (File: "map_52.dat"; status: OK)........... 14/64
Scanned map_33.dat (File: "map_33.dat"; status: OK)........... 15/64
Scanned map_31.dat (File: "map_31.dat"; status: OK)........... 16/64
Scanned map_53.dat (File: "map_53.dat"; status: OK)........... 17/64
Scanned map_2.dat (File: "map_2.dat"; status: OK)............ 18/64
Scanned map_27.dat (File: "map_27.dat"; status: OK)........... 19/64
Scanned map_28.dat (File: "map_28.dat"; status: OK)........... 20/64
Scanned map_15.dat (File: "map_15.dat"; status: OK)........... 21/64
Scanned map_54.dat (File: "map_54.dat"; status: OK)........... 22/64
Scanned map_22.dat (File: "map_22.dat"; status: OK)........... 23/64
Scanned map_49.dat (File: "map_49.dat"; status: OK)........... 24/64
Scanned map_14.dat (File: "map_14.dat"; status: OK)........... 25/64
Scanned map_29.dat (File: "map_29.dat"; status: OK)........... 26/64
Scanned map_59.dat (File: "map_59.dat"; status: OK)........... 27/64
Scanned map_45.dat (File: "map_45.dat"; status: OK)........... 28/64
Scanned map_5.dat (File: "map_5.dat"; status: OK)............ 29/64
Scanned map_4.dat (File: "map_4.dat"; status: OK)............ 30/64
Scanned map_41.dat (File: "map_41.dat"; status: OK)........... 31/64
Scanned map_23.dat (File: "map_23.dat"; status: OK)........... 32/64
Scanned map_24.dat (File: "map_24.dat"; status: OK)........... 33/64
Scanned map_47.dat (File: "map_47.dat"; status: OK)........... 34/64
Scanned map_56.dat (File: "map_56.dat"; status: OK)........... 35/64
Scanned map_13.dat (File: "map_13.dat"; status: OK)........... 36/64
Scanned map_18.dat (File: "map_18.dat"; status: OK)........... 37/64
Scanned map_32.dat (File: "map_32.dat"; status: OK)........... 38/64
Scanned map_17.dat (File: "map_17.dat"; status: OK)........... 39/64
Scanned map_51.dat (File: "map_51.dat"; status: OK)........... 40/64
Scanned map_3.dat (File: "map_3.dat"; status: OK)............ 41/64
Scanned map_0.dat (File: "map_0.dat"; status: OK)............ 42/64
Scanned map_46.dat (File: "map_46.dat"; status: OK)........... 43/64
Scanned map_16.dat (File: "map_16.dat"; status: OK)........... 44/64
Scanned map_25.dat (File: "map_25.dat"; status: OK)........... 45/64
Scanned map_8.dat (File: "map_8.dat"; status: OK)............ 46/64
Scanned map_57.dat (File: "map_57.dat"; status: OK)........... 47/64
Scanned map_50.dat (File: "map_50.dat"; status: OK)........... 48/64
Scanned map_48.dat (File: "map_48.dat"; status: OK)........... 49/64
Scanned map_42.dat (File: "map_42.dat"; status: OK)........... 50/64
Scanned map_38.dat (File: "map_38.dat"; status: OK)........... 51/64
Scanned map_37.dat (File: "map_37.dat"; status: OK)........... 52/64
Scanned map_58.dat (File: "map_58.dat"; status: OK)........... 53/64
Scanned map_6.dat (File: "map_6.dat"; status: OK)............ 54/64
Scanned map_34.dat (File: "map_34.dat"; status: OK)........... 55/64
Scanned map_40.dat (File: "map_40.dat"; status: OK)........... 56/64
Scanned chunks.dat (File: "chunks.dat"; status: OK)........... 57/64
Scanned map_43.dat (File: "map_43.dat"; status: OK)........... 58/64
Scanned map_35.dat (File: "map_35.dat"; status: OK)........... 59/64
Scanned map_10.dat (File: "map_10.dat"; status: OK)........... 60/64
Scanned idcounts.dat (File: "idcounts.dat"; status: The data file cannot be read) 61/64
Scanned map_9.dat (File: "map_9.dat"; status: OK)............ 62/64
Scanned map_20.dat (File: "map_20.dat"; status: OK)........... 63/64
Scanned map_55.dat (File: "map_55.dat"; status: OK)........... 64/64
------------------ Scanning region files -------------------
Scanned r.20.21.mca (c:0, w:0, tme:0, so:0, mt:0, t:0)......... 1/1846
Scanned r.21.13.mca (c:0, w:0, tme:0, so:0, mt:0, t:0)......... 2/1846
Scanned r.-2.3.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 3/1846
Scanned r.-15.-13.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 4/1846
Scanned r.-20.21.mca (c:0, w:0, tme:0, so:0, mt:0, t:0)......... 5/1846
Scanned r.-18.10.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 6/1846
Scanned r.16.-15.mca (c:0, w:0, tme:0, so:0, mt:0, t:33)........ 7/1846
Scanned r.8.16.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 8/1846
[...]
Scanned r.6.17.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1823/1846
Scanned r.-13.-22.mca (c:0, w:0, tme:0, so:0, mt:0, t:0)......... 1824/1846
Scanned r.21.11.mca (c:0, w:0, tme:0, so:0, mt:0, t:0)......... 1825/1846
Scanned r.-14.8.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1826/1846
Scanned r.9.11.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1827/1846
Scanned r.-18.6.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1828/1846
Scanned r.-18.9.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1829/1846
Scanned r.-22.-1.mca (c:0, w:0, tme:0, so:0, mt:0, t:288)....... 1830/1846
Scanned r.0.15.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1831/1846
Scanned r.8.13.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1832/1846
Scanned r.-10.5.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1833/1846
Scanned r.2.-6.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1834/1846
Scanned r.3.-4.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1835/1846
Scanned r.11.-14.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1836/1846
Scanned r.-17.-4.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1837/1846
Scanned r.4.21.mca (c:0, w:0, tme:0, so:0, mt:0, t:10)........ 1838/1846
Scanned r.-14.20.mca (c:0, w:0, tme:0, so:0, mt:0, t:0)......... 1839/1846
Scanned r.-13.8.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1840/1846
Scanned r.13.12.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1841/1846
Scanned r.7.10.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1842/1846
Scanned r.-14.-17.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1843/1846
Scanned r.16.5.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1844/1846
Scanned r.-17.8.mca (c:0, w:0, tme:0, so:0, mt:0, t:1024)...... 1845/1846
Scanned r.-21.21.mca (c:0, w:0, tme:0, so:0, mt:0, t:0)......... 1846/1846
############################################################
################# Scan results for: REDRED #################
############################################################
Unreadable player files:
No problems found.
Unreadable data files:
idcounts.dat
Chunk problems:
No problems found.
Region problems:
No problems found.
No chunks to delete with status: Corrupted
I misreading the output?
Is this the real life? Or just fantasy?
If the region files are zero bytes, wouldn't that technically mean they ARE corrupted, even if they aren't flagged as corrupted?
Should I just have players /tp around until all the regions get regenerated by the server? And just in case the question of space comes up, there's still 300 GB free on the EXT4 partition where I store the server, which only takes up about 10 GB.
Hello!
-You are not misreading the output.
-Although strange, as far as I know, this is real life.
-If I recall correctly, if the region files are zero bytes in size they should be flagged as "too small" and that doesn't seem to be the case.
There is no doubt that something strange is going on. Could you please send me one of the offending mca files? I'll give it a look and see what's going on.
EDIT: regenerating the region files will most probably solve the issue but that maybe too drastic (depending on your players)
I've downloaded the server world to my local machine, then zipped all the "corrupted" .mca files and dropped them into my Google Drive.
Some of them I did delete, then had the server regenerate them, and re-ran MCRF, but I got the same result, where t=0 for the new files, but it still didn't complain about them.
According to my file manager, those .mca files are 8K on disk.
Further note: Elon Musk seems to think this isn't actually the real life, but that it's a SIM Life, but you can take that as you will. Either way, I'm still seeing something unexpected. 😛
Hello again!
I gave your mca files a look and regionfixer is kind of right. There is nothing wrong with them. Actually there is nothing in them at all. All the files that you sent me are exactly 8192 bytes in size and that is exact size of a region file header. The only problem is that the headers in those files are completely empty.
And with empty I mean full of zeros, theres is nothing else but zeros (check it yourself! Open them in a hex editor). All the files are just 8192 bytes with zeros written in them, and that, as far as I can tell, is just an empty region file with no chunks in it. So, there is no problem with it, no corrupted chunks or anything like that... now we can ask other questions:
- Why is Minecraft creating those files without chunks?
- Why is overviewer reading them as bad region files?
- I ran a md5 sum of all your files, and all of them have the same checksum, so all of them are the same file, except for the name. Did you send me only some of the broken region files and maybe there are others with different problems?
Anyway, looking at the region files theres is nothing that can be done because there is nothing, not information whatsoever, in them.
If this become somekind usual then I could add detection and/or an option to delete this region files but right now this looks quite unusual.
Do you have any idea of what is causing this? Is this still happening to you? Anything else I can help you?
Finally, if this is a simulation I really hope that the player is not bored of this game and trying to end it up in a epic way...
Edit: changed some formating
All the files I sent are the ones Overviewer had a problem with. None of the other files were problematic.
I used the Minecraft Land Generator to create the landmass, and as far as I can tell from MLG's documentation, and watching it run, it loads the Minecraft server at specific overlapping intervals to generate a landmass.
MLG takes X and Z sizes (in blocks) and starting location (X, Z coordinates) as command line options. The Minecraft server then does what it's supposed to (usually, it seems) and creates the chunks.
The process has never been problematic before and I've used it off and on for years.
So, since the problem isn't MRF, and it's not Overviewer, that means it's either the 1.15.2 Server JAR, or it's MLG.
It's definitely a strange issue, and I can't recall ever seeing Region Files with no chunks before. Perhaps something changed in the Server itself... I've verified that the chunks do fill in properly once visited by a player, so maybe the empty .MCA files are simply placeholders because the generated landmass was too big to fit in memory?
It's definitely not a MRF issue, so feel free to close this issue if you like and I'll pursue the matter in other avenues.
Edit: misc formatting fixes
Some of my conclusions/speculations/opinions:
-
In my opinion Overviewer should not complaint about these files as broken/corrupted/problematic, they are just empty region files, and the holes in the map are completely correct. At least the files are correct as you sent them to me. Maybe something funny was happening in the file system when overviewer and the server were a accessing the file at the same time and that is why they are like this?
-
(Warning! Wild speculation!) Without more data I would say the problem is how MLG is interacting with the server. Some update of the server is doing funny things with how MLG does things. Maybe is not giving enough time between runs to create the terrain, or maybe the server doesn't generate as much terrain as expected and the overlapping is not enough. So, the MCA files are created, ready to be populated later but never filled up.
-
I don't think it has to do anything with not enough memory, but who knows.
Thanks a lot for the well documented bug report and your patience, it took me a lot of time to answer. I'm closing this, feel free to reopen or open a new issue if you have more problems.