jonnenauha/obj-simplify

software Killed on very large file

Closed this issue · 2 comments

Adidushi commented

I'm trying to run the program on a very large obj file (~8.5gb) and it prints this:
`
obj-simplify -in 3mx_1.obj -out 3mx_1_optimized.obj

obj-simplify dev {
"Input": "/home/rlb/Documents/STLS/3mx_1.obj",
"Output": "/home/rlb/Documents/STLS/3mx_1_optimized.obj",
"Workers": 64,
"Gzip": -1,
"Epsilon": 0.000001,
"Strict": false,
"Stdout": false,
"Quiet": false,
"NoProgress": false,
"CpuProfile": false
}
1 000 000 lines parsed - Forced GC took 0.02s
2 000 000 lines parsed - Forced GC took 0.05s
3 000 000 lines parsed - Forced GC took 0.15s
4 000 000 lines parsed - Forced GC took 0.08s
5 000 000 lines parsed - Forced GC took 0.09s
6 000 000 lines parsed - Forced GC took 0.13s
7 000 000 lines parsed - Forced GC took 0.15s
8 000 000 lines parsed - Forced GC took 0.19s
9 000 000 lines parsed - Forced GC took 0.18s
10 000 000 lines parsed - Forced GC took 0.23s
11 000 000 lines parsed - Forced GC took 0.22s
12 000 000 lines parsed - Forced GC took 0.29s
13 000 000 lines parsed - Forced GC took 0.29s
14 000 000 lines parsed - Forced GC took 0.28s
15 000 000 lines parsed - Forced GC took 1.10s
16 000 000 lines parsed - Forced GC took 0.27s
17 000 000 lines parsed - Forced GC took 1.26s
18 000 000 lines parsed - Forced GC took 0.29s
19 000 000 lines parsed - Forced GC took 0.32s
20 000 000 lines parsed - Forced GC took 0.34s
21 000 000 lines parsed - Forced GC took 1.56s
22 000 000 lines parsed - Forced GC took 0.37s
23 000 000 lines parsed - Forced GC took 0.39s
24 000 000 lines parsed - Forced GC took 1.79s
25 000 000 lines parsed - Forced GC took 0.48s
26 000 000 lines parsed - Forced GC took 0.42s
27 000 000 lines parsed - Forced GC took 2.04s
28 000 000 lines parsed - Forced GC took 2.08s
29 000 000 lines parsed - Forced GC took 0.50s
30 000 000 lines parsed - Forced GC took 0.45s
31 000 000 lines parsed - Forced GC took 2.34s
32 000 000 lines parsed - Forced GC took 0.47s
33 000 000 lines parsed - Forced GC took 0.52s
34 000 000 lines parsed - Forced GC took 0.51s
35 000 000 lines parsed - Forced GC took 2.64s
36 000 000 lines parsed - Forced GC took 0.57s
37 000 000 lines parsed - Forced GC took 2.77s
38 000 000 lines parsed - Forced GC took 2.85s
39 000 000 lines parsed - Forced GC took 2.92s
40 000 000 lines parsed - Forced GC took 3.00s
41 000 000 lines parsed - Forced GC took 3.07s
42 000 000 lines parsed - Forced GC took 3.15s
43 000 000 lines parsed - Forced GC took 0.66s
44 000 000 lines parsed - Forced GC took 0.71s
45 000 000 lines parsed - Forced GC took 0.67s
46 000 000 lines parsed - Forced GC took 0.72s
47 000 000 lines parsed - Forced GC took 0.75s
48 000 000 lines parsed - Forced GC took 0.77s
49 000 000 lines parsed - Forced GC took 0.80s
50 000 000 lines parsed - Forced GC took 3.74s
51 000 000 lines parsed - Forced GC took 3.83s
52 000 000 lines parsed - Forced GC took 3.91s
53 000 000 lines parsed - Forced GC took 4.00s
54 000 000 lines parsed - Forced GC took 4.05s
55 000 000 lines parsed - Forced GC took 0.85s
56 000 000 lines parsed - Forced GC took 4.22s
57 000 000 lines parsed - Forced GC took 4.30s
58 000 000 lines parsed - Forced GC took 0.90s
59 000 000 lines parsed - Forced GC took 0.90s
60 000 000 lines parsed - Forced GC took 0.92s
61 000 000 lines parsed - Forced GC took 0.94s
62 000 000 lines parsed - Forced GC took 0.96s
63 000 000 lines parsed - Forced GC took 4.74s
64 000 000 lines parsed - Forced GC took 0.97s
65 000 000 lines parsed - Forced GC took 0.99s
66 000 000 lines parsed - Forced GC took 1.01s
67 000 000 lines parsed - Forced GC took 1.08s
68 000 000 lines parsed - Forced GC took 5.13s
69 000 000 lines parsed - Forced GC took 5.16s
70 000 000 lines parsed - Forced GC took 1.07s
71 000 000 lines parsed - Forced GC took 5.33s
72 000 000 lines parsed - Forced GC took 1.08s
73 000 000 lines parsed - Forced GC took 5.49s
74 000 000 lines parsed - Forced GC took 1.14s
75 000 000 lines parsed - Forced GC took 1.15s
76 000 000 lines parsed - Forced GC took 5.75s
77 000 000 lines parsed - Forced GC took 5.82s
78 000 000 lines parsed - Forced GC took 6.08s
79 000 000 lines parsed - Forced GC took 5.98s
80 000 000 lines parsed - Forced GC took 1.24s
81 000 000 lines parsed - Forced GC took 1.27s
82 000 000 lines parsed - Forced GC took 1.35s
83 000 000 lines parsed - Forced GC took 1.28s
84 000 000 lines parsed - Forced GC took 1.30s
85 000 000 lines parsed - Forced GC took 1.30s
86 000 000 lines parsed - Forced GC took 6.47s
87 000 000 lines parsed - Forced GC took 6.59s
88 000 000 lines parsed - Forced GC took 1.35s
89 000 000 lines parsed - Forced GC took 6.76s
90 000 000 lines parsed - Forced GC took 6.84s
91 000 000 lines parsed - Forced GC took 6.84s
92 000 000 lines parsed - Forced GC took 1.68s
93 000 000 lines parsed - Forced GC took 6.98s
94 000 000 lines parsed - Forced GC took 1.42s
95 000 000 lines parsed - Forced GC took 1.47s
96 000 000 lines parsed - Forced GC took 7.54s
97 000 000 lines parsed - Forced GC took 1.49s
98 000 000 lines parsed - Forced GC took 1.51s
99 000 000 lines parsed - Forced GC took 7.46s
100 000 000 lines parsed - Forced GC took 1.57s
101 000 000 lines parsed - Forced GC took 1.88s
102 000 000 lines parsed - Forced GC took 7.98s
103 000 000 lines parsed - Forced GC took 1.61s
104 000 000 lines parsed - Forced GC took 1.99s
105 000 000 lines parsed - Forced GC took 1.64s
106 000 000 lines parsed - Forced GC took 1.62s
107 000 000 lines parsed - Forced GC took 8.11s
108 000 000 lines parsed - Forced GC took 1.66s
109 000 000 lines parsed - Forced GC took 1.66s
110 000 000 lines parsed - Forced GC took 1.89s
111 000 000 lines parsed - Forced GC took 1.73s
112 000 000 lines parsed - Forced GC took 8.76s
113 000 000 lines parsed - Forced GC took 1.72s
114 000 000 lines parsed - Forced GC took 1.72s
115 000 000 lines parsed - Forced GC took 8.78s
116 000 000 lines parsed - Forced GC took 9.01s
117 000 000 lines parsed - Forced GC took 1.88s
118 000 000 lines parsed - Forced GC took 9.15s
119 000 000 lines parsed - Forced GC took 9.24s
120 000 000 lines parsed - Forced GC took 1.93s
121 000 000 lines parsed - Forced GC took 2.09s
122 000 000 lines parsed - Forced GC took 2.05s
123 000 000 lines parsed - Forced GC took 9.65s
124 000 000 lines parsed - Forced GC took 1.96s
125 000 000 lines parsed - Forced GC took 1.95s
126 000 000 lines parsed - Forced GC took 9.67s
127 000 000 lines parsed - Forced GC took 1.96s
128 000 000 lines parsed - Forced GC took 2.01s
129 000 000 lines parsed - Forced GC took 2.20s
130 000 000 lines parsed - Forced GC took 2.07s
131 000 000 lines parsed - Forced GC took 2.20s
132 000 000 lines parsed - Forced GC took 2.28s
133 000 000 lines parsed - Forced GC took 10.23s
134 000 000 lines parsed - Forced GC took 2.05s
135 000 000 lines parsed - Forced GC took 2.09s
136 000 000 lines parsed - Forced GC took 2.11s
137 000 000 lines parsed - Forced GC took 10.39s
138 000 000 lines parsed - Forced GC took 2.15s
139 000 000 lines parsed - Forced GC took 10.53s
140 000 000 lines parsed - Forced GC took 10.71s
141 000 000 lines parsed - Forced GC took 2.17s
142 000 000 lines parsed - Forced GC took 2.20s
143 000 000 lines parsed - Forced GC took 10.79s
144 000 000 lines parsed - Forced GC took 10.99s
Killed
`

Does anyone know what the issue is?

I know it's a very large file but that's why i'm using the program

antont commented

I'd guess you run out of memory - this util does not do anything fancy to read only segments or anything to mem, but IIRC the whole model uses mem and thus there's a limit depending on your system.

Adidushi commented

thanks - I ran on 16gb of ram, i'll try on a computer with 64gb