mikeizbicki/deep-tda

How does ReLU affect metric entropy?

ranyishere opened this issue · 1 comments

Related to: #3

How do you relate the decrease in point sizes to holes? We are inclined to look into doubling dimension and covering number as a way to explain distance and holes.

The n-dimensional ball has doubling dimension n. Situated in the appropriate orthant, ReLU transforms it to a metric space with doubling dimension 0.

Similarly, ReLU can increase the doubling dimension by 1, e.g. {(x,x+1) : for all real numbers x}.
.
Both of these results are yet to be proven explicitly.