stan-dev/math

Inconsistent type used for size of containers

WardBrian opened this issue · 0 comments

Description

The various functions in the Math library that deal with sizes of containers return different types than each other:

  • size() returns size_t (equivalent to unsigned long int)
  • rows() returns int
  • cols() returns an Eigen::Index which at least on GCC is equivalent to long int
  • num_elements() returns int
  • dims() returns a vector of int

In particular, size_t is problematic because it is an unsized type, which leads to bad semantics when the result is used in a mathematical expression as seen in this forum post

We should probably use the same type for all of these functions. Because int is what is used in the language, that seems like a reasonable choice. If we're worried about internal overflows, we should at least used a signed type like ssize_t/long int, rather than size_t

Current Version:

v4.9.0