typelevel/squants

Generic quantities support

Opened this issue · 33 comments

non commented

It would be nice if Squants provided a parallel set of generic types for working with other kinds of numbers besides Double, or supported generic types in some other way.

I'm not sure exactly what this would look like, or if this is a realistic goal for the project, but there are definitely types in Spire (Rational, Real, Complex[_], Interval[_], and so on) that would be really useful for dimensional analysis.

garyKeorkunian commented

Erik, I was recently informed of Spires and have been looking at it. I agree it would be nice to let users choose the type for the underlying value. I will play with the idea in a branch and see what I come up with.

Thanks.

garyKeorkunian commented

This work is in progress.

The current development version (0.3.1-SNAPSHOT) offers support for arbitrary numeric types as the argument for all Quantity factory methods. The underlying value and unit multipliers are still Doubles, but this change is a significant step towards providing generic support there as well.

This change will allow user code to begin using arbitrary numeric types now (when creating Quantities) and benefit from the full generics once it is available in the future.

garyKeorkunian commented

Significant progress has been made on this effort. It is a goal for the 0.5 series to have this fully implemented.

There is currently a wip branch that includes this work in progress within the object model replica. This replica can be found in the experimental package in the test code.

Most of the work has been complete and at this time I am playing a bit of whack-a-mole with Scala's type system.

The general strategy is to create a SquantsNumeric trait that can be implemented for any generic type. Implementations for Int, Long, Double and BigDecimal have been created. Implementations for Spire and other types will be included in a contrib project - once all of this working.

The main sticking point is a type conflict between specific UnitOfMeasure implementations and the type required by a Quantity valueUnit.

Quantities are now typed not only on themselves (as in previous versions) but also on a generic value type.

abstract class Quantity[T <: Quantity[T, N], N] ... {
  implicit val num: SquantsNumeric[N]  // provides required operations on N
  def value: N  // previously Double
  def valueUnit: UnitOfMeasure[T]
  ...
}

Since UnitOfMeasure is typed on Quantity, it's signature needed to change to ...

trait UnitOfMeasure[T <: Quantity[T, _]] {
}

The reason for the placeholder in the Value Type position is that it shouldn't matter to the UOM's what generic number type a quantity's underlying value is using. However, this leads to the following type incompatibility:

Quantity.valueUnit must be a UnitOfMeasure[T] where T <: Quantity[T, N]

however

Implementations of UnitOfMeasure are actually UnitOfMeasure[T] where T <: Quantity[T, _]. That is the UOM's are not fixed to specific underlying Quantity value.

In the end Quantity[T, N] != Quantity[T, _], and that is what I need to work through.

I suspect this could be remedied by applying the correct variance, but I haven't found a solution yet.

Any advice or suggests for solving this would be greatly appreciated.

garyKeorkunian commented

Well after a year of dabbling with this off and on I finally have a functioning prototype, which can be found in the squants.experimental package in the test code.

There is still work to do ...

  • update the QuantityRange and TimeDerivative / Integral classes to support the change.
  • port all of the Quantity Types (there are currently 55 of these).
  • create SquantsNumeric Type Classes for Spire types - likely in a companion project.
cquiroz commented

I got inspired by @zainab-ali to add spire/generic support to squants. Would you think the wip branch is a good place to start?

garyKeorkunian commented

@cquiroz I think so. A good amount of work has been done there. The blocker was around the typing for QuantityRanges. Let me know if you have any questions or want to go over it.

garyKeorkunian commented

@cquiroz Actually, the more complete work is in the shared/test/scala/experimental folder of the master branch.

derekmorr commented

I had tried to grab Erik from the Spire project at NEScala to talk about this, but we never connected. He seemed to think there were issues with the current approach, but he didn't elaborate.

hunterpayne commented

I've created something like what this issue requests here...https://github.com/hunterpayne/terra
Its an entirely different project as it required rewriting most of the source to make it work but enjoy...

  • 👍1
garyKeorkunian commented

@hunterpayne Nice. That work in the experimental package needed a whole lot of refactoring to get as far as did, too. I'll take a look at this. It's something we are informally targeting for version 2.0. Thanks!!

hunterpayne commented

Thank you Gary. It should be noted that to make it all work I had to resort to using ClassTags in a couple of places which has drawbacks for native and JS builds. So tread carefully on what you want to bring in from Terra. The simplicity of the Squants code has advantages that are lost when you refactor out the numeric types from Quantities (and the types get significantly more complicated). This is a really good case where the cure might be worse than the disease.

hunterpayne commented

Update, I've successfully removed the ClassTag dependency and gotten Scala-JS and Scala-native versions working. Its still a more complex source base to manage but it now has the same platform support.

garyKeorkunian commented

I've created an alternative model that supports generic numerics in quantities.

If can be found here: https://github.com/garyKeorkunian/squants-generic

This one inverts the type stack so that Dimension is at the "root" and UnitOfMeasure and Quantity have a type-dependency on that. This seems to be more semantically correct for the domain.

The README outlines the goals, current state and roadmap.

Please let me know your thoughts. I'd like to vet this a bit before the major refactoring of classes begins.

hunterpayne commented
garyKeorkunian commented

@hunterpayne Thanks for taking a look and the feedback.

I have looked at terra. I agree terra will be a bit heavier on the maintenance side, but I like many of the things you did there. I need to study it more as it's not quite clear to me how we can extend it with additional types like spire, although I'm sure you have considered it. I do like what you've done, however, I also want to explore further the idea of Quantity being typed on a Dimension instead of the other way around. It seems more semantically correct to me.

I do like that users don't need to change much about their code besides an import. I do think minimizing that impact is important. One part of terra that I like, that seems to get you there, is the [DimensionName]Like naming you use, with specific implementations getting the normal dimensional name. That inspires me to something like this:

final class MassGen[A: SquantsNumeric] // actual class implementation

then provide different imports like

package {
 
  object SquantsDouble {
      type Mass = MassGen[Double]
  }

  object SquantsGeneric {
     type Mass[A] = MassGen[A]
  }
}

Importing SquantsDouble would provide the same typing as exists now. And of course, more can be created that provide specific numerics for each dimension.

SquantsNumeric does provide for interoperability between numerics. It's the fromSquantsNumeric method that provides this and the implementations can choose how best to do that.

As I continue to look at terra, I expect further inspiration and hopefully we can get to something that is both easy to maintain and use. Thanks again!

Gary

garyKeorkunian commented

@hunterpayne Also, I agree I need to refactor a broader set of Dimensions and other features to validate this approach. I just wanted to get some eyes on it to ensure I wasn't doing anything too far off base before I go further.

hunterpayne commented

So there are a few classes in Terra that you might want to look at to really see what is going on inside of there.

  1. TerraOps.scala which is the interface that defines how Terra interacts with types and converts between them. This class is admittedly very messy and could use some clean-up but you get the idea.
  2. AbstractDoubleTerraOps.scala which glues together all the scopes (*Ops) for each subclass of Dimension and UnitOfMeasure into one big scope
  3. StandardTerraOps.scala which is an example of an implementation of the AbstractDoubleTerraOps. Pay special attention to lines 185-206 which use the type aliases to dynamically generate a package hierarchy containing all the types present in Squants.
  4. InformationSymbols.scala which is an example of a package mapping which aliases the type parametrized Dimensions and UnitsOfMeasures into nicer types like Squants currently uses (e.g. Mass, Energy instead of MassLike[Tuple] which is the real type).
hunterpayne commented

@garyKeorkunian Your general approach seems sound. Investigate Quantity being typed on a Dimension further, its a good idea. The thing I discovered as I was working on Terra was that the interactions between values coming from different dimensions happens quite often and you need a good solution for that. Also, I learned that the Scala type inference engine really doesn't do well with multiple type parameters which is why there is a TypeContext which holds types which normally might be their own type parameters. Hope this helps.

  • 👍1
garyKeorkunian commented

I've update squants-generic to support better backward compatibility with 1.x.

There's some sample code in the README here: https://github.com/garyKeorkunian/squants-generic#current-state

  • 👍1
hunterpayne commented

@garyKeorkunian Looks good so far. For extra types to test on maybe consider sigfigs which is a significant digits library for chemistry and engineering.

  • 👍1
TomasPuverle commented

Hi, would mind giving an update on this, please?

garyKeorkunian commented

Unfortunately, not much progress since the last comments above.

Contributors are welcome.

  • 👍1
garyKeorkunian commented

Hello, all.  

It's been a while, but I think I might have a solution to this.

I created a branch with a POC inside of a new squants2 package.

You can see a write up here: https://github.com/typelevel/squants/tree/generic-value-poc/shared/src/main/scala/squants2

Most of the core is refactored and working as hoped.  I converted several dimensions to validate it.  Of course, there is more to do.

If this approach seems satisfactory, I will continue on ... with as much help as I can get :-)

hunterpayne commented
garyKeorkunian commented

Hi @hunterpayne!

Thanks for the quick feedback!  I'll take a look through it this weekend.

garyKeorkunian commented

@hunterpayne 

Thanks for the comments.

  1.  Self typing is used in the current model.  I found it difficult to get the numeric working the way I wanted, but I will revisit it.  Returning this.type seems to work OK, but it is ugly, and also requires some type-casting.  This is mostly because the unit.apply method is returning Quantity[A, Mass.type] instead of Mass[A], which is really just a sub-type of the former.
  2. I think for the non-commutative operations, it's OK to return the result using the numeric type of the LHS.  The user can be explicit if necessary.  Is there something I am missing as to why we would want both variants?
  3. User code can mix types, so some quantities can be Long and others Double.  These should use appropriate conversions when computed, but some "yet to be written" tests will confirm that.
  4. The reason for creating the new QNumeric was primarily to support mixed-type operations.  Numeric has binary operations that require type A on both sides.  I wanted to get around that.  That said, I did follow your advice and created an implicit conversion that creates a QNumeric from any Numeric, which allowed me to eliminate about half that code.  That should allow the use of Spire (and others) out of the box.  Some of the default methods, however, are a bit hacky.  Like the trig and rounding functions do a conversion to and from Double to make use of the math lib.  That could be overwritten for specific Numeric's where necessary.  There's now an example of QBigDecimal that uses a more specific rounded function.  More improvements to come.

I agree, there's never one single solution that works for everyone.  I am happy to share that link.

garyKeorkunian commented

I just pushed an update that eliminates the QNumeric and uses the standard Numeric throughout. Much simpler and more flexible.

I had to supply some default implementations for the rounding stuff, but an alternative could be supplied to the map function if the user code requires something different. The trig functions are only limited places (Angle and SVector) so they return Double (for now). Same thing there, user code could provide alternatives.

@hunterpayne ... and that link is up on the README.

hunterpayne commented

For non-commutative operations, I was worried about usages like Long / Double => Long which is likely not what you want. Forcing the numerator to be a Double would probably be what a user would want there. Not sure which side the compiler would force an upcast for. Most CPUs don't allow mixed type math operations so the compiler would just force casting somewhere. Making that casting explicit is probably for the best as it makes debugging much easier for you.

The trig functions probably can't be well defined for things that are not floating point numbers so that's probably good. I am not sure if sin(i) is defined according to a mathematician.

Glad the implicit QNumeric conversion worked for you. Now that I think about it an implicit conversion there is probably better than an implicit class.

Thanks for the link. And nice progress overall.

  • 👍1
AtelierSnek commented

The trig functions probably can't be well defined for things that are not floating point numbers so that's probably good. I am not sure if sin(i) is defined according to a mathematician.

The trig operations are all defined for complex numbers, yes, however there are rules that are not the same as real trig.
sin(i) == i * sinh(1) for example

So complex trig would likely need its own implementation

  • 👍1
hunterpayne commented
GregoryEssertel commented

Hey there,

What is the status of this development?

Thanks,

hunterpayne commented
hunterpayne commented