nanoFramework Nuget automatic generation
Closed this issue · 32 comments
Is your feature request related to a problem? Please describe.
I'm using nanoFramework (see https://github.com/nanoframework/Home) and would like to benefit of UnitsNet unique ability to convert a temperature or a length to almost anything. There are couple of constraints even if nanoFramework is a .NET implementation using C# and capable of using Nugets. IT's very small footprint and on tiny embedded devices like ESP32 with few kilo bytes of memory and storage.
Describe the solution you'd like
In very small embedded solution, size matters and the smallest the best. Ideally, you just bring what you need and have a reasonable balance between size and features. Also nanoFramework has lilimited capacity like no generic, no Linq support.
Ideally, it would be great to generate automatically 1 Nuget package, like for most of the nanoFramework packages, one per unit in a simplified way.
I've implemented with couple of measurements (so they are all grouped in the same project, ideally should be separated) with a minimum set of features that seems to be sufficient in most cases.
I generated those filesby mainly doing copy/paste from the automatically generated content and having few adjustments to make all the FromXXX static factory functions to just take a double as argument.
Example available here: https://github.com/Ellerbach/nanoFrameworkIoT/tree/master/UnitsNet
What can be available and not is adjustable. The static Factory and the properties are the most valuable elements.
Describe alternatives you've considered
Producing manually a specific set of measures, based on the example I gave above. Updated not often. I would prefer the automatic solution.
Additional context
This automatic code generation would make it easier as well to support reusability of bindings (devices) present in .NET IoT (see https://github.com/dotnet/iot/tree/master/src/devices). All sensors are natively using UnitsNet. That would allow more .NET developers to access smaller .NET capable embedded boards like ESP32 using nanoFramework. The example link include some prototype on how this can be achieved. UnuitsNet are part of this all up path.
Hi and thanks for a great description.
Ideally, it would be great to generate automatically 1 Nuget package, like for most of the nanoFramework packages, one per unit in a simplified way.
Do you mean multiple nugets like UnitsNet.NanoFramework.Length and UnitsNet.NanoFramework.Mass or something else?
I'm open to supporting this target, it seems widely used enough and solves a real need. However, there are a lot of constraints here so I think the easiest approach is to write a separate code generator for this target, similar to how we did for Windows Runtime Component.
See our two implementations here:
https://github.com/angularsen/UnitsNet/tree/master/CodeGen/Generators
Would you be willing to attempt a pull request on this?
I'm happy to assist.
Do you mean multiple nugets like
UnitsNet.NanoFramework.LengthandUnitsNet.NanoFramework.Mass
Yes, that would be most likely ideal, the smallest the best.
The namespace should still be UnitsNet for compatibility with .NET Core/5 code when it comes to reuse code on nanoFramework. But for the name, yes, it should contain nanoFramework, UnitsNet and the unit. For the order, any will do I guess :-)
I'm open to supporting this target, it seems widely used enough and solves a real need.
That's as well the the reason why I'm asking :-)
I think the easiest approach is to write a separate code generator for this target, similar to how we did for Windows Runtime Component.
Yes, I guess that would be a good option. Also, except if some new units or conversion would appear I expect those nanoFramework nuget to be very stable all up.
Would you be willing to attempt a pull request on this?
I'll need to find time for this. Now I understand what to do, the critical part will be the time :-)
I'm happy to assist.
Thanks! I'll need for sure! Especially when it will come to create the nugets and publish them automatically. I'm less stressed by the automatic generator than by this part and best way to create all those small units nuget.
Right, nuget per quantity is a new concept. I guess we could maybe auto-generate the .csproj files needed to build and pack the individual nugets, plus a core nuget package that has all the shared code among them.
We would lose some things, like Length * Duration = Speed calculations, because it would add some complicated dependencies to make such inter-quantity calculations work that I don't see how we can solve.
I look forward to see what you come up with 🚀
Another benefit of this all, is that the library becomes more modular. I can see some benefit in being able to pull in only the quantities you care about, in particular where binary size matters.
We could then consider doing the same for the main library, publishing quantity nugets for that target - in addition to the current package that simply brings it all in one package.
We could also publish meta packages that brings collections of the most typical packages in certain domains, such Length, Mass, Force, Pressure and a handful others that 80% of applications care about. Or more specific domains like electrical engineering bringing in ElectricCurrent, ElectricPotential etc.
I guess we could maybe auto-generate the
.csprojfiles needed to build and pack the individual nugets, plus a core nuget package that has all the shared code among them.
Yes, so far I generated automatically the njproj format needed, once this is done, it should be easier to add the nuget elements to auto create directly the nuget.
We would lose some things, like
Length * Duration = Speedcalculations
Yes, now, in the device itself when you have small sensors, it's usually not where those maths are done. They are done on an aggregated way, on an edge device or app. And this is where having those base units is interesting as they'll match perfectly with the advanced .NET code from this edge device.
I can see some benefit in being able to pull in only the quantities you care about, in particular where binary size matters.
This is THE main benefit, just use what you need. When you have few kbytes available but still need to make some conversions because some sensors are in specific units, but you want to convert for international units, that's where it's super interesting. Also it standardise the way to transfer the data without having to care about some conversion.
We could then consider doing the same for the main library, publishing quantity nugets for that target - in addition to the current package that simply brings it all in one package.
Yes, that would be possible. Honestly, once you know what you want in your Quantity and Unit file, adding a bit more properties or removing some, adding some extra is quite easy. It took me only few hours to get there.
We could also publish meta packages that brings collections of the most typical packages in certain domains, such
Length,Mass,Force,Pressureand a handful others that 80% of applications care about. Or more specific domains like electrical engineering bringing inElectricCurrent,ElectricPotentialetc.
Absolutely. That's totally doable. Now, back to one of your previous point, where I get rid of some of the conversion, used directly the base type, that will make things a bit more complicated maybe but all up, the modularisation is something not too complicated to do now. All the elements are presents, so it's a matter of extracting compatible domains.
I though it will take me much more time, in fact some coding yesterday evening, some early this morning, and testing all this evening and finalizing was largely enough to achieve the most complicated part.
Now, see the notes and limitation into the PR, it's not perfect yet and missing all the build chain. For this, I'll clearly need your help, as it should be quite similar than for the rest. Still nanoFramewok elements needs to be somehow installed for the project to compile. Also the directory artifact needs to be addressed.
I am not sure a NuGet package per unit is a good idea. Imagine the discoverability issues when you have to wade through them on nuget.org.
Isn't in-memory size what you mainly care about? I think an assembly per-unit inside one NuGet package would accomplish that, no?
Both size matters. Now, we can as well choose the most "useful" units, those that are all the time used: Length, Ratio, Mass and some others like this. Like 15 or 20 of them. That may do the trick. Also
Look at today on nuget with nanoFramework as keyword to search, and you'll see that 1 little nuget for 1 little namespace :-)
And we keep all other units just generated with source code. If anyone needs, a copy/paste will do the trick. But at least they'll exist. So a mixt solution.
@angularsen I will need your help on the build side. I don't want to mess up with how things are configured for the rest of the platforms.
Also, we should decide which one to publish and which one not to publish and just leave as source code for specific needs.
What I've identified so far and used a lot:
- Duration
- ElectricResistance
- Length
- Pressure
- Ratio
- Temperature
There are for sure others, will find them while moving forward.
Thanks!
I like @tmilnthorp's suggestion of having multiple assemblies in a single nuget if that is possible.
https://blog.maartenballiauw.be/post/2020/04/22/referencing-specific-assembly-nuget-package.html
Not sure if .njproj has restrictions on nuget usage that would prevent this from working and I'm also not sure if this can be used for referencing multiple assemblies, but it's a neat idea.
If not, then I suppose we could spam the nuget directory with 100+ nanoframework nugets. Not loving that idea, but if all you got is a hammer... 😄
I will need your help on the build side.
I commented on #837 . Will continue the discussion related to getting things built and working there.
I like @tmilnthorp's suggestion of having multiple assemblies in a single nuget if that is possible
That would be nice clearly! We just should make sure then only the needed unit dll will go to the board. Size does matter, the smallest, the best!
If not, then I suppose we could spam the nuget directory with 100+ nanoframework nugets
That's how it's done today for every single code nanoFrmaewok component: https://www.nuget.org/packages?q=nanoframework
That's how it's done today for every single code nanoFrmaewok component:
Actually, the total count on that search today seems to be around 60 so I do think it seems excessive if we added 100+ more just for our small library.
Still, I don't know if the approach with multiple assemblies in a single nuget will work for us. You will just have to try, I guess 😄
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
This is still active :-) We "just" need to find time!
Yes, pinning it so it won't be auto-closed.
The proposed approach above for packaging multiple assemblies in a single NuGet package doesn't work for us. Our project system does support SDKs.
Even if it would work it would still require a manual edition on the projec file to point to the Units assembly(ies) that the project would be consuming.
Unless there is another idea for tackling this, I'm afraid we are doomed to go for the approach of a NuGet per assembly.
As a side comment: I don't think that will impact on discoverability because the search will be performed by either a specify package name or, if the dev is browsing, he(she) will be using a broad search filter and then can narrow the search. No big deal...
Also, to start with, we can publish the main units only. There are a lot of the which is great. Now, for embedded usage, most related to sensors (Temperature, pressure, energy, angles, acceleration and few others like that) are the one most people are using.
Those are the ones used in .NET IoT for example:
- ElectricPotential
- Temperature
- Ratio
- RelativeHumidity
- Illuminance
- Length
- Pressure
- Duration
- ElectricResistance
- ElectricCurrent
- VolumeConcentration
- Power
- Frequency
- ElectricPotentialDc
Ok, so I believe there are three options on the table:
1. A single nuget with the most widely used quantities Length, Force, Pressure etc.
Pro: Simple, does not spam nuget.org.
Con: There will always be someone wanting that extra quantity included. Binary size might grow beyond what is desirable.
2. A single nuget with all the quantities packed as separate assemblies
Then manually referencing individual assemblies from the nuget.
Pro: Simple for us. Simple to install if end-user accepts getting all quantities.
Con: Complicated to install for end-user if only wanting individual packages. It seems nuget upgrades should be painless though.
Uncertainty: The article references a tool nuget. Does installing a regular nuget with multiple assemblies automatically reference all the assemblies, or would that also require a manual step?
3. Multiple nugets, one per quantity + one core nuget.
14 in your proposal so far, could grow upwards of 100+ over time.
Pro: Modular, easy to install with standard nuget tools.
Con: Complicated for us, spams nuget.org that could hurt discoverability.
4. Source generator nuget (experimental)
There was a suggestion not long ago about using source generators (#902), which are nugget-publishable, and allow for some nice tweaking at the client-side.
My thinking
From the UnitsNet contributor perspective, I favor option 2 given that it works like I expect.
From an end-user perspective, who we serve as a library, then I believe option 3 is the way to go. I don't believe it would be very problematic even if we end up publishing 100+ nugets in the end. Those packages would be less popular than the main package, plus with a naming scheme UnitsNet.nanoFramework.Length it becomes clear what nugets belong together.
My conclusion: Option 3 - multiple nugets.
@lipchev @tmilnthorp thoughts? I want to get this ball rolling while @Ellerbach and @josesimoes are on fire and motivated to get this completed.
Update: Added option 4.
Not sure if this is an actually an option- but i reckon it would be possible to have some flavor of the QuantityGenerator published as a separate assembly, allowing some customization of the resulting assembly, which could then be referenced internally in the target solution. There was a suggestion not long ago about using source generators (#902), which are nugget-publishable, and allow for some nice tweaking at the client-side.
@angularsen like I've mentioned before 2) doesn't work because our project system is not SDK based. So, unfortunately, we don't have the luxury of PackageReference therefore no GeneratePathProperty...
My vote goes for option 3. 😉
Sorry I haven't had the time to review much of the proposed changes- but have you already considered what would go into the base package? Would the operators be supported by option 3?
I'm thinking the base package includes
- Pretty much everything;
IQuantity,UnitConverter,UnitAbbreviationsCache,QuantityFormatteretc. - Does not include quantities, such as
Length,LengthUnitenum etc.
Each quantity nuget, beyond types like Length and LengthUnit, would also have to automatically
- Register its unit abbreviations in the static lookup instance
- Register its unit conversions in the static lookup instance
- Probably more I'm not thinking of
Regarding overloads, it will not be possible to do Force = Mass * Acceleration out of the box, but it will be possible to do Mass = Mass + Mass etc.
There was a suggestion not long ago about using source generators
This is a very appealing idea, to publish source generator nugets, but I have no experience with this yet and how well it would work for the nuget consumer trying to create a nanoFramework app.
I'm thinking the base package includes
* Pretty much everything; `IQuantity`, `UnitConverter`, `UnitAbbreviationsCache`, `QuantityFormatter` etc. * Does not include quantities, such as `Length`, `LengthUnit` enum etc.Each quantity nuget, beyond types like
LengthandLengthUnit, would also have to automatically* Register its unit abbreviations in the static lookup instance * Register its unit conversions in the static lookup instance * Probably more I'm not thinking ofRegarding overloads, it will not be possible to do
Force = Mass * Accelerationout of the box, but it will be possible to doMass = Mass + Massetc.
If memory is an issue- then it may not be ideal that something like Quantity has the full list of Quantity/UnitInfo's (we'd have to have each Quantity self-register left and right).
If memory is an issue- then it may not be ideal that something like Quantity has the full list of Quantity/UnitInfo's (we'd have to have each Quantity self-register left and right).
Yes, the current generator for nano isolate every one.
Regarding overloads, it will not be possible to do Force = Mass * Acceleration out of the box, but it will be possible to do Mass = Mass + Mass etc.
So far the inter quantity scenario is not possible but the scenario per quantity is.
We may think of generating specific ones with a subset including few easy to integrate quantities. I would not do it just right now to see how things go first with individual ones. And if there is the need for the inter quantity integration.
@josesimoes @Ellerbach @lipchev @tmilnthorp
I want to keep the ball rolling.
It seems we mostly agree that option 3 (multiple nugets) is the preferred option to try out first.
Option 4 (source generators) sounds interesting, but I can't speak as to whether this will be feasible or a good approach.
I propose you either investigate option 4 - or go ahead with option 3.
I propose you either investigate option 4 - or go ahead with option 3.
I'll look into this and will add the nuspecs to the PR.
Guess you prefer these to be listed as UnitsNet owner on NuGet, correct?
Yes, UnitsNet organization on nuget.org should be the owner, but put yourselves as the authors. No need to include my name in there.
As an example:
https://github.com/angularsen/UnitsNet/blob/master/UnitsNet.NumberExtensions/UnitsNet.NumberExtensions.csproj
@angularsen just pushed a "test" nuspec for Temperature, along with the changes in the build PS1's to support this.
Please review.
When this is good, I'll add the nuspecs for the other units.
And one more time, thanks a lot @angularsen for the help and huge thanks @josesimoes for the work on packaging all this. First heavy lifting migration from .NET IoT to .NET nanoFramework is next week. That will clearly help!
Thanks to both of you, I'm impressed by how quickly you pulled off all this work 🚀🙌