Wrong library.properties placement

[00alis]

Hi!
On https://create.arduino.cc/editor (the Arduino Web IDE) we include all the libraries in the Library Manager, your library is giving a warning for every single compilation, and many users have complained about it on our Forum.
WARNING: Category '' in library ArduinoUnit is not valid. Setting to 'Uncategorized'

This is due to the wrong placement of library.properties, you can find the guidelines on the right placement here https://github.com/arduino/Arduino/wiki/Library-Manager-FAQ

We could really appreciate if you could move it and make a release, so that the updated version will be automatically displayed in the Library Manager.

Thanks a lot!

[mMerlin]

It looks like the examples folder and library.properties file in the src folder need to be move to the root of the repo. I do not know if utility belongs in the root or src folder. That is going to leave a lot of extraneous (not used to run arduinounit) files in the automatically installed library.

A possibly better alternative, would be to split this into 2 repos. arduinounit and something like ardinounit_testing. arduinounit would need structure and contents something like:

library.properties
keywords.txt
src/
src/ArduinoUnit.h
src/ArduinoUnit.cpp
src/FakeStream.cpp
src/FakeStream.h
src/FakeStreamBuffer.cpp
src/FakeStreamBuffer.h
src/FreeMemory.cpp
src/FreeMemory.h
examples/
examples/...

An extras/ folder could be added to hold things like developer.md and porting-guide.md

Everything else goes in the testing repo. I don't know if the FakeStream and FreeMemory files can be in a sub folder, or have to stay directly in src.

[wmacevoy]

I would love to call this a library policies bug --- how can you provide a library and not have the harness to build and test it (this may be not legal for GPL code). However, since I'm not on that committee, I flipped the project and put a working build tree under "extras". I also merged iss47 since that is just an enhancement. Look at branch iss50 and that should fix both issues.

[mMerlin]

True, though in this context the library is essentially straight source code anyway. Installation is just drop the files in the right folder. Build is handled by the IDE. Which is where this problem comes in, since the online ide library manager effectively gets the installation wrong for this repo.

Not sure that extra branch is of any use here. How does that get turned into a release that the crawler script will find and use going forward? What I read says it uses a (version) tag, but isn't that going to be the master branch?

From my perspective, testing should remain a separate project. There could be several testing projects, none of which have anything directly to do with the actual product (the library). The only real advantage of keeping them together, is making the synchronization between library and testing version visible. But with a source code library, a separate testing project for each target ide makes sense. Nothing wrong with including tests in the examples either.

[matteosuppo]

We need a new release, like 2.1.2, or it won't update on our side.

[wmacevoy]

@mMerlin --- I strongly disagree. From the GPL,

The “Corresponding Source” for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities.

This does not specifically mention testing; but any project with a goal of reliability must have tests. I have never seen a project where the tests for a project and the source code were considered separate (releases are another question). Asking the arduino community to take a different point of view makes for less reliable and reusable toolsets.

Having a library policy that encourages the management of components separate from build tools runs against a very common open source license. Having a policy that discourages the distribution of tests harms reliability.

This seems easy to fix: the root of the install directory is the folder with the library.properties file in it. The rest is project build/test infrastructure. It is NOT extra, the current library policy is broken.

[matteosuppo]

We're not asking you to remove the tests. Just to move the library.properties (and the examples folder) in the root of the repo. Why do you say the library policy is broken?

[mMerlin]

Here is scenario / structure that would seem to cover your concerns, and still fit the existing library policy (actually the library manager API). In your full development project repo, include a build script that would collect the subset needed by the library manager, and send / copy that to a separate repository. The separate repo would be what the library manager is pointed at. It is a totally automatically generated release. The base repo should fulfill all of your concerns about gpl, testing, management. The second provides the run time release version that the library manager would use. To keep them together, make the release repo a submodule. Doing a release involves running the script, commiting and pushing the submodule. Normally a submodule is a dependency for the main project. I do not know of any 'rule' that says it can not be an (automatically generated) output instead.

• aunit
  • .git
  • .gitignore
  • arduinounit
    • .git
    • library.properties
    • keywords.txt
    • src
      • Arduinounit.h
      • Arduinounit.cpp
      • …
    • examples
      • …
    • extras
      • …
  • tests
  • bin
  • …

The library management tool gets pointed at the arduinounit repo. Anyone doing development grabs the aunit repo:

git clone https://github.com/mmurdoch/aunit.git
cd aunit
git submodule init
git submodule update
cd arduinounit
git checkout master

I have not done enough work with git submodules to provide a 'definitive' command sequence, but there are multiple ways of updating the submodule, committing, and pushing, either directly from the submodule, or as part of the main project. And that should be scriptable, to avoid needing to use a bunch of 'arcane' commands to get it right. Though attention to error messages will be needed, if anyone else could be updating the submodule repo.
https://git-scm.com/book/en/v2/Git-Tools-Submodules

That structure also handles my comment about several / separate testing projects for different environments. Those could just be different folders in the main project (or sub folders of tests), or additional submodules.

As you say, the quote from GPL does not mention testing. It would seem that libraries that meet the Arduino 3rd party library specification
https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification
meet the GPL requirements. Not all project best practices, but at least the technical license requirements. Since the libraries are themselves source code. As mentioned in that link, the "library format is intended to be used in tandem with an automatic Library Manager". It is really a form of API.

One of things that annoys me about some of the library (and application too) GPL projects, is the need to install 'everything', just to get run time capabilities. Sure, if I have the need, skills, interest in making changes, I will want everything. But I want that to be kept separate from the main run time installation. Downloading a full source repo, running build and install scripts is fine. If I do not need to make changes, I can delete the repo after installing. For manual installation, the existing arduinounit repository structure works fine for me. Installation amounts to clone the repo, copy the src folder. That just does not work well with the automated library manager tool. Which is interested (only) in the run time requirements.

[wmacevoy]

Because a typical build has the source of a project in a source directory, with the root of the project containing the build tools, Makefiles (or whatever turns your crank), and tests. Since Arduino deploys source for compile-time efficiency (I completely understand this), that is the part that should be installed into a typical users’ library directory. Saying the properties file belongs at the top of the tree (even though the source is usually not at the top of a project directory tree), encourages people to make repos with just source. This is actually against the letter and spirit of the GPL, and bad software development practice.

[wmacevoy]

@mMerlin --- thanks I will try a refactor like this. But it is not typical - it seems ironic to create advanced infrastructure to support a development environment aimed at beginners. It means people thinking about deploying an library will probably not do it this way.

[wmacevoy]

@matteosuppo --- I apologize --- I misunderstood the format requirements (and have seen projects with just the src data). I just pushed an iss50 version which is compliant but I do not have time to build at the moment. I will try to test it soon; can you please tell me if there is an issue with the format?

[matteosuppo]

I think it would work, yes. Thank you.

[wmacevoy]

Ok I made a new release 2.2.0 that fixes #47 and this issue. Thank you for helping me understand the layout. As I see it, the repo can now be cloned into the libraries folder and it would work.

[matteosuppo]

Thank you very much. It's working now

[00alis]

Thanks a lot!