Nucleus release issue

Question

Nucleus release issue

Hywan opened this issue 9 years ago · 66 comments

Hello :-),

We have a big issues with the not-finalized libraries. They are tagged as 0.*. We require ~0.0 in this case. This is wrong. Here is why.

Try this:

{
    "require": {
        "hoa/ruler": "~1.0"
    }
}

and then

$ composer install
Loading composer repositories with package information
Installing dependencies (including require-dev)
  - Installing hoa/core (2.15.11.09)
    Loading from cache

  - Installing hoa/visitor (1.15.08.17)
    Loading from cache

  - Installing hoa/exception (1.16.01.11)
    Loading from cache

  - Installing hoa/consistency (1.16.01.11)
    Loading from cache

  - Installing hoa/event (1.16.01.11)
    Loading from cache

  - Installing hoa/protocol (1.16.01.11)
    Loading from cache

  - Installing hoa/stream (0.16.01.11)
    Loading from cache

  - Installing hoa/iterator (1.15.10.29)
    Loading from cache

  - Installing hoa/file (0.15.11.09)
    Loading from cache

  - Installing hoa/ustring (3.15.11.09)
    Loading from cache

  - Installing hoa/compiler (2.15.10.29)
    Loading from cache

  - Installing hoa/regex (0.15.08.13)
    Loading from cache

  - Installing hoa/math (0.15.10.26)
    Loading from cache

  - Installing hoa/ruler (1.15.11.09)
    Loading from cache

See what's wrong here? Because of hoa/stream in 0.…, it will load 0.16.01.11 and then hoa/consistency etc., and it will create a conflict with hoa/core.

To solve this, this is easy. Track all composer.json files with a ~0.0 dependency, and add this: ~0.0,<=0.15, it will work. However, I don't want to create another branch just for that, so here is my proposal:

Track all composer.json with ~0.0 dependencies,
Change them to ~0.15.0 or ~0.0,<=0.15,
Commit on master,
Release a new 0.15.12.31 snapshot,
Undo the change in composer.json and re-commit,
Maybe release a new snapshot, but it's not necessary.

What do you think @hoaproject/hoackers?

Answer 1 · 2016-01-14T10:21:03.000Z

Hello @Hywan!

Is there a way to find quickly all the wrong composer.json repos to update ?

Why rolling back to previous composer version ?

Answer 2 · 2016-01-14T10:21:11.000Z

No it won't work with this workflow. We need new branches. cc @CircleCode

Answer 3 · 2016-01-14T10:21:50.000Z

@shulard ag is your friend.

Answer 4 · 2016-01-14T10:23:02.000Z

I also don't get the point in rolling back the change
what's the point about having new branches?

Answer 5 · 2016-01-14T10:23:30.000Z

Forget my proposal. How will you fix this situation? Do you understand it?

Answer 6 · 2016-01-14T10:33:39.000Z

I'm not really aware of what is possible to do with composer but is there a way to have a conditional rules in composer.json like if Hoa\Core > ~3.0 then use Hoa\SomeLib ~0.16 else Hoa\SomeLib ~0.15

Answer 7 · 2016-01-14T10:34:06.000Z

It seems ok now with the new consistency snapshot... I tried to install ruler: ~1.0 and it works.

Answer 8 · 2016-01-14T10:41:00.000Z

@shulard No it cannot. This is not related.

Answer 9 · 2016-01-14T10:42:07.000Z

To be honest, I think the real problem is that we have stable (tagged) libraries requiring unstable ones.
This mean the assertion "this stable library won't do any BC break" is false since a bc break can occur in any 0.* required library causing the code using the stable library to stop working…

Answer 10 · 2016-01-14T10:42:50.000Z

ATM, I don't see any elegant way of solving this.

Answer 11 · 2016-01-14T10:42:52.000Z

@CircleCode Exactly, and we are working hard on finalizing (making them stable). But we have an issue right now :-). How to solve it?

Answer 12 · 2016-01-14T10:43:08.000Z

@CircleCode Maybe just throw a notice? With apologies.

Answer 13 · 2016-01-14T10:44:35.000Z

@Hywan we have ton find a way to flag BC on unstable lib

Answer 14 · 2016-01-14T10:45:02.000Z

@CircleCode Here is a new proposal. Let's consider hoa/ruler. Checkout 1.15.* (the latest before the BC break), then make the change in the composer.json file to only require ~0.0,<0.15 for instance, then commit, release a new snapshot, and here we go. We will have another branch. This is eleguant, isn't it?

Answer 15 · 2016-01-14T10:45:17.000Z

@Jir4 We should not have unstable libraries in fact.

Answer 16 · 2016-01-14T10:45:24.000Z

@Hywan We need tell them to specify the release of all the beta dependencies.

It's like that i solve my problem when i reporting the issue Monday morning.

Answer 17 · 2016-01-14T10:46:02.000Z

until they have drop Hoa\Core

Answer 18 · 2016-01-14T10:47:56.000Z

@Hywan a lib cannot be stable from the beginning, so for each lib that we will able to add, it will have some times where the lib will be unstable

Answer 19 · 2016-01-14T10:49:54.000Z

@Pierozi I'm not sure Hoa\Core is THE point: future BC breaks in 0.* libs will do the same.
We have to find a future proof way of handling this.
But your points is interesting: isn't it a way of solving it by letting the user know this point and doing something in its composer file?

@Hywan @Jir4

We should not have unstable libraries in fact.

a lib cannot be stable from the beginning…

I think we MUST NOT (rather than should not) have unstable libraries required by stable libraries.
How many unstable libraries are concerned?
On the opposite, if a user explicitely requires a unstable library, up to him to handle its composer requirements.

Answer 20 · 2016-01-14T10:51:55.000Z

@CircleCode i agree with your last sentence

Answer 21 · 2016-01-14T10:54:19.000Z

So

@Hywan
Maybe just throw a notice? With apologies.

And expain what to do ? (explain in #32 (comment))

Answer 22 · 2016-01-14T10:55:15.000Z

@CircleCode you right future BC between dependencies can still exist.

We cannot specify in composer.json a conflit version ?
When trying to load Hoa\Core and Hoa\Stream > 0.16 throw an warning for refuse install

Answer 23 · 2016-01-14T10:56:21.000Z

the problem is: is there a way to fix it on the user side?
since the problem occurs on indirect dependencies (requirements from a lub required by the user), i'm not sure its master composer can restrict versions

Answer 24 · 2016-01-14T10:57:13.000Z

@CircleCode checking git tag ? ^^

Answer 25 · 2016-01-14T10:57:29.000Z

if it can be fixed on user side, so we juste have to apologize, explain it, and stabilize any lib required by a stable lib.

Again, how many unstable libraries are concerned? (I mean, how many unstable libs arerequired by stable ones?)

Answer 26 · 2016-01-14T10:59:07.000Z

@CircleCode on user side yes, instead let composer install dependencies, specify ourself the last compatible version like i did.

Answer 27 · 2016-01-14T11:00:13.000Z

so this seems to be the right and easy way to solve it, isn't it?

Answer 28 · 2016-01-14T11:02:24.000Z

@CircleCode several ... :s
I'll check to make a list

Answer 29 · 2016-01-14T11:04:51.000Z

@Pierozi it will no solve the problem in other project dependencies

Answer 30 · 2016-01-14T11:26:24.000Z

Please, see #32 (comment). This is how to fix it. Thoughts?

Answer 31 · 2016-01-14T11:26:52.000Z

It's strictly equivalent as maintaining 2 versions of the same software.

Answer 32 · 2016-01-14T11:29:04.000Z

@Hywan but if it can be fixed on the user side, it would be better than having to maintain n versions (one for each bc break in unstable libs)

I think we should not follow the multi-versions approach because we don't have enough manpower for this

Answer 33 · 2016-01-14T11:30:49.000Z

@Hywan it seems the most clean way that we can use but @CircleCode is right

Answer 34 · 2016-01-14T11:31:07.000Z

@CircleCode Here the list of unstable lib :

I maybe forgot some

Answer 35 · 2016-01-14T11:47:19.000Z

@CircleCode Yes it can be fixed on the user-side but not easily. We must not ask our users to fix our fails. It's not like maintaining 2 versions of a software, this is just a branch with a small fix living in there, and it solves everything. When hoa/core will be removed, we will be able to remove all these branches too.

Answer 36 · 2016-01-14T11:49:55.000Z

@Hywan so we have to choose a date and announce it as soon as possible

Answer 37 · 2016-01-14T12:05:50.000Z

@Jir4 Why as soon as possible? We must fix this first. Then it will be calm and safe and we will think about it.

@CircleCode Is it possible with Bhoat to push to another branch than master?

Answer 38 · 2016-01-14T12:07:33.000Z

@Hywan just wanna say we have to announce as soon as the date is choosen

Answer 39 · 2016-01-14T12:19:17.000Z

When hoa/core will be removed, we will be able to remove all these branches too.

@Hywan only until next BC break…

Here the list of unstable lib…

@Jir4 but which of them is required by any stable lib?

Is it possible with Bhoat to push to another branch than master?

not now, this is a work that has not been done on Hoa, I think.
IIRC, the push is done with --mirror so all branches are pushed, but only master is listened.
I have to check

Answer 40 · 2016-01-14T12:27:59.000Z

@CircleCode mainly thoose with an exclamation mark

Answer 41 · 2016-01-14T12:54:27.000Z

@CircleCode only until next BC break…

Nop, because we will avoid having unstable dependencies.

So, @hoaproject/hoackers, what do you suggest? We have to take a decision. Maybe we have to make a vote?

Answer 42 · 2016-01-14T12:56:05.000Z

Nop, because we will avoid having unstable dependencies.

what does it mean for stable libs currently depending of unstable libs?

Answer 43 · 2016-01-14T12:58:43.000Z

@CircleCode We have to stabilize unstable libs and this situation will not happen anymore.

Answer 44 · 2016-01-14T13:00:21.000Z

Make an snapshot with "0.15.0" make sense as an hotfix :D.
And its lets us some time to stabilize unstable libs

Answer 45 · 2016-01-14T14:13:27.000Z

I have another idea.

We can delete the latest tags of unstable libraries. Thus, the problem disappears. Then, we create another snapshot but as stable. Most libraries are stables now.

Thoughts @hoaproject/hoackers?

Answer 46 · 2016-01-14T14:23:37.000Z

I love this idea, but have some remarks:

we don't have to do this on libs not required by any stable lib
thus, this will imply the following rule: a lib can pass to a stable status only if all its dependencies are marked as stable (I love this rule)
we will have to communicate on the tag deletion (this is not a problem, but it has to be done)

Answer 47 · 2016-01-14T14:29:06.000Z

@Hywan 👍 but we have to be sure that the lib is really stable

@CircleCode 👍 on all your points ( and an other 👍 for the 2, it will avoid further issues )

Answer 48 · 2016-01-14T14:54:27.000Z

@Hywan 👍
@CircleCode 👍

Answer 49 · 2016-01-14T15:14:35.000Z

Hey guys, it seems that you have found the simpler way to handle the current problem 😄.

For me the 3 points mentioned by @CircleCode are the right rules to be followed.

Answer 50 · 2016-01-14T15:23:30.000Z

That mean if i call Hoa\Ruler at version ~1.0 the Hoa\Stream dependency will be call at 0.* instead new stable 1.* ? i'm not sure

Answer 51 · 2016-01-14T16:15:32.000Z

@Pierozi Currently, hoa/ruler 1.* and 2.* require hoa/stream 0.*. What we are going to do is to bump hoa/stream to 1.*, and update hoa/ruler 2.* to use hoa/stream 1.* instead of 0.*. This way, it will work.

Answer 52 · 2016-01-14T16:16:59.000Z

Got it ! 👍

Answer 53 · 2016-01-14T16:23:25.000Z

@CircleCode @Jir4 Some unstable libraries are not required at all, like hoa/irc or hoa/math. Maybe we should keep them at unstable. But hoa/stream, hoa/socket, hoa/stringbuffer or hoa/file can be promoted as stable. I will start by these ones. Same for hoa/regex etc. Actually, the list is:

$ cat */composer.json | grep '~0.' | cut -d\  -f 9 | grep hoa | sed -e 's/[": ]*//g' | sort | uniq
hoa/devtools
hoa/dispatcher
hoa/file
hoa/graph
hoa/http
hoa/math
hoa/praspel
hoa/realdom
hoa/regex
hoa/socket
hoa/stream
hoa/stringbuffer
hoa/xml
hoa/xyl
hoa/zformat

This is all the libraries we need to update as stables.

Here we go:

It should not take a lot of time.

Reminder of the strategy:

Cancel the last snapshot,
Create a new snapshot with devtools:snapshot -b (-b for BC break, it will bump 0 to 1),
Update all (needed) libraries to use these new snapshots,
Release them.

Agree?

Answer 54 · 2016-01-14T16:25:14.000Z

Actually the strategy is not that simple. For instance, hoa/graph is required by hoa/acl, which is unstable. An unstable library can depends of unstable libraries. So we can remove it from the list (applied).

Answer 55 · 2016-01-14T16:26:34.000Z

Anyway, we can have a better strategy but the final list seems correct. All these libraries are stables. The API didn't change since a long time. Even if it changes, it will be a BC break. It's not a big deal.

Is everyone agree?

Answer 56 · 2016-01-14T16:27:04.000Z

(just hoa/http which is a pure nightmare currently, but I don't care. PSR-7 will be BC break and it will be a good one).

Answer 57 · 2016-01-14T16:36:57.000Z

👍 Agree, about hoa/zformat she's the new one, but don't really a problem for me.

Answer 58 · 2016-01-14T17:41:21.000Z

@CircleCode, @Hywan Agree, seem good for me

@Pierozi Hoa\Zformat is maybe young but it's code isn't ;)

Answer 59 · 2016-01-14T18:05:43.000Z

@Metalaka Exact about Hoa\Zformat.

I am starting. Then lunch, then night and I will finish this night.

Answer 60 · 2016-01-14T18:12:58.000Z

Note: I don't change the .State file.

Answer 61 · 2016-01-14T20:23:35.000Z

Done. All dependencies, except hoa/graph, are not 0.*.
Next step: Update composer.json files to use these new snapshots. I am going to prepare all the commits tonight and create the snapshot tomorrow.

Answer 62 · 2016-01-14T20:29:44.000Z

To get the list of all libraries to update:

$ for i in `\ls`; do if test ! -f $i/composer.json; then continue; fi; cd $i; if test 0 != $(cat composer.json | grep '~0.' | wc -l); then echo $i; fi; cd ..; done

Hmm nop, actually we must filter by stable/unstable libraries. An unstable library (~0.*) can have unstable dependencies. So:

$ for i in `\ls`; do if test ! -f $i/composer.json; then continue; fi; cd $i; if test 0 != $(git tag | grep '0.16' | wc -l); then cd .. && continue; fi; if test 0 != $(cat composer.json | grep '~0.' | wc -l); then echo $i; fi; cd ..; done
Cli
Compiler
Console
Devtools
Dispatcher
Eventsource
File
Http
Json
Locale
Math
Mime
Praspel
Realdom
Regex
Router
Ruler
Socket
Stringbuffer
Test
Websocket
Worker
Xml
Xyl

So here is the progression list:

Answer 63 · 2016-01-14T21:39:55.000Z

Now, if I install hoa/ruler ~1.0, I will have:

$ composer install
Loading composer repositories with package information
Installing dependencies (including require-dev)
  - Installing hoa/core (2.15.11.09)
    Loading from cache

  - Installing hoa/visitor (1.15.08.17)
    Loading from cache

  - Installing hoa/iterator (1.15.10.29)
    Loading from cache

  - Installing hoa/stream (0.15.10.26)
    Loading from cache

  - Installing hoa/file (0.15.11.09)
    Loading from cache

  - Installing hoa/ustring (3.15.11.09)
    Loading from cache

  - Installing hoa/compiler (2.15.10.29)
    Loading from cache

  - Installing hoa/regex (0.15.08.13)
    Loading from cache

  - Installing hoa/math (0.15.10.26)
    Loading from cache

  - Installing hoa/ruler (1.15.11.09)
    Loading from cache

which is what we expect!
And now, if I change ~1.0 for ~2.0 and then update, it works great too:

$ composer update
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Removing hoa/core (2.15.11.09)
  - Installing hoa/exception (1.16.01.11)
    Loading from cache

  - Installing hoa/consistency (1.16.01.14)
    Loading from cache

  - Installing hoa/event (1.16.01.11)
    Loading from cache

  - Removing hoa/visitor (1.15.08.17)
  - Installing hoa/visitor (2.16.01.11)
    Loading from cache

  - Removing hoa/iterator (1.15.10.29)
  - Installing hoa/iterator (2.16.01.11)
    Loading from cache

  - Installing hoa/protocol (1.16.01.11)
    Loading from cache

  - Removing hoa/stream (0.15.10.26)
  - Installing hoa/stream (1.16.01.14)
    Loading from cache

  - Removing hoa/file (0.15.11.09)
  - Installing hoa/file (1.16.01.15)
    Loading from cache

  - Removing hoa/ustring (3.15.11.09)
  - Installing hoa/ustring (4.16.01.11)
    Loading from cache

  - Installing hoa/zformat (1.16.01.14)
    Loading from cache

  - Removing hoa/compiler (2.15.10.29)
  - Installing hoa/compiler (3.16.01.14)
    Loading from cache

  - Removing hoa/math (0.15.10.26)
  - Installing hoa/math (1.16.01.15)
    Loading from cache

  - Removing hoa/regex (0.15.08.13)
  - Installing hoa/regex (1.16.01.15)
    Loading from cache

  - Removing hoa/ruler (1.15.11.09)
  - Installing hoa/ruler (2.16.01.15)
    Downloading: 100%

This is what we expect!

Answer 64 · 2016-01-14T21:41:39.000Z

@Hywan Good job !

Answer 65 · 2016-01-14T21:45:50.000Z

Thank you everyone for your participations! Awesome community ❤️!

Answer 66 · 2016-01-15T07:55:52.000Z

@Pierozi 👍

@Hywan Thanks for all your work !