Regular zarr/n5 community conference calls (Open to all)
jakirkham opened this issue · 139 comments
tl;dr:
- biweekly calls have been 7-8pm Europe/London time on Wednesdays, beginning in 2020 from January 8
- Zoom link: https://zoom.us/j/300670033?pwd=OFhjV0FHQmhHK2FYbGFRVnBPMVNJdz09 (Passcode: 558943)
- rolling agenda/minutes doc
More information as well as a Google calendar (ics) are available under https://zarr.dev/#community-meetings
Original comment
Raised in issue ( https://github.com/zarr-developers/zarr/issues/291 ) was the idea of having regular conference calls to discuss more complex issues in a high bandwidth setting. Opening here to discuss the logistics (e.g. how often, how long, what day, making agendas, what medium, how to join, etc.).
Following up for on the recent adhoc call, is there any interest in starting this process up (this week?) even if it needs adjustment. Happy to add initial agenda items once @jakirkham's questions are worked out. I've copied them here with some ideas and my (relatively unimportant) preferences:
- how often: weekly, semi-weekly, monthly
- how long: 30m, 60m, ...
- what day: M T W R
- what time (PST): 9 10 11
- making agendas: gdoc, issue on this repo, (or even an issue on another repo)
- what medium: appear, zoom, hangout, blue jeans (happy to donate my zoom account)
- how to join: find the right agenda and click on the link?
tl;dr chosen meeting plan
- every two weeks for one hour
- on Wednesdays at 19:00 UK (both GMT and BST)
- agenda and notes are in this single gdoc
- which contains the zoom link
- with rotating chairs
Thanks @joshmoore. What's the best way of collecting preferences for something like this? Google sheet with a form? Some other voting tool?
Why don't we do what @joshmoore suggested above, unless anyone chimes in that it doesn't work?
Day+time is presumably the only real sticking point; if it needs to be put to a vote, Doodle and Google Forms (dumping into a Google Sheet) are good ways to do that, in my experience.
OK, how about this...
- We start telecons from next week, and continue from there every two weeks at least initially.
- We use zoom (thanks @joshmoore).
- Schedule for 60 minutes each call.
- We create a new gdoc for the agenda for each meeting.
- Link to the zoom telecon plus any other joining instructions gets added to the gdoc.
- Link to the gdoc gets posted here to this issue prior to each telecon, as part of an invitation and reminder that the telecon is happening.
- The call is open to anyone, however the focus of the calls is development of zarr specs, implementations and community/sustainability (i.e., it's not a forum for asking usage questions).
Regarding the day/time for the first call, I made a doodle: https://doodle.com/poll/4nwfqqske8bqyh3i
Can I nudge anyone who hasn't completed the doodle poll to add their preferences (@zarr-developers/core-devs). Current best option is Thursday 14th Feb at 20:00 GMT.
I've made a google doc for the agenda for the next call, please feel free to propose agenda items.
Sorry for the radio silence. Just recovering from transatlantic travel. 👍 for the doodle (which I've added my preferences to). I'll add the zoom info to the gdoc once a choice has been made.
Just wanted to add that a big part of the driver for getting these calls going was to build links between zarr and n5 communities (both developers and users), and to discuss ways of aligning efforts. Coming from the zarr side I can be a bit too close to the code sometimes, so apologies if at any point I don't seem to be giving enough recognition to this important point.
I've change the title of this issue to "zarr/n5 community conference calls", and I've also change the title of the google doc with the agenda to the same.
About the meeting, @rabernat did a great job of chairing the last call, and I was thinking it might be a good idea to rotate the chair round several people so we get a diversity of influences. I'd be happy to chair the meeting this week, but I was then wondering if anyone else might be willing to volunteer to be part of the rotation and chair a future meeting.
Personally I would be happy for anyone with an interest in using and/or developing in/on/around zarr/n5 to act as chair. I'd also be happy for whoever is chair to reorder the agenda to reflect their views on priorities.
Does anyone have any objections to this idea?
Btw it looks like Wednesday and Thursday are currently tied for best option, I'll close the poll at end of today.
Thanks for the ping; I'll fill out the doodle poll on our end!
OK, the most popular option is Wednesday (13th) at 20:00 GMT, let's go with that for this week at least.
@WardF, @ambrosejcarr, I'm very sorry you can't make that slot, is that day/time always a problem or would you be able to make the following meeting if it was the same day/time (Wednesday 27th at 20:00 GMT)?
It is just a problem this week; internal meeting conflict. I can make the same time on the 27th, no problem!
Very sorry I missed this call. Something came up at the last minute that I had to deal with. Are there notes available?
The call will only start in 20 minutes, won't it?
Damn time zones! You're right, sorry for the confusion! See you all soon.
Thanks again for the call, it was great to meet everyone.
I commented on the doc just now suggesting we create a calendar event; consider adding your email here or there if you'd like to be added to that.
I'd also weakly vote for reusing the same google doc week after week, and just prepending each new week's agenda (and then minutes) at the top. It's nice for scrollback, searchability, and browser-history recall of the doc when it's time for the call. Obviously whoever's doing the work (thanks Alistair!) should just do what's best for them though.
mdurant at anaconda dot com
Thanks @ryan-williams, good suggestion to create a calendar event, I made a calendar event but ironically don't have your email address :-) Also @funkey I don't have yours either.
Also 👍 on re-using the same google doc for agenda and minutes.
The next call will be Wednesday 27 Feb at 20:00 GMT. @joshmoore has kindly agreed to chair.
The main focus of the call today was how to set up a governance framework so we have a clear process for evolving specs and implementations and for new members of the community to become involved and get recognition for contributing. Pangeo, Jupyter, NumPy and Dask were discussed as possible models. Dask is currently establishing a governance framework and pursuing NumFOCUS affiliation, so there may be relevant discussions there. There was general feeling that we'd like to establish something lightweight and quickly, but we should consider possible models, including those outside the PyData/NumFOCUS space. I proposed we create a zarr-developers/governance repo as a place for governance discussions, and we all took a loose action to read up on relevant governance & process models, with the goal of getting something agreed ASAP.
BTW if anyone ever wants to email me I'm alimanfoo at googlemail dot com.
I think everyone was cool with the idea that we'd have a rotating chair for the calls. I'd like to propose that anyone can add items to the agenda, but it's up to the chair to decide how much time to allocate to which agenda items, and what order to take them in. It's only a 60 min call, so this will probably mean only 1 or 2 substantive agenda items will get covered, but I think that's OK as conversations can also continue in parallel on GitHub around any items that don't get covered.
@joshmoore is up to chair next, then I'd like to nominate @jakirkham, @WardF, @axtimwalde, @ryan-williams, @ambrosejcarr, @constantinpape, @martindurant, @rabernat. Please feel free to decline or nominate someone else. Also anyone else please feel free to volunteer. In an ideal world we'd have representation from all zarr/n5 implementations and user groups.
The only standing agenda item I'd suggest is that we allow time at the start of the call for anyone joining for the first time to introduce themselves and their interest/involvement with zarr/n5.
I am generally up for chairing a session.
Am very busy until end of March though, but I can do it in April or May.
2019-02-27 section has been added to the running document if anyone has agenda items. I've added an "intend to attend" section: I'd suggest that everyone show up a few minutes early for sound check. If you can't get there early, add yourself and we'll wait a bit.
I am confused. I thought the call is on 2019-02-27. Why does the doc say 2019-02-25? Is that a typo?
I think so. We've been doing Wednesdays thus far. @joshmoore, can you please confirm?
Bad math. Very sorry. See everyone this evening. (Note: I've updated the description to include the coordinates so mistakes like mine can be more authoritatively shouted down 😄)
Hi @axtimwalde, @funkey, would either of you be willing to chair the next call? It feels like we created some good momentum to start having technical conversations around the zarr/n5 spec issues raised in the call today, and so I wondered if you'd be willing to lead a continuation of that at the next call?
I won't be here for the next call, sorry :/
@alimanfoo yes, I will be happy to do that.
Great, thanks @axtimwalde, please feel free to structure the meeting as you think best.
No worries @funkey, I'll keep you on the list to chair a future meeting if that's ok.
Hi all, just a reminder of the conference call later today, agenda and joining instructions in the google doc.
I also can't make it, still need to prepare a talk.
Will have a look at the notes in the doc!
I also probably can't make it 😔
In the midst of this bomb cyclone, we’ve lost power several times today so far. Unless it comes back on soon I probably won’t make it either.
Had a good conversation today about architecture, hopefully some useful notes in the google doc, I also raised zarr-developers/zarr-specs#8 to follow up the conversation.
Hi all, we have a call scheduled for tomorrow (27th) but I don't think we nominated a chair. @jakirkham I know it's short notice, but would you be willing?
Also wanted to xref an issue I just raised on the zarr-specs repo - zarr-developers/zarr-specs#11 - with a proposal for how to organise the spec docs.
I would. Unfortunately am hopping on a cross country flight tomorrow afternoon; so, won't make this meeting. Should be able to do the next one.
No problem @jakirkham. Btw I'll be on leave for the next meeting (10 April) but please go ahead without me.
If there is anyone else who would like to chair tomorrow please feel free to volunteer, if not I'll volunteer myself as fallback to chair.
Btw I've also been thinking a bit about defining a simple process for development of new specs - zarr-developers/zarr-specs#12 - any comments welcome.
Hi all, reminder of call today at 20:00 GMT, please feel free to add to the agenda.
@joshmoore can we use your zoom again? If so could you add joining instructions? Thanks.
I would join today as well.
Is there any update on the joining instructions?
Meeting should start in 5.
I was wondering the same thing. If there is no other option, this might work:
Many of us are in this Zoom from the agenda doc: https://zoom.us/j/300670033
We're heading to zoom...
Is anything planned for today?
Yes, we are meeting today. I'll be chairing the meeting today. Though I missed the last meeting, so will be relying on people from the last meeting to direct us to specific topics. From a higher level view, this will be a continuation of the spec discussions we have had thus far.
Have added a section in the docs if people would like to add any topics there. If not, I plan to walk through the issues on the zarr-specs repo.
Thanks for taking the lead @jakirkham!
Hey @jakirkham - we are waiting for you!
We are now discussing whether we have a time-zone-related bug in our calendars.
We have concluded that we must have showed up early for the meeting. Checking back in 45 minutes.
We are blaming Google Calendar.
@ryan-williams, @martindurant - we are back in Zoom now waiting for you. Are you planning to join?
Sorry for the time zone/daylight savings time issues yesterday. We've adjusted the time to be 20:00 BST (15:00 EDT) going forward.
Hi all, just getting back into things here. Re the meeting tomorrow (24 April)...
- Do we have a chair?
- Were there any strong feelings about where the discussion should go next?
While on leave I had a few thoughts about how to simplify the proposed organisation of the zarr-specs repo a bit, and also the associated process for developing spec documents. I'll drop some comments into the relevant issues/PRs (zarr-developers/zarr-specs#11, zarr-developers/zarr-specs#12).
- No chair selected
- Generally more spec discussion (people are eager to move to something more concrete)
Re specs, I've made a revised proposal for how to organise the spec documents, and have also pushed an edit to the WIP process guide to match. I'd appreciate any comments.
I think we need to get to the point of having a skeleton for the 3.0 core protocol spec in a PR. That would give us something concrete to focus discussions on. The main idea in the changes above is that we set up a simple division between core protocol versus protocol extensions, which would give us a mechanism for focusing down on a minimal set of features that are in the core and that we think we can get all implementations to support, while still allowing for extensions that support interesting features but which might be harder to get into all implementations.
Re the call tonight, I feel bad asking anyone specific to chair given it is late notice. Would anyone like to volunteer? If not but there is interest to attend, I'm happy to chair again.
If you intend to attend the call later today please add your name to the attendees section of the agenda. If attendance is light I'm tempted to cancel today's call and try to build up a bit more steam for the next call on 8 May, but if anyone is interested to talk today I'm happy to go ahead.
Btw I've created an initial placeholder for a v3.0 core protocol spec in zarr-developers/zarr-specs#16, and have done a bit of writing trying to try and capture the conceptual model in zarr-developers/zarr-specs#17. Comments welcome of course.
Given light attendance I propose we cancel tonight and reconvene on 8 May, aiming to focus that meeting on spec development.
Thanks @joshmoore for commenting on the agenda. If you have some spec feedback could you put it in an issue on the zarr-specs repo? I'll try to put some more time into specs tonight and this week so would be interested to hear feedback.
Hi all, I've added some suggested items to the agenda for the call today. If you have anything you'd like to discuss please do edit the agenda doc.
I've done a bit of work on the core protocol v3.0 draft, really just creating some structure and straw man content as a focus for discussion. Here's links to relevant PRs:
- Core protocol v3.0 - conceptual model : zarr-developers/zarr-specs#17 -- has been merged into core-protocol-v3.0-dev branch but comments and discussion very welcome, can be revisited
- Core protocol v3.0 - data types : zarr-developers/zarr-specs#18 -- comments and discussion welcome, particularly regarding division between core and extensions
- Core protocol v3.0 - chunk grids : zarr-developers/zarr-specs#22 -- comments and discussion welcome, particularly regarding division between core and extensions
In the interests of keeping things simple, I'd like to suggest we don't worry about nominating a chair person for each meeting, at least for the time being. If anyone would like to set the agenda and/or lead discussion on any particular topic then please do say, otherwise we can just operate informally for a while. If at any point in future it seems that the informal approach is not working so well and not everyone is getting a fair say then I'd be very happy to revisit this, but I think we're OK for the moment.
I will not be able to join the call today. I'm pleased to see the spec discussions moving forward. I think that the Unidata folks are the best people to participate in these detailed discussions on behalf of the netCDF community, rather than me (it's not truly my area of expertise).
Sorry, I will not be able to join today either. I hope very much to be able toc catch up during the coming two weeks and join the next call.
Thanks @rabernat, @axtimwalde, no worries.
We had some good discussion around the core protocol v3.0 spec and exploring ideas around what goes into the core protocol versus what goes into protocol extensions. I think there was general support for the idea that the core protocol is minimal and as simple as possible to do a complete implementation in a range of programming languages. Each implementation can then decide where to invest further effort, which could be in any/all of implementing protocol extensions, storage backends or codecs.
My plan for the next couple of weeks is to bash out some straw man draft content for remaining sections of the v3.0 spec, to provide more substrate for discussion.
Notes on this and other topics in the agenda doc.
Hi folks, just to mention, for the next call (2019-05-22 20:00 BST) I've added a couple of agenda items. One to discuss whether we'd like to apply to the recent CZI call, if so what would we ask for. Another to review work towards the v3.0 protocol spec. I've tried to list out some open questions there, please feel free to add. Some decisions to be made around the store interface and storage keys in particular, lots of subtle issues there. Very happy to discuss other topics too in any order.
Thanks all for discussion today. Regarding the "tricky bit" of the v3.0 spec that we discussed at the end, here's the open PR I mentioned: zarr-developers/zarr-specs#30. @DennisHeimbigner or anyone if you have any thoughts/ideas/alternatives please do chime in.
Apologies I won't be able to call in today, I'm at a meeting. I'll try to post some info shortly on current status and issues regarding spec development.
I’d be happy to run the meeting if you like or we can postpone. It would be good to discuss SciPy at some point (either this meeting or next).
Unfortunately I can't make it today either. I have to give an oral exam all afternoon.
I really think we should go for the CZI call. Zarr would be a perfect target for this, since it has strong bioimaging applications but also benefits a broader scientific ecosystem. We need to figure out who among the devs can actually submit the proposal and host whatever effort is proposed.
I agree with @rabernat and hope that people have reached out to @alimanfoo to declare their availability.
I will also miss today's call because I will be flying home from a meeting then.
Yeah just to clarify @rabernat (and anyone that missed the last meeting), we discussed the CZI call a bit last time. I think everyone is in favor of it. Though the intent of the CZI call is to fund a person to do this work half or possibly full time. Right now we don't know who this person is. So we decided to let people anonymously come to Alistair and voice interest in being this person.
Meeting was somewhat brief. We discussed meeting next week at the usual time. Does this seem reasonable to others (@alimanfoo?)?
Hi all, apologies I missed the call this evening.
I just wanted to mention that I've started working on some slides that give an overview and update on the current status of the v3 spec development work. I still have more topics to cover but there's WIP here. If you have any questions about the v3 spec that you'd like me to address then I'd love to know, please drop any comments here: zarr-developers/zarr-developers.github.io#4
Hi all, apologies for cross-posting this in a few different places, but I finished up a slide deck with an overview of current status of the v3 design work. If you're interested in the v3 work then please do take a look and let me know your thoughts. I'd also be happy to talk to some/any of this at a future call if that would be useful.
I also made a short blog post just to share the slides a bit more broadly and provide some context for anyone outside the current community.
I'm actually finding it a little difficult to make the calls every two weeks at the moment, purely for logistical reasons. The next call is scheduled for Wed 3 July, I wonder if we could keep that, but then maybe drop down to once a month?
Is it the frequency that is the issue or is it the time? If it's the latter, maybe we should just move it?
Makes sense, @alimanfoo. Seems like that's how several of us are faring at the moment.
I was pondering where to store the rota/roster but couldn't convince myself to use gdoc or gspreadsheets. A new repo named z5-meetings or similar was the best I came up with so far. If someone with rights ends up creating that or something similar, I'd be happy to be assigned Jul 3 or Aug 7.
No objection to changing the frequency.
As another consideration, we may consider the length. Perhaps we would be happier with a 30 min meeting for instance?
I think it is worth rechecking that the weekday/time works for people that want to come. Some people have dropped off from the meetings. Also we have occasionally seen some new faces since. As we are also proposing changing frequency, I think it would be good to assess if the time still makes sense for people that want to be there.
No objections to having a chair and a predefined schedule. I think one challenge is we have been discussing the spec primarily, which you have been driving forward, @alimanfoo. This sort of makes you the natural chair for most meetings. Though I can understand that shouldering both burdens might be a bit much. How can we distribute this responsibility a bit?
I'll propose one option (though please feel free to share others). We should have each meeting focus on a specific part of the spec we want to discuss for that meeting. As there are many topics to discuss, we can schedule each topic for a specific meeting along with a chair. It will be up to everyone beforehand to read up on that piece, make some comments in the issue or PR, and be prepared to discuss. This way there is already some discussion. Also everyone has the topic fresh in their minds and anyone can chair it. Plus any tricky points have surfaced in that discussion so we can focus on those in our high bandwidth session. It will be up to the chair to act as a PM a bit and steer the group to avoid going off topic. This may also help accelerate things since we are breaking the spec down into digestible bits that we can work through and solve.
Thoughts?
Thanks @jakirkham for sharing these suggestions, much appreciated. I'd be very happy to take SciPy as a natural break and revisit the scheduling for the meetings afterwards. I also like the suggestion to have a scheduled topic for each meeting along with a chair. Re length of meeting, I suspect we'd struggle to cover enough ground in 30 mins, but I'd also be happy to make 60 mins a hard cutoff for example.
Can I suggest we go ahead with the meeting as scheduled tonight for those who are able to attend, and include two things in the agenda:
- Telecon frequency, length, format.
- How to break up the spec into a series of discussion topics, and what order to take them in.
Regarding the spec discussions, the latest draft of the v3 spec is here, and the design update slides are here, these should hopefully give some ideas about how to break things up.
I'd also be very happy to make some time to answer any questions regarding the current design of the spec. The more we can do to build a common understanding of the problems and possible solutions the better.
Also very happy if anyone has other suggestions for the agenda.
One thing I should raise is that I'm going to be flat out on other projects for a couple of months after SciPy, plus some vacation in August. I am keen to keep Zarr spec discussions going during that time and so can certainly put time into that, but won't be able to do much technical work. So there is a natural period where everyone could take some time to fully digest the current state of the v3 spec, and start to formulate some consensus on where the next phase of work should be directed.
On the last call we agreed to revisit the scheduling of the conference calls. So there will be no conference call today. @jakirkham kindly agreed to setup a new doodle.
Sorry for the delay. Have been out with a cold since SciPy. Will add the poll Monday.
Here's a poll. I left the times unbounded to avoid timezone bias. Also did this poll for the next 2 weeks given that will be the new frequency.
cc @llllllllll
It seems we had only one response to the poll. Though a fair bit of time has also passed. Should we create a new poll? Or should we stick with the original meeting time?
@jakirkham : @WardF and I hung out 2 weeks ago and are in the zoom now if you want to join. It's pretty last minute though. I'd assume due to the turn out that we either need the poll or at least a definitive statement of the next attempt.
Thoughts @alimanfoo?
Next meeting is planned for Oct 23 Nov. 6. at the same time (2000 UK). Agenda items can be added to https://docs.google.com/document/d/1o-xSVe1iAKNIfz2a9U0-90-z96XwwTT9jzp-ZuyOJQc/edit#heading=h.mw5m9r245ow0
Thanks for doing that Josh! 😄
Updated my post that was made on the 24th of October to say that it's tonight the 6th of Nov. Sheesh.
I have a conflict for this meeting, unfortunately. I will hopefully be on for at least part of it.
Just a reminder to add anything to the agenda that you'd like to potentially have discussed tomorrow: Nov. 20 (I triple checked my dates this time!)
Blank agenda added for tomorrow, Wed. the 4th, if anyone would like to add items.
And one added for Wed, the 18th, just in case anyone is around. Other than that, 🎄(or similar). I won't be back around until the 8th of January at the earliest.
Unfortunately I won't be able to make it. Let's try to meet on the 8th in the New Year. Happy Holidays all! 😄
seems like folks didn't meet this past Weds 2020-01-22; should we start again next Weds, 2020-01-29?
If I'll make a cal event again if so (that was how I was usually reminded of them previously, thx @joshmoore 😀)
If you do make an event @ryan-williams, could you please send it out to the group? 🙂
(I made the cal event and invited everyone who I had on the previous one)
This evening @sofroniewn (napari) and @thewtex (ITK) will be joining to discuss spatial dimensions (especially of images) in Zarr and the related ecosystem. Other topics and everyone are welcome.
It's that time again, this evening half the agenda is up in the air for community thoughts/comments/use-cases but we'll try to focus at least half of the time on moving the current governance push forward. See you there.