Discussion: What does it take to defragment an open source community?
campagnola opened this issue · 40 comments
Current situation: several python projects already exist, partially competing and partially complementary. For the most part we agree this is wasteful, that we could potentially do much more by working together, but actually accomplishing that can be very difficult.
In this thread I'd like to collect thoughts about this process:
- How to decide whether successfully defragmenting would actually be a net benefit to the community
- How to decide whether a community is capable of defragmenting, and how much effort it will take
- How to ensure the project has the resources and attention necessary to succeed
- How / whether to ensure older projects are deprecated to avoid exacerbating the problem (https://xkcd.com/927/)
- Pitfalls that have doomed projects like these in the past
My favorite success story has to be NumPy, but I wasn't close enough to that to know what made it work (perhaps @njsmith has some ideas?).
@aquilesC mentioned the PyLab project via this discussion, which is a joint effort between the developers of lantz, pymeasure, and instrumentkit (maybe others). That seems to have had some success, but again I can't really tell you that story (I welcome comments from @hgrecco, @bilderbuchi, @scasagrande).
From my own experience working on vispy, I can say that project was not easy, but ultimately successful in some of its goals (importantly, we reduced the number of active projects and created a self-sustaining community project). The existence of large, prior codebases and strong, competing views about architecture were a major challenge. I and others have worked to avoid that by keeping the scope of this project minimal as a first step toward establishing a cooperative community.
I think one of the most important things we'll need is a core set of developers who are able to commit significant time to the project. Typically a project like this is driven by necessity--either someone is developing code because they need it for their own research, or because someone is paying them to do it. Many of the people in this group have already invested in their own acquisition systems and will not have the resources or incentive necessary to push the project through. So perhaps an early question to resolve is whether anyone in this group has the necessary resources and incentive, or what organizations would be willing to support the project.
Thank you for the invitation to contribute. I'll try to be brief to keep focus (a lot can be said about this topic), so if my comments appear curt that's probably because of that.
Also pinging @cjermain and @cgranade who might have something to contribute.
So, from my perspective the effort to unify you mentioned has not really worked. I invite you to look through pymeasure/pymeasure#53 (warning: long thread), which is what the effort to catalogue you mention above came out of. It shows basically the course of the effort to unify several hardware interaction projects, with a lot of discussion and reasoning from a variety of contributors.
I think the whole thing fizzled out primarily because of too little developer/maintainer availability and political/technical reasons (people not wanting to fold their project into another one).
One thing that made the unification effort hard is that there are so many ways to segment the space of interest (see the online sheet in the OP of LabPy/labpy-discussion#23 for a try to make structured sense of the landscape), so it's very easy for multiple groups to have different focus (and projects), instead of collaborating on one package.
Only having quickly looked from the outside, this also seems to have happened here - you seem to want to start a package focused around microscopy, but instead of using and contributing to an existing project (e.g. https://itom.bitbucket.io/ or https://github.com/python-microscopy/python-microscopy), you initiate a separate thing. Instead of trying to contribute to another effort to unify (e.g. https://github.com/LabPy/ which you might not have found before), you start a fresh community. This will probably lead to more fragmentation in the space, even though you cite the imo extremely relevant https://xkcd.com/927/.
Please know that I am not judging you for that! There are a multitude of reasons to do your own thing (not having known about others; it's more difficult to attract funding for contributing to another project instead of building your own, slightly different focus, license preferences, commercial considerations,...), as I also found in the previous discussions, so maybe this is tragically and inherent property of the space we are operating in.
Sadly, this exacerbates the one primary problem that I see OSS project having - lack of developer resources. Therefore, we all end up with N projects with slightly different scope and each with ~ 0.M time-starved maintainers, instead of one project with a more sustainable pool of N*0.M maintainers.
This is also very relevant with contributions from academia - often projects are started by grad students, postdocs or other temporary staff - when they finish their work the project often ends up abandoned (hello, bus factor). If there is no professor/permanent staff to focus efforts, or a whole group that uses a package, or several involved institutions, it's very hard to end up with a sustainable pool of developers - critical mass is extremely important IMO.
This is also how I read the situation from LabPy/labpy-discussion#23. Personally, from a candidate pool of InstrumentKit, Lantz and Pymeasure, I have drifted towards using (and trying to contribute to) Pymeasure mainly because there is at least some activity there, so there is a chance of getting your bugs fixed.
Nowadays I have come to regard developer activity as one of the primary rubrics when evaluating if I should use a project or package.
So, what is the solution? I would love to see a Numpy for data acquisition :-)
I think, considering the tendency of potential contributors to have different foci, it could be worthwhile to evaluate what is out there, and
- find a common set of overlaps, and try to build solutions for that (domain-specific things should probably be elsewhere)
- plus some kind of plugin infrastructure to make it easy for others to adapt your solution to their needs (pytest is a good reference for that I think)
- plus making it easy for people to contribute small things back
IMO, one of these kernels is device drivers for data acquisition, i.e. something on top of VISA, IVI, etc. that implements and tests(!) lots of ready-made interfaces for the many instruments out there! GUI and procedure automation would be different things and out of scope - there, people already have wildly differing needs/wants. Everybody likes pre-made instrument drivers e.g. for the oscilloscope in their lab, though. This is still a hard problem, and lots of different way to "skin the cat" (e.g. what a common API should be)!
I think InstrumentKit is a good example for that, especially the way to test against instrument protocols, but sadly there's very little activity. Lantz is basically in the same situation.
As mentioned, I am using Pymeasure's instrument drivers features for this (although Pymeasure's focus is larger than that). I try to give back as much as possible, but sadly I am also not in a situation where I can invest much time into it - a pool of contributors/maintainers would definitely help!
Oh man, this still grew too long, sorry for the wall of text. >.<
Also, keep in mind that this is primarily coloured by my own experience, I also don't have the wisdom of the ancients. There already seems to be a number of people collected here, so your opinions might contrast with my own.
I guess my tl,dr could be:
- We couldn't pull this off before, but please use what we found and learn from us
- Gather a large amount/pool of contributors
- Identify a common need
- Build something as generic as possible
- Make sure it's extensible (e.g. via plugins)
- Build specific things on top/elsewhere
Many thanks for your comments, @bilderbuchi !
One note I want to throw in: although we have already started some work on a new project, I consider this just research in that it is intended to help us decide whether we can agree on the kind of API we need, and from there whether any of the existing systems are adequate, or can be adapted to our needs, or would be willing to deprecate their own code in favor of another.
Sure, I hope it helps!
I think in the frame of device drivers, my personally ideal package would have
- a driver definition style like pymeasure,
- unitful support like InstrumentKit or Lantz,
- a test harness like InstrumentKit (❤️),
- auto-GUI construction like Lantz (or Pymeasure),
- concurrency features and querying all properties like Lantz.
Ideally also then the
- unified users and
- driver collection (maybe wrapped or autoconverted, but that's probably a metaprogramming nightmare)
from all three to get nearer to critical mass. :dreams:
Of course, the resources or funding for such an endeavour are just not there, we can't even keep up with PR review for Pymeasure in our free time. :-/ At work, barring some funding (EU/research grant), actual research tasks take precedence.
Also, it's probable that the next person over will have different ideas or priorities, and that's when things start drifting apart or into discussions, again.
I think what is important for attracting a sizeable crowd is that what you are rallying people around is generic and not focused on a specific use case. For example, if the direction goes into "control for microscopy", or Quantum computing something, this puts me on a sceptical footing concerning the usefulness for my usecase (I am mostly controlling vacuum chambers and high-voltage power supplies), and I think other people will react similarly. If I think about the larger, well-established projects in this space (numpy, PyVISA, pyserial, matplotlib/pyqtgraph,...), they offer building blocks, not specialist tools.
I think making building blocks for "every"one, and building more specialised tooling on top, would be the more promising approach.
In any case, I recommend sitting down in a comfy chair, making a glass of your favourite concoction, and read through the long but hopefully illuminating pymeasure/pymeasure#53, I feel we have covered similar ground over there.
@bilderbuchi, I think you summarized the situation very well, and definitely captured my feeling on this. That issue is a great display of multiple failed attempts to consolidate the space. :)
@bilderbuchi very interesting read on pymeasure#53, especially considering the range of timestamps traversed by that discussion!
I think that there is a natural tension between a specialist group (could be single molecule microscopists, or slice electrophysiologists or whatever) trying to do this for their use cases and the desire you expressed to
Build something as generic as possible
And then the impedance mismatch between project scope and contributor hours is probably easiest adjusted by starting small and specialized and building up, and that path of least resistance has led to the hordes (?) of dead or zombie projects in this space staggering around github.
I still think that an organic approach based on specialist projects wanting to remove driver dependencies from their projects might have the right combination of risk-reward to gain community buy-in incrementally. It sounds like I'm not the first person to naively hope for this...
One more thing Iearned from heading down the amazing array of python hardware control rabbit holes (some from your comments I had no idea about!): There are wildly disparate use-cases represented here and wildly diverse sets of hardware represented in these labs. I think our initial ideas and discussions implicitly included only a tiny fraction of this diversity. Given the resource problems you spelled out above, it seems like carving out the scope of this project (like a list of hardware devices that we're considering and not considering) is critical to making it manageable to even have a discussion.
@berl thank you - basically, 👍 on what you said!
I agree that unifying the driver situation could be most promising approach. Drivers are where starting to use Python needs additional effort compared to, e.g. Labview, where drivers are often already implemented by the manufacturer. In addition, I feel like drivers are still a low-level enough unifying component to concern most users irrespective of background/application, so it should be easier to get critical mass.
I think the complex thing here would be to define a generic but extensible API and usage patterns fitting as many device types as possible, so that everybody wanting to contribute a device driver can find their slot. IVI instrument classes are one way to approach this/learn from, but I am not sure how well this translates to Python.
Then, people can put other stuff on top (classical GUI, notebook interfaces, multiprocessing|threading|concurrency, distributed solutions, scripting/automation)
The application-specific stuff (things for analysing black holes, microscopy, controlling lathes, or whatnot) can go on top, and everybody can profit from what is below.
But yeah, that's probably a hard/large undertaking...
Hey everyone,
I'm actually still working on InstrumentKit, but I no longer work at a job that involves interfacing with hardware. I work on IK as I have time, and some of my bigger ideas can take long enough that people might not notice progress is being made. Such as moving to Pint, implementing a pluggy based plug-in structure, etc. It also doesn't help that I don't get Github notifications unless I manually check.
As to why I personally dropped off any discussion on combining efforts, its pretty simple; I just don't have the capacity to carry these sorts of conversations. My day job is to lead a test automation effort (in Python even) and so nearly all of my development energy is focused on that. When I program for fun, I'd rather work on InstrumentKit at my own pace and deal with the few ideas people bring me, instead of having to manage yet more developers. Many people have requested an interface layer for IK to use pymeasure's drivers, but no one has submitted anything at all. And without any tests on the drivers, well, what am I to do 🤷♂️
And to be honest, I didn't see the point me moving to another project that is missing core features that originally spurred the creation of InstrumentKit.
Also relevant I think, if you are not aware: https://github.com/QCoDeS/Qcodes
@scasagrande thank you for your thoughts! I am not sure if I read between the lines too much, but I want you to know that my remarks regarding IK were not made out of a sense of entitlement to your development/discussion effort. I understand the situation, and it's of course fine to work on this when you want. For me the thinking was mainly "if I run into problems, will I be able to reach somebody to help me fix it?", and for that very little repo activity was a bit of a warning flag.
From my point of view, it is also important to keep the code pythonic, readable and as short as possible. (Which was already noted by LabPy/labpy-discussion#23 (comment))
I am one of the experimenters who are OK with reading driver source code to get things working, but expect it to be clear like import xxx, __init__, move() or get_voltage(). If it fits on a single screen, good.
The reality is truly frustrating. Many of existing projects need some fixing (which is usually not documented), and more than once I wasted a whole day analyzing some interface's logic, only to find out the project does not address my needs.
Further, for mass adoption, the project should keep mandatory dependencies at absolute minimum. This means module detection should be dynamic; if e.g. pyvisa is not installed in the computer, the module would only skip detection of corresponding devices, printing a warning about it.
Also this stipulates that the graphical frontend should not require
- any particular plotting library (which actually presents a similar redundancy trouble, see https://wiki.python.org/moin/NumericAndScientific/Plotting),
- any particular widget toolkit excepting TkInter (which is included with every python installation).
But a simple TkInter GUI would be very useful, if it would
- show all supported device classes in some ListBox, try to autodetect the presence of such instruments, and if found, show their unique serial number(s), or
not foundor, e.g.,module visa missing - serve as simple boilerplate application for individual projects,
- possibly allow some rudimentary lab control from scratch (i.e. when an instrument is selected in the ListBox, use code introspection to auto-generate buttons like [Identify], [Move], [Measure] etc.)
It's been a long time! The pandemic certainly rearranged a lot of priorities for me..
There's the beginning of an interesting discussion here that I encourage y'all to contribute to: https://github.com/micro-manager/futureMMCore
@campagnola, thanks for the link to another relevant project, I added it to the table: https://ethercalc.net/1anmq248ktu6
It is amazing how much duplicated human effort could have been saved if there was one central repository for instrumentation...
Added https://github.com/AlexShkarin/pyLabLib to theh table: https://ethercalc.net/1anmq248ktu6 This seems like one of the more elaborated projects here.
@roesel I noticed your issue over at the pyLabLib and wanted to draw your attention to this thread- some of the projects linked above may be interesting to you
@berl Thanks for the tag! Apart from the list of projects, there is definitely an interesting discussion going on here (and even with some familiar names, hi @FilipDominec). Although judging from the list, it looks like pyLabLib is still closest to what I am looking for right now.
From briefly scanning through this thread and a few linked ones, I would say there should also be a clear separation between libraries (that you can import when writing your own python code) and programs (which you can immediately use). It's fairly straightforward to make libraries modular and reusable, but when it's a finished program with a singular purpose, it is not only harder to extend and reuse, but also requires making more decisions about dependencies such as a GUI framework.
I think we as a community still have a visibility problem. By now, every time someone starts a new, greenfield project (like pylablib, which is not yet a year old) instead of contributing/pooling resources/adapting an existing one, I get a heavy sense of https://xkcd.com/927/. 😞
Obviously it's still not easy enough to get an overview over the available tools/capabilities out there, or we developers don't cultivate an inviting enough environment to encourage people to stand on our shoulders, our tools are still not generic enough, or something else. I wonder how we could change that.
Over at PyMeasure, it's more of a maintainer manpower problem -- we don't manage to work down all the incoming PRs in a ~reasonable amount of time. It's getting better, though, slowly.
@bilderbuchi I totally get what you are saying, but I think there is a bit more going on than someone wanting to make "yet another standard". Speaking as an employee tasked with automating a setup in a lab, which contains everything from servo motors to photon-counting cameras with their own proprietary software, I would like to be able to confidently say "I'll do it all in Python" rather than grudgingly choose to do it in LabView, which is how it's usually done around me.
The LabView approach also doesn't come with a "one-for-all" solution from one source, but somehow all the manufacturers provide a VI that I can glue together with the rest and get something working fast. Maybe what I would actually like is not one swiss-knife library that does it all, but simply an up-to-date list, where I can find each one of my instruments and see which libraries have implemented their control. That seems like a much more useful list (for my use case at least!) than a list of projects with descriptions of which 1-5% of areas each project is able to cover.
That's not to say it wouldn't be great if all this code eventually merged into one library that would support a wide range of devices! I don't have experience with PyMeasure (I wonder if it existed already when I was writing my own code to control a Keithley 237 six years ago?), but despite it's age, pyLabLib already looks to cover a very wide span of device types, while letting you use code for any single device without having to even know about the existence of the rest. That looks like a very promising approach to me!
If I need to move a Thorlabs stage, it's great that there are multiple other "open source microscope control" packages that have the ability, but if I cannot find the import library; stage = library.Stage(); stage.move(10) snippet in their docs with a google search, it might as well not exist for me, if you know what I mean. I think it is clear PyMeasure is not one of those packages, but rather another excellent example of how it should be done. What do you think?
Previously, I was in a similar boat as you are, although with a different (simpler?) set of instruments (mostly SCPI-compatible, few modbus-based, some dll-based). I pulled it off, so don't lose hope! :-)
somehow all the manufacturers provide a VI that I can glue together with the rest and get something working fast.
Yeah, I fear the network effect is strong here. That definitely puts any Python-only (or rather "non-Labview"/"non-institutional") effort at a disadvantage.
(I wonder if it existed already when I was writing my own code to control a Keithley 237 six years ago?)
Possibly, but only barely; first commit was in April 2014.
pyLabLib already looks to cover a very wide span of device types, while letting you use code for any single device without having to even know about the existence of the rest. That looks like a very promising approach to me!
Maybe I don't understand, are there libraries that necessitate you to "know" about other devices when using one specific model? Or what are you trying to say?
simply an up-to-date list, where I can find each one of my instruments and see which libraries have implemented their control. That seems like a much more useful list (for my use case at least!) than a list of projects with descriptions of which 1-5% of areas each project is able to cover.
I agree, this would be quite useful (if kept up-to-date automatically), and would definitely be an improvement!
The above-cited spreadsheet was a first effort to make sense of the quite diverse landscape.
In the end, we could use something like https://pyviz.org/overviews/index.html, just for controlling hardware.
Also, I don't necessarily mean or want to unify everything in one package, just to pool scarce developer resources, and (ideally) not add new ones where existing ones could be adapted. ☮️
Maybe the take-away has to be that any abstraction beyond PyVISA/pyserial/nidaqmx/minimalmodbus is ultimately doomed because people's needs and interests are too diverse? It's already visible e.g. in the distinction between miscroscopy folks and others, and in the difference in character/scope between something like EPICS/Taurus and simpler packages like ours.
Code heritage/legacy/back-compatibility is also an issue, of course. Most packages are quite different in architecture, so are hard/impossible to merge even if interest/drive were there.
Decisions from the deep past sometimes restrict what you can currently do, and how generic your package can become. I surely see this in pymeasure, too. In that vein, younger projects have an advantage -- they can study the available solutions, learn from their mistakes, and build it up all over again, better (ofc, at a large time investment).
PyMeasure is not one of those packages
We're getting there (slowly) -- we just merged a contribution with our first (I think) stepper motor driver. 🥳 We have few Thorlabs instruments, and that is not gonna change for quite some time I fear -- by necessity we have to restrict our scope to non-dll drivers for now.
Maybe I don't understand, are there libraries that necessitate you to "know" about other devices when using one specific model? Or what are you trying to say?
I don't have specific links, but I remember coming across several different projects last year that tried to implement a program for controlling a microscope, which had support for (among other things) various stages internally, but unless you were comfortable with diving into the source code and extracting the implementation yourself (which can cost a lot of time in some cases), there was no easy way to just move a stage without spinning up their full GUI or setting up objects that would never be used. Maybe it was even possible, but since the docs did not talk about it, I imagine a person looking for a solution would simply move on rather than invest their time towards an unknown outcome.
PyMeasure is not one of those packages
Please don't misunderstand, I was trying to say that PyMeasure looks like the opposite of what I am describing above! It seems to be an exemplary library that you can use for your device without the need to commit to any particular model or ecosystem.
Please don't misunderstand
Not at all! I took this to refer to "moving a Thorlabs stage", which we can't help with at this point, due to lack of drivers. All good 😃
Just piling up in here, if I understand correctly @roesel is looking for some simple implementations on which to build (like getting a simple UI for a stage, or a simple UI for a camera, etc.) This allows you to extract and glue things together. Pretty much in the same way you can use VI's that companies provide for LabView. I think this is perhaps something missing, where very large projects don't allow you to extract just the bits you need, and many projects focus on being very generalistic (from developing drivers all the way to visualizing data in real time). How can we build and, more importantly, share those building blocks is another type of discussion, but that can be very valuable to have at some point.
In between the challenges, the most pressing one is the lack of interest from companies in releasing boilerplate code that just works. The other, is that the use cases of hardware vary so much: the same camera used for machine vision in production lines of a pharmaceutical is the one I use for a low cost microscope. It makes it hard to settle on the major guidelines. Not all communities favor Python over C#, or LabView, for example. And here is where the interest of companies can't be steered unless the use case is large enough (I've even seen a company that sells analysis software for microscope images forcing users into TCL because if they needed to learn to script, choosing one or another language does not provide any value).
@aquilesC That's exactly what I was trying to express! The other thing buried in my thoughts above is that these building blocks should be GUI agnostic, or even better provide a GUI which is decoupled from the business logic and easily usable on its own. Where one person has a Qt interface, another uses GTK, a third one needs to write a plugin to augment an existing C# program, and a fourth person simply wants to run a one-time script. That's another point where Python loses to LabView, since all the VIs from all manufacturers already share both the language and the interface, and are easy to combine.
To comment on your second point, one thing we can do is to ask the manufacturer for a Python implementation when setting up new equipment. While I have myself rarely actually received one, I hope it has at least raised the awareness that this is something people are interested in. Do you think there is another way?
@roesel the original repo here was made as initial research into some of the key issues you have brought up. The name python-data-acquisition was clearly a little too general, since we're focused on microscopy and neuroscience applications, but we identified some needs shared across a broad range of hardware + python developers:
- separate out drivers/libraries from GUI and acquisition/experiment coordination code
- develop minimal base classes for devices and provide a sane structure for them. I think acq4 does this well.
- make sure people can import The Thing They Want without getting a whole bunch of extra stuff.
important context/history here is the micromanager project, which through pymmcore or pycromanager can provide access to hardware that can be otherwise difficult to control because of proprietary drivers, NDAs, etc. The C drivers in micromanager represent a huge value that has been built up over years of working with individual contributors and manufacturers, and to some extent pymmcore fits the bill of being able to import and do stuff, but because its wrapping C drivers, it doesn't have the benefit of broader community contributions to the driver layer itself. Also, you do get a lot of acquisition control code coming along for the ride, which is good or bad depending on your perspective.
I think IVI could be something of a labview Alternative, with device vendors sometimes providing drivers, and with a generic driver structure in place, but I don't know it's current status.
Maybe python-ivi, despite some of the existing technical shortcomings, could be revived to hook into/wrap IVI drivers without having to implement/define instrument APIs in detail.
@bilderbuchi You are right we should now collect a universal list of all supported instruments. The structure of all cca. 38 projects in the overview table seems different, so they cannot be scraped automatically. The list is going to contain hundreds of items, so a directory of YAML files may initially be easier to make than a huge online table.
It should be noted that it's not all about supported hardware - some projects implement clever hardware autodetection, measurement control & visualisation etc. So there is also stuff that cannot be easily put into a table.
@roesel David, please don't forget about TkInter - it is distributed by default with Cpython (all versions, on all platforms), so it takes no extra space and loads immediately. It integrates well with Matplotlib, and also with direct numpy.array drawing. Also Tkinter+TTK is not as ugly visually minimalistic as it used to be, and programming a basic GUI is a matter of few readable commands.
In an ideal world authors of all the competing projects would recognize that it would be beneficial to humankind to share one wisely selected repository of instrument drivers, and would gradually port their labs to it. World is not ideal⁽ᶜⁱᵗᵃᵗⁱᵒⁿ ⁿᵉᵉᵈᵉᵈ⁾, there is nothing like a selected repository, but still we could ask them kindly for:
- a check that the list of supported instruments describes their project's capabilities correctly,
- adding a link to this list on their project's website so that newcomers can choose,
- writing some short update on their vision of their project's future, its level of maturity and whether they intended to make it work outside their own lab.
It is going to be human-resource sensitive, as it challenges the appreciable effort people put in their own laboratory control software. I know the feeling from my own experience (e.g.). But as soon as the list of instruments is sketched up, we should openly communicate across the projects. We are facing a tough problem here, an it would grow if it stayed neglected.
would be beneficial to humankind to share one wisely selected repository of instrument drivers, and would gradually port their labs to it. World is not ideal⁽ᶜⁱᵗᵃᵗⁱᵒⁿ ⁿᵉᵉᵈᵉᵈ⁾
🤣 congratulations, you are where I was when I started assembling that table years ago 😁
so they cannot be scraped automatically.
I suspect what should be feasible would be to create a couple of lookup rules that scrape instrument names from the repositories of the projects (or maybe any html docs). For example, for pymeasure, we collect all instruments in one folder, with one subfolder per manufacturer, with one python module per device model. I assume most projects will have a structure that is similarly simple to express, and this has to be done only once.
I think it's important to have automation in place to harvest that information, otherwise this places quite the maintenance burden on whomever volunteers to maintain that (i.e., it will not be updated).
I would have started something long ago, but I'm very much not a web person, and pymeasure maintenance eats all my spare time budget for this already.
I expect (it's not my call alone) that I can offer a space/repo for this in the pymeasure org org if you want. We might want to base this info page off a Github repo for easy publishing/updating/CI machinery.
please don't forget about TkInter
@FilipDominec I have nothing against TkInter, but I still believe all projects should strive for the library/program separation I tried to describe above. It's great if the author writes a GUI, and TkInter is definitely the most universal approach, but having business logic decoupled from the interface makes adoption much easier no matter the circumstances.
In an ideal world authors of all the competing projects would recognize that it would be beneficial to humankind to share one wisely selected repository of instrument drivers, and would gradually port their labs to it.
This might just be my personal opinion, but I believe most scientists want to get the job done, hence the widespread use of LabView despite its shortcomings. The start a new project step only comes when no easy-to-use implementation is to be found, in most cases anyway. It seems there really is much more fragmentation than there is competition, so perhaps once the existing implementations area simpler to find, all this could get easier and easier?
A table update: Added https://github.com/caizikun/Instruments-1 to the big list https://ethercalc.net/1anmq248ktu6, reaching total 43 projects of mostly overlapping scope.
Hm, did you notice the big comment in that sheet that says
THIS LIST IS DEPRECATED/NOT UP-TO-DATE, PLEASE SEE https://github.com/pymeasure/python-lab-automation-landscape INSTEAD
I added that a while ago when I had completed the first iteration of https://github.com/pymeasure/python-lab-automation-landscape, so that we have a saner (and citeable) way of tracking these than a file at some file sharing service that might go away at any time.
I guess it needs to be even more prominent? I'll see if I can add a yellow background. 😁
Did you check out the link? There is a readme, but I guess I can make the part about how to contribute more prominent. (you basically have to add one very short yaml file).
I hope your project the table refers to can somehow process the new entries.
No, I migrated all the previous ones by hand/via some editor search/replace. I'm not set up to process new entries, but I did not want to delete the contents of the Ethercalc sheet totally. Maybe i should, to avoid this going out of sync.