Sources incomplete
Opened this issue · 20 comments
I want to fix FreeDos Keyboard Layouts for Czech/Slovak Keyboards which is defined incorrectly.
The sources here on GitHub are incomplete. Following is missing:
- Key definition compiler KC and KLIB for KEYBOARD.SYS creation from layout definition files
- Keyboard layout definitions for KEYBOARD.SYS creation
- build scripts/makefiles/batch files
The necessary sources are fragmented between FreeDos 1.1 CD (some missing in newer FD version), GitHub and GitLab.
What is proper location for them?
I will try to complete source code here, but I am not sure if it is right position.
I would like to migrate KEYB from Pascal to C or ASM language, but I don't know if in-line assembly code is it preferred method or should be separated from high-language source to separate ASM source file?
I suppose prefered C compiler is Open Watcom and prefered Assembler is NASM.
What is prefered build platform?
GitHub is the ones I support, but it is only base programs. Gitlab is the home of fd-nls which is meant to be the primary site for translations. Base programs here may only be a mirror if it is still being maintained by someone and they have their own hosted site. Keyb is maintained by Aitor. I will send him an email. Note, the GitHub fdos is derived from when I assembled the official FreeDOS distribution, with this being where I kept updates and work on help maintaining programs. FreeDOS Distribution Of Sorts - FDOS. Originally on source forge and a decade or so switched to GitHub. I don't do distribution in a long while, the gitlab site seems to be along the same idea, a single place to keep updates and maintenance of non base programs. I wish base programs were not duplicated there to avoid confusion, but in the spirit of open source, the more the merrier. I personally try to work upstream maintainer and mirror to GitHub, unfortunately there really aren't many around any more. I do not believe Aitor has any public repository for keyb, just source archives published; though I could be mistaken. I will verify with him and work on getting missing sources.
Not specifically about keyb since it is in pascal, open watcom and ia16-gcc are the preferred c compilers. Nasm is the preferred assembler for cross platform compabililty (and source license), but if a masm like syntax is preferred then jwasm or uasm can be used. For ci and others, Linux seems to be preferred build platform, I use windows and DOS but if limited prefer Linux so can have automated builds and testing.
Thanks for your info.
For newcomers it is now very confusing, especialy KEYB which required more components for creating run-time files.
I don't need to change anything in KEYB but I need to change layout definition files and recompile them to *.SYS files to check it.
If anybody want to do similar change in Keyboard layout it requires to collect all tools spread over world and create some script for recompilation run-time definition files.
Maybe OK if I will add all these tools to KEYB repository even if they need not to be distributed, it is development/customization tools.
I submit change for default Czech Keyboard to be Czech standard and small change in AltGr pane to assign < and > keys as is more current (Windows and Linux).
It is on GitLab as pull-requests
https://gitlab.com/FDOS/base/keyb_lay/-/merge_requests
It change KEYBOARD:SYS and appropriate source files.
Hi AItor,
Thanks for all your info.
I collected all necessary sources and binaries already.
I ported KLIB to OpenWatcom and get list of layouts in KEYBOARD.SYS.
I did changes in layout definition and recompile new KEYBOARD.SYS with new layout definition.
All looks OK and new layout works OK.
It looks like GitLab repository is more complete.
KEYB - binary and sources
KEYB_LAY - binary and sources
only KC and KLIB is missing that it should be put there to be complete and anybody could do changes in any layout.
I agree KEYB core (resident part) must be as small as possible that it must be in assembly code. Migration fromTASM to WASM or MASM could not be a big problem all use intel syntax only directives/macros are little different.
Jiri
Firstly I should say that I'm a fairly recent contributor to FreeDOS so I might not get the proper demarcation and responsibilities within the community. That said I don't understand why the recent Gitlab copies of the repos hosted here actually exist? If they are provided to mirror those at Github all well and good, but accepting changes rather than marking themselves as purely a backup and forwarding the contributor to github/FDOS seems counterproductive. This only means that at some point somebody has to work out what changes should be propagated to a release. There's nothing new there that couldn't have been achieved by a PR to github/FDOS. @shidel, @PerditionC I know this is coming over as a rant, that's not my intention, but is there no way you can share the maintenance of a single site?
@andrewbird I feel the same. As new contributor I have a problem to identify where release code is located and where to put PR.
Anyway if anybody try to contribute it is as wall between FreeDOS and contributor that contributor probably stop any work on FreeDOS. I supposed that FreeDOS has one repository where all necessary parts are placed. But it looks like only kernel is somehow maintained in central repository (GitHub) and rest of FreeDOS is fragmented over world. Also it is not clear for me what is part of FreeDOS distribution (as OS) and what is applications. One simple change take me a few days and I am not sure if I put it on right place to be passed to dietribution. I am thinking I abandone any other work on FD, it is wasting of my time.
The FreeDOS Orphanage serves multiple purposes. Here are the primary reasons for it.
First, Every single package that "ships" with the FreeDOS release is there. However, only those projects that are believed not to have an official home elsewhere are visible to the non-project members. This provides a single location to assemble packages the Official FreeDOS Repository. For example, both FreeCOM and the Kernel are there. But, neither of those is visible to the general public because they are known to be actively maintained elsewhere.
However, I've been considering changing that to make all projects there visible. Then for the ones still known to be actively maintained, place a note in the topmost level of that project to point any potential contributors to the official site for that project.
Second, All packages in the Orphanage are kept in a directory structure that is ready to "zip-and-ship" as a FreeDOS package. Some programs require little or no work. While others require major restructuring or hours of gathering the proper files to build it from source. Doing that is not fun, easy or quick. The FreeDOS 1.1 release was roughly 40MB. However, what gets included has grown significantly since then. With FreeDOS 1.2, it was about 418MB. Now with 1.3, it has expanded even more resulting in many things being removed from the main release media and placed into an over 600MB BonusCD. So, keeping all of those in a ready to go format is critical to the next reason.
Third, packages that go onto the Official Repo (not the mirrors), go through the Orphanage in preparation for upload to the server. While not all of the packages in the official repo end up on the release media, that is where they come from. Other than a few things that get auto-generated, no packages that are not in that repo get included in a release. The release media itself is generated using the RBE (release build environment) and has been completely automated to reduce the tedium and complexity of putting together a release. The RBE pulls down the latest packages from the repo, incorporates language translations, processes and updates the installer, generates the release media and several other things.
A side benefit of GitLab is that it has an extremely flexible permissions system. Wether or not any particular project is visible to the public, if an actual maintainer exists, they can request or be given direct access to just that particular project. I have no problem giving them direct access to their projects. At present, there are only a couple of developers who have requested direct access to their projects there.
@shidel I understand it is not simple to handle overall FreeDOS distribution due to lot of different packages.
But for me as potential contributor it is absolutely unclear where all components of FreeDOS are, list of them and how to contribute to them.
If you are prefering GitLab for distribution creation then all additional application can be there. But FreeDOS components (as full OS) should be on one place to simplify contribution. It is not problem replicate or compile FreeDOS on GitHub and pass binary on GitLab for creation FreeDos distribution, or any other place.
I am maintaining OpenWatcom project on GitHub which is also complex, but I can't imagine doing it by hand.
Most of things is automated by GitHub Actions, CI (Continuous Integration) and Azure Pipelines as example replication of all changes to mirror git repositories that only one source repository is used, compilation of documentation to be available on WEB pages, full OW distribution build one per day if some change happened, CI after each change etc.
Simply no manual intervention is necessary to create new Release on GitHub for users. If anybody submit some change then next day you get new release automatically for testing.
Anyway It is not clear for me what all should be on GitHub and what should be on GitLab.
Please correct me if I am wrong.
GitHub should hold FreeDOS as OS only and GitLab should hold all things for distribution.
It means binary distribution of FreeDOS and all additional package should be on GitLab and FreeDOS sources should be on GitHub.
Next unclear thing is what components are FreeDOS OS.
@shidel it seems you have a well thought out process to packaging and of course your translation collation efforts are excellent. I do think though that packaging and development are two distinct activities. The issue I see is that I, amongst others, have been contributing to the various Base sources on Github's FDOS, but many of those (not including Kernel and FreeCOM) are also hosted at GitLab's FDOS/Base. So since they are in the Orphanage are they not considered to be maintained at Github FDOS?
@jmalak I am also an advocate of CI, I added simple Github Actions to make each commit in the Kernel and FreeCOM get built and zipped into a snapshot, see https://github.com/FDOS/kernel/actions/runs/1290778325. So whilst it's not a full release, and I'm not sure that would ever be possible due to the various proprietary compilers we'd need to use, at least people can test their / others' changes without having to setup a build environment.
But FreeDOS components (as full OS) should be on one place to simplify contribution.
Unfortunately, that will probably never happen. It is kinda like saying all of Linux OS should be in one place. Like Linux, FreeDOS is basically just the kernel. Then you have FreeDOS BASE, which is a about 55 packages made up from numerous different developers to replicate the functionality of MS-DOS plus a couple improvements. Some of those programs are still actively maintained on the web somewhere or through other means. While other programs in the BASE have long since been abandoned by their original or later developers. The you have FreeDOS FULL, containing megabytes of additional packages in similar states to those in BASE. And of course there are the packages provided as a value add on the BonusCD. And finally, don't forget about the programs that exist on repository and don't ever get included on any release media.
Each package contains a LSM meta data file. In that file are links to the project website that is maintaining it. If no such place exists, they end up pointing to either the Online Repo or the Orphanage.
It is not problem replicate or compile FreeDOS on GitHub and pass binary on GitLab for creation FreeDos distribution, or any other place.
I've got no problem with @PerditionC managing as many packages as he wants. Either on Github or in the Orphanage. It is completely up to him. I can easily grant him access there. Or, if he wishes to do it on GitHub, I can "hide" them on the Orphanage or put a obvious note pointing would-be contributors to the GitHub versions. I'm completely fine with however Kenneth wants to deal with those packages.
I'm already way over-committed. So, the less I need to do--the better.
Anyway It is not clear for me what all should be on GitHub and what should be on GitLab.
Please correct me if I am wrong.
GitHub should hold FreeDOS as OS only and GitLab should hold all things for distribution.
It means binary distribution of FreeDOS and all additional package should be on GitLab and FreeDOS sources should be on GitHub.
They only place that ALL FreeDOS package sources exist. That one place is the Orphanage. It was a great deal of work getting several hundred different projects up there and with the proper directory hierarchy. In my opinion, packages without an official home elsewhere should probably be maintained there. However, for packages maintained elsewhere, I think I am going to make ALL projects there visible and include a note to point potential contributors to the current website for such submissions.
Next unclear thing is what components are FreeDOS OS.
Like I said earlier, it depends on what you mean by FreeDOS.
@shidel it seems you have a well thought out process to packaging and of course your translation collation efforts are excellent. I do think though that packaging and development are two distinct activities. The issue I see is that I, amongst others, have been contributing to the various Base sources on Github's FDOS, but many of those (not including Kernel and FreeCOM) are also hosted at GitLab's FDOS/Base. So since they are in the Orphanage are they not considered to be maintained at Github FDOS?
As a general rule... At present, if a project is visible in the Orphanage, it has been abandoned by its most recent maintainer or at least as far as I could tell, is no longer actively maintained elsewhere. For example, there are some programs whose latest maintainer still hosts them on their website. However, they no longer update those programs. So, after a discussion with them regarding their work, I made those projects visible on the Orphanage and granted them a controlling level of access over those projects on the Orphanage. Allowing those projects to continue being updated, yet letting the original maintainer keep an eye on them.
When it comes to the packages in Github FDOS, most don't have official homes anywhere. Yet some, like edlin are still maintained. I really am fine with however @PerditionC wants to handle the GitHub/GitLab issue. I could probably even "do lunch" with him sometime to discuss it. Unlike the rest of the FreeDOS community which is spread out across the globe, I think he only lives a couple miles from me.
Hello there!
The idea behind the division is that 90% of the guys may only need the binaries of KEYB and the binaries of the layouts as separate.
Most of the folks won't ever use KC/KLIB or modify layouts (developers will), so I see no point in adding them to KEYB and increasing the size for most of the users that will download it.
Thanks for porting KC/KLIB to OpenWatcom, I will do that too in the future, but plan to add them (jointly with other small tools), to a PowerToys kind-of package, that developers may download and enjoy.
Aitor
With the stated above, I consider this NOT a bug, but just a planned feature.
It should be somehow one development package with all necessary sources and second binary distribution which could be automated build from development package. But it requires two well defined repositories one for development and second distribution where binary or development package is prepared for users (in package form).
I am only random contributor and for me it is hard to do some small contribution, because it take me lot of time to get appropriate package for development that I will abandone FDOS and install MS-DOS 6.22 because I lost a days for a fix and any such similar small change is a few days again. Change in kernel is much simpler because it is compact repository.
Main FreeDOS repository has moved to GitLab ( https://gitlab.com/FreeDOS/base/keyb/-/issues ).
Please, if still interested, open it there as Feature request.
If that's the case then the project owner should ask @PerditionC to remove this repo here to avoid confusion.