Should performances be in their own (sub)project on Github?
diyelectromusic opened this issue · 26 comments
It's great to see the performances continually evolving, but having so many files changing so much does make managing branches with code in development quite challenging.
Is it worth creating another repository for them so they can be maintained independently? Perhaps it would be included in the main repository as a submodule or maybe they should be pulled into the build when images are created?
I don't really know enough about how Github works to know what is the best solution, but having so many files changing in a way that doesn't affect the code but seems to really confuse the merging/syncing process does seem to add quite a bit of overhead to anyone updating code. Although, I'd be the first to admit I'm perhaps not using Github in the most optimal way!
Kevin
@diyelectromusic
You have my vote
My thoughts so far were that I would put together 128 “factory performances” and then they would permanently belong to the project. And if there is an opportunity to work with multiple performance folders or performance banks, I wanted to create additional topic-related performance banks.
I created a performance update as a new repository. I hope it makes your work easier.
https://github.com/Banana71/Soundplantage
I think @probonopd should link this into his miniDexed project as submodule. !?
My conversions from DX/TX-series will go here:
https://github.com/BobanSpasic/MDX_Vault
wow, that's nice. @BobanSpasic
Do you want to include the TX816 Performances? Then all Yamaha Performances are in one place.
https://github.com/probonopd/MiniDexed/files/10717692/Performance.TX816.Factory.zip
Thanks @Banana71 , I'll include TX816 too.
I still have a loooot of banks to translate. This commit today is just the beginning.
I've got the memory dumps of DX7II cartridges like Decay, Percussion, Plucked etc.
I created a performance update as a new repository. I hope it makes your work easier.
https://github.com/Banana71/Soundplantage
Thanks @Banana71. Indeed, should make my work easier. So I take it that the "performance" directory contains the MiniDexed "factory" performances. If I could have one wish for those, all of them should support aftertouch.
What is supposed to happen with the B-Side directory?
My conversions from DX/TX-series will go here:
https://github.com/BobanSpasic/MDX_Vault
Thanks @BobanSpasic, very helpful! For the conversions, wouldn't it be even cooler if they were generated on-the-fly by your conversion software, as we had started to do?
With this, it looks like it is time to think about supporting more than one directory (folder) full of performances?
My conversions from DX/TX-series will go here:
https://github.com/BobanSpasic/MDX_VaultThanks @BobanSpasic, very helpful! For the conversions, wouldn't it be even cooler if they were generated on-the-fly by your conversion software, as we had started to do?
Can you make one example script for me, that will work on the files from the directory Test_files from the same repository?
I am fighting with conversion from SoundDiver 3 files to SysEx at the moment, as all the files I've got (and that are worth of converting) are in Sound Diver 3 format.
Can you make one example script for me
Check this:
BobanSpasic/MDX_PerfConv#9
I am fighting with conversion from SoundDiver 3 files to SysEx at the moment
You probably already have found this:
https://gist.github.com/arteme/76f59209db3fe263c82ee151e35a1723
MIDI-OX is said to be able to open SoundDiver lib files and save them as syx, although I have not tried it.
I've been chewing over some possibilities, and the simplest I think would be a system where:
- Performance number is encoded in the filename, so that when PC messages are selecting performances the 1..128 number that is user visible matches the file. If there is a gap, then that gap exists in the MIDI table of performances too. At the moment, this isn't the case, so things are quite confusing selecting performances over MIDI! There are numbers right now, but they are only used for ordering, not for linking performances to actual visible or MIDI performance numbers.
- Actual performances can go from 1 to max (currently 256) but the number of performances per bank is fixed (64? 128?). Banks are only used over MIDI not in the UI in order to allow selection of more than 128 performances via MIDI.
- We keep the current text file structure for simplicity.
- We keep PerformanceSelectChannel as the "system" MIDI channel for use when selecting performances.
Note: We need a maximum number of supported performances as structures in MiniDexed tend to be preallocated for the most part - in this case the record of the filenames.
Some choices or variations that increase the complexity but might still be worth considering:
- Should bank number be encoded in the filename too? Note: there are a few places where the filename structure is assumed and hardcoded...
- Subdirectories could be added, but they'd probably just be a convenience with no real meaning in the system, so I'm not sure its really worth the effort unless there was a huge speed increase in loading/etc (as there were for voices). Bank numbers could potentially be taken from the directory structure tough - e.g. have directories 00_bank1, 01_secondbank, 02_anotherbank, 03_lastbank etc?
- We could move to a Sysex performance function (via MIDI and files) maybe as described here: #533.
General Issues:
- Performances are saved with a "create next new slot" type method which doesn't work if performances have numbered slots, so that will need looking at. Ditto with Delete and maybe Save too.
- We'd need the uimenu to skip blank/unused performance slots.
If subdirectories are introduced:
- At present we only retain a list of performance file names and store them in m_nPerformanceFileName[]. This would need to expand to include the subdirectory too if we want them, but assuming ID handling stays the same (i.e. a single index/array of all performances) then that is probably all that has to change here.
- Note: there are access methods for the performance name in both CMiniDexed and CPerformanceConfig which are used for the menus, but these aren't designed to know about subdirectories (they return the name not the filename) - presumably that will be ok... Having said that, the version in performance.cpp will need to be able to extract the name from the stored subdir/filename.
- There are also access methods for the performance filename in both too - but I don't think these are used anywhere?
If we switch to a sysex model:
- Well, lets just say that is pretty much a re-implementation of the whole setup at this point, so probably worth just starting from scratch!
So yes, the simplest is as per the first set of points with no subdirectories or sysex, but just performance numbers in filenames and banks only being meaningful via MIDI.
Kevin
Yes, I think we should map file names to MIDI numbers for performances (and voices).
How about folders being numbered and being mapped to MIDI banks in the same way?
Doing everything via sysex would be nice, but it'd probably be a huge piece of work? And if we do it, I think we should at least try to implement one (the most capable) of the Yamaha performance sysex formats (if not all of them)? Looks like @BobanSpasic has figured out how they work (but his code is Pascal rather than C++).
How about folders being numbered and being mapped to MIDI banks in the same way?
Well, for all the issues I list about the use of subdirectories, whilst ideally I think this would be a good way to go, I think it will be a lot of complications in the code for very little real gain.
Doing everything via sysex would be nice, but it'd probably be a huge piece of work?
As I say, I think it is a complete re-implementation.
About SysEx - the nearest device to MiniDexed is TX802, but this would need a re-write from scratch. Voices need to be in DX7II format and performances something like TX802 Performances, where TX802 uses a very ugly format for performance files.
SynthDexed engine is capable of this, but MiniDexed needs to be rewritten.
By using DX7II format for voices, all the controller settings (ModWheel, Aftertouch etc.), Portamento, Poly/Mono are then a part of the voice.
NoteLimits, MidiChannel, Pan, and additional MiniDexed parameters for effects - part of the performance.
I would not go for it.
What we could optimize is:
- have a directory "cartridges" instead of sysex and performances
- cartridge contains subfolders (actual "cartridges")
- every "cartridge" contains sysex banks, and performances using these banks
- now we can strip voice data from the performance files, because the linked banks are provided in the same directory
- deploying voices and performances is now deploying the directory content of the "cartridge"
Actually what I like about the current format of our performances is that they contain the needed voices, makes it easy to share/move just 1 file, and you don't have to keep track of matching the correct voices with the corresponding performances in the filesystem.
So maybe we should just somehow "binary serialize" our performance files for sending/receiving them via sysex over MIDI... (e.g., base64 encode, then send inside a sysex - something along these lines). Wouldn't be compatible to other existing instruments, but would probably be the easiest and most flexible way?
If I could have one wish for those, all of them should support aftertouch.
Um yeah, are you really serious? Not every dessert has raisins, not every picture has all the colors, not every song uses all the notes,.... 😅
There are sounds where aftertouch doesn't bring any improvement: drums, pianos, all sounds with a short decay time.
For example, a piano played hard and rocking would trigger aftertouch and change the sound without the player wanting it.
I'll be reworking all the modulations, especially the settings for the ModWheel are often too drastic (default, pitch). Aftertouch is also used where it is useful. But I actually wanted to wait until problem #562 was solved before starting work.
What is supposed to happen with the B-Side directory?
I use the B-Side folder for discarded performances, which is neither sorted nor documented. When I think about it, it doesn't make sense at this point and I'll remove the folder again.
If I could have one wish for those, all of them should support aftertouch.
...where it makes sense of course ;-)
It's not a probem to have a B-Side folder in the repository, but given your explanation let's not include it in the MiniDexed download.
It's not a probem to have a B-Side folder in the repository, but given your explanation let's not include it in the MiniDexed download.
Does that mean I can create additional folders, but you only pick up what you need?
And please include the miniDexed Performance List.pdf in the root directory.
Does that mean I can create additional folders, but you only pick up what you need?
Yes, as long there is one folder that I should pick. Ideally the folder would be called performance and everything that should not be shipped would be in folders outside of that folder.
btw - should @Banana71's performance github project be included in the main MiniDexed github too? I don't really know how these things work, but it is certainly useful to be able to get from MiniDexed to the list of performances from the same github starting point.
should @Banana71's performance github project be included in the main MiniDexed github too?
It already is:
MiniDexed/.github/workflows/build.yml
Lines 73 to 74 in c6325cd
It is included in the build, but not in the project...?
It is not a git submodule if you mean that by "included in the project". To be honest, I find git submodules a pain and try to avoid them if not absolutely needed.
Ok - it was more that it allows one to click through from the main project to any sub-projects easily for browsing the repositories :)
I guess a prominent link in the readme would do just as well!
Kevin
Good idea, done!
Closing this now as I think it is pretty much covered by #581
If elements are still outstanding, then I suggest they are picked up as new issues.
Kevin