Porting 0CC-Famitracker to Qt (using Nesicide)
nyanpasu64 opened this issue · 33 comments
I'm trying to build 0CC on Android.
Building 0CC for ARM using Visual Studio failed because there's no prebuilt MFCxxxx.lib for ARM, and mfc110.dll can be converted into a .lib (but MSVC++11 is too old for 0CC).
Building 0CC for ARM using Winelib failed because I can't build MFC in Winelib.
I built Nesicide-Famitracker on a Raspberry Pi using updated Wine headers (you must compile Wine to convert .idl to .h) and QT 5.7 or something.
Now I'm trying to add an Android target using QT Creator and latest QT 5.10 on Windows...
I tried compiling for Windows in QT Creator, using MSVC++15 (2017) instead of MinGW, but it complains hundreds of times about TCHAR*=_T("asdf") and TCHAR=_T('c') since the RHS is const (I couldn't figure out how to replace _T with an unconst cast). Maybe 0CC 0.3.14.5 is a good idea? Also I like MinGW-w64 or msys2 over mingw.
Now I'm trying to use QT Creator and NDK 10e (QT requires an old NDK version) gcc to compile for Android. I just moved over the Wine headers from Pi to my Windows laptop, and will be trying to setup and build over the next few days. Maybe I'll throw in 0CC instead of FT. How difficult would adapting 0CC to Qt be?
I took a brief look at 0CC and found it to be more complicated of a migration into the existing Qt Famitracker than I'd hoped. I haven't made much progress on it. There's quite a departure in styles and language capability usage between the two versions.
0CC 0.3.14.5 or jimbo1qaz/0CC-Famitracker should have an older codebase. I was trying to port to Android, but had trouble compiling https://github.com/pelya/commandergenius on Windows (SDL 1.2 port to Android). I researched Qt or Android audio libraries, but stopped working (gave up) on the port.
SDL 2.0 has an official port, and HertzDevil mentioned on Famitracker discord, that he was experimenting with 0CC and SDL.
i'm trying to document your exact changes from 0.4.6 at https://docs.google.com/document/d/1jLTmbk_hjBHMOUzrLjfe5m_-ZyKg7LmbBeHxD3MjBg8/edit, which should help me incorporate those changes into 0CC. Not quite done yet.
seems you're using outdated versions of emu2149 and emu2413, relative to 0.4.6 https://github.com/Camano/FamiTracker.
Interesting. Perhaps the emu2149 and emu2413 files got dropped somewhere along the migration from 0.4.1 to 0.4.6. Looks like a pretty comprehensive list of file differences. I'd tried to make it as minimally invasive as possible for the original FT code. Not sure if you got that sense from your review?
This actually isn't the complete list of changes...
There were a large number of style changes... So I threw both projects through clang-format... And removed all empty lines... (which made diff -r think the entire file was a block, so I added empty lines everywhere ;) ).
oof... I spent hours normalizing the case of .cpp files, header files, and #include statements, when I could've just used diff -i to ignore those changes. For that matter, man diff reveals a smorgasbord of whitespace-dropping options I forgot to use.
You removed all occurrences of ON_EVENT(something, &CClass::Something) and one occurrence of ON_EVENT(something, CClass::Something), why was that? Did the compatibility layer not support that well? In any case, I have a fork of 0.4.6 and j0CC with the same change applied, to reduce unimportant diffs.
On the topic of fun , I accidentally corrupted my "source code folder" by adding blank lines into every .cpp and .h file, while working on diff-simplfying sed scripts (instead of using the appropriate diff option :( ). And somehow Git generated N163.h and N163.H in the same folder. On Windows. On a case-insensitive system (except Windows Ubuntu is kinda-sorta case sensitive, and will tend to record deletion but not addition when I git add case changes yay).
The compatibility layer shit itself whenever it came across ON_EVENT(something, &CClass::Something), specifically the scope portion (CClass::). I couldn't figure out why and since there were only a few instances where the scope-specifier was present I just removed them. It'd be nice to figure that out so I wouldn't have to touch the message maps AT ALL. In my many-moons-ago implementation I'd attempted to replace the message map architecture with signal/slot, but that led me down a path of waaaay too much context-specific glue code, which I'd set out to avoid at all costs.
diff is a fantastic tool! :)
I noticed you removed or omiited #include <algorithm> in several files, even though (for example) APU.cpp used std::min and std::max. Is that intentional or not?
I think i've hit a roadblock. cqtmfc_famitracker.cpp appears to be a translation of FamiTracker.rc, but they don't match exactly. Was that done programmatically or by hand (all 244 KB typed, one line at a time?)
(Also many source files seem to omit , stdafx.h, MFC headers, but still compile in Visual Studio. They don't compile in Qt Creator unfortunately. Why is that?)
Originally the compile failed because of missing . When I removed it to see if it was even needed and the compile succeeded, well...I left it removed. :)
As for cqtmfc_famitracker.cpp, yes, that was entirely hand coded. I thought about creating a .rc->.moc translator tool but when it came down to the details it was quicker to just manually do it. I'm still interested in the possibility of a .rc->.moc tool, in general as an expansion of the idea that any MFC app can be run atop Qt. The file isn't that hard to maintain against .rc file changes now especially with a meld of the old/new .rc files open.
Looking back on it I'm not sure I got far enough in the port to even bother with the .rc files.
I am/was working on a .rc to .cpp Python script (but I'm unsure if it's a good use of my time). How is that different from .rc to .moc, and is the latter a better approach? (I'm not too familiar with C++ Qt)
Looking back on it I'm not sure I got far enough in the port to even bother with the .rc files.
cqtmfc_famitracker.cpp seems like a rearranged translation of FamiTracker.rc. What do you mean you didn't use .rc?
How easy would it be for you to port 0CC's .rc file (I don't know how)? It has a huge number of changes (hundreds of lines, maybe thousands).
I meant I hadn't got far enough in the 0CC port (0CC-ft branch, not master) to where I'd even tackled the updates to cqtmfc_famitracker.cpp needed by the changes between 0CC-FamiTracker.rc and FamiTracker.rc.
For example, there is a control item labeled "Include grooves" in 0CC-FamiTracker.rc. I don't have that in cqtmfc_famitracker.cpp.
I can do the .rc porting, yes.
As for a tool, I misspoke. moc is the signal/slot translator. I meant a tool that translates .rc->.ui files. A .ui file is XML of the UI.
https://github.com/jimbo1qaz/j0CC-Famitracker-Qt (Note it's based off my own fork of 0.3.14.5, and some things, like the properties dialog, differ from HertzDevil)
Only tested on Qt Creator, MSVC, Windows. qmake works, but compilation crashes and burns (one reason is because cqtmfc_famitracker.cpp).
(Also many source files seem to omit \<algorithm\> (thanks markdown), MFC headers and class definitions, but still compile in Visual Studio. They don't compile in Qt Creator unfortunately. Why is that?)
Probably because I only got part way through the 0CC port before deciding it was too much work to tackle in the original timeframe I'd given myself.
By the way, cqtmfc.cpp/.h are also 'glue'.
How did you get past this:
In file included from cqtmfc_famitracker.cpp:7:
Source/FamiTrackerDoc.h:324:22: error: no template named 'is_invocable_v' in namespace 'std'; did you mean '__invokable_r'?
if constexpr (std::is_invocable_v<F, CSongData &>)
~~~~~^~~~~~~~~~~~~~
__invokable_r
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/type_traits:4336:8: note: '__invokable_r' declared here
struct __invokable_r
^
It appears to be a c++17 thing which I've specified in my .pro but OSX/Xcode don't appear to have the right stuff for c++17.
Source/FamiTrackerDoc.h:324:22: error: no template named 'is_invocable_v' in namespace 'std'; did you mean '__invokable_r'?
are you trying to build my new repo? It doesn't build for me, even.
so i tried compiling the original nesicide in Qt Creator (with latest Qt 5.10.1) (not using shell scripts)... after a pile of pain (turns out you should open build\famitracker\famitracker.pro (and no other projects) in qt creator, and msvc doesn't work)... slow compilation with make, and errors with jom... Finally finished building.
I get dll errors when opening famitracker.exe in Windows Explorer. and 0xc0000139 when launching from Qt Creator.
Qt Installer: "MinGW 5.3.0 32 bit" Qt library, and "MinGW 5.3.0" toolchain from MinGW-builds. Compiler should be compatible, Qt Creator doesn't complain that Qt doesn't match the compiler, but the app won't launch.
I'll try downgrading to 5.6.2 5.6.3 was found in the online Qt installer, and 5.6.2 added an old version of Qt creator I didn't use... 5.6.3 also has same issue.
I have some questions, In Qt Creator, if I don't open the subprojects, I get this, and Ctrl+K doesn't "see" any of the embedded files.
If I open the subprojects, I get this, and every project has its own build configuration. And Ctrl-Shift-B causes FT to be unable to find rtmidi.lib when linking.
Right-clicking famitracker/famitracker-lib -> Build, right-clicking famitracker -> Build, started compiling the exact same code again. Aaand apparently managed to build rtmidi and famitracker (maybe rename to famitracker-app?) using mingw, but the others with msvc. Why is Qt Creator so weird?
Seems Projects tab, "Shadow Build" checkbox routes builds into a separate folder, which breaks linking (LNK1104 .lib symbols not found)
Trying to compile with Qt 5.10.1 and MSVC. qmake insists on passing -Zc:strictStrings and nothing I do (removing flag, removing flag from debug, clearing all flags entirely) will convince it otherwise. I tried deleting all .qmake.stash files to no success.
Issue persists if I make the operations run, unconditionally even if win32 is unset.
Issue persists if I add those flag modifications to build/famitracker.pro.
The problematic flag is due to Qt's msvc-version.conf . https://forum.qt.io/topic/11565/qmake_cxxflags-overriden/2 https://stackoverflow.com/a/32179647
message($$QMAKE_CXXFLAGS) shows my modifications work. But the new flags don't reach cl.exe. https://stackoverflow.com/a/14236984 ?
Turns out qmake forgets to rewrite the makefile because it's a defective piece of software. https://stackoverflow.com/a/32179647 Clearing the makefile manually fixed the issue.
Everything compiles. Linking can't find the .lib file, since for shared libraries: "Q_DECL_EXPORT must be added to the declarations of symbols used when compiling a shared library." (https://doc.qt.io/qt-5/sharedlibrary.html)
Apparently different projects treat each other as shared libraries (.lib files on MSVC). Famitracker originally was a single project. HertzDevil has partially split 0CC 0.3.15.1 and master into projects, and I don't know if he uses export/extern/etc.
EDIT: apparently not.
https://github.com/HertzDevil/0CC-FamiTracker/blob/master/libft0cc/include/ft0cc/doc/dpcm_sample.hpp
https://github.com/HertzDevil/0CC-FamiTracker/blob/master/libft0cc/vc/libft0cc.vcxproj
Using procmon, I found that Visual Studio uses lib.exe to extract all symbols from the obj file (Tracker.exe ... "...\Community\VC\Tools\MSVC\14.14.26428\bin\HostX86\x86\Lib.exe" /OUT:"...\Debug\libft0cc.lib" /NOLOGO Debug\dpcm_sample.obj). Similarly Qt Creator and MinGW makes all symbols visible when linking the different projects. However, Qt Creator and MSVC requires manually Q_DECL_EXPORT ing all symbols in order to be visible... Maybe qmake can be persuaded to use Lib.exe to extract all-symbols .lib files out of .obj or .dll?
The Qt not showing subproject content thing is an issue I've seen but it generally clears up after a while -- like it's syncing the project in the background. I'm not a keyboard shortcut guy so I have no idea what Ctrl+K and Ctrl+Shift+B are supposed to do.
You're supposed to open the .pro files in nesicide/build/. Those are subdirs .pro files which 'contain' all of the subprojects required to build the whole project. So famitracker.pro contains all of the projects for famitracker. ide.pro contains all of the projects for nesicide, etc. You can build the whole thing by building at the top level, or build individual subprojects by building from their level. That is how I've been doing the Qt/0CC merge thus far -- just rebuilding the famitracker-lib project, since the famitracker(-app) project is irrelevant until the -lib project compiles.
I can't bother with MSVC I don't have a working Qt installation in Windows that I have regular access to at the moment. I'm either using MinGW in a Windows machine I have occasional (read: lunch breaks at work) access to, or clang on my MacBook.
No, I haven't tried to build your new repo, I was just curious if you'd been able to get past that issue. It seems it might be a clang issue, so you might not see it.
(Using MSVC) Seems some functions in cqtmfc.cpp cause linking conflicts with the real windows.h.
kernel32.lib(KERNEL32.dll) : error LNK2005: CloseHandle already defined in cqtmfc.obj
kernel32.lib(KERNEL32.dll) : error LNK2005: SetEvent already defined in cqtmfc.obj
kernel32.lib(KERNEL32.dll) : error LNK2005: ResetEvent already defined in cqtmfc.obj
If I remove those functions from the .cpp file, it compiles...
CVisualizerWnd::FlushSamples(...) calls Windows's SetEvent(m_hNewSamples). The value is around 0x300.
CVisualizerWnd::ThreadProc() calls cqtmfc's WaitForSingleObject(hHandle=m_hNewSamples, dwMilliseconds=INFINITE).
- CObject* pObject = (CObject*)hHandle;
- CCmdTarget* pCmdTarget = dynamic_cast<CCmdTarget*>(pObject);
- crashes since hHandle is not a valid pointer.
honestly I think trying to make this MSVC-compatible is a waste of my own time. Should I push some of my partial MSVC fixes (however far I got so far)?
EDIT: #33 meh
I finally got back to this and got past the C++17 issues I was having before. I can now start the resource file conversion. There is a huge pile of differences that will take some time to parse through. It's reinvigorating my desire to create a tool to do it for me.
I wrote up a "vanilla vs nesicide" diff at https://hackmd.io/@IFuHNFVvQXWfCLfmOaJA1w/S1L_6G71S . My plan is to copy the entire j0cc repo in place of vanilla, regenerate the cqtmfc...cpp, and then port over all changes to vanilla to j0cc.
Git comparison repo (including methodology) is at https://github.com/jimbo1qaz/famitracker-vs-nesicide. Files have been lowercased, line endings converted to Unix, comments and empty lines removed, and clang-format applied.
Why did 883097c change libs/famitracker/Source/ColorScheme.h and you extracted _STATIC_COLOR_SCHEME from STATIC_COLOR_SCHEME? This change is not in vanilla 0.4.6 or j0cc.
883097c#diff-80b1c26f4f22c9d6a938d0c4f71f8fdeR29 claims to be updating nesicide to vanilla 0.4.2.
Also why is it a struct and not an enum/namespace?
Do you have any other means of communication other than issue trackers, or is this thread OK?
Why does https://github.com/christopherpow/nesicide/blob/master/libs/famitracker/Source/stdafx.h include #include "cqtmfc.h" twice? I think the second include is ignored because #ifndef CQTMFC_FAMITRACKER_H.
Guess that was a typo. Fixed.
https://github.com/christopherpow/nesicide/blob/master/libs/famitracker/Source/SoundGen.h has #include "Common.h" twice.
I think CImageList pushes new CBitmap into _images: QList<CBitmap*>, but never deletes them. I think this is a memory leak. Maybe you should switch to unique_ptr and maybe make_unique (c++14).
Actually that won't work, CImageList::commonAdd calls new, but CImageList::Add(HICON hIcon) borrows the argument.
(I'm implementing CImageList::Replace(int nImage, CBitmap* pbmImage, CBitmap* pbmMask) among many other things.)
You're probably right but please keep in mind that missing or unexpected (no deletion/removal) functionality is likely intentional -- I only implemented the bare minimum of what I would need to get the job done. It is by no means a perfect MFC emulation. Anything you add to make it better will only do that...make it better. Thanks.
I'm thinking of starting from scratch and porting j0cc to WxWidgets, which has a similar API to MFC, including a WxImageList.
I think the current code base is seriously flawed and hacky (though possible to somewhat clean up with lots of extra work and investigation). Does CImageList essentially hold void* which can either be CBitmap or QIcon, and the ExtractImage caller hopefully uses the right cast?
You're free to do whatever you want, of course. I never implied that my Qt emulation is perfect. It has a lot of half/unimplemented functionality because I didn't need it all. I'm not sure I follow your trouble with CImageList. It holds a QList<CBitmap*>...
If you want me to implement CImageList::Replace, I can take a look at that. Seems pretty straight forward to me.
If I were being paid to implement a MFC emulation on Qt, that'd be different.
Can this issue be closed?
I'm no longer working on "0cc to qt", so go ahead.
Are you working on Occ to WxWidgets? How goes?