Great, but needs some things.
chorman0773 opened this issue · 29 comments
If I can request, it would be good if the Compiler was designed with all major features. One thing is that the primitive types in C include all, including a 64 bit long. It could be setup to operate on 2 ints at once. Also The c standard library should be provided, the other integral types could be setup to do something similar to java, where char, and short are promoted to int, with unsigned values being 0-extended, and signed values being sign-extended. (IE. FFFF would be FFFFFFFF if it starts as a (signed) short but 0000FFFF if it starts as an unsigned short.) Also if possible pointers and indirection would be greatly appreciated. (Like MOV 1, [] -> in most assembly languages, when a memory address is given, but is enclosed in [], it refers to using the memory address stored within .
I know that function-pointers are probably not possible, but all other parts of C should be possible to implement. The last thing is indexing (in ASM). It should be possible to index a memory address from a given.
Finally, this should probably be updated to Minecraft 1.13 command-set, because 1.12 commands are incompatible with 1.13, and 1.13 will be more versitile.
All in all, this is an amazing thing, and assuming that I can utilize the standard library and all of the internals, It would because possible to implement the VM of a High-Level Language, and total control MC.
It's possible to add different sized integer types, but note that it would add some overhead to arithmetic operations. Integers >32 bits would be even more complex (but possible).
I agree this would be a nice feature to have. using int will always yield best performance though.
The c standard library should be provided
maybe portions could be, but this is not something I think will be possible (at least with reasonable performance). Someone other than me could attempt this if they wish.
Note that floating points are not natively supported in scoreboards, so a floating point implementation will have to be written (not exactly very performant).
Also if possible pointers and indirection would be greatly appreciated
Already possible :) Dereferencing structs etc work as expected. This behaviour is an extension to the assembly language only possible using C. (See these instruction in asm_extensions.py)
I know that function-pointers are probably not possible
They actually are possible, with the use of a function lookup table. The compiler doesn't currently support it but a table is generated when using SYNC (see add_lookup_table). Each function is given a unique number and the table just does a load of if-elseif-elseif to find the corresponding function.
The last thing is indexing (in ASM). It should be possible to index a memory address from a given.
Looks like you didn't finish the sentence. I assume you're talking about an offset index from a base address (like MOV [esp+0x04], eax in x86). This is possible with the new instructions in asm_extensions.py and the compiler makes use of this.
I would add it to the assembly language, except it requires the use of a memory region. There is no actual addressable memory with the assembler, which I mentioned in the readme. With the C compiler, it defines a continuous memory region using scoreboard objectives (i.e. mem_0, mem_1, mem_2) and adds a lookup table to find the correct slot (using a binary search).
This is something I wanted to avoid in the assembler, but was required for the compiler. (The stack PUSH/POP actually already do this, but for a relatively small stack size, so use this if you need this with the assembly language).
Finally, this should probably be updated to Minecraft 1.13 command-set, because 1.12 commands are incompatible with 1.13, and 1.13 will be more versitile.
Yep, will be updated. Probably will wait for it to stabilise first.
It would because possible to implement the VM of a High-Level Language
theoretically :p
Even a simple tetris game has a large amount of overhead, and runs really slowly (and that's even without time-sharing with the rest of the Minecraft's tick cycle)
Adding all the fundamental types would make 2 things better:
1 -> It would complete the c standard
2 -> It would make it easier to use pointers,
Also, it allows for more descriptive uses types. Yes have type promotion would have some overhead, but it would make programs more readable.
One potential change, would be to use NBT tags with /data and /execute store, over objectives. It would make indirection easier (assuming that we get variables in functions).
Also just curious how string literals are stored, since char is not a type in your compiler, and also curious if your compiler supports Variadic functions?
One potential change, would be to use NBT tags with /data and /execute store, over objectives.
I wanted to use NBT, but I have not been able to find a way to transfer data to/from scoreboards. Scoreboards are required to perform the actual arithmetic on values.
Also just curious how string literals are stored, since char is not a type in your compiler.
Currently there is not a real string type. Strings are injected directly into the commands and are not treated as data (which is why you can't perform any string operations).
As for transferring data, I know that in 1.13 it is possible to convert nbt into scoreboards with /execute store score run data get
As for strings, I would store it as a pointer to a memory address, which has n characters, and ceil(n/4) bytes. You could stored char*'s as pointers to multiple characters, which will keep reintrepretation state correct (IE, if a char* consists of 4 chars, an int* consists of 1 int which is stored that way). I would recommend using big-endian, as that will keep it consistant with other things. Also an Idea I had, stems from this, that you store multiple <4 byte data-types in a single scoreboard objective>. Though even with NBT you would still have to do operations in integers, you would have to implement special floating-point operations.
BTW I have to ask how TEST works in c code with the asm tag, and what instructions allow for dereferencing, in the extended ASM that C uses?
As for transferring data, I know that in 1.13 it is possible to convert nbt into scoreboards with /execute store score run data get
I've taken a quick look at this, it's almost possible make use of /execute but I don't see a way to dynamically specify the path, which means there isn't any advantage over a scoreboard objective.
Storing multiple values in a single objective is possible with bitwise operations, but they are much less efficient than a typical CPU. It has to loop over each bit (so 32 iterations) for every operation, which gets expensive. (this is what it generates)
BTW I have to ask how TEST works in c code with the asm tag
There is a built in function, __test_command which returns 1 if the command succeeds or 0 if it fails.
what instructions allow for dereferencing, in the extended ASM that C uses?
I linked to them earlier
They are:
So the use of NBT would enable you to store data in a Byte-Array. The advantage to using it is space. While a single byte in java takes a full int (as java does not store byte, short, or char types in VM code), a Byte array, takes 1/4 of the size of the same length int [] (assuming that the array length is in multiples of 4. Plus, using alot of scoreboard objectives would be extremely problemattic as it makes it hard for people running it to decipher objectives. The use of a few objectives, say the SP, the current handle function number, and a few registers for operations places a slight overhead (for transfer and decoding), but all in all an advantage for high-level programs (which use a massive ammount of data). It would also serve to enable dynamic allocation with functions like malloc and calloc. (You would also have to store information about allocation, but that can also use a byte[].)
If you do add this, the way I have pseudo-implemented the allocations, is to flag memory map with flags. I would keep the flags the same. The flags I use are M_read (0x01), and M_write (0x02).
Also using byte[]'s would allow you to store unaligned 4-bytes, and also store other values.
For 8-byte values, I would keep the same manipulation registers (Say X and Y), but also add LX_H and LY_H. These registers represent the high-byte of the 8-byte value, and cannot be used directly.
Adding processor flags would allow you to "enable"-disable standard manipulation of the X and Y registers. I would also do this for your acc register.
It would add a few instructions for 8-byte usage, but other than that, would work the same way.
Another question is how function symbols are stored, and what the function mapping is for the call lookup table (which the stack uses)? Also to ask, is there an option to output the symbol table with compilation into a file? (So that I can call a C function from an advancement)
I know that the basic answer is (probably) within the code, but I don't use python, and don't really understand it. My primary Background is in Java, then Lua, followed by C and C++.
I will take a look at using byte arrays when I next work on CBA.
Not sure that providing low/hi parts of a register has any advantage here, other than more flexible register manipulation. Unlike real hardware where this doesn't have additional overhead, in the context of scoreboards it'll be less performant.
Another question is how function symbols are stored, and what the function mapping is for the call lookup table (which the stack uses)?
Each function is given a numerical ID, which is a CRC32 of the function name, see here.
The func_lookup_table function is generated as a if-elseif-else block on all the function IDs.
Also to ask, is there an option to output the symbol table with compilation into a file?
Good idea, there's no specific way to do that, I will add that as an option.
You can figure out the function name if you know the namespace given.
It will be: {namespace}:sub_{function_name}
e.g. in the tetris example, using --namespace c_generated, the new_shape function is called c_generated:sub_new_shape
Took a look at your new API.
I already pointed this out in a related question, but from examining the python code (have a bit more experience now then I did previously), it looks like many of the things require literal expressions. While this is fine, it may be prudent to extend it to constant expressions. For example: the following code would be valid (if I understand the python correctly this doesn't work currently)
const char myMSG[]="Hello World";
printf(myMSG); //Would like to use puts here but you cant have it allThere are other use cases, where the feature makes sense, such as having the same basic format string being used in multiple places. There could also be a __format intrinsic which gives you back a constant string based on format arguments.
Question: are bitfields supported? As in
struct S{
unsigned A:16;
unsigned B:8;
unsigned C:7;
_Bool D:1;
};
If so, I can use my qc hack:
typedef union __qc_float32{
unsigned bits:32; //Yes this is required, __qc_float32 basically is a float and aliasing rules apply here. Thankfully you can't take the address of a bitfield or bind a bitfield to a reference in C++.
float val;
struct{
unsigned sign:1;
unsigned exp:8;
unsigned sig:23;
} rep;
} __qcfloat;
Also can you extend bitfields in C to allow (un)signed long long when they get added? So I can set up double the same way.
Bitfields are not currently supported. I figured it'd not be a widely used feature. Bitwise operations are expensive compared to arithmetic, due to all values in a scoreboard are ints, you'd need to divide or multiply by 2 in a loop to get to the desired bit. It should still be possible to do this with << and >>.
If I was to add decimal values, I'd probably go for a fixed point instead. e.g. scale everything by 1000 so it can be kept as integer arithmetic.
qc is a decent library. It also happens to have optimizations that are specific to floating point numbers.
And yeah, most bitfields would be stupid w/o bitwise operations in scoreboards (that will probably go into my next proposal). Though yeah, I do have a bitfield there to protect from taking the address, and violating aliasing rules (even though qc basically violates every type-punning rule in the book but hey, its a library designed to be used with a compiler or backend support (the original use was a SoftFP library for a Snes Homebrew llvm backend). Also just a question, but have you considered an llvm backend for CBA, then having the C support of clang?
have you considered an llvm backend for CBA, then having the C support of clang?
Someone else suggested that too:
https://www.reddit.com/r/Minecraft/comments/7fnzv4/i_wrote_a_c_compiler_that_creates_minecraft/dqd9osn/
The thing I'm having most difficulty with at the moment is optimisations and a good IR, so LLVM would definitely be useful for that.
I may try out creating an LLVM backend if I don't get much further with the current implementation.
Using llvm would be good, you could also add support for your intrinsics in clang++.
As I pointed out in the other open issue, __scoreboard_objective should be added to allow the general access of scoreboard objectives (via a volatile lvalue) . The specific "kind" of lvalue it results in is similar to the register intrinsics I use in SNES-Dev (my snes homebrew thing I mentioned eariler).
The definition is somewhere in This Document, under the "Register Intrinsics" heading.
Basically, you can't take there address, can't bind them to a reference, except a const reference, and if you bind them to a const-reference in C++, it acts the same way that rvalues act when bound to a const-reference.
Also as much as I hate to be a pendant, use of sync as a reserved word on its own should not be done. If you want it to be a single special declaration, it should either be __sync or _Sync, but I would have it as an intrinsic, rather than a free-floating declaration. The double-underscore or underscore-capital letter makes it a reserved identifier, so you can do whatever you want with it, without conflicting with other identifiers people may want to define.
What you're describing with __scoreboard_objective sounds very similar to the entity_local variable type I made in this commit:
6e52bee
I need to create documentation for it, but basically it implicitly assumes the value of an entity's scoreboard objective under a selector context (intrinsics created in this commit)
I still persist that you need to make sync into either __sync, __sync__, or _Sync. Otherwise, your steeling an identifier which normal C code might want to make use of
I would like to request an extensions system be added at some point. Basically, at the directory of the compiler, you would have a folder called ./extensions. Inside is a bunch of python files that can override part of the assembler or compiler. When you invoke either the assembler or compiler, you should be able to pass -x<extension> and have the extension named by <extension> loaded. Extensions are loaded in the order they are passed, so if you pass -xa -xb and b overrives some functionality added or overriden by a, b would take presedence over a.
My main use cases is implementing optimizations which cannot be performed with vanilla, but can be performed with standerized modded extensions, such as the basic bitops in scoreboards, or more complex, the function stack defined by [cmd.sys.stack] in the Gac14 Specification which would basically allow for 2 thirds of the stack access code to disappear, and even returning from functions after a sync.
Clearly such things wouldn't make it into the real code (at least, not until they become vanilla), but would be useful to have available, and possible to enable or disable with a command line switch.
There should also be built-in/standard extensions for breaking ABI Changes, such as if we ever get actual function stacks (ha ha yeah right), that would force cba to use the old ABI. That way, if someone had a heck ton of code (cough me cough) and had to recompile a tiny piece with the new version after such a change, they don't have to recompile the entire project and everything still works fine.
It's kind of possible to add extensions already - in the compiler/lib directory there are python scripts which implement an API, for example block.py implements the is_block and set_block functions.
I'm currently writing a new compiler for a new language (based on C/C++) that is backed by Command IR so once the basic implementation is done I'll see how I can create extensions for it.
The main thing I was asking for was the ability to enable/disable extensions. Also the extensions are mostly intended to apply to the assembly, forexample, replacing the expensive CALL and RET instructions with [cmd.sys.stack]:
CALL fn
/system stack push resourceref
/function fn
RET
/system stack pop frame -> given that the prolog invokes push frame
/system stack function pop
Though they could certainly also introduce intrinsics.
Just commenting here to show how the new language (CBL) is coming along https://gist.github.com/simon816/0a68380b250b6e20526eac7d24943cb7
I have a question. I assume that compiling with --enable-sync affects the ABI, so functions compiled with it cannot call functions compiled without it, and functions compiled without it cannot call functions compiled with it (or you get dreaded UB). Is this a correct assumption?
You are correct in that the ABI is different. --enable-sync hasn't been needed since I moved over to Command IR so it's not an issue any more.
Regarding types smaller than ints:
Instead of forcing them to be within their ranges, I think it's better to just treat them as syntactic sugar. Make the compiler check for correct type usage but treat them as ints internally (in generated commands).
There is nothing to gain by simulating side effects like over/underflow and checking their size on every step and all you'll do is just slow them down.
The reason why different types exist is that they actually consume less memory. If short consumes the same amount of space as int, then it's effectively an int.
That is how C compiler would behave if it was operating on a platform that only supports int types. It wouldn't generate extra assembly to check for size.
Long and floating point types will require a custom implementation however, no way around it.
This works for any signed type, where its UB to overflow, but not unsigned
types which are defined to wrap.
What happens in vanilla when you overflow the scoreboard value? Also, it's undefined behaviour for signed types only because it makes it easier to optimize compilers and it's platform-specific how signed types are stored. As unsigned types are stored the same everywhere it's expected of them to wrap around when all bits are 1 and the first flip-flop gets incremented triggering other circuits to reset to 0 as well. It makes no sense to fight against how numbers are stored "internally". My argument is not that wrapping shouldn't be used, but that types shouldn't be forced to be smaller than what they actually are. Reason why cinttypes.h exists is because on some platforms int types were being stored as shorts - if they're "allowed" to do that, no extra effort should be put into validating number sizes conform to standard sizes used by common CPU arhitectures.
Unsigned types are completely artificial and up to the compiler to decide how they should behave. Addition and subtraction work the same, multiplication cannot be implemented the same way as for signed types and would require loops.
A possible alternative would be to define CHAR_BIT as 32, in which case,
everything up to int is the same size.
This should be done for types smaller than int IMO. As well as defining their MAX and MIN to the same values.
I also wanted to suggest that bools get stored as tags instead of taking up size of ints.
Regarding non-integer types, I removed it due to being incomplete but I started working on type casting: bbb9475#diff-2159fe19f79152c7a315904c89d8c523R167
It used the /execute store command: https://minecraft.gamepedia.com/Commands/execute#store
… entity <target> <path> (byte|short|int|long|float|double) <scale> -> executeUse the data tag path from one targeted entity to save the return value. Store as a byte, short, int, long, float, or double.
With this command, it is possible to use Java's type casting to handle truncation of data by using NBT data.
The disadvantage is that every write access must cast via NBT before storing in a scoreboard