Missing features

[cemeyer]

I'm making a simple small project in ctypes.sh and there are a couple missing features I notice that really hurt:

1. Something like alloca() or automatic stack memory that we can reference via pointer. (I should clarify — it does not matter at all where the memory lives. Just that the release is automatic on function return.)
2. sizeof(), both for primitives and structs.

Additionally, "structs" in ctypes.sh are more like prototype objects than types. That's ok, just sort of confusing coming from C. I don't know if this could be done better. It seems very desirable to not have to hand-code libc structures. Maybe we could infer it from DWARF debuginfo? This would be a major project. Just something to consider.

Documentation is also kind of sparse. I'm digging into the C sources in order to find usage of unpack, for example.

[cemeyer]

Something seems to be wonky with errno as well. For example, calling a routine that sets errno and then using err(3) prints "Success".

[cemeyer]

Another common issue that may be worth considering changing:

Because the default integer size is 'int', on 64-bit architectures bare pointer values are truncated to 32-bit values and cause instant segfaults. If we change the default integer size to intptr_t, it's closer to the naive C ABI (at least on AMD64).

[cemeyer]

As long as I'm here — it would be nice to have some easy conversions back and forth between bash and C strings.

[cemeyer]

Anywhere, here's the little project: https://github.com/cemeyer/httpd.sh

There's a start at some C stdlib API wrappers in there that you're welcome to use or adapt in some way for ctypes.sh.

[taviso]

Nice!

Parsing dwarf data is an interesting idea, maybe we could at least make templates for humans to edit....I'll think about it.
The wrappers look good, let me think about the api to include them. Maybe something like source ctypes.sh/socket.sh.

[cemeyer]

Sure, it's just a sketch of what stdlib-ey stuff might look like. It'd sure be nice to make structs automated, though. It'd also be nice to have a short-cut way of creating a zeroed instance of a struct.

Today you must:

1. Copy the struct prototype (fairly verbose, too: foo=( "${bar[@]}" ))
2. Allocate memory the size of the struct
3. Zero it
4. Unpack into the prototype

[taviso]

I've been thinking about making -g the default, if you want something else you can do -h $handle, what do you think? This will break existing scripts, but best to get it out of the way before first release :-)

I think it will make things less verbose, which is usually a plus for shell scripts.

[taviso]

Well, I went ahead and did it. I think it makes things look cleaner.

I updated the documentation and sent you a pull request to fix httpd.sh. :-)

Do you think we should just get a first release out, or should we finish a basic stdlib first?

[cemeyer]

Sounds fine to me.

I don't have a preference on ordering release vs stdlib. This'll be fairly
niche, v1.0 or not, for a while. I don't think we should commit to any kind
of API stability yet.

Thanks,
Conrad

On Sun, Aug 23, 2015, 15:44 Tavis Ormandy notifications@github.com wrote:

Well, I went ahead and did it. I think it makes things look cleaner.

I updated the documentation and sent you a pull request to fix httpd.sh.
:-)

Do you think we should just get a first release out, or should we finish a
basic stdlib first?

—
Reply to this email directly or view it on GitHub
#27 (comment).

[taviso]

I thought about your dwarf idea today. The utility pahole almost does what we want. Here's an example of pahole output:

$ ./pahole -C stat /usr/lib/debug/lib64/libc.so.6.debug 
struct stat {
    __dev_t                    st_dev;               /*     0     8 */
    __ino_t                    st_ino;               /*     8     8 */
    __nlink_t                  st_nlink;             /*    16     8 */
    __mode_t                   st_mode;              /*    24     4 */
    __uid_t                    st_uid;               /*    28     4 */
    __gid_t                    st_gid;               /*    32     4 */
    int                        __pad0;               /*    36     4 */
    __dev_t                    st_rdev;              /*    40     8 */
    __off_t                    st_size;              /*    48     8 */
    __blksize_t                st_blksize;           /*    56     8 */
    /* --- cacheline 1 boundary (64 bytes) --- */
    __blkcnt_t                 st_blocks;            /*    64     8 */
    struct timespec            st_atim;              /*    72    16 */
    struct timespec            st_mtim;              /*    88    16 */
    struct timespec            st_ctim;              /*   104    16 */
    __syscall_slong_t          __glibc_reserved[3];  /*   120    24 */
    /* --- cacheline 2 boundary (128 bytes) was 16 bytes ago --- */

    /* size: 144, cachelines: 3, members: 15 */
    /* last cacheline: 16 bytes */
};

it can also expand types into primitive types (like long, int, etc) with --expand-types. I think if we import it we can make builtin command struct that does something like this:

# make foo an array of the right size and type prefixes populated correctly
$ struct stat foo 
$ echo ${foo[@]}
long long unsigned unsigned int ..etc

# struct command can also define a bunch of integers like _structname_membername so this works:
$ echo ${foo[_stat_st_uid]}
int:0
$ echo ${foo[_stat_sts_size]}
long:8192

Where struct is a builtin loaded by ctypes.sh that does what pahole does. What do you think?

[taviso]

I'll move this into a separate issue.