tcncloud/protoc-gen-persist

remove unneeded arguments option, and positional dependency in result protobuf

iamneal opened this issue · 2 comments

A big confusing part of persist is the arguments option, and the requirement to match field ordering on the output proto to the pointers used in rows.Scan(). Both of these problems deal with translating proto-position to query-position, and I think we can fix both.

The arguments option goes away if we are willing to preprocess the query string.
There is a number of ways we could do this, but for example, we could follow how spanner does fields in queries, and use "@" annotated names that must match fields in the input proto.

at generation time, we can search for the "@", note its name and position, and replace it with a ?, $<#>, leave it, or whatever else.

The positional dependency of fields for the output proto is tricky to solve while being as fast as our current solution. Currently I am thinking that we implement a scanner that looks like this:

type alwaysScanner struct {
	i *interface{}
}

func (s *alwaysScanner) Scan(src interface{}) error {
	s.i = src
	return nil
}

Then we need to update the <output>From<service>DatabaseRow function to work with the sql.Rows.Columns() array, and the value scanned from the alwaysScanner. We know all the type conversions at generation time, so this should be trivial.

As an added bonus, our code doesn't explode with cryptic scan errors at runtime. We only scan out of the database, and convert to the output proto, the exact number of values that are returned from rows.Columns()

the new function to map output types should look a lot like this:

func UserFromUServDatabaseRow(serv UServTypeMapping, row persist_lib.Scanable) (*User, error) {
	res := &User{}
	cols, err := row.Columns()
	if err != nil {
		return nil, err
	}
	scanned := make([]alwaysScanner, len(cols))
	if err := row.Scan(scanned...); err != nil && err != sql.ErrNoRows {
		return nil, err
	}
	for i, col := range cols {
		switch col {
		case "id":
			res.Id = int64(scanned[i].i)
		case "name":
			res.Name = string(scanned[i].i)
		case "friends":
			{
				var converted = new(Friends)
				if err := proto.Unmarshal([]byte(scanned[i].i), converted); err != nil {
					return nil, err
				}
				res.Friends = converted
			}
		case "created_on":
			{
				var converted = serv.TimestampTimestamp().Empty()
				if err := converted.Scan(scanned[i].i); err != nil {
					return nil, err
				}
				if err := converted.ToProto(&res.CreatedOn); err != nil {
					return nil, err
				}
			}
		default:
			return nil, fmt.Errorf("unsupported column in output: %s", col)
		}
	}
	return res, nil
}

arguments option was removed in v3