about how to achieve protoc --include_import

[guochenghui13]

Hello, I am wonder how I can use protoparse to generate fileDescriptorSet like "protoc -- descriptor_set_out" .

Since sometimes I will include import proto but I no compile, but I want to see the import proto file in fileDescriptorSet

[timostamm]

Are you asking about protoparse, or about this repository, and the protocompile module?

It is certainly possible to generate a google.protobuf.FileDescriptorSet with this module. If you take a look at the godoc, you'll see that compiler.Compile returns files that implement protoreflect.FileDescriptor.

You will have to convert these files into google.protobuf.FileDescriptorSet - a protobuf message. To learn more about the relationship between the protobuf descriptor messages and protoreflect, see the godoc for the protoreflect package.

For a minimalistic example, see here:

package main

import (
	"context"
	"github.com/bufbuild/protocompile"
	"github.com/bufbuild/protocompile/protoutil"
	"github.com/golang/protobuf/proto"
	"google.golang.org/protobuf/types/descriptorpb"
	"os"
)

func main() {
	compiler := protocompile.Compiler{
		Resolver: &protocompile.SourceResolver{},
	}
	files, err := compiler.Compile(context.Background(), "a.proto")
	if err != nil {
		panic(err)
	}
	set := &descriptorpb.FileDescriptorSet{}
	for _, file := range files {
		set.File = append(set.File, protoutil.ProtoFromFileDescriptor(file))
	}
	bytes, _ := proto.Marshal(set)
	os.WriteFile("x.bin", bytes, 0644)
}

Please note that this example does not handle errors correctly, and takes many shortcuts, it really is just a minimal example. You will likely have to study the godoc, the source and tests to get a good understanding.

[jhump]

Since sometimes I will include import proto but I no compile, but I want to see the import proto file in fileDescriptorSet

Does this mean you want something like protoc's --include_imports?

If so, you'd need a slight tweak to @timostamm's excellent example above: instead of only adding the files returned from compiler.Compile to the set, you'd also need to add their dependencies, and do so in dependency order (so a file's imports always appear before that file in the set).

You can accomplish this with a little recursive function:

func addToSet(file protoreflect.FileDescriptor, set *descriptorpb.FileDescriptorSet, added map[string]struct{}) {
	if _, alreadyAdded := added[file.Path()]; alreadyAdded {
		// A file may appear more than once in the graph of transitive imports.
		// But we only want it to appear in the set once. So don't add it again.
		return
	}
	added[file.Path()] = struct{}{}
	// Add dependencies first...
	imports := file.Imports()
	for i, length := 0, imports.Len(); i < length; i++ {
		addToSet(imports.Get(i), set, added)
	}
	// and then the file itself.
	set.File = append(set.File, protoutil.ProtoFromFileDescriptor(file))
}

Then use from the example snippet above like so:

	set := &descriptorpb.FileDescriptorSet{}
+	added := map[string]struct{}{}
 	for _, file := range files {
-		set.File = append(set.File, protoutil.ProtoFromFileDescriptor(file))
+		addToSet(file, set, added)
 	}
 	bytes, _ := proto.Marshal(set)

[guochenghui13]

Thanks a lot!
When I using "parser.ParseFiles(protos...)", the filename in protos slice will always be concatenated with ImportPath. But I think import path is only related to the import proto in proto file. So how can I avoid this?

[jhump]

When I using "parser.ParseFiles(protos...)"

parser.ParseFiles is not in this repo. This question really belongs on the repo where that API lives: https://github.com/jhump/protoreflect/issues

FWIW, the behavior you are describing does not sound right. Neither this package, nor github.com/jhump/protoreflect/desc/protoparse include the import path in the file name. If you are seeing something different, you may have found a bug -- but in protoparse, not here. In that case, please file a bug in the other repo and please include a small repro case that demonstrates the issue.