about how to achieve protoc --include_import
guochenghui13 opened this issue · 4 comments
Hello, I am wonder how I can use protoparse to generate fileDescriptorSet like "protoc -- descriptor_set_out" .
Since sometimes I will include import proto but I no compile, but I want to see the import proto file in fileDescriptorSet
Are you asking about protoparse, or about this repository, and the protocompile module?
It is certainly possible to generate a google.protobuf.FileDescriptorSet with this module. If you take a look at the godoc, you'll see that compiler.Compile
returns files that implement protoreflect.FileDescriptor
.
You will have to convert these files into google.protobuf.FileDescriptorSet
- a protobuf message. To learn more about the relationship between the protobuf descriptor messages and protoreflect, see the godoc for the protoreflect package.
For a minimalistic example, see here:
package main
import (
"context"
"github.com/bufbuild/protocompile"
"github.com/bufbuild/protocompile/protoutil"
"github.com/golang/protobuf/proto"
"google.golang.org/protobuf/types/descriptorpb"
"os"
)
func main() {
compiler := protocompile.Compiler{
Resolver: &protocompile.SourceResolver{},
}
files, err := compiler.Compile(context.Background(), "a.proto")
if err != nil {
panic(err)
}
set := &descriptorpb.FileDescriptorSet{}
for _, file := range files {
set.File = append(set.File, protoutil.ProtoFromFileDescriptor(file))
}
bytes, _ := proto.Marshal(set)
os.WriteFile("x.bin", bytes, 0644)
}
Please note that this example does not handle errors correctly, and takes many shortcuts, it really is just a minimal example. You will likely have to study the godoc, the source and tests to get a good understanding.
Since sometimes I will include import proto but I no compile, but I want to see the import proto file in fileDescriptorSet
Does this mean you want something like protoc's --include_imports
?
If so, you'd need a slight tweak to @timostamm's excellent example above: instead of only adding the files returned from compiler.Compile
to the set, you'd also need to add their dependencies, and do so in dependency order (so a file's imports always appear before that file in the set).
You can accomplish this with a little recursive function:
func addToSet(file protoreflect.FileDescriptor, set *descriptorpb.FileDescriptorSet, added map[string]struct{}) {
if _, alreadyAdded := added[file.Path()]; alreadyAdded {
// A file may appear more than once in the graph of transitive imports.
// But we only want it to appear in the set once. So don't add it again.
return
}
added[file.Path()] = struct{}{}
// Add dependencies first...
imports := file.Imports()
for i, length := 0, imports.Len(); i < length; i++ {
addToSet(imports.Get(i), set, added)
}
// and then the file itself.
set.File = append(set.File, protoutil.ProtoFromFileDescriptor(file))
}
Then use from the example snippet above like so:
set := &descriptorpb.FileDescriptorSet{}
+ added := map[string]struct{}{}
for _, file := range files {
- set.File = append(set.File, protoutil.ProtoFromFileDescriptor(file))
+ addToSet(file, set, added)
}
bytes, _ := proto.Marshal(set)
Thanks a lot!
When I using "parser.ParseFiles(protos...)", the filename in protos slice will always be concatenated with ImportPath. But I think import path is only related to the import proto in proto file. So how can I avoid this?
When I using "parser.ParseFiles(protos...)"
parser.ParseFiles
is not in this repo. This question really belongs on the repo where that API lives: https://github.com/jhump/protoreflect/issues
FWIW, the behavior you are describing does not sound right. Neither this package, nor github.com/jhump/protoreflect/desc/protoparse
include the import path in the file name. If you are seeing something different, you may have found a bug -- but in protoparse
, not here. In that case, please file a bug in the other repo and please include a small repro case that demonstrates the issue.