Add Generative Language API
Closed this issue · 8 comments
Request
Generative AI support on Vertex AI is now available in (GA).
https://cloud.google.com/vertex-ai/docs/generative-ai/release-notes
@kevmoo Can we generate this API client and add it to the package? I dont see any Vertex-AI support in this package. I assume that this API has discovery endpoints as well that can be used for code generation?
API
I know this is new and constantly changing right now, but its quite confusing. Im not sure where this API actually lives. It looks like there is a PaLM API as well as the same capabilities in Vertex-AI that seemingly use the same models and do the same things, albeit the response schema is different. This package is probably only interested in vertex-ai/generative-ai
See two examples below:
vertex-ai/generative-ai
https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings
POST https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/textembedding-gecko:predict
generative-ai
https://developers.generativeai.google/api/rest/generativelanguage/models/embedText
POST https://generativelanguage.googleapis.com/v1beta2/models/embedding-gecko-001:embedText?key=$PALM_API_KEY
The vertex-ai/generative-ai can be used via the typical gcloud auth access token.
Whereas generativeai link uses a PALM_API_KEY, different from every other GCP API. The PALM_API_KEY needs to be requested through the trusted developer program.
Sadly, I can't do anything unless we have a discovery API endpoint. 🤷
It seems Vertex AI now has a discovery API endpoint:
https://aiplatform.googleapis.com/$discovery/rest
Looking now!
That was fast!
Works like a charm 🚀
final predictApi = AiplatformApi(
client,
rootUrl: 'https://us-central1-aiplatform.googleapis.com/',
).projects.locations.publishers.models.predict;
final result = await predictApi(
GoogleCloudAiplatformV1PredictRequest(
instances: [
{'prompt': 'How cool is googleapis Dart package?'},
],
parameters: {'temperature': 0.5, 'maxOutputTokens': 256},
),
'projects/my-project/locations/us-central1/publishers/google/models/text-bison@001',
);
final prediction = result.predictions?.first as Map<String, dynamic>?;
print(prediction?['content']);
// The googleapis Dart package is a really cool package that provides access to Google APIs.
// It's easy to use and has a lot of features, making it a great choice for developers who need to work with Google APIs.@davidmigloz I wish the API surface was "prettier". The projects.locations.publishers.models bit is kinda crazy. This is what we get for having an auto-generated API. Glad it works!
Yeah.. it is not very pretty nor provides type-safety, but it's a good starting point. I'll provide a nice wrapper around it in LangChain.dart 🙂