Maxim-Mazurok/google-api-typings-generator

[docs] Add usage info to the README

dhakan opened this issue · 12 comments

dhakan commented

Hi! I'm sorry if this is written somewhere and I'm just not understanding it, but:

  • I'm looking for YouTube Data V3 typings for a project I'm building. I'm not actually using gapi, but I'd still like the types for the response payload
  • Since the type definitions within @types/gapi.client.*, in my case * being youtube-v3 is not available, I'm guessing I need to use this project to generate them somehow? I was thinking to download it as a dev dependency to my project, and then run the compile scripts documented to generate local type definitions. Since this package is not on NPM I suspect I'm thinking about this the wrong way. I was thinking that I could generate them on the fly on my CI server
  • The other approach is to clone this repository and create the typings separate from my project, and then include them in version control?
  • Are the built types published somewhere so that I can just use them instead?

Once again, sorry for my confusion. I'm quite new to the TypeScript world so would appreciate any guidance :)

Maxim-Mazurok commented

Hello Dannie, thanks for your interest in the project!

I'm not actually using gapi, but I'd still like the types for the response payload

Make sure you're writing a script for the browser and not for the server, as NodeJS API packages provided by google are shipped with types included, see https://github.com/Maxim-Mazurok/google-api-typings-generator#javascript-vs-nodejs-clients

type definitions within @types/gapi.client.*, in my case * being youtube-v3 is not available

They are available: https://www.npmjs.com/package/@types/gapi.client.youtube-v3
They are also generated, tested, updated, and published hourly :)

I can totally see how it may be confusing, I'll try to add a note about usage in the README, thanks for bringing that up!

dhakan commented

Hi Maxim, thanks for the reply!

It seems that after looking I do actually have access to gapi.client.youtube.* within my codebase. I think my struggle was that I was trying to import VideoListResponse, but this doesn't seem to be exported as an individual interface. This made me think that the types weren't visible. That assumption, on top of the fact that inside of the type definitions file of the project, I found

// Type definitions for non-npm package YouTube Data API v3 v3 0.0
// Project: https://developers.google.com/youtube/
// Definitions by: Maxim Mazurok <https://github.com/Maxim-Mazurok>
//                 Nick Amoscato <https://github.com/namoscato>
//                 Declan Vong <https://github.com/declanvong>
// Definitions: https://github.com/DefinitelyTyped/DefinitelyTyped

// Referenced type definitions are generated by https://github.com/Maxim-Mazurok/google-api-typings-generator
// In case of any problems please open issue in https://github.com/Maxim-Mazurok/google-api-typings-generator/issues/new

/// <reference types="@maxim_mazurok/gapi.client.youtube-v3" />

As per

Referenced type definitions are generated by https://github.com/Maxim-Mazurok/google-api-typings-generator

made me believe that I needed to build them myself. I can now see that what's being intended here is for the reader to understand that the contents of the referenced file is created via the typings generator project, NOT that I'm supposed to generate them.

I'm building a browser plugin, so I do believe that I'm using the right project.

Perhaps I'm just better off using gapi at runtime instead of trying to hack types into my fetch calls?

dhakan commented

For reference, this is the code I'm trying to apply these types to:

// helpers/api.ts

import settings from "../config/settings";

// gapi.client.youtube.videos.list

export async function getVideoMetadata(id: string) {
  const url = new URL(`${settings.YT_API_BASE_URL}/videos`);
  url.searchParams.append("id", id);
  url.searchParams.append("part", "snippet");
  url.searchParams.append("part", "contentDetails");
  url.searchParams.append("key", settings.YT_API_KEY);
  const response = await fetch(url.toString());
  return await response.json();
}

export async function getCommentThreads(id: string) {
  const url = new URL(`${settings.YT_API_BASE_URL}/commentThreads`);
  url.searchParams.append("videoId", id);
  url.searchParams.append("part", "snippet");
  url.searchParams.append("order", "relevance");
  // Activate this if we find videos where the original 20 don't cover it
  // url.searchParams.append("maxResults", 100);
  url.searchParams.append("key", settings.YT_API_KEY);
  const response = await fetch(url.toString());
  return await response.json();
}
Maxim-Mazurok commented

that the contents of the referenced file is created via the typings generator project, NOT that I'm supposed to generate them

Ah, I see what you mean, I'll try to clarify that as well.

Regarding your usage, I do recommend using gapi for that. That way you won't depend much on the internal naming of interfaces and only use the public API, getting all the auto-complete goodies, argument types validation, etc. Also, you'll avoid forcefully casting responses using as.

Please refer to the docs on how to use YouTube Data API with gapi. Here's a sample JS code from there:

<script src="https://apis.google.com/js/api.js"></script>
<script>
  /**
   * Sample JavaScript code for youtube.videos.list
   * See instructions for running APIs Explorer code samples locally:
   * https://developers.google.com/explorer-help/code-samples#javascript
   */

  function authenticate() {
    return gapi.auth2.getAuthInstance()
        .signIn({scope: "https://www.googleapis.com/auth/youtube.readonly"})
        .then(function() { console.log("Sign-in successful"); },
              function(err) { console.error("Error signing in", err); });
  }
  function loadClient() {
    gapi.client.setApiKey("YOUR_API_KEY");
    return gapi.client.load("https://www.googleapis.com/discovery/v1/apis/youtube/v3/rest")
        .then(function() { console.log("GAPI client loaded for API"); },
              function(err) { console.error("Error loading GAPI client for API", err); });
  }
  // Make sure the client is loaded and sign-in is complete before calling this method.
  function execute() {
    return gapi.client.youtube.videos.list({
      "part": [
        "snippet,contentDetails,statistics"
      ],
      "id": [
        "Ks-_Mh1QhMc"
      ]
    })
        .then(function(response) {
                // Handle the results here (response.result has the parsed body).
                console.log("Response", response);
              },
              function(err) { console.error("Execute error", err); });
  }
  gapi.load("client:auth2", function() {
    gapi.auth2.init({client_id: "YOUR_CLIENT_ID"});
  });
</script>
<button onclick="authenticate().then(loadClient)">authorize and load</button>
<button onclick="execute()">execute</button>
dhakan commented

I think it's just an unfortunate case of me not being that aware of how type definitions are included as dependencies :)

Your reasoning makes lots of sense. Thanks for the input. I'll consider switching to gapi, since what I'm doing ought to be very easy to refactor.

Thanks again for all of the help!

  • ❤️1
dhakan commented

Hi again! I've revisited this trying to make gapi work for my project. Unfortunately it seems that gapi does not support Chrome Extensions under Manifest V3, making the efforts of using this library futile.

Since figuring this out, what I've currently been trying to do is to refactor the types I found for this library in order to make it work for my use case, under a local youtube.d.ts project file.

Something I'm struggling with is that I found, for instance, a VideoListResource which returns one or several Video:

interface Video {
    /** Age restriction details related to a video. This data can only be retrieved by the video owner. */
    ageGating?: VideoAgeGating;
    /** The contentDetails object contains information about the video content, including the length of the video and its aspect ratio. */
    contentDetails?: VideoContentDetails;
    /** Etag of this resource. */
    etag?: string;
    /**
     * The fileDetails object encapsulates information about the video file that was uploaded to YouTube, including the file's resolution, duration, audio and video codecs, stream
     * bitrates, and more. This data can only be retrieved by the video owner.
     */
    fileDetails?: VideoFileDetails;
    /** The ID that YouTube uses to uniquely identify the video. */
    id?: string;
    /** Identifies what kind of resource this is. Value: the fixed string "youtube#video". */
    kind?: string;
    /**
     * The liveStreamingDetails object contains metadata about a live video broadcast. The object will only be present in a video resource if the video is an upcoming, live, or completed
     * live broadcast.
     */
    liveStreamingDetails?: VideoLiveStreamingDetails;
    /** The localizations object contains localized versions of the basic details about the video, such as its title and description. */
    localizations?: { [P in string]: VideoLocalization };
    /** The monetizationDetails object encapsulates information about the monetization status of the video. */
    monetizationDetails?: VideoMonetizationDetails;
    /** The player object contains information that you would use to play the video in an embedded player. */
    player?: VideoPlayer;
    /**
     * The processingDetails object encapsulates information about YouTube's progress in processing the uploaded video file. The properties in the object identify the current processing
     * status and an estimate of the time remaining until YouTube finishes processing the video. This part also indicates whether different types of data or content, such as file details
     * or thumbnail images, are available for the video. The processingProgress object is designed to be polled so that the video uploaded can track the progress that YouTube has made in
     * processing the uploaded video file. This data can only be retrieved by the video owner.
     */
    processingDetails?: VideoProcessingDetails;
    /**
     * The projectDetails object contains information about the project specific video metadata. b/157517979: This part was never populated after it was added. However, it sees non-zero
     * traffic because there is generated client code in the wild that refers to it [1]. We keep this field and do NOT remove it because otherwise V3 would return an error when this part
     * gets requested [2]. [1]
     * https://developers.google.com/resources/api-libraries/documentation/youtube/v3/csharp/latest/classGoogle_1_1Apis_1_1YouTube_1_1v3_1_1Data_1_1VideoProjectDetails.html [2]
     * http://google3/video/youtube/src/python/servers/data_api/common.py?l=1565-1569&rcl=344141677
     */
    projectDetails?: any;
    /** The recordingDetails object encapsulates information about the location, date and address where the video was recorded. */
    recordingDetails?: VideoRecordingDetails;
    /** The snippet object contains basic details about the video, such as its title, description, and category. */
    snippet?: VideoSnippet;
    /** The statistics object contains statistics about the video. */
    statistics?: VideoStatistics;
    /** The status object contains information about the video's uploading, processing, and privacy statuses. */
    status?: VideoStatus;
    /**
     * The suggestions object encapsulates suggestions that identify opportunities to improve the video quality or the metadata for the uploaded video. This data can only be retrieved by
     * the video owner.
     */
    suggestions?: VideoSuggestions;
    /** The topicDetails object encapsulates information about Freebase topics associated with the video. */
    topicDetails?: VideoTopicDetails;
  }

What surprises me is that every field is possibly undefined. If I look at the official Resource representation I can't seem to find anything that points to something like the id being undefined.

This might be a separate issue, but do you have any further guidance on how to approach defining types for my scenario? I'm not a fan of what I'm currently planning to do, but I'm not sure what else I can do except create these types myself?

Much thanks in advance again.

Maxim-Mazurok commented

Unfortunately it seems that gapi does not support Chrome Extensions under Manifest V3, making the efforts of using this library futile.

They say "Instead of remote code, we recommend the use of remote configuration files", so you could include gapi and client scripts into your extension as static files (I think they rarely change and probably do not break). Then the client script just downloads json schema (config) to generate all the API methods. The only challenge is to modify gapi code to fetch client from your local files instead of google. If you're interested in exploring this approach further, this might help: https://github.com/Maxim-Mazurok/gapi

What surprises me is that every field is possibly undefined. If I look at the official Resource representation I can't seem to find anything that points to something like the id being undefined.

Yeah, I feel your pain. I'm pretty sure that videos from VideoListResponse should all have id defined. However, if you look at how they structured their schema, they're using the same Video interface for responses and for requests. For example, take a look at the schema for inserting a video:

/*...*/
"videos": {
  "methods": {
    "insert": {
      "response": {
        "$ref": "Video"
      },
      "request": {
        "$ref": "Video"
      },
      "id": "youtube.videos.insert"
/*...*/

As you can see, it accepts the same Video interface for requests as the one being returned. Obviously, it doesn't make sense to pass id when creating a video as it's generated on the back-end side, so in order to accommodate this they had to make id optional. But now as a consequence id remains optional even for responses and this doesn't make sense.

In order for a parameter to be non-undefined it has to have "required": true property. For example, VideoResource.rate has id: string and rating: string because both of them have required: true in the schema:

/*...*/
"rate": {
  "path": "youtube/v3/videos/rate",
  "parameters": {
    "id": {
      "required": true,
      "type": "string",
      "location": "query"
    },
    "rating": {
      "required": true,
      "type": "string",
/*...*/

However if we take a look at the `id` from `Video`, we'll notice `"annotations"`:
```json5
/*...*/
"id": {
  "type": "string",
  "description": "The ID that YouTube uses to uniquely identify the video.",
  "annotations": {
    "required": ["youtube.videos.update"]
  }
},
/*...*/

We could probably get some use of it, but it'll probably require creating a separate interface for that method... Or using something like Video & Required<Pick<Video, "id">>. I'll create an issue to look into this, but it likely won't help in your case anyway because there are no annotations for list method.

I'm not a fan of what I'm currently planning to do, but I'm not sure what else I can do except create these types myself?

I see 3 possible approaches for you:

  1. If you're building a critical enterprise solution - only trust the types and actually handle the possibility of id being undefined even though we understand that it can't be undefined when you're listing existing videos.
  2. If you're only going to refer to the id once, like const id = await bla.bla.id - use non-null assertion, like const id: string = await bla.bla!.id.
  3. If you find yourself having to use non-null assertion too often - extend the existing interface, like VideoResponse = Video & Required<Pick<Video, "id">>. Since you can't use gapi you'll likely end up casting responses anyway, like const video = await fetch(...).data as VideoResponse or something like that.

Hope it helps!

dhakan commented

Thanks a lot @Maxim-Mazurok! you're doing gods work in helping me understand this jungle.

I'm not a fan of creating a fork of gapi where the config files are loaded. I'm thankful you suggested that approach though since I didn't think of it myself.

Regarding your three proposals:

  1. I understand this, and I've no reason to believe that the fields I'm reading are potentially undefined for existing videos.
  2. I did start using the non-null assertion as per your suggestion, but I started questioning why I even tried to use the type definitions since I'm opting out of the type safety anyways for several fields.
  3. Let me see if I can understand this clearly. If we go from the top of VideoListResponse and down we can play with this idea a bit (I've trimmed the interfaces to only include the fields I'm reading):
interface VideoListResponse {
    items?: Video[]; 
}

All of the following fields of Video are also declared as optional:

interface Video {
    id?: string;
    contentDetails?: VideoContentDetails;
    snippet?: VideoSnippet;
}

which goes deeper into:

interface VideoContentDetails {
    duration?: string;
}

and:

interface VideoSnippet {
    description?: string;
}

None of the fields above will probably ever be undefined based on my understanding of the API docs. Since I'm also utilising the comment threads API, I'll need to do a similar thing there.

I suppose I can extend these interfaces with Required and Pick like you mentioned, so that they become somewhat useful for my use case. I've never used these constructs in TS :)

How do I access the interfaces when extending though?

gapi.client.youtube.VideoListResponse // Property 'VideoListResponse' does not exist on type 'typeof youtube'.

One last thing is that I noticed that when I fetch Comment Threads for a video that has its comment section disabled, I get a response such like:

{
  "error": {
    "code": 403,
    "message": "The video identified by the <code><a href=\"/youtube/v3/docs/commentThreads/list#videoId\">videoId</a></code> parameter has disabled comments.",
    "errors": [
      {
        "message": "The video identified by the <code><a href=\"/youtube/v3/docs/commentThreads/list#videoId\">videoId</a></code> parameter has disabled comments.",
        "domain": "youtube.commentThread",
        "reason": "commentsDisabled",
        "location": "videoId",
        "locationType": "parameter"
      }
    ]
  }
}

This is not something that I could find an interface for. Is this handled by type definitions found elsewhere?

Maxim-Mazurok commented

I started questioning why I even tried to use the type definitions since I'm opting out of the type safety anyways

Well, you still get autocompletion which will potentially save you from typos and such, and also when making requests some fields will be required which also can help. Also, it will hopefully make it easier to refactor if the API changes, etc. The main idea is that you're just making existing properties non-nullable instead of allowing yourself to use any random properties of any random types.

For example, I can do:

const etag: string = ({} as gapi.client.youtube.VideoListResponse).etag!;

but I can't use non-existing property:

const etag: string = ({} as gapi.client.youtube.VideoListResponse).blabla!;

or assign it to a non-compatible type:

const etag: number = ({} as gapi.client.youtube.VideoListResponse).etag!;

Hopefully, you do see the value, but yeah, I can see how this experience is less than ideal...

This is not something that I could find an interface for. Is this handled by type definitions found elsewhere?

That's an excellent question, I didn't find any interfaces for errors either. I'll create an issue to look into that.

How do I access the interfaces when extending though?

Your code worked fine for me:
image

Are you sure you're using the latest @types/gapi.client.youtube-v3? If you try to Ctrl+Click on youtube proprty in VS Code - what does it resolve to? It should resolve to this:
image

I suppose I can extend these interfaces with Required and Pick like you mentioned, so that they become somewhat useful for my use case. I've never used these constructs in TS :)

This is what I had in mind, two approaches:

  1. Using default interfaces + non-null assertions: 
    const videoListResponse = {} as gapi.client.youtube.VideoListResponse;
const durations: string[] = videoListResponse.items!.map(
  (x) => x.contentDetails!.duration!
);
  2. Defining own VideoListResponse to eliminate need for non-null assertions: 
    type DeepRequired<T> = { [K in keyof T]: DeepRequired<T[K]> } & Required<T>; // credit: https://stackoverflow.com/questions/57835286/deep-recursive-requiredt-on-specific-properties#comment125977921_67833840
type VideoListResponse = DeepRequired<gapi.client.youtube.VideoListResponse>;
const videoListResponse = {} as VideoListResponse;
const durations: string[] = videoListResponse.items.map(
  (x) => x.contentDetails.duration
);

Hope this helps!

dhakan commented

Thanks a bunch. It does help!

Regarding the non-null assertions, I can see how it does help more than just leaving it to JS with no types :)

Regarding the non-accessible VideoListResponse, it seems that

gapi.client.youtube.VideoListResponse; // Property 'VideoListResponse' does not exist on type 'typeof youtube'.

vs

const listResponse: gapi.client.youtube.VideoListResponse = {};

makes a difference. I think it's because TypeScript is looking for runtime code in the first example? Like it's looking for a value and not a type.

Regarding

That's an excellent question, I didn't find any interfaces for errors either. I'll create an issue to look into that.

Thanks! I'll create my own interface in the meantime, since it's not too complex.

I'll soldier on and see how it goes. Big thanks again Maxim!

Maxim-Mazurok commented

I think it's because TypeScript is looking for runtime code in the first example? Like it's looking for a value and not a type.

Ah, yes, exactly :)

I'll create my own interface in the meantime

If you do - please share it in #806 - I might include it in the gapi.client if it's the same for all APIs, in any case, it'd be a good starting point, thank you!

Feel free to reopen if you'll have further related questions or open a new issue to make it easier for others to find exact answers without having to go through all the comments, cheers!

dhakan commented

I'll take all of this into account. I'll post whatever I come up with in that other thread. Thanks again @Maxim-Mazurok !

  • ❤️1