Pull Types For File

Question

Pull Types For File

jsfour opened this issue 2 years ago · 31 comments

We need to create a subclass of DependencyGraphParser here
https://github.com/octaviuslabs/walter/blob/main/src/dependency-graph-parser.ts#L49 where the build function pulls in the types (not the full code blocks) for the target file.

Answer 1 · 2023-03-21T18:39:24.000Z

@imwalterbot thoughts on how to go about this?

Answer 2 · 2023-03-21T18:39:26.000Z

@imwalterbot thoughts on how to go about this?

Processing this now

Answer 3 · 2023-03-21T18:39:26.000Z

@imwalterbot thoughts on how to go about this?

Queued for processing...

Answer 4 · 2023-03-21T18:40:06.000Z

Based on the provided context, you can create a subclass of DependencyGraphParser that overrides the build function to only pull in the types for the target file. Here's an example of how to create the subclass:

class TypeOnlyDependencyGraphParser extends DependencyGraphParser {
  constructor(gitHubFileUrl: string) {
    super(gitHubFileUrl);
  }

  public async build(): Promise<DepGraph<ts.SourceFile>> {
    Log.info(
      `Building type-only dependency graph for ${this.targetFile.url} excluding target`
    );
    const deps = await this.getDependencies(this.targetFile);

    for (const dep of deps.deps) {
      await this.processFileForTypes(dep);
    }
    return this.graph;
  }

  private async processFileForTypes(url: ParsedGitHubURL): Promise<void> {
    Log.info(`Processing file for types ${url.url}`);
    if (this.graph.hasNode(url.url)) {
      Log.info(`Node exists ${url.url}`);
      return;
    }

    const src = await this.getDependencies(url);
    const typeOnlySourceFile = this.extractTypeDeclarations(src.sourceFile);
    this.graph.addNode(url.url, typeOnlySourceFile);

    for (const dep of src.deps) {
      await this.processFileForTypes(dep);
      Log.info(`Adding edge ${src.url.url} -> ${dep.url}`);
      this.graph.addDependency(src.url.url, dep.url);
    }
  }

  private extractTypeDeclarations(sourceFile: ts.SourceFile): ts.SourceFile {
    // Pseudocode for extracting only type declarations from the source file
    // 1. Iterate through the nodes of the source file
    // 2. If the node is a type declaration, add it to a new source file
    // 3. Return the new source file containing only type declarations

    // Implement the actual logic for extracting type declarations here
  }
}

This subclass, TypeOnlyDependencyGraphParser, overrides the build function to use a new private function processFileForTypes which processes the file and adds only the type declarations to the dependency graph. The extractTypeDeclarations function is a placeholder for the logic to extract type declarations from the source file. You'll need to implement this logic based on your requirements.

Answer 5 · 2023-03-21T18:51:00.000Z

Could we use the typescript compiler in extractTypeDeclarations ?

Answer 6 · 2023-03-21T18:51:02.000Z

Could we use the typescript compiler in extractTypeDeclarations ?

Queued for processing...

Answer 7 · 2023-03-21T18:51:02.000Z

Could we use the typescript compiler in extractTypeDeclarations ?

Processing this now

Answer 8 · 2023-03-21T18:52:06.000Z

Yes, you can use the TypeScript compiler in the extractTypeDeclarations function to extract type declarations from the source file. Here's an example of how to do this:

private extractTypeDeclarations(sourceFile: ts.SourceFile): ts.SourceFile {
  const typeDeclarations: ts.Node[] = [];

  const visitNode = (node: ts.Node): void => {
    if (ts.isTypeAliasDeclaration(node) || ts.isInterfaceDeclaration(node) || ts.isEnumDeclaration(node)) {
      typeDeclarations.push(node);
    } else {
      ts.forEachChild(node, visitNode);
    }
  };

  ts.forEachChild(sourceFile, visitNode);

  // Create a new source file with only type declarations
  const typeDeclarationSourceFile = ts.updateSourceFileNode(sourceFile, typeDeclarations);

  return typeDeclarationSourceFile;
}

In this example, we define a visitNode function that checks if the current node is a type alias, interface, or enum declaration. If it is, we add it to the typeDeclarations array. We then use ts.forEachChild to recursively visit all nodes in the source file. Finally, we create a new source file with only the type declarations using ts.updateSourceFileNode.

You can then use this extractTypeDeclarations function in the TypeOnlyDependencyGraphParser subclass as described in the previous answer.

Answer 9 · 2023-03-21T18:53:42.000Z

What if we only waned the types that the gitHubFileUrl fiile depends on? I think you would need to modify getDependencies.

Answer 10 · 2023-03-21T18:53:43.000Z

What if we only waned the types that the gitHubFileUrl fiile depends on? I think you would need to modify getDependencies.

Queued for processing...

Answer 11 · 2023-03-21T18:53:44.000Z

What if we only waned the types that the gitHubFileUrl fiile depends on? I think you would need to modify getDependencies.

Processing this now

Answer 12 · 2023-03-21T18:54:33.000Z

If you want to modify the getDependencies function to only include the types that the gitHubFileUrl file depends on, you can do the following:

Traverse the sourceFile AST to find the import declarations.
For each import declaration, check if it imports a type.
If it imports a type, add it to the deps array.

Here's an example of how to modify the getDependencies function:

private async getDependencies(targetFile: ParsedGitHubURL): Promise<{
  sourceFile: ts.SourceFile;
  deps: ParsedGitHubURL[];
  url: ParsedGitHubURL;
}> {
  Log.info(`Getting dependencies for ${targetFile.url}`);
  const fileContent = await utils.getFileFromUrl(targetFile.url);

  const sourceFile = ts.createSourceFile(
    fileContent.parsedUrl.filePath,
    fileContent.body,
    ts.ScriptTarget.ES2015,
    true
  );

  const deps: ParsedGitHubURL[] = [];
  const branch = this.branch;

  sourceFile.forEachChild((node) => {
    if (ts.isImportDeclaration(node)) {
      const importPath = (node.moduleSpecifier as ts.StringLiteral).text;
      if (!importPath.startsWith(".") && !importPath.startsWith("/")) {
        Log.info(`Skipping import ${importPath}`);
        return;
      }

      // Check if the import declaration imports a type
      const importClause = node.importClause;
      if (importClause && importClause.namedBindings) {
        const namedBindings = importClause.namedBindings;
        if (ts.isNamedImports(namedBindings)) {
          for (const element of namedBindings.elements) {
            if (element.name && ts.isIdentifier(element.name)) {
              // If the imported element has a type keyword, add it to deps
              if (element.typeKeyword) {
                const imported = calculateAbsolutePath(
                  targetFile.filePath,
                  importPath + ".ts"
                );

                Log.info(`Adding import ${imported}`);
                deps.push(branch.urlForPath(imported));
              }
            }
          }
        }
      }
    }
  });

  return {
    sourceFile,
    deps,
    url: targetFile,
  };
}

This modification will ensure that the deps array only contains the types that the gitHubFileUrl file depends on. Note that this assumes the imports are using named imports. If there are other import styles, you might need to handle them accordingly.

Answer 13 · 2023-03-21T18:59:19.000Z

Ok summarize what we are looking to do.

Answer 14 · 2023-03-21T18:59:20.000Z

Ok summarize what we are looking to do.

Queued for processing...

Answer 15 · 2023-03-21T18:59:22.000Z

Ok summarize what we are looking to do.

Processing this now

Answer 16 · 2023-03-21T19:00:16.000Z

Based on the provided DependencyGraphParser class, you want to create a subclass that only includes the types that the gitHubFileUrl file depends on. To achieve this, you need to modify the getDependencies function to only include type dependencies, and then use this modified function in the subclass.

Here's a summary of the steps:

Create a subclass of DependencyGraphParser called TypeDependencyGraphParser.
In the subclass, override the build function to use a new private function processFileForTypes which processes the file and adds only the type dependencies to the dependency graph.
Modify the getDependencies function to only include the types that the gitHubFileUrl file depends on. This can be done by checking if the import declarations in the source file import a type, and if they do, add it to the deps array.

Here's an example of the TypeDependencyGraphParser subclass:

class TypeDependencyGraphParser extends DependencyGraphParser {
  constructor(gitHubFileUrl: string) {
    super(gitHubFileUrl);
  }

  public async build(): Promise<DepGraph<ts.SourceFile>> {
    Log.info(
      `Building type dependency graph for ${this.targetFile.url} excluding target`
    );
    const deps = await this.getDependencies(this.targetFile);

    for (const dep of deps.deps) {
      await this.processFileForTypes(dep);
    }
    return this.graph;
  }

  private async processFileForTypes(url: ParsedGitHubURL): Promise<void> {
    Log.info(`Processing file for types ${url.url}`);
    if (this.graph.hasNode(url.url)) {
      Log.info(`Node exists ${url.url}`);
      return;
    }

    const src = await this.getDependencies(url);
    this.graph.addNode(url.url, src.sourceFile);

    for (const dep of src.deps) {
      await this.processFileForTypes(dep);
      Log.info(`Adding edge ${src.url.url} -> ${dep.url}`);
      this.graph.addDependency(src.url.url, dep.url);
    }
  }

  // Modify the getDependencies function to only include type dependencies
  private async getDependencies(targetFile: ParsedGitHubURL): Promise<{
    sourceFile: ts.SourceFile;
    deps: ParsedGitHubURL[];
    url: ParsedGitHubURL;
  }> {
    // (Include the modified getDependencies function from the previous answer here)
  }
}

This subclass, TypeDependencyGraphParser, overrides the build function to use the new private function processFileForTypes which processes the file and adds only the type dependencies to the dependency graph. The modified getDependencies function ensures that the deps array only contains the types that the gitHubFileUrl file depends on.

Answer 17 · 2023-03-21T19:21:33.000Z

Seems like you can use the existing getDependencies right?

Answer 18 · 2023-03-21T19:21:34.000Z

Seems like you can use the existing getDependencies right?

Queued for processing...

Answer 19 · 2023-03-21T19:21:35.000Z

Seems like you can use the existing getDependencies right?

Processing this now

Answer 20 · 2023-03-21T19:22:13.000Z

Yes, you're right. You can use the existing getDependencies function and create a new subclass TypeDependencyGraphParser that extends DependencyGraphParser. Override the build function in the subclass to process files for types and build a dependency graph containing only the type dependencies.

Here's an example of the TypeDependencyGraphParser subclass:

class TypeDependencyGraphParser extends DependencyGraphParser {
  constructor(gitHubFileUrl: string) {
    super(gitHubFileUrl);
  }

  public async build(): Promise<DepGraph<ts.SourceFile>> {
    Log.info(
      `Building type dependency graph for ${this.targetFile.url} excluding target`
    );
    const deps = await this.getDependencies(this.targetFile);

    for (const dep of deps.deps) {
      await this.processFileForTypes(dep);
    }
    return this.graph;
  }

  private async processFileForTypes(url: ParsedGitHubURL): Promise<void> {
    Log.info(`Processing file for types ${url.url}`);
    if (this.graph.hasNode(url.url)) {
      Log.info(`Node exists ${url.url}`);
      return;
    }

    const src = await this.getDependencies(url);
    const typeOnlySourceFile = this.extractTypeDeclarations(src.sourceFile);
    this.graph.addNode(url.url, typeOnlySourceFile);

    for (const dep of src.deps) {
      await this.processFileForTypes(dep);
      Log.info(`Adding edge ${src.url.url} -> ${dep.url}`);
      this.graph.addDependency(src.url.url, dep.url);
    }
  }

  private extractTypeDeclarations(sourceFile: ts.SourceFile): ts.SourceFile {
    // (Include the extractTypeDeclarations function from the previous answer here)
  }
}

This subclass, TypeDependencyGraphParser, overrides the build function to use the new private function processFileForTypes which processes the file and adds only the type dependencies to the dependency graph. The extractTypeDeclarations function will extract type declarations from the source file, which can be added to the graph.

Answer 21 · 2023-03-21T19:29:06.000Z

@imwalterbot APPROVED for https://github.com/octaviuslabs/walter/blob/main/src/dependency-graph-parser.ts changes

Answer 22 · 2023-03-21T19:29:07.000Z

@imwalterbot APPROVED for https://github.com/octaviuslabs/walter/blob/main/src/dependency-graph-parser.ts changes

Queued for processing...

Answer 23 · 2023-03-21T19:29:08.000Z

@imwalterbot APPROVED for https://github.com/octaviuslabs/walter/blob/main/src/dependency-graph-parser.ts changes

Processing this now

Answer 24 · 2023-03-21T19:36:08.000Z

Will extractTypeDeclarations also include function declarations? I'm trying to build a type hint system and it would be good to have those as well.

Answer 25 · 2023-03-21T19:36:09.000Z

Will extractTypeDeclarations also include function declarations? I'm trying to build a type hint system and it would be good to have those as well.

Queued for processing...

Answer 26 · 2023-03-21T19:36:09.000Z

Will extractTypeDeclarations also include function declarations? I'm trying to build a type hint system and it would be good to have those as well.

Processing this now

Answer 27 · 2023-03-21T19:36:33.000Z

To include function declarations in the extractTypeDeclarations function, you can modify the function to also check for function declarations while traversing the sourceFile AST. Here's an updated version of the extractTypeDeclarations function that includes function declarations:

private extractTypeDeclarations(sourceFile: ts.SourceFile): ts.SourceFile {
  const typeDeclarations: ts.Node[] = [];

  const visitNode = (node: ts.Node): void => {
    if (
      ts.isTypeAliasDeclaration(node) ||
      ts.isInterfaceDeclaration(node) ||
      ts.isEnumDeclaration(node) ||
      ts.isFunctionDeclaration(node)
    ) {
      typeDeclarations.push(node);
    } else {
      ts.forEachChild(node, visitNode);
    }
  };

  ts.forEachChild(sourceFile, visitNode);

  // Create a new source file with only type declarations and function declarations
  const typeDeclarationSourceFile = ts.updateSourceFileNode(sourceFile, typeDeclarations);

  return typeDeclarationSourceFile;
}

In this updated version, we've added ts.isFunctionDeclaration(node) to the condition in the visitNode function. This will include function declarations along with type declarations when building the type hint system.

Answer 28 · 2023-03-22T04:28:40.000Z

@imwalterbot given the above how would we output the only the definitions (not the full function) in the toString method on the class?

Answer 29 · 2023-03-22T04:28:41.000Z

@imwalterbot given the above how would we output the only the definitions (not the full function) in the toString method on the class?

Queued for processing...

Answer 30 · 2023-03-22T04:28:42.000Z

@imwalterbot given the above how would we output the only the definitions (not the full function) in the toString method on the class?

Processing this now

Answer 31 · 2023-03-22T04:29:15.000Z

To output only the definitions (and not the full function) in the toString method, you can modify the toStrings method to print only the type and function signatures instead of the entire source file. You can achieve this by using a custom printing function that extracts the type and function signatures from the source file.

Here's an example of a custom printing function called printTypeAndFunctionSignatures:

private printTypeAndFunctionSignatures(sourceFile: ts.SourceFile): string {
  const printer = ts.createPrinter();
  let signatures: string[] = [];

  const visitNode = (node: ts.Node): void => {
    if (
      ts.isTypeAliasDeclaration(node) ||
      ts.isInterfaceDeclaration(node) ||
      ts.isEnumDeclaration(node) ||
      ts.isFunctionDeclaration(node)
    ) {
      signatures.push(printer.printNode(ts.EmitHint.Unspecified, node, sourceFile));
    } else {
      ts.forEachChild(node, visitNode);
    }
  };

  ts.forEachChild(sourceFile, visitNode);

  return signatures.join("\n");
}

Now, modify the toStrings method to use this custom printing function:

public toStrings(): string[] {
  const order = this.graph.overallOrder();

  let out: string[] = [];
  for (const node of order) {
    const data = this.graph.getNodeData(node);
    out.push(`// SOURCE_FILE: ${node}`);
    out.push(this.printTypeAndFunctionSignatures(data));
    out.push(`// END: ${node}`);
  }
  return out;
}

With these changes, the toString method will output only the type and function signatures, which can be useful for building a type hint system.