jin-qu/jinqu-odata

Consider code generation from OData metadata

umutozel opened this issue · 14 comments

umutozel commented

Code generation from OData Metadata would be great.
OpenAPI/Swagger could be next step.

solenya-group commented

This will gen classes (could easily be extended):
https://stackblitz.com/edit/odata?file=convert.ts

I'm wondering whether requiring additional framework(s) for this translation is overkill. Nowadays if something doesn't pass the "can I whip up a prototype in 5 minutes on stackblitz test" I start getting skeptical of the approach. Having said that obviously I realize the benefits of a more declarative approach.

umutozel commented

Looks great.
Maybe code generation could be in another package(s), using every package that make translation easier. Could be only a dev-dependency, we all ignore how many packages dev-dependencies use :)

umutozel commented

Also, with your Decorator suggestion in use, we wouldn't need Service generation (I always thought they would be necessary).

@jinquEndpoint('companies')
export class Company {
}

const service = new ODataService(url);
const companies = await service.createQuery(Company).toArrayAsync();

looks great and feels right.

Btw, I completed jinqu development for proto fixing. I'm using existing cast method. Working on jinqu-odata now. With above approach, we can automatically call cast for the query before returning.

umutozel commented

I created a very simple code generator and considering to publish it alongside npm as a bin script.

solenya-group commented

I'm not actually familiar with best strategies for distributing code generators on npm, and how to those would potentially be wired automatically into the build process. Some very rough thoughts anyway...

In terms of a user of such a process, if my schema was fairly stable I'd probably prefer a one-off web tool w/ a UI to tweak the generator output, as I can avoid the steps/headache of installing, configuring, learning, running the tool. OTOH, if I was simultaneously developing a schema while developing the client, automatically re-generating the client classes could potentially be a time-saver. OTOH again, we don't have partial classes in javascript, so if I've customized those classes in any way, we're dealing with a one-time generation step. Also, part of the benefit of generation is that it teaches you how to build the types. Once you know how, it's easy to add a new field etc.

umutozel commented

Yeah, these were exact same scenarios in my mind.

By the way, to avoid unnecessary version updates to jinqu-odata, I decided to create a new npm for metadata generation, likely named 'jinqu-odata-metadata' or something like that.

NetTecture commented

Not really. The main benefit is that you can regenerate in the build process or as separate EASY step. If multiple teams actively develop an API, then there are times it is really tricky to keep all clients in sync. DTO - and odata classes are pure data transfer objects - should not really require customization. The benfit outweights the possible costs.

solenya-group commented

Hi NetTecture,

Agree it's nice as a build step!

Umut - something that occurred to me regarding not having partial classes in Typescript... Assuming in a particular scenario you can't or don't want to use inheritance, then mixins can be used to customize the generated classes. So you implement (rather than extend) the generated class with your customized class, then copy the prototype across from the generated class to the customized class. Explained here:

https://www.typescriptlang.org/docs/handbook/mixins.html

NetTecture commented

There is also more to it than just entities. What about actions and functions? We use both extensively. I would love a decent typescript based odata client - all around seem to be VERY partial solutions.

solenya-group commented

Well Umut (who's the project owner) can give you more info, but the work so far on jinqu has involved a lot of foundational stuff. One thing I appreciate about what jinqu is striving for, is to provide a consistent experience for LINQ queries across multiple providers, where odata is one of several possible providers. It's quite an undertaking, and you're right that there's more work ahead, but the approach looks pretty solid to me. Umut has been really responsive to feedback I've given so far - if you can spare a little time it would probably be really helpful if you can actually poke as many holes in the solution so far as possible, to help define the road map.

umutozel commented

Hi @NetTecture, first of all, thanks for the feedbacks.

I pushed a new repo (jinqu-odata-cli), still experimenting with the idea. With this -dev dependency- approach we can use it in the build process.

Also, I'm trying out generation options, for now we have "generate interfaces instead of classes", "use decorators" etc. We can always add a new option for service type generation and with this enabled, we can create a class with actions and functions. In fact, this was the approach I took for my company's one of internal framework(ish) project. Having concrete service types for REST API's make life really easier for developers. Later on, we switched to Swagger (it was for the best, I didn't like to use reflection on REST Controllers).

I'm still updating my old OData knowledge with v4, now reading this. I'm sure we can implement them somehow.

@solenya-group, thanks for your support and kind words. Mixins would fill the gaps, I've used them for global injections, I'm sure there wil be lots of useful scenarios for jinqu.

I always missed a real LINQ counterpart for TypeScript (and JavaScript), I am trying my best to keep all API LINQ way. Last few weeks I was a little busy, hope this will change soon, there's still a lot to do.

NetTecture commented

@umutozel
Actually i have spent the last day implementing a base bones odata client generator in dotnet - we do not use node as main platform, and a .net tool has many advantages:

  • Integratble into our build chain
  • it is based on a config file, so I can generate client and proxy classes for multiple odata endpoints (all in one project - and yes, we pan using about 3-4 diferenet ones in the same client) with one command
  • .NET ;) WHich means odata backend which means microsoft edm libraries which means I have a validatable object model instead of dealing with the raw xml files ;) There is something to say about calling ONE command and all proxies are regenerated ;)

We now need to nail down which client to use in the generator and there the mess starts. I like jinqu, but it is extremely limited. Particualrly missing for us are:

  • Inherited entities
  • Actions and Functions, bound and unbound (and yes, we have a LOT of actions)
  • Contained entities (i.e. /invoice(2233)/Line(2) - which is an entity without entity set.
  • Post, ut, patch? Documentation only mentions select (get), which is like read only.

Given taht whatever we go with needs to replace existing messy solutions NOW... you get the idea. I think optionally I would love to get away from all the generic style of odata and go towards an entity framework style - i.e. pull in ONE service and then ahve all the functions available on the right endpoints, possibly via some helpers. OData itself is very dynamic, but one endpoint in a specific time is not normally - the functions etc. are all predefined and in the metadata.

And that is just baseline. There are a lot of more detailed internals - as much as I love linq style queries, the functional limitations are right now a breaker because we use actions and functions extensively.

umutozel commented

@NetTecture
We have that in common, I LOVE .Net too :) Especially where C# is going and .Net Core.

I think we can support inherited entities and actions/functions too. We already support Post, Patch etc (via request options). But contained entities, that's tricky. But I believe we can handle it in generated code. I contributed to swagger code generator for linquest, they have some similar solution. We can use request method on ODataService class for non-query actions/functions (just thinking out loud).

To be honest, I would love to contribute a project like yours, maybe develop a jinqu implementation over a fork. I share your frustration and I started with a goal in mind, to solve this mess once and for all as best as I can. Yeah, this sentence always reminds me a Randall Munroe cartoon on xkcd.

Standards
xkcd - Standards

NetTecture commented

You name it.

Ok, lets straight.

  • Actions/Functions - definitely needed. We start having a lot of actions and functions in some API. Remember Functions support $filter and $expand ;) This is relevant because we do use them for compelx search. Building/Search('term')?$filter for additional filters. Return: Collection of buildings, and search cascades into a dozen of additional fields doing full text search - not one ;(
  • IContainment also needed. As well as entities without key (which is now valid assuming the entity is uniquely identified,. Example: We track events (think: event like eventlog, not event like party). Every event has a possible response (0:1). It is contained - there is no EventResponse URL to query them. As every event (key: guid) has zero or one responses, there is no response level key. You address them (i.e. pu publish them) via i.e.

POST to /Event({guidgoeshere})/Response

Now, always a good reference: Simple.Odata.Client:

something along the line

.For()
.Key({guidgoeshere})
.NavigateTo(x => x.Response)
.Set ({postdagoeshere})
..InsertEntryAsync()

  • Swagger- I do NOT like swagger for this. Seriously, all the information you can have and should have is in the metadata document, which in dotNET also has an EDM library which means objects. No need to use something like swagger, PARTICULARLY because a lot of things can not be expressed in swagger as the odata->visual mapping is not complete ;) Sorry to say. There are annotations going missing etc. IU am actually working on a backend library that moves attributes (many already existing) into standard (as per odata standard) annotations for properties and entities (indicating read only etc.) which then can be used on the generated side to i.e. filter out data, theoretically.

  • And let's do not get into batches. Like lets do not for now ;) I do not support them and server side support for them is TRICKY.

  • But what is also "needed" is handing return data outside of an array. Either by having the returned array castable to a Response object (which likely breaks down the moment you get related objects) or by returning the "raw" responses with a "GetObjects" function. See, there may be a LOT more in there than objects.

    • Annotations for download links for images i.e. (which can be named - this url for preview, this url for raw, this url for small and this url for large format)
    • Item count (total, not returned in case returned is limited) and links for next items, which may be in the root AND ANY EXPAND COLLECTION. The standard is extremely complicated on that, sadly. So, a way to access the raw returned collection for "one invoice line item" is needed (i.e. you ask invoice, expand lines, but a line may have x packing items and yes, those can be limited with a next items link).