Can Context generation be automated using POCO classes with attributes?
Closed this issue · 21 comments
I am just thinking loud now, but to me it seems that context can be generated from plain C# classes annotated with custom attributes. Is this something that can be useful for this library?
We've thought about that too, but I'm not thrilled about the idea. It feels like we'd be turning this into a serialization library, which is an anti-goal for me.
I agree this functionality doesn't belong to core JSON-LD, but such needs may often arise in projects where JSON-LD is used to expose domain entities.
I actually started such wrtiting a library to support POCO serialization in our project, if it goes well I can turn it into a JSON-LD contrib extension.
BTW I was wondering why JSON-LD.Net library originated inside NuGet team. What was the business motivation for you to start this work? Since you consider POCO serialization as an anti-goal for this project, what is the goal for you?
I fully agree with @jeffhandley, it doesn't belong into this library... but it would be very useful to have a separate specialized library. I've implemented a prototype (in PHP) offering that functionality to simplify the creation of Hydra-powered APIs. With json-ld.net and @object's serialization library it would be quite trivial to port that to C#. Is that something you would be interested in? If so, you should consider joining the W3C Community Group working on Hydra, which is of course completely free.
This sounds interesting. I will have to validate that the scope of my current project gives me enough time to work on a general purpose library like this, but since I am already doing some bits and pieces for our internal development, it will definitely help checking out a prototype for PHP. I will also check Hydra community group.
Cool.. Don’t hesitate to ask either directly on the mailing list or me personally if you have any questions.
Thanks! I believe I still have some way to go to better understand JSON-LD, but it's very exciting concept.
Regarding potential contrib pieces. We also have some integration with dotNetRdf. Take a look at the code in this repo.
This piece allows us to translate from regular XML to regular JSON using the RDF model as the intermediary format. Breaking things down to RDF has some significant benefits, not least of which is that you can trivially merge documents (as long as you avoid blank nodes). The ability to translate and merge arbitrary data in either XML or JSON is very useful. This is a component part of an architecture that puts more emphasis on cloud storage technology and less reliance on the relational database. Cloud storage (like Azure Storage for example) has some very appealing characteristics (availability, performance, cost) especially when combined with other caching like CDN's. But less reliance on the database means that we need other technology to fill in the gaps.
Anyhow, I think it would be quite doable and definitely interesting to see some serialization. I don't think we have an immediate need for that on NuGet because we are becoming a little more data driven, so it's not a priority for us. It would be cool to see however, and might be useful one day.
@object As far as why this library was created within the NuGet organization...
The NuGet team is working on NuGet API v3, which will be a RESTful API to replace today's RPC-oriented API v2. We wanted to use JSON as our over-the-wire format, but with an emphasis on linked data. JSON-LD is a great fit for us. The problem was that there weren't any .NET libraries for processing JSON-LD, so we created one.
Whether or not this library stays under the NuGet organization long-term is certainly open for discussion. We anticipate that it would move out to a JSON-LD organization at some point, but for now it's a lot easier for us to build it here until its long-term home is identified.
John and Jeff.
Thank you for the explanation of what was the motivation for the project. We are in initial phase of using JSON-LD in our development, so it's very valuable to understand how others plan to use this, especially since it's NuGet team.
FYI, if anyone's interested I've just recently started a JSON-LD serialization library for .NET: https://github.com/wikibus/JsonLD.Entities. At the moment I probably won't need dynamically created @context for POCOs but it's definitely something I'd like to see.
I'd very much appreciate if you guys could have a look and give me some opinion about how you think it should work. The main readme shows some simple examples, which are a starting point for further features.
Oh and @object, regarding your earlier questions about json-ld.net origins, you will definitely like the recent presentation by @jeffhandley: http://vimeo.com/111285814
Thanks for the information, @tpluscode. This sounds very interesting. In the project I am participating we have written our own JSON-LD serializer, it is not open sourced unfortunately but I will have a look at your project, perhaps we can exchange some ideas. BTW in our serializer we create context dynamically.
I though you would have since March :). I'd love to hear about your experience so that I won't make same mistakes you may have made.
Just to give you an insight about what our serializer does, here's an example of our POCO classes:
[JsonLdContext(ContextId = "https://w3id.org/schema.org/address")]
class Address
{
[JsonLdContext(ContextId = "http://localhost/city")]
public string city { get; set; }
[JsonLdContext(ContextId = "http://localhost/postalcode")]
public string postalcode { get; set; }
[JsonLdContext(ContextId = "http://localhost/addresslines")]
public string[] addresslines { get; set; }
[JsonLdContext(ContextId = "http://localhost/confidential")]
public string confidential { get; set; }
}
class PersonWithAddressList
{
[JsonLdContext(ContextId = "https://w3id.org/schema.org/name")]
public string name { get; set; }
[JsonLdContext(ContextId = "https://w3id.org/schema.org/addresses", ContainerType = JsonLdContainerType.Set)]
public List<Address> addresses { get; set; }
}
Here's a test code:
var person = new PersonWithAddressList()
{
name = "John",
addresses = new List<Address>()
{
new Address { city = "Oslo", postalcode = "0123", addresslines = new []{"Postboks 51", "Storgata 2"}},
new Address { city = "Kolbotn", postalcode = "1410", addresslines = new []{"Skiveien 10"}},
}
};
And here's generated JSDON-LD compacted:
{
"addresses": [
{
"addresslines": [
"Postboks 51",
"Storgata 2"
],
"city": "Oslo",
"postalcode": "0123"
},
{
"addresslines": [
"Skiveien 10"
],
"city": "Kolbotn",
"postalcode": "1410"
}
],
"name": "John",
"@context": {
"name": "https://w3id.org/schema.org/name",
"addresses": {
"@id": "https://w3id.org/schema.org/addresses",
"@container": "@set"
},
"city": "http://localhost/city",
"postalcode": "http://localhost/postalcode",
"addresslines": {
"@id": "http://localhost/addresslines",
"@container": "@list"
},
"confidential": "http://localhost/confidential"
}
}
@tpluscode @object Did you reach some sort of conclusion on how to unite the work so we can all contribute to a single library with attributes we can decorate our serialization entities with?
I'd like to know where this is going also... The example of what @object showed us makes a lot of sense. Would be great to have one library that can use POCOs and rely on schema.org's data definitions. I'm thinking of taking this idea one step further for the masses by giving ANY user who doesn't know a thing about schema.org or JSON-LD a way of storing their content as raw json-ld (whether it be an event, product, etc.), storing it in a datastore (mongo, sqlserver, redis, etc.) and rendering to the browser the JSON-LD object as well as some corresponding "html helper" generated JavaScript code that would yield pre-formatted HTML text nuggets to be dynamically embed/rendered to containing DIVs into their webpages easily via JQuery and AJAX. I know that goes way beyond what json-ld provides, but the rendering of these final view code snippets is helpful for many I believe. This would ultimately help many with getting their products, events, etc. modeled correctly so that SEO can pick of the content and index their sites effectively.
You may be interested in the library I started - wikibus/JsonLD.Entities. It certainly shares some similarities with what @object illustrated above. There's a documentation link in the readme. Maybe I should emphasize it more...
For quick-and-dirty JSON-LD serialization there is a context helper, which simply applies a given namespace, like schema.org, to you object structure.
public class Person
{
public string GivenName { get; set; }
public string LastName { get; set; }
public static JObject Context { get; } => new VocabContext<Person>("http://schema.org");
}
When serialized, that would produce a JSON like
{
"@context": {
"givenName": "http://schema.org/givenName",
"lastName": "http://schema.org/lastName"
},
"givenName": "Tomasz",
"lastName": "Pluskiewicz"
}
For the HTML part you may like the idea that I'm experimenting with using Web Components.
The idea is that you can package HTML template for given data (for example for a matching @type
<!-- template for your data @type = schema:Person -->
<template is="ags-type-template"
type="http://schema.org/Person" as="person"
compact-with='{ "@vocab": "http://schema.org/" }'>
<!-- your-html goes here and supports data binding -->
<div>
{{person.givenName}}
</div>
</template>
<!-- this will render your data -->
<ags-view id="view"></ags-view>
<!-- to render, set the property on ags-view -->
<script>
document.getElementById('view').object = {
"http://schema.org/givenName": "Tomasz",
"http://schema.org/lastName": "Pluskiewicz"
};
</script>
You can see my blog posting for a more detailed example. Note though that the elements' names have changed since. The code is at wikibus/augeas. Unfortunately no docs at this point yet.
@markchipman Do these ideas somewhat resonate with your needs?
@tpluscode ... thanks for getting back to me.
Hmmm, I'm not quite sure how it might fit in. I want to investigate wikibus/JsonLD.Entities more, but as far as the html rendering goes, I'm not sure what I want to do. I tested the execution of the templating mechanism that you linked to in Plunker, and although I like the outcome, the performance was rather sluggish... and that was a small data-set... how does it perform with something rather complex and large?
What I am looking at is having a solution where the JSON-LD is auto-generated based off of the server-side POCO model.
I should regress abit and explain what I'm really going after... awhile ago I purchased the domain names usedotnetwithschema.org as well as usingdotnetwithschema.org with the intent of exposing the hierarchy of schema.org via POCOs and providing options of generating those models as pre-generated Microdata view snippets that could be used with the razor view engine.
When JSON-LD came to my attention as the evidently preferred way of expressing schema.org classes, properties, actions, etc my plans changed as I saw that its the suggested way of 1.) having web crawlers index site/web-service data and 2.) provide sophisticated useful options to yield better rendered views (Google has taken great strides in this -- i.e. Google Rich Snippets). I also saw how things like email can be enhanced now using JSON-LD to add auto-magically generated content links within email messages themselves... ie GMail. Cool stuff!!
Now, my goal is to provide a way to allow a .Net developer easily populate and render the POCO schema.org class models in any way they see fit... ie Microdata AND/OR JSON-LD. I consider this as really the "core" of an initial offering.
Then I want to take this one step further... which I alluded to in my first comment. This is specifically a direct JSON-LD data and JSON-LD enhanced HTML code snippet generation website/web-service that would act somewhat as a web-resource for ANY third party website to provide pre-rendered SEO (JSON-LD enhanced) friendly content for inclusion into their own websites. Something that 'any Joe' can use to have the tricky Microdata/JSON-LD stuff done for them and they just embed the specified content on-the-fly in their own webpages via AJAX,/JQuery direct RESTful calls from within script tags. using a templating engine like Handlebars or called within server-side rendering implementations.
Let me explain this a bit better... Imagine I built a publicly available multi-tenanted, secure, encrypted web-service; where numerous well-known "most common" schema.org representations of their data for specific things like users/products/events/etc. is securely stored in the cloud in some encrypted backing data store like Mongo, SQL Server or Redis, Consider that the customers of this site/web-service utilize it as the primary data store for things that they would want represented in their website like products, people, events... you know, the exact stuff that Schema.org models.
This wouldn't simply be another 'Shopify' kind of back-end but instead will be much broader; encompassing the scope of schema.org.
Customers in essence need to have their entity data managed and stored using familiar methods like a simple oAuth protected RESTful CRUD API, or also perhaps they wish to utilize corresponding easy-to-use forms to perform CRUD operations directly as well (in case they don't want to use an API). Each form would need to be templated as required to address all the properties of a comprehensive schema.org class models (ie: a set of individual products and their corresponding offers/variants of specific SKUs).
There would be lots of different schema.org classes and their respective Microdata/JSON-LD representations included as "standard" entities to choose from as data-stores to cover the most popular schema.org @type's used for various use-cases and industries.
I believe that naturally, these same customers also want an easy way to get the properly formatted HTML and JSON-LD back from this web-service so that all of their products etc would be greatly enhanced to take advantage of SEO features that would increase their sites' view-ability, information, statistics and more --directly benefiting in their inclusion in 'enhanced' presentation within crawler result pages such as Google, Bing, Yahoo, etc. Google uses JSON-LD as reference examples for Structured Data SEO for their Knowledge Graph (companies and people). There is a HUGE need for this!
As such the customer's websites could take advantage of things like Google's Rich Snippets that will leverage the JSON-LD data that the web-service yields.
So, to sum it up, I would like to provide a straight forward way for ANYONE to make RESTful API requests, routed by schema.org @type classes (i.e. individual-product), and get back either:
1.) Raw populated JSON-LD objects of any specified schema.org POCO model (which they can then do whatever they want with like embed as a link into their web pages
2.) Pre-built Microdata formatted HTML snippets of a laundry list of possible templated schema.org class @type models (and the properties)... this of coarse would include hierarchical embedded properties so the rendered Microdata enhance HTML might be 'quite complicated'.
I know that this was long-winded... but I hope it explains the general BIG idea a bit better. I wouldn't mind partnering up with some other folks like yourself to provide this as starter project to be eventually folded into a real commercial venture.
This library will now aim to be more-or-less equivalent to the JavaScript counterpart (https://github.com/digitalbazaar/jsonld.js/) which has become a pseudo-reference for implementations. An additional library to add POCO bindings might be a welcome addition.