link to stackshare from industrial users
Opened this issue · 23 comments
In addition to the locally maintained lists, stackshare has a list of companies using ocaml
Consider linking to that website from the top or bottom of the industrial users page.
The API might also expose the data directly. If that is the case, one could get the names and urls for the companies for now, and create a follow up issue to generate a page with that data.
I would like to work on this.
@patricoferris @pitag-ha, is this a medium-level issue? Shall I start working on this ??
@Bhavika-Gianey , I haven't had a look at the stackshare API yet, but you can have a look and see if the following is possible
The API might also expose the data directly. If that is the case, one could get the names and urls for the companies
I think that would be a medium issue. Let us know :)
@Bhavika-Gianey , I haven't had a look at the stackshare API yet, but you can have a look and see if the following is possible
Hii @pitag-ha , stackshre has a GraphQL API, I have never used that before but I will try to implement it and let you know in case of any difficulties :).
Perfect @Bhavika-Gianey! :) Do you know if using the stackshare API is for free?
Perfect @Bhavika-Gianey! :) Do you know if using the stackshare API is for free?
I was going to discuss this with you @pitag-ha. It's free for the first 2 months allowing 100 API calls. And after that, there are monthly plans.
What do you think ?? Shall we proceed using this?
Thanks for investigating that, @Bhavika-Gianey! I think that should be ok, yes :) I don't think we would pay the monthly plan, but I'm hoping that 100 API calls will be enough to get the current list of names and urls of companies using OCaml (we simply wouldn't keep it updated).
I spent a few minutes trying to find this data in the API and I didn't find it immediately. It may or may not exist there, so I would investigate that first and then come up with an implementation approach after finishing that research.
I spent a few minutes trying to find this data in the API and I didn't find it immediately. It may or may not exist there, so I would investigate that first and then come up with an implementation approach after finishing that research.
@pitag-ha @kanishka-work , I have gone through the stackshare data and extracted the list .
Shall I proceed using this data??
Also, I have one doubt, the current hard-coded companies page already has a lot of companies listed and in the stackshare API, there is no such data of the image or logo and description of the respective companies. It only contains names and related GitHub repos of the companies using Ocaml,
So, shall I have to make a new page and delete the existing page or if I have to render data on the existing page, then do I need to change the layout of that page as I can't see image URLs and descriptions from the current API data.
This is the existing companies page for reference:
@pitag-ha Perhaps this work can aim to produce a script that generates a yaml file with company name, stackshare url, company website, and a few other attributes, and stop there for now?
I'm sorry, I've totally missed this somehow.
I have gone through the stackshare data and extracted the list .
Thanks! That's great :)
Perhaps this work can aim to produce a script that generates a yaml file with company name, stackshare url, company website, and a few other attributes, and stop there for now?
That sounds like a good idea to me! @Bhavika-Gianey , do you want to do that? @kanishka-work, would you include that script or that yaml file somewhere into the ocaml.org code base?
I would include the script only and add documentation in the script for how to run it. We can rename this issue as "create script to fetch company data from stackshare api" and create another issue for follow-up steps as we get time to think of more concrete requirements.
I'm sorry, I've totally missed this somehow.
I have gone through the stackshare data and extracted the list .
Thanks! That's great :)
Perhaps this work can aim to produce a script that generates a yaml file with company name, stackshare url, company website, and a few other attributes, and stop there for now?
That sounds like a good idea to me! @Bhavika-Gianey, do you want to do that? @kanishka-work, would you include that script or that YAML file somewhere into the ocaml.org code base?
@pitag-ha, Yes I would like to proceed further.
I want to clarify the issue as per my understanding, so,
- I need to extract the company name, stack share URL, company website, etc from the API, etc of all the companies using Ocaml, so I need to fetch data simultaneously from two endpoints, one which gives the company name and from the other endpoint the corresponding URL and other resources of those companies ??
- The data will then be stored in the YAML format.
I am still a beginner in Ocaml, I want to ask which modules or tools in Ocaml are used for HTTP requests and API calls in grphQl and parsing the data in YAML ??
I have till now seen that I have to use the Lwt, Cohttp, and Cohttp_lwt_unix libraries.
Hi @Bhavika-Gianey, yes, what you've understood is right :) If you want, as a first step you can generate a YAML containing the list of all companies using OCaml and, if there's time, later improve that by adding also the information about the company to it.
About your question which libraries to use: I've used curly for HTTP requests in OCaml. If you're familiar with the command line tool curl, that's easy to use. If I remember correctly, it doesn't use Lwt, which also makes it simpler.
I'm not sure what to recommend for graphql and YAML myself. I'm currently asking around. I will come back to you on that. or @kanishka-work, do you happen to know what to recommend?
@pitag-ha I don't see any polished graphql clients in ocaml. Maybe this task can drop down a level of abstraction and and use an http rest client library with this method https://graphql.org/learn/serving-over-http/ ? For yaml, I just skimmed opam and I see https://opam.ocaml.org/packages/yaml/
Thanks @kanishka-work, both ideas sound great to me!
Also, @Bhavika-Gianey, I've just talked to @patricoferris and he has mentioned that using cohttp as you've suggested (instead of curly as I've suggested) is a good idea since the current page is already using cohttp.
Hi @Bhavika-Gianey,
I think for simplicities sake we should narrow the initial scope of this:
- You can just Cohttp for the requests as we already use that for the RSS feeds, have a look at: https://github.com/ocaml/ocaml.org/blob/master/script/http.ml#L32. For GraphQL you just need to put the GraphQL in the body of the request (I made a small example to help) you can also have a look at the slightly more complicated approach used by "Coq Bot" https://github.com/coq/bot/blob/master/bot-components/GitHub_GraphQL.ml which uses a PPX to get "typed" queries based on a schema. This would be neat, but it's not required right now.
- If you could get the content just to JSON for now, that would be a great first step (the conversion to Yaml after that will probably be trivial as Ezjsonm shares the same basic type as yaml).
- Yes you will have to use Lwt if you use
Cohttp_lwt, this is just a promise library, but for this script (like the example I sent) it will be probably self-contained into one executable.
Re:two endpoints, which ones are you talking about. I don't know the stackshare API but if it is all GraphQL then usually there's a single endpoint that you can send one big query too.
Hopefully this helps narrow it down a little and focus the scope of this initial prototype. Be sure to ask questions as I'm sure you have some. Thanks :))
Thank You @pitag-ha, @patricoferris, @kanishka-work :) , Yesterday my whole family tested Covid positive , so I will not be able to give full time for a few days but I will ty to complete work on this issue as much as possible .
Hello, @patricoferris,
I tried running the sample code on the stackshare API; I am facing an error message in the final response and I am not able to debug the error. Can you please look into this ;
#use "topfind";;
#thread;;
#require "lwt";;
#require "cohttp-lwt-unix";;
#require "ezjsonm";;
open Lwt
open Cohttp
open Cohttp_lwt_unix
open Printf
let uri = "https://api.stackshare.io/graphql"
let key = "hmTBPOT6qizPthv9BVFjFg"
let query = {| query {
leads(usingToolSlugs: ["ocaml"], toolMatch: "any",
after:""){
edges{
node{
companyId
companyName
domain
}
}
}
}|}
let send_graphql_query query =
let open Lwt.Infix in
let headers =
Cohttp.Header.of_list ["Content-type", "application/json"; "Accept", "application/json"; "x-api-key" , key]
in
let uri = Uri.of_string uri in
let body =
O [("query", String query)]
in
let serialized_body = Ezjsonm.to_string body in
Cohttp_lwt_unix.Client.post ~headers ~body:(`String serialized_body) uri >>= fun (_resp, body) ->
Cohttp_lwt.Body.to_string body >|= fun s ->
print_endline s
let () =
Lwt_main.run (send_graphql_query query)
The error msg is:
Also, what is this piece of code doing ??
i.e . Why are we using Ezjsonm.to_string body if the above expression is already converting the query to string form.
You might find it easier to ask questions and get help by creating a Draft Pull Request and asking questions using comments on the Pull Request.
You might find it easier to ask questions and get help by creating a Draft Pull Request and asking questions using comments on the Pull Request.
Thanks, I will make one.