Add Kina (www.kinaknowledge.com)

Question

Add Kina (www.kinaknowledge.com)

alexnygren opened this issue 3 years ago · 13 comments

We use Common Lisp extensively in our document processing software core for classification, extraction and other aspects of our service delivery and technology stack.

Answer 1 · 2021-10-13T16:53:59.000Z

Thank for the information. A few more words if you have time? I would share the answers on my blog (https://lisp-journey.gitlab.io/), reddit and it will certainly be on Twitter (Lisp Advocates).

What implementation(s) are you using?
What is not Lisp in your stack? For example, in "Kina extracts information from PDF, TIFFs, Excel, Word and more"
What's your position on open-source: do you use open-source Lisp libraries, do you (plan to) open-source some?
How did hiring and forming lisp or non-lisp developers go? Did you look for experienced lispers or did you seek experienced engineers, even with little to no prior Lisp background?
How many lispers are working together, how big a codebase to you manage?
Do you sometimes wish the CL (pro) world was more structured? (we have a CL Foundation but not so much active)
And so, how did you end up with CL?

ignore and add what you want, thanks!

Answer 2 · 2021-10-14T09:56:27.000Z

Hi Vindarel, These are great questions, and we plan on providing solid responses. This week is extremely busy for us, but I hope to have some time this weekend and early next week to prepare some responses. Through my career in software and banking operations, I have grown into a Common Lisp advocate, and more broadly a prefix notation language advocate. Our six year old company is in the computer vision space, and we use SBCL to implement our machine learning algorithms for document classification and information extraction. Specifically, we use a novel neural network algorithm implemented in Common Lisp to learn spatial features in a semantic way and more recently spatial-temporal pattern recognition and prediction. Our software has been licensed at the federal government level, as well as by small to large commercial organizations. Lisp is our secret weapon, both on the server, and in the browser, where we can rapidly compose user interface design using a Lisp-1 that compiles to JavaScript. Our UI is in a sense like Emacs in the browser where the interface is completely introspectable and can be modified quickly to handle new operational use cases. We plan on open sourcing the UI elements after some cleanup work. Expect more next week! By the way, love the Lisp Cookbook. Thank you! Thanks Alex Alex Nygren Founder and CEO Kina www.kinaknowledge.com<http://www.kinaknowledge.com>

…

________________________________ From: vindarel ***@***.***> Sent: Wednesday, October 13, 2021 12:54 PM To: azzamsa/awesome-lisp-companies ***@***.***> Cc: Alex Nygren ***@***.***>; Author ***@***.***> Subject: Re: [azzamsa/awesome-lisp-companies] Add Kina (www.kinaknowledge.com) (#66) Thank for the information. A few more words if you have time? I would share the answers on my blog (https://lisp-journey.gitlab.io/), reddit and it will certainly be on Twitter (Lisp Advocates). 1. What implementation(s) are you using? 2. What is not Lisp in your stack? For example, in "Kina extracts information from PDF, TIFFs, Excel, Word and more" 3. What's your position on open-source: do you use open-source Lisp libraries, do you (plan to) open-source some? 4. How did hiring and forming lisp or non-lisp developers go? Did you look for experienced lispers or did you seek experienced engineers, even with little to no prior Lisp background? 5. How many lispers are working together, how big a codebase to you manage? 6. Do you sometimes wish the CL (pro) world was more structured? (we have a CL Foundation but not so much active) 7. And so, how did you end up with CL? ignore and add what you want, thanks! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#66 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AG4XEL4TYPX5IQEAX3JF36TUGW2TDANCNFSM5FZDQGPA>.

Answer 3 · 2021-10-21T18:06:25.000Z

Thank for the information. A few more words if you have time? I would share the answers on my blog (https://lisp-journey.gitlab.io/), reddit and it will certainly be on Twitter (Lisp Advocates). What implementation(s) are you using? We use SBCL for all our Common Lisp processes. It's easier with the standardization on a single engine, but we also have gotten tied to it in some of our code base due to using the built in SBCL specific extensions. I would like, but have no bandwidth, to evaluate CCL as well, especially on the Windows platform, where SBCL is weakest. Since our clients use Windows systems attached to scanners, we need to be able to support it with a client runtime. Development is on MacOS with Emacs or Ubuntu with Emacs for CL, and then JetBrains IDEs for Ruby and JS and Visual Studio for some interface code to SAP and such. We develop the Kina UI in Kina itself using our internal Lisp, which provides a similar experience to Emacs/SLY. What is not Lisp in your stack? For example, in "Kina extracts information from PDF, TIFFs, Excel, Word and more". Presently we use a Rails/Ruby environment for driving our JSON based API, and some legacy web functions. However, increasingly, once the user is logged in, they are interacting with a Common Lisp back end via a web socket (Hunchentoot and Hunchensocket) interacting with a Lisp based front end. Depending on the type of information extraction, the system uses Javascript, Ruby and Common Lisp. Ideally, I'd like to get all the code refactored into a prefix notation, targeting Common Lisp or DLisp (what we call our internal Lisp that compiles into Javascript). What's your position on open-source: do you use open-source Lisp libraries, do you (plan to) open-source some? Yes. We recently put our JSON-LIB (https://github.com/KinaKnowledge/json-lib) out on Github, which is our internal JSON parser and encoder and we want to open source DLisp after some clean-up work. Architecturally, DLisp can run in the browser, or in sandboxed Deno containers on the server side, so we can reuse libraries easily. It's not dependent on a server-side component though to run. Library wise, we strictly try and limit how many third party (especially from the NPM ecosystem) libraries we are dependent on, especially in the Javascript world. In CL, we use the standard stuff like Alexandria, Hunchentoot, Bordeaux Threads, and things like zip. How did hiring and forming lisp or non-lisp developers go? Did you look for experienced lispers or did you seek experienced engineers, even with little to no prior Lisp background? Because we operate a lot in Latin America, I trained non-lisper engineers who speak Spanish on how to program Lisp, specifically our DLisp, since most customizations occur specifically for user interface and workflows around document centric processes, such as presenting linked documents and their data in specific ways. How the lisp way of thinking really depended on their aptitude with programming, and their English capabilities to understand me and the system. The user system is multilingual, but the development documentation is all in English. But it was really amazing when I saw folks who are experienced with Javascript and .Net get the ideas of Lisp and how compositional it can be as you build up towards a goal. Besides, with DLisp, you can on the fly construct a totally new UI interaction - live - in minutes and see changes in the running app without the dreadful recompile-and-reload everything cycle that is typical. Instead, just recompile the function (analogous to C-c, C-c in Emacs), in the browser, and see the change. Then these guys would go out and interact with clients and build stuff. I knew once I saw Spanish functions and little DSLs showing up in organizational instances that they were able to make progress. I think it is a good way to introduce people to Lisp concepts without having to deal with the overhead of learning Emacs at the same time. I pushed myself through that experience when I first was learning CL, and now use Emacs every day for a TON of work tasks, but at the beginning it was tough, and I had to intentionally practice getting to the muscle memory that is required to be truly productive in a tool. How many lispers are working together, how big a codebase to you manage? Right now, in our core company we have three people, two here in Virginia and one in Mexico City. We use partners that provide services such as scanning and client integration work. We are self-funded and have grown organically, which is freeing because we are not beholden to investor needs. We maintain maximum flexibility, at the expense of capital. Which is OK for us right now. Lisp allows us to scale dramatically and manage a large code base. I haven't line counted recently, but it exceeds 100K lines across server and client, with > 50% in Lisp. Do you sometimes wish the CL (pro) world was more structured? (we have a CL Foundation but not so much active). I really like the Common Lisp world. I would like it to be more popular, but at the same time, it is a differentiator for us. It is fast - our spatial classifier takes only milliseconds to come to a conclusion about a page (there is additional time prior to this step due to the OpenCV processing - but not too much) and identify it and doesn't require expensive hardware. Most of our instances run on ARM-64, which at least at AWS, is 30% or so cheaper than x86-64. The s-expression structures align to document structures nicely and allow a nice representation that doesn't lose fidelity to the original layouts and hierarchies. I am not as active as I would like to be in the Common Lisp community, mainly due to time and other commitments. I don't know much about the CL foundation. And so, how did you end up with CL? Our UI was first with the DLisp concepts. I was intrigued by Clojure for the server portion, but I couldn't come to terms with the JVM and the heavyweight of it. The server-side application was outgrowing the Rails architecture in terms of what we wanted to do with it, and, at the time, 4 years ago, Ruby was slower. In fact, Ruby had become a processing bottleneck for us (though I am certain the code could have been improved too). I liked the idea of distributing binary applications as well, which we needed to do in some instances, and building a binary runtime of the software was a great draw, too. I also liked how well CL is thought out, from a spec standpoint. It is stable both in terms of performance and change. I had been building components with TensorFlow and Python 3, but for what I wanted to do, I couldn't see how I could get there with back propagation and the traditional "lets calculate the entire network state". If you don't have access to high end graphic cards, it's just too slow and too heavy. I was able to get what we needed to do in CL after several iterations and dramatically improve speed and resource utilization. I am very happy with that outcome. We are in what I consider to be a hard problem space: we take analog representations of information, a lot of it being poor quality and convert it to clean, structured digital information. CL is the core of that for us. Here is an example of our UI, where extractions and classification can be managed. This is described in DLisp which interacts with a Common Lisp back end via a web socket. [cid:b38f0e5a-f032-4c84-aa90-9969273f39a9] Here is the function for the above view being edited in Kina itself. We do not obfuscate our client code, and all code that runs on our clients' computers is fully available to view and, with the right privileges, to modify and customize. You can see the Extract Instruction Language in the center pane, which takes ideas from the Logo language in terms of a cursor (aka the turtle) that can be moved around relative to the document. We build this software to be used by operations teams and having a description language that is understandable by non-programmers such as auditors and operations personnel, is very useful. You can redefine aspects of the view or running environment and the change can take effect on the fly. Beyond the Javascript boot scaffolding to get the system started up in the browser, everything is DLisp communicating with Common Lisp and, depending on the operation, Rails. [cid:166f2532-f850-4b59-9818-bc9762cbc941] I hope this information is helpful! Thanks, Alex

…

________________________________ From: Alex Nygren ***@***.***> Sent: Thursday, October 14, 2021 5:56 AM To: azzamsa/awesome-lisp-companies ***@***.***>; azzamsa/awesome-lisp-companies ***@***.***> Cc: Author ***@***.***> Subject: Re: [azzamsa/awesome-lisp-companies] Add Kina (www.kinaknowledge.com) (#66) Hi Vindarel, These are great questions, and we plan on providing solid responses. This week is extremely busy for us, but I hope to have some time this weekend and early next week to prepare some responses. Through my career in software and banking operations, I have grown into a Common Lisp advocate, and more broadly a prefix notation language advocate. Our six year old company is in the computer vision space, and we use SBCL to implement our machine learning algorithms for document classification and information extraction. Specifically, we use a novel neural network algorithm implemented in Common Lisp to learn spatial features in a semantic way and more recently spatial-temporal pattern recognition and prediction. Our software has been licensed at the federal government level, as well as by small to large commercial organizations. Lisp is our secret weapon, both on the server, and in the browser, where we can rapidly compose user interface design using a Lisp-1 that compiles to JavaScript. Our UI is in a sense like Emacs in the browser where the interface is completely introspectable and can be modified quickly to handle new operational use cases. We plan on open sourcing the UI elements after some cleanup work. Expect more next week! By the way, love the Lisp Cookbook. Thank you! Thanks Alex Alex Nygren Founder and CEO Kina www.kinaknowledge.com<http://www.kinaknowledge.com>

________________________________ From: vindarel ***@***.***> Sent: Wednesday, October 13, 2021 12:54 PM To: azzamsa/awesome-lisp-companies ***@***.***> Cc: Alex Nygren ***@***.***>; Author ***@***.***> Subject: Re: [azzamsa/awesome-lisp-companies] Add Kina (www.kinaknowledge.com) (#66) Thank for the information. A few more words if you have time? I would share the answers on my blog (https://lisp-journey.gitlab.io/), reddit and it will certainly be on Twitter (Lisp Advocates). 1. What implementation(s) are you using? 2. What is not Lisp in your stack? For example, in "Kina extracts information from PDF, TIFFs, Excel, Word and more" 3. What's your position on open-source: do you use open-source Lisp libraries, do you (plan to) open-source some? 4. How did hiring and forming lisp or non-lisp developers go? Did you look for experienced lispers or did you seek experienced engineers, even with little to no prior Lisp background? 5. How many lispers are working together, how big a codebase to you manage? 6. Do you sometimes wish the CL (pro) world was more structured? (we have a CL Foundation but not so much active) 7. And so, how did you end up with CL? ignore and add what you want, thanks! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#66 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AG4XEL4TYPX5IQEAX3JF36TUGW2TDANCNFSM5FZDQGPA>.

Answer 4 · 2021-10-22T08:27:45.000Z

Thank you very much, great to hear all that. Looking forward to DLisp!!

I can't see the images though.

Answer 5 · 2021-10-22T09:42:25.000Z

Send me your email address, or a link that I can upload to so you can receive the images. Alex

…

________________________________ From: vindarel ***@***.***> Sent: Friday, October 22, 2021 4:27 AM To: azzamsa/awesome-lisp-companies ***@***.***> Cc: Alex Nygren ***@***.***>; Author ***@***.***> Subject: Re: [azzamsa/awesome-lisp-companies] Add Kina (www.kinaknowledge.com) (#66) Thank you very much, great to hear all that. Looking forward to DLisp!! I can't see the images though. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#66 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AG4XEL2XKCIZVU53NO6US3LUIEOAXANCNFSM5FZDQGPA>.

Answer 6 · 2021-10-22T10:20:44.000Z

but an image should display correctly on Github so what's with them?

You can write me at vindarel at mailz dot org

and I use this to upload images: https://drop.infini.fr/

Answer 7 · 2021-10-22T10:32:33.000Z

I inlined them vs. attached them. Not sure if that makes a difference or not to whatever processes these emails. I sent you the pictures to your email address. Alex

…

________________________________ From: vindarel ***@***.***> Sent: Friday, October 22, 2021 6:20 AM To: azzamsa/awesome-lisp-companies ***@***.***> Cc: Alex Nygren ***@***.***>; Author ***@***.***> Subject: Re: [azzamsa/awesome-lisp-companies] Add Kina (www.kinaknowledge.com) (#66) but an image should display correctly on Github so what's with them? You can write me at vindarel at mailz dot org and I use this to upload images: https://drop.infini.fr/ — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#66 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AG4XEL2H7BQ4626TJQRUXSTUIE3INANCNFSM5FZDQGPA>.

Answer 8 · 2021-10-22T10:45:37.000Z

oh you used email. I drop them in the UI:

Answer 9 · 2021-10-22T11:59:43.000Z

The production:

https://lisp-journey.gitlab.io/blog/lisp-interview-kina/

https://www.reddit.com/r/lisp/comments/qdfcb7/lisp_interview_questions_to_alex_nygren_of_kina/?

and Lisp Advocates relaying on Twitter (https://twitter.com/LispAdvocates/status/1451503639105769525)

Answer 10 · 2021-10-22T17:09:06.000Z

Amazing stuff!! Thank you to you both for your dedication.

Note that GitHub's email replies feature offers quite degraded functionality, as it supports plain text only.
Please post from the site or the mobile app for best results.

Answer 11 · 2021-10-23T00:52:02.000Z

And Kina is under HN's lights: https://news.ycombinator.com/item?id=28961987

thanks @Hexstream !

Answer 12 · 2021-10-25T12:56:22.000Z

We have translated the blog interview to Spanish. Would you prefer to post the Spanish language version as well on your blog and us link to it, or would you prefer us to post it directly in our LinkedIn and our own site and then provide a link back to the Lisp Journey blog for the source material?

Answer 13 · 2021-10-25T15:56:43.000Z

nice. Personally I'd enjoy the article on your website, it's even more impressive to read it on the company's official website.