Why typeclasses?
Opened this issue · 201 comments
Is less verbosity the only main benefit of typeclasses over just inputting a set of functions?
Typeclasses can do where bar is a member of Function:
foo(x: A) where Function<A>
bar(x)
Without typeclasses we could do:
foo(x: A, y: Function<A>)
y.bar(x)
I understand @keean proposed other functionality for the where clause, but as for the main functionality of providing pluggable (i.e. dependency injection) interfaces, what is the big benefit that justifies creating a new language?
I think the question is why not just pass all the interfaces as arguments? You can of course do this, after all type classes are implemented as implicit dictionary arguments.
The advantage of type classes is that they are implicit, hence you have shorter more readable function signatures. They can also be inferred, so you don't have to explicitly type the constraints. You don't want to have to explicitly thread y through every function call where you pass type A because you might want to call bar at some point. Further when you realise you need to call bar on a value of type A you don't want to have to refactor every call in the program to pass an implementation.
Thanks for the reply. I appreciate those advantages. I am just thinking about near-term priorities. I think achieving the "perfect" language is a major endeavor that needs a significant amount of funding and time. I've been thinking that your goal is maximum genericity (i.e. the ability to write some code that can parametrized over many types and uses cases) and minimum explicitness (although in other cases you argue for more explicitness where I argue for less).
My near-term goal is not that. That is far too much for me to think about and digest at this point. I am wanting some improvements on JavaScript so I can go use the same code base both on the client and the (Node.js) server. For example a cryptographic library.
- promises (ES6 has this)
- anonymous unions (TypeScript has this)
- guards or match (TypeScript has this)
- integer types (u8, u16, u32, 64)
- structures (handles all the boilerplate on top of
TypedArrays andArrayBuffers) - everything as an expression
- Python-like code block indenting instead of curly brace delimited blocks
I've been thinking about achieving a much simpler transpiler to TypeScript that can meet those near-term objectives.
Later can experiment with moving to higher levels of genericity and less explicitness, if warranted by the use cases that are prominent. And with more funding and momentum behind the initial simpler transpiler.
Further when you realise you need to call bar on a value of type A you don't want to have to refactor every call in the program to pass an implementation.
That works if you only allow every data type to implemented only one way for each typeclasses, but as we discussed in the past (with @skaller) that has limitations also. Otherwise, you could get unintended implicit functionality. So explicitly passing has its benefits.
Edit: Dec. 4, 2017
- integrated low-level C-like coding with the high-level language and memory allocation mgmt (aka GC)
Perhaps we will eventually think of some low-complexity abstraction that gives us non-marshaled integration for tree data structures between HLL and LLL, and perhaps we can even prove borrowing and lifetimes statically. But I suspect we can not without heading towards the complexity of C++ and Rust.
I would not write a web service in 'C' or even 'Rust' ever, this is really ideal ground for a garbage collected language. I would use Node (TypeScript), Django(Python) or 'Go'.
My view is there needs to be a high-level-language and a low-level-language that share enough memory-layout and data-structure that data can easily be passed backward and forward.
@shelby3 wrote:
I am wanting to move away from such clusterfucks of never ending aggregation of complexity.
My quest in @keean’s repository has to been to get the advantage of GC and perhaps some other aspects of FP such as first-class functions and closures in a simple language that also can do some low-level things such as unboxed data structures. To further erode the use cases where C/C++ would be chosen, to relegate that clusterfucked C++ language tsuris (and perhaps Rust also but the jury is still out on that) more and more to the extreme fringe of use cases. For example, It is ridiculous that I must employ a cumbersome Node.js Buffer library to build an unboxed data structure.
[…] so that code which would otherwise be written in for example TypeScript and C/Emscripten would not have to be rewritten later in Lucid. More optimum compiled targets could be developed such as WebAssembly which leverage the full power of the design laid out above, including more optimum compilation of the C-like code. One of my test use cases would be to translate my Blake C code to Lucid.
Regarding one implementation per class, that is the advantage of type classes :-) If you want more you need to pass an object or function (because you want the implementation to depend on the value of the object not the type).
To me it seems you can do nearly everything you want in typescript, so why not use it? I am working on a project in typescript at the moment, and for the ES6 stuff plus types it's good enough.
Regarding goals, I agree maximum genericity is my goal, and minimum boilerplate. However I also want to prioritise readability and understandability over conciseness, as you spend more time debugging and maintaining code than we do writing it. However I am happy for explicitness to be optional, which includes type-inference (type annotations optional) typeclass constraint inference (so constraints/bounds are optional). The only place we need types would be modules, where we want a stable interface that enables separate compilation.
Regarding one implementation per class, that is the advantage of type classes :-) If you want more you need to pass an object or function (because you want the implementation to depend on the value of the object not the type).
If one universal implementation for each universal class was an absolute advantage or even feasible, then we wouldn't need scoping for (type) name conflicts in programming. It is impossible to guarantee one global implementation for the universe, because total orders don't exist.
I am not arguing that it can not be a feature that is desirable in some cases in a partial order.
To me it seems you can do nearly everything you want in typescript, so why not use it?
I guess you didn't read my list above, or didn't realize that the items I listed which TypeScript doesn't do are extremely important for the work I need to do.
However I also want to prioritise readability and understandability over conciseness
Ditto myself. I take this even further than you as I am thinking all types shall be explicitly declared, except in an assignment where either the destination or source is explicit, but not required for both. Since passing function arguments is assignment, then anonymous lambda functions can be implicitly typed by the function's argument type.
I thought typescript included ES6 features, and has integer types as well as strucures (classes). The only one it does not have is "everything is an expression" I think, so it seems close to what you want?
Regarding type classes, it is clear that each function can only have one implementation for any given set of argument types. In other words we can only define one implementation for (+)<Int, Int>
Regarding type classes, it is clear that each function can only have one implementation for any given set of argument types.
Disagree (and my reason was already stated). But I don't want to repeat that argument now. I have said all I want to say about it for now. It isn't a priority issue for me at the moment. We can revisit that later if ever get to the point where I want to implement a "typeclass" language.
Edit: we had a long discussion about this in another thread in 2016.
I thought typescript included ES6 features, and has integer types as well as strucures (classes).
Afaik, there are no integer types in ES6. Are you thinking of 64-bit math methods in ESNEXT?
http://kangax.github.io/compat-table/esnext/
Interfaces and classes do not pack into ArrayBuffers. Perhaps you were thinking of the following proposal which has gained no traction:
http://wiki.ecmascript.org/doku.php?id=harmony:typed_objects
In 2016, I wrote some very complex (untested) JavaScript code to improve on that concept and provide a polyfill. But it has some drawbacks. I need to revisit that code to refresh my memory.
I looked again at Nim, but it is getting too far from JavaScript (wants to be a systems language competing with C, C++, Rust, and Go). For example, lose JavaScript's Promises, ArrayBuffers, etc..
And it doesn't offer 64-bit integer support on JavaScript.
And I'd much prefer the JavaScript output is recognizeable, so the JavaScript debugger can be effectively used.
I think I envision a robust way to transpile from a syntax and type system I want into comprehensible TypeScript. For example, I can use TypeScript's getters and setters to access the fields by name of a DataView on an ArrayBuffer. When not packed into a binary ArrayBuffer, 64-bit integers can be represented with TypeScript's tuples.
It may be slower than normal JS objects due to packing issues, or it may be faster due to non-string property lookup. It certainly seems the right way to process binary network data, but I would probably use native JS objects everywhere memory layout is not important.
but I would probably use native JS objects everywhere memory layout is not important.
Agreed. I didn't intend to imply otherwise. Not only network data, but file data. You know I am working a blockchain design.
Edit: Also memory data. With a blockchain, you need the UXTO stored in RAM and this becomes huge.
nodejs can mmap files from JavaScipt which would work well with this too. You don't have too many choices in-browser, you are limited to Blobs with a streaming interface, and the file IO API is not supported by all browsers.
nodejs can mmap files from JavaScipt which would work well with this too.
Right agreed. Interacting with binary data in JavaScript on Node.js is via very verbose APIs. I'd prefer a language native solution.
You don't have too many choices in-browser
I wrote "client side" but that doesn't mean we are limited to the browser. I am thinking Android and iOS.
Most in the blockchain realm use a systems programming language with good binary control, such as C or C++. But this is painful in other aspects. There is a lot of verbosity that isn't always needed. Need complex libraries such as Boost for asynchrony, etc... Do not have GC by default, etc.. I think it is overkill.
I am searching for a better balance. Rapid coding with less needless verbosity, yet still be able to handle binary data and integer math without verbose APIs. And also some (explicit) typing for correctness checks and better readability (open source).
Others have integrated C/C++ with JavaScript via Emscripten separating out code that requires that control from that can be done more elegantly in JavaScript, but this IMO isn't ideal.
Most code doesn't need to be that heavily optimized (or at least not the first priority). I am thinking there should be something in between a toy language such as JavaScript and a full blown systems language such as C++.
node-webkit could be an option then (npm nw). You can still compile typescript ( for example: https://basarat.gitbooks.io/typescript/content/docs/quick/nodejs.html )
node-webkit could be an option then (npm nw).
Good point and agreed if Node.js APIs are needed client-side.
Well if you want to read and write binary files, its pretty much required. In a browser, FileReader ( http://caniuse.com/#search=FileReader ) is well supported, but FileWriter is not ( http://caniuse.com/#search=FileWriter ).
Well if you want to read and write binary files, its pretty much required.
Obviously Node.js isn't the only way to read and write files on Android and iOS (and even can access it via JavaScript in a WebBrowser instance). Whether one wants to carry all the baggage of node-webkit just for file APIs is not a certainty. But yeah, it is apparently one of the options.
My focus in on the notion that we shouldn't be forced to use a systems language just to do integer operations and packed binary structures. That we should be able to combine these capabilities with a rapid-coding paradigm that has GC, simple-as-pie asynchronous programming, and less optimum but less painful ways to for example manipulate strings (because we don't always need that to be highly optimized).
C++ is too tedious and overly detailed when in most cases we don't need that. Yet we shouldn't have to revert to C/C++ just to get integer operations and packed binary structures.
Premature optimization often means projects that don't get finished.
Yeah performance is battery life and that is important on mobile. But projects that get to market are more important than projects which don't.
For Android and iOS there are things like Cordova, is that the sort of thing you are thinking about?
Yeah there are also Nativescript and Tabrisjs.
But I am not sure if I choose those, because may also want to think about desktop apps as well.
This is a wide open field of experimentation, because we also have convergence of the laptop/desktop and mobile on the horizon perhaps...
Using HTML5, JavaScipt and IndexedDB you can already write software that runs across mobile, pad, laptop, desktop, and big-TVs. It works pretty well, except for annoying corner cases, like iOS limiting IndexedDB to 50MB of storage. It's definitely fast enough for a lot of applications, and HTML/CSS makes good looking UI reasonably easy.
Writing to the least common denominator controlled by gatekeepers who squat on and stall innovation seems to me to be a losing paradigm. What about SD cards, cameras, various multi-touch swipe gestures, etc... Consumers care about features not about how you achieved it.
Well you have to write your own "drivers" for each platform. On Android you have to write in Java to call all the native APIs, but you can call Java you have written from JavaScript if you create an android application shell in Java. Same for iOS (9+) where you can call native APIs from objective-c or Swift, and you can call your own objective-c or Swift code from JavaScript, so you would need a different objective-c or Swift wrapper app. These wrapper apps need to get into the respective app-stores, which means getting Google/Apple to approve your code. You can distribute outside of the app store on both devices, but it requires the user to jump through hoops (android you need to enable a device setting to allow non-app-store apps to install, iOS you need to "trust" the developer of each non-app-store app in the device settings).
Well you have to write your own "drivers" for each platform. On Android you have to write in Java to call all the native APIs, but you can call Java you have written from JavaScript if you create an android application shell in Java. Same for iOS (9+) where you can call native APIs from objective-c or Swift, and you can call your own objective-c or Swift code from JavaScript, so you would need a different objective-c or Swift wrapper app.
Agreed and I was fully aware of that. But good you've summarized for readers.
These wrapper apps need to get into the respective app-stores, which means getting Google/Apple to approve your code. You can distribute outside of the app store on both devices, but it requires the user to jump through hoops
These companies are trying to create walled gardens and be gatekeepers. Me thinks the free market is going to route around these Coasian costs. They are trying to carefully position themselves with "safety" and "convenience" to acquire monopoly control.
In fact, that is one of my other longer-term goals.
If you have an open source app which acts as a base for all other apps and doesn't abuse as a gatekeeper, then it could alleviate this problem. Once everyone has that base app installed, then it takes care of overriding all these toll bridges. Toll because the costs are paid but indirectly.
(android you need to enable a device setting to allow non-app-store apps to install, iOS you need to "trust" the developer of each non-app-store app in the device settings).
Clicking Ok for a confirmation dialog is not problem for users. It is when they make the enabling action modal and hidden, that the achieve insidious control.
Popping up a red alert, "many other users have reported issues with this app" (with a link off to details) would be a sufficient free market solution to safety.
Even Google realizes that ads can't sustain the company. So they are going to have to become monopolistic. Open source should slay this, as it did to Microsoft Windows.
Android defeated iOS in global installed units (not dollar weighted) marketshare because it was more open. Ditto to Android if they make it difficult for users to have freedom.
As I assume you know, Android still lags (or perhaps near parity with) iOS on a dollar weighted basis (such as $ spent on apps), because iOS simply worked better. For example the egregious latency with Android audio was unacceptable for some scenarios and many people aren't even aware of issues like this. Apple's cathedral-style engineering integration was superior, but eventually bazaar-style open source catches up. Takes a while.
@keean I am not actively coding again yet (only the 19th day of my 8 week intensive 4-drug phase of TB treatment)
I find this comment to be relevant:
Maybe what Rust really needs is fewer programmers writing Rust crates and features, and more programmers who use those features to write end-user programs that won’t be redistributed.
This.
The Rust community looks, from the outside, a bit ingrown and subject to a kind of “can you top this (complexity)” syndrome. I don’t mean this observation in a hostile way; if I were eyeball-deep in the development of a young language I would probably behave the same way – and arguably I did, back in Python’s earlier days when I was briefly on that devteam.
But yes, I think more feedback from people who just want to get stuff done and have a lower complexity tolerance would be good for the language.
Complexity is even more insidious than that, I’m sad to say. It’s not enough to avoid being excessively clever. Even if you restrain yourself, complexity will still creep on you anyway. You need to actively fight it whenever your can, and periodically review your “complexity budget.”
Rust has already, in my opinion as a production user, overdrawn the complexity budget in one place. This happened accidentally, very early on. Specifically, it’s the interaction between several Rust features: (1) The auto-deref features provided by
Deref. (2) The split between “owned” types likeStringand reference/slice types like&str. (3) The unification-based type inference that allows you to writelet i: u32 = "10".parse()? and automatically choose the right version ofparse. (4) TheIntotrait (and related traits), which allows you to write functions that take arguments of typeInto<String>, which essentially means, “Oh, just give me any type that advertises conversion to aString.”Any one of these features individually makes Rust much nicer to program in. But if several of them all gang up in the same piece of code, a novice Rust user will get burnt. If I’m reading between the lines correctly, this might actually be what bit you with
bind. The bind function takes an argument of typeToSocketAddrs, which is basically one of thoseInto-like traits which allow you to pass in several different ways of representing a socket address. It’s cute and clever (yuck), and it works most of the time. But if it gets combined with (1-3) above, it’s going to generate crappy error messages. The fix is relatively simple for an experienced Rust programmer, if annoying: Add explicit type declarations all over the place. But at work, my guideline is “just don’t useInto-style conversion traits in your APIs unless you have a damn good reason. They don’t actually improve API ergonomics.”If this is the only place where Rust exceeds it’s complexity budget, well, users will just learn this one bad interaction and go about their lives. C++ has dozens of warts worse than this. But any further expansions of Rust in this area need to be handled very carefully, and production Rust users need to participate in the RFC process.
But let’s dig deeper.
There are two libraries in the Rust space which worry me: Diesel and Tokio. Diesel looks like an ORM, but it’s really not—it’s just a typed version of the relational algebra which can dump output into Rust data types. It results in really concise and efficient code once it’s working. But the error messages are just horrendous (though not yet in modern C++ territory, though that’s nothing to be proud of). Diesel has chosen to push Rust’s type system to its limits in the name of speed and expressiveness. I’m not sure it’s worth it. We had a long debate at work and I paired on Diesel code with one of our other Ruby backend guys, and he said the tradeoffs with Diesel’s error messages were worth it. I’m not 100% sold.
Where I’m more concerned is
tokio. As everybody has told you,tokiois central to the Rust async I/O story, and lots of popular crates are moving totokio-based backends (though most will still export synchronous APIs). And from what I’m hearing,tokiois currently generating bad error messages for some common use cases. In my opinion, this needs to be fixed—and the core team is discussing pushing up a language feature that’s been in the RFC process for a while now, which will hopefully make the error messages much clearer.Still, I’m holding off on
tokio-based async I/O for at least 6 months in production code, to see how this all plays out.
And the comments complaining about the JavaScript ecosystem chaos such as this one:
The situation with JavaScript and Node.js is more complex. At the time of writing, there are 399,773 packages on npmjs.com. And yet, there are really obvious categories with no good alternatives. For example, I needed a CLI option parser the other day, and I needed it to support git-style subcommands. I searched npmjs.com for 30 minutes and spoke to our in-house Node guru, and everybody said, “No, every single Node package for your use case is terrible.” This makes me sad and frustrated. I eventually found something that was, frankly, pretty bad and cudgeled it until it worked.
And as I had pointed out on the Rust forum back in 2016, all the complexity doesn't really provide us anything that really helps us that much because most programming errors are not constrained to the sort of safety Rust is checking for (yet those safety checks are very debilitating to code in), as reiterated by this comment:
What you described is probably true for a lot of user-space code, but it is not true for the kernel. If you look at Linux kernel CVEs, you will see that they have nothing to do buffer overflows, stack smashes, use-after-free vulnerabilities. They are more likely to be logical errors, which can be exploited under some condition. Even when you hear a data race condition found in the kernel, it is unlikely to be caused by a missing lock. In most cases, it is a logical error of some kind. For example, here is the patch for a recently discovered data race in Linux (CVE-2016-5195): https://lkml.org/lkml/2016/10/19/860. As you see, it was not about locking, but about proper checking different flags.
Moreover, Linux developers actively use a static analyzer to catch the most common mistakes in their code. So Rust can’t help there much. Also, it is completely unrealistic to write a general purpose kernel without unsafe code. Even if you look at the Rust standard library, you will find a lot of unsafe code there (e.g. all collections). When you work with hardware at low-level, you have to do a lot of things that can be potentially unsafe. But let’s suppose that it was possible to write the whole kernel in a safe programming language, would it eliminate security exploits? Of course, not.
You may look at long list of CVEs for many web-frameworks written in safe languages (such as PHP, Python, Ruby). By its nature, the kernel works at a higher privilege than the user-code, which means many logical errors in the kernel can be exploited to do something that the user should not be allowed to do. So writing a safe kernel is a far more complex task then writing a kernel in a safe language. First of all, you have to have a formal definition of what you mean by “safe” and then decide how you are going to prove that. It is very tricky even for a very small toy kernel.
@keean you might find it useful to read these perspectives:
I think I could do a transpiler that doesn't even need to compute the types, if I am willing to use special infix operators for the 64-bit integer types, e.g. '+' instead of + because Typescript doesn't have operator overloading. I will make the 64-bit integer types interfaces. Ditto what I stated about employing getters and setters for modeled structured data. Then let Typescript do all the type checking.
But I will probably need a type checker to do everything well, including ideas about abstracting concurrency and parallelism boilerplate from the other thread.
The transpiler does not need to check types, but the idea of checking types is to not accept any input that will break the transpiler. In other words the types are used to validate the assumptions on which the transpilation works.
So if you make no additional assumptions over type script you can do without type checking, however the error messages would relate to the target generated code, not the source code. It is probably better for the transpiler to type check and give error message referencing the source code line and position, rather than letting the program transpiler, and then have typescript throw an error referring to a line and code the author has never seen because it's the output of transpilation.
I completely agree it is better to type check in the transpiler. As I said, would also enable the use of + instead of some hokey '+' for an overloaded infix operators on 64-bit integer types.However, the '+' could be replaced with + in the future with a simple search and replace. And agree, it would enable better error messages.
Note I am also factoring in that I want to be coding with this on a blockchain project within a month, if possible. So I don't think that is enough time to build a type checker. Also I have a lot of commitments from the community to fund my blockchain project if I can deliver it asap. So then from that funding, we could possibly fund the development of the type checker and possibly even the other features we've been considering.
Tangentially, I recently asked Eric S. Raymond (the creator of the term "open source") if he was interested in paid work improving the JavaScript ecosystem and he replied it might be interesting. He seems to eat architectural complexity with ease.
I think perhaps an incremental approach is most realistic. But perhaps I am mistaken. Perhaps I should just start coding in Typescript instead.
Is less verbosity the only main benefit of typeclasses over just inputting a set of functions?
...
I understand @keean proposed other functionality for thewhereclause, but as for the main functionality of providing pluggable (i.e. dependency injection) interfaces, what is the big benefit that justifies creating a new language?I think the question is why not just pass all the interfaces as arguments? You can of course do this, after all type classes are implemented as implicit dictionary arguments.
The advantage of type classes is that they are implicit, hence you have shorter more readable function signatures. They can also be inferred, so you don't have to explicitly type the constraints. You don't want to have to explicitly thread
ythrough every function call where you pass typeAbecause you might want to callbarat some point. Further when you realise you need to callbaron a value of typeAyou don't want to have to refactor every call in the program to pass an implementation.
Another benefit is that instead of the function taking an input argument reference to the interface, the function can be optionally monomorphised to an optimized variant of the function for each interface dependence injection.
So instead of the caller creating an object which has the structure of the interface and passing it as an argument, the compiler can create a variant of the called function which calls the interface methods directly, i.e. monomorphized.
You may have intended to imply this where you wrote ‘implicit’. I am stating the benefit more explicitly.
P.S. I am conflicted whether to use British or American spelling, such as the ‘s’ or ‘z’ in monomorphised.
Optional implicit parameters (which Scala has) provide a way of unifying runtime interfaces (passed by value) and monomorphisable interfaces (passed implicitly by type).
I like that summary (in the context of what I had written). We should make sure we frame that and use it in our documentation.
Regarding type classes, it is clear that each function can only have one implementation for any given set of argument types.
Disagree (and my reason was already stated). But I don't want to repeat that argument now. I have said all I want to say about it for now. It isn't a priority issue for me at the moment. We can revisit that later if ever get to the point where I want to implement a "typeclass" language.
Edit: we had a long discussion about this in another thread in 2016.
https://www.reddit.com/r/programming/comments/1ptiy2/martin_odersky_the_trouble_with_types/cd6dni7/
It may be slower than normal JS objects due to packing issues, or it may be faster due to non-string property lookup. It certainly seems the right way to process binary network data, but I would probably use native JS objects everywhere memory layout is not important.
I just wrote:
Of course this means extra overhead when for example adding and removing elements from an array, except for the binary packed structures (i.e. stored by value in place instead of by reference) I proposed (and note that store by value is better for cache locality performance).
Is less verbosity the only main benefit of typeclasses over just inputting a set of functions?
...
I understand @keean proposed other functionality for thewhereclause, but as for the main functionality of providing pluggable (i.e. dependency injection) interfaces, what is the big benefit that justifies creating a new language?I think the question is why not just pass all the interfaces as arguments? You can of course do this, after all type classes are implemented as implicit dictionary arguments.
The advantage of type classes is that they are implicit, hence you have shorter more readable function signatures.
The function definitions aren’t shorter, because the required interfaces would be in the where clause.
The function call site would be more concise but lack local reasoning.
They can also be inferred, so you don't have to explicitly type the constraints.
I have explained that I want to move away from principal typings inference.
You don't want to have to explicitly thread
ythrough every function call where you pass typeAbecause you might want to callbarat some point. Further when you realise you need to callbaron a value of typeAyou don't want to have to refactor every call in the program to pass an implementation.
Correct and agreed. Even if not inferring the where clause, explicitness would require refactoring all the call sites turtles all the way up.
But maybe that is a good thing. It is more explicit when changes are made, which call sites are affected. Otherwise this is obscured from version control systems by the implicit capability of the compiler.
And the function definition sites do need be refactored any way, unless there where were somehow inferred.
Another benefit is that instead of the function taking an input argument reference to the interface, the function can be optionally monomorphised to an optimized variant of the function for each interface dependence injection.
So instead of the caller creating an object which has the structure of the interface and passing it as an argument, the compiler can create a variant of the called function which calls the interface methods directly, i.e. monomorphized.
On further thought, the smart compiler could still make the monomorphized optimization with explicit interface object arguments, especially if the construction of object arguments is done inline in the function call and the objects are never assigned to any reference then the compiler.
Also monomorphization is not my near-term nor any where near my highest priority.
Another advantage of explicitly passing the interfaces instead of the implicit action of typeclasses, is that it will eliminate the syntax (and thus eliminated the obtuse implicit action of the) proposed forall B (aka for<B>) syntax:
So instead of needing to understand how for<B> implicitly selects the interfaces (either at the call site scope or in the scope of the function being called), the required interfaces will be explicitly written in the function definition signatures and the call site arguments. IMO, this will improve the readability by not having obtuse, implicit action and instead explicitly written action. Thus afaics, it would eliminate that issue of whether to use first-class polymorphism.
Edit: HRT have been clarified and apparently the forall<B> annotation can be inferred.
I abandoned Ada because it took the interfaces (modules as ML calls them), all the explicit instantiations were just too much boilerplate for me. Have a look at the 'Go' board with monte-carlo simulation in Ada on my GitHub. I would not want to create another language like that.
On a side note, the Rust guys have realised the ad-hoc approach to type system design is getting them in trouble, and have started implementing a logic engine (Chalk) to help them tidy and understand it. This is one of the ideas central to my design is to start from a clean logic, so you avoid all the mess.
I can’t read Ada, so it is meaningless to me when I looked at your Ada code.
If we have special syntax for interface objects being explicitly passed in, then the IDE could optionally hide these on toggle.
Again my point about not obscuring changes from version control systems may be an important one. With implicit selection of interfaces at the call sites (with typeclasses instead of explicit passed interface objects), a change which causes cascaded changes turtles up through the call hierarchy, will not be seen by the version control system at the call sites even though the function definitions have changed. Although some might argue this improves the version control reasoning as the changesets are only at the S.P.O.T. of the function definitions (on the where clauses) instead of also all the “noise” of the matching explicit call site changes, the fact is those call sites have changed and those files must be recompiled. Implicit changes are obscuring information. I am contemplating that the main priority of version control is not brevity but rather completeness and transparency of changes. We are aiming for correctness of understanding of changes here, not reducing the number of files with changes.
With explicitness, if some call site was not updated, there would be a compiler error. This forces the programmer to manually contemplate the effect of these changes on the call site logic. Whereas with implicits, things change that programmer is not forced to approve and check. Implicit changes seem more dangerous.
I haven’t decided. I am brainstorming.
The function definitions aren’t shorter, because the required interfaces would be in the
whereclause.The function call site would be more concise but lack local reasoning.
You don't want to have to explicitly thread
ythrough every function call where you pass typeAbecause you might want to callbarat some point. Further when you realise you need to callbaron a value of typeAyou don't want to have to refactor every call in the program to pass an implementation.Correct and agreed. Even if not inferring the
whereclause, explicitness would require refactoring all the call sites turtles all the way up.But maybe that is a good thing. It is more explicit when changes are made, which call sites are affected. Otherwise this is obscured from version control systems by the implicit capability of the compiler.
I am thinking there is no conceptual difference between the syntax of the required interfaces as typeclass bounds in where clauses versus explicit arguments that have the type of type parametrised interfaces with static methods. Note TypeScript’s handling of static methods fits perfectly (and note the restriction that static methods may not employ the class type parameters is irrelevant because we can parametrise the interface ClockConstructor<A> for the static methods).
The programmer can explicitly supply these arguments at the call site or the compiler the can implicitly supply them at the call site. The former provides a way to get rolling quicker with an initially simpler compiler and that code would not be incompatible with an improvement in the compiler to optionally to do the latter. The compiler can detect these arguments are optionally implicit because the interfaces are parametrised only by type parameters which are quantified at the call site.
It will be a boilerplate pita to always manually (explicitly) supply the interfaces at the call site, especially when the data type is a union and thus the interface has to be constructed as type-case dispatch for each interface defined for each data type in the union.
The above proposal addresses @skaller’s criticism about importance of local reasoning:
Shelby3's chosing by selecting instances by providing candidates in a particular context, and different instances in other contexts mixes both models, so the selection is partly implicit, and partly explicit. This is a bad idea because it makes it hard to reason about what is going on: when you look at a call site, you also have to look at the top of the file to see which instances are made available. The ability to reason about code is dependent on local knowledge.
The problem with putting the interface where the type should go, is that you lose information, consider:
f : A -> A where Window[A]
vs
f : Window -> Window
The first is a function that is passed a type 'A' and returns a type 'A' where 'A' is a Window. The second is a function that is passed some unknown type that is a Window, and returns another possibly different type which is also a Window. Consider:
let x : SquareWindow = new SquareWindow()
let y = f(x)
With the first signature we know 'y' has the type 'SquareWindow', but with the second we do not know the type of 'y' only that it implements the Window interface, in other words we have turned 'y' into an existential type (or a trait-object as Rust calls it). Once something is an existential (trait-object) there is no way for the type system to ever recover the static type.
f : Window -> Window
I am not proposing that syntax. Instead it would be:
f : A → Window[A] → A
The syntax you presumed is incongruent with what I wrote as follows:
… versus explicit arguments that have the type of type parametrised
interfaces with static methods. Note TypeScript’s handling of static methods fits perfectly.
The compiler can detect these arguments are optionally implicit because the interfaces are parametrised only by type parameters which are quantified at the call site.
P.S. I understand you were presuming the incorrect syntax because it was formerly a proposal of mine from a long time ago before I started this thread.
I guess I don't understand this:
f : A → Window[A] → A
because a one argument function has turned into a two argument function?
because a one argument function has turned into a two argument function?
With an implicit 2nd argument, meaning the compiler can supply it if the call site does not. Scala’s implicits are an example of this.
I wrote before:
The programmer can explicitly supply these arguments at the call site or the compiler the can implicitly supply them at the call site. The former provides a way to get rolling quicker with an initially simpler compiler and that code would not be incompatible with an improvement in the compiler to optionally to do the latter. The compiler can detect these arguments are optionally implicit because the interfaces are parametrised only by type parameters which are quantified at the call site.
Don't implicit arguments have to come first? Normally they have a special annotation like:
f : ?Window[A] -> A -> A
Actually i'm not totally opposed to implicit arguments, as long as their scoping is clear.
Don't implicit arguments have to come first?
In Scala no, because afair due to curried partial application you need to have quantified the type parameters first.
as long as their scoping is clear
That is one of the big issues with implicits is they are too implicit (which was one of my arguments upthread for being explicit). But that is also the case for typeclass bounds unless one total order for all open modularity is presumed (which I argued is not possible but let us not re-argue that now). In any case, it should not be any worse. And I am proposing the caller can optionally be explicit when they want to.
I am not proposing an implicit keyword such as in Scala, but rather limiting them to this one use case which I posit can be inferred without a keyword. Point being not to create a proliferation of use cases of implicits and that implicit scoping confusion.
I don't think implicits are a bad solution, but I prefer type classes because they are simpler, and express the requirements of a function in a clear way. This is also why I prefer 'requires' as a keyword rather than 'where'.
Implicits have the advantage that a single definition can be used in two different ways, which increases generality, however I don't think its useful generality. If type classes represent interfaces, you define functions and algorithms in terms of an interface, why do I need anything more? For example, if I specify in my function requirements that certain objects must be 'openable' then they must support the 'open' interface and I can call 'open' on those objects. Why would I ever need to explicitly pass a dictionary?
@keean regarding your prior comment post in this thread, I finally figured out the holistic meaning, context, and implications of what you were writing last year.
This exemplifies how much special effort and care must be put into communication, else the reader’s eyes may gloss over due to technobabble and they may really have not comprehended the writer’s point. Communication needs to be not worded in a way that would make sense to someone who is not inside the writer’s head. For example, yesterday I employed private messaging with you to figure out what you meant by “inhabited types” in the context you employed (it required considering if the function was pure then the callback was useless— a detail which you did not mention). When I re-read my own prose, I often find similar lapses in clear communication. Even when we write down many details as you did last year, we fail communicate the key essence of the point cogently for the bewildered reader.
Apparently you intend to convey that you think if an algorithm is reduced to its algebraic essence, then it is fully generalized without any need for overloading typeclass instances (i.e. without multiple different implementations of the same typeclass for the same data type), i.e. you wish you have a total ordering for typeclass implementations. Any generality that can not be expressed in the typeclass total ordering, is separated into non-typeclasses (i.e. function arguments) dependency injection (aka callbacks). In your sort example, I remember your algorithm input a relation function for ordering the sort instead of a monolithic sort method for a typeclass. In other words, you choose to never employ typeclasses wherein a data type could be implemented in more than one way for a typeclass. It is a (probably advantageous) design choice you make to separate the concerns of typeclasses, which in your criteria should (be a total ordering, i.e.) only ever be implemented one way for each data type, from more generalized dependence injection. I have read some snippets of Alexander Stepanov books and I want to eventually buy them and read them entirely, as they seem to be very astute and an interesting mathematical perspective.
So with that new understanding, I agree with you that explicitly providing typeclasses enables the programmer to violate the total ordering design criteria for typeclasses. However, as I had explained to you in the past, with open modularity there is no way to enforce a total ordering on implementations for data types of a typeclass. For example, imagine a request made over the Internet wherein the data types and dictionaries are dynamically injected by the remote host making the request (and dynamic open modularity is why I am trying to design a language such that script injection attacks are not possible). It’s impossible to guarantee statically (compiler nor linker) that the remote host is employing the same implementation (for a specific data type and typeclass) as any other remote host. A dynamic check could be made based on a cryptographic hash of a canonical implementation, but this is useless because since remote hosts can inject new data types of their own, then it is ambiguous which implementation of data type for a typeclass is the canonical reference implementation. This is why total orderings do not exist in unbounded non-determinism (i.e. do not exist in our universe). Since I have been working on blockchain consensus research for the past years, I want to help others understand the relevance of relativistic concepts in the new world order of decentralized programming and computer systems
Therefor, an explicit option for supplying typeclass instances does not introduce any new loopholes that do not already exists with open modularity.
And as I already pointed out, the explicit option enables a way to code with typeclasses without needing a special compiler, yet the compiler can still be upgraded to optional implicit functionality later remaining compatible with the legacy code. And the explicit option is available for cases where for version control system justifications, the programmer wishes to be more explicit.
Explicitness also provides one way to work around the problem with two typeclasses containing a function with the same name. This could of course also be handled in other ways such as allowing a renaming syntax in a where clause and/or a name scoping prefix (e.g. MyTypeClass::function(…)).
P.S. regarding trust in running external code:
- code might be sandboxed and only the remote host’s data (on the server or blockchain) is impacted.
- code might have been run externally and only a proof of execution along with state changes is provided, e.g. Intel trusted code initiative or cryptographic methods of proving execution.
- remote host may have been issued an API key for reputation and malicious activity tracking.
An actual TypeScript code example finally!
@shelby3 wrote:
@keean wrote:
@shelby3 wrote:
- structures (handles all the boilerplate on top of
TypedArrays andArrayBuffers)It may be slower than normal JS objects due to packing issues, or it may be faster due to non-string property lookup. It certainly seems the right way to process binary network data, but I would probably use native JS objects everywhere memory layout is not important.
Agreed. I didn't intend to imply otherwise. Not only network data, but file data. You know I am working a blockchain design.
Edit: Also memory data. With a blockchain, you need the UXTO stored in RAM and this becomes huge.
I’m contemplating an unboxed annotation on instances of records data which forces the data structure to be packed into memory. Any incompatible types would generate a compiler error. Variable length fields (e.g. String) must populate the last fields. The binary format would be default little endian and utf8, with support for other data formats added in the future. I would have compiler generated functions for converting between the unboxed and normal variants.
In JavaScript there is no way to store a JavaScript reference in an ArrayBuffer or TypeArray. So there is no need to have a finer grained control over which members of a record data are unboxed. This might cause a performance inefficiency is some cases, where it would be more efficient to store a pointer directly to an object in packed binary data structure (sort of complexity explosion that is exposed in Rust), than an index into an external array of objects. But this is the price to pay for the clean abstraction layer which provides security and other benefits by not exposing low-level control (although it might be plausible to have GC reference in such a binary packed data structure, just not in JavaScript so we can not target that feature now[actually GopherJS is able to emulate Go pointers with id keys which are looked up in a hashmap of object references]).
ivg claims typeclasses are just a special case of nominal subtyping:
Yeah, I was trying to be brief, because I don't want to go into peculiarities of the ad-hoc typing, as a completely different question. An ad-hoc polymorphic type denotes a family of types which nominally (i.e., via binding to some name, not via the structure) places themselves into the class of applicable types for this name. This is basically a special case of nominal subtyping, except that the interface is specified via the name of the overloaded function, and the hierarchy height is one.
@keean wrote:
you will be able to do more with 100,000 generic lines than you could with 100,000 non-generic lines (due to the repetition)
I'm curious, from your hypothetical example, what % of those lines would be a generic function?
In my experience this is less than 2% (from a codebase with 115k LOC), so I don't think this is a good argument unless you write a library for "collections"
Generic implementations mean you don’t forget to fix the same bug or make the same optimization in every variant of your code that exists for every variant of data type. Moreover, so your implementations don’t diverge in general.
Probably more importantly, genericity forces the programmer to think in terms of data independence first.
And ad-hoc polymorphism with typeclasses makes dependency injection (at function call sites) implicit, which is otherwise quite gnarly because for example typeclass implementation can depend on other typeclasses recursively. I think @keean has a much weaker argument in the Go issues thread, because @keean punted on ad-hoc polymorphism (which I still don’t understand how typeclasses can exist without ad hoc polymorphism?).
Dependency injection is also writing code in terms of data independence. So @Goodwine is incorrect. This genericity propagates out into all of the code, if the code is written correctly so that we don’t have to refactor all the dependent code when we change or introduce a new data type.
My Quora comment is also relevant:
Take an extra 10 minutes to think about the small thing because each one is going to be an hour of your time to fix later on (assuming you can even do it, in which case you have to call someone else to do it).
This is the salient bit. The programmer should always be aware of all the costs and tradeoffs of the code written.
Premature optimization is bad if actually optimize something which doesn’t require it, which causes other tradeoffs such as forsaking ease-of-refactoring, readability, and maintainability. But premature awareness of possible optimizations and tradeoffs is the antithesis of bad.
I wrote in the Subtyping issues thread:
However it is possible that you meant what you wrote today in the Go issues thread:
Sometimes I think that people that don't get the need for generics just haven't written a large enough program yet. Of course you can instead have a large team of developers working on something and only have each developer responsible for a small part of the total code, but I am interested in making a single developer (or small team) as effective as possible.
So in that case I agree with your statement. I’m not against genericity. I wrote on Quora:
Ideally all programmers should be owners. Their reputation and income should be tied to the long-term success of what they have coded.
I also wrote on Quora:
Indeed the lack of dedication to the holistic craft is noticeable. For example, when I copy and paste a url into Quora one after the other, the second and subsequent instances end up with the same url as the first one, although all are rewritten to have the correct title. This is entirely inexcusable for a site of Quora’s stature.
I also wrote on Quora:
It’s true that being a great programmer doesn’t necessarily correlate with IQ, but 2 SD or so is going to help you, if you apply it. You’re also correct that programming is an applied art that requires significant dedication and effort.
But I disagree that the best programmers aren’t smart or that programming is just bumbling around in the dark.
https://en.wikipedia.org/wiki/Ad_hoc_polymorphism
In programming languages, ad hoc polymorphism[1] is a kind of polymorphism in which polymorphic functions can be applied to arguments of different types, because a polymorphic function can denote a number of distinct and potentially heterogeneous implementations depending on the type of argument(s) to which it is applied. It is also known as function overloading or operator overloading. The term ad hoc in this context is not intended to be pejorative; it refers simply to the fact that this type of polymorphism is not a fundamental feature of the type system. This is in contrast to parametric polymorphism, in which polymorphic functions are written without mention of any specific type, and can thus apply a single abstract implementation to any number of types in a transparent way. This classification was introduced by Christopher Strachey in 1967.
@shelby3 what I was considering may be to much for Go was supporting multiple-dispatch. It still has type-class style polymorphism, but only the first argument (the receiver) takes part in dispatch, so multiple parameter type-classes would not really work. Given that only one type can take part in dispatch, associated types (types that depend on types) is really the only option to deal with things like the type of the contents of a collection, the type a reference points to etc.
Of course for the language we are working on I would go for multiple-dispatch functions, with multi-parameter type-classes, and either associated types or functional dependencies. Alternatively modules with implicit parameters is still an interesting alternative.
@keean so just to make it clear for readers, you’re admitting that your use of the term ad-hoc polymorphism was incorrect (making too strong of an implied meaning) in that context. Go has ad-hoc polymorphism (although it’s structural not nominal). You have clarified that what really meant is that Go doesn’t have to support multiple-dispatch. Presumably you also meant Go doesn’t have to support recursive typeclass resolution? [I presume “recursive typeclass resolution” meant type class instance implementations which require type class interfaces on (class-scope or member function-scope) type parameters]
Of course for the language we are working on I would go for multiple-dispatch functions, with multi-parameter type-classes, and either associated types or functional dependencies.
I’m not really excited about the very small chance that Go could add single-parameter typeclasses without recursive implicit resolution. Why would I want to invest effort writing code only to become frustrated by such an artificial limitation. I’m tired of using crappy PLs. We need to build the next mainstream PL. Nobody else is going to do it for us. I suppose for those who must use Go, then incremental improvements are better than nothing. I am hoping Zer0 will compile to Go and those anyone who needs a full typeclass implementation within the Go ecosystem can use it instead. However, this will probably involve incompatibilities between the two ecosystems, because for example with typeclasses then maps, arrays, and slices will be probably be user configurable libraries instead of built into the compiler.
I mean it would help if Go actually added typeclasses completely, but I really think their community (collectively, not to insinuate that individuals there aren’t knowledgeable about Haskell, Rust, C++ templates, etc) is not as far along as we are on understanding the issues, so I doubt very much they can bite the bullet.
Go has attracted those who wanted a better C, who need managed M:N green threads, and who dislike the OOP clusterfucks and who also probably don’t like Haskell’s abstruseness. Whereas, you and I are trying extricate the best from Haskell and merge into an imperative PL.
@Goodwine wrote:
The post I quoted had a hypothetical and vague argument. It would have been different if the argument pointed to a real example. How was I wrong in calling it bad?
Thanks for coming over here to discuss, because I had contributed to that thread (before the more complete understanding I have gained in discussions over here hence), but somehow I am banned from it.
I am pointing out that dependency injection so that functions operate with data independence, is something we want to do for many functions (procedures) in our programs. Not because we are necessarily going to call those with different data types today, but because we can do so in the future. Dependency injection means the function asks for data of a parametrised type and the necessary interfaces are injected separately, instead as in OOP of those interfaces being bound to those data types at the definition site of the data type. And instead of hard-coding our functions to only work with specific data types. The interfaces are thus bound to the data types at the call sites.
Imagine you write code that works with a certain record type. And then after that programmer has left the project, it is decided that some other data structure should have a similar semantics, so you write a typeclass interface instance to map that new data structure to the interfaces required by the code that was already written.
So I was asserting that you’re incorrect to think of genericity as only applying to the core libraries that for example implement collections.
Consequently I think @Merovius is overplaying the “don’t need” perspective. Actually I think we need typeclasses (or some variant of ML modules but I have argued against that in the Modularity issues there over here) in order to program correctly, efficiently, and with reuse. In the Subtyping issues thread over here, I have postulated that maybe he and many others are justifiably concerned about doing genericity in way that ends up like the clusterfuck of OOP (i.e. class inheritance, multiple inheritance mess) or the abstruseness of Haskell’s type family type-level programming. I'm concerned that Go will follow @andrewcmyers and go off on some complex merging of subtyping with the typeclass concept and I think we have discussed over here why that’s not desirable. Of course PL design is a craft (not a pure science nor purely engineering discipline, although I think it’s closer to an engineering discipline than some others claim) so I don’t claim to be omniscient. I am reading the arguments that others are making and remaining open-minded.
Btw, ostensibly before I was banned from the Go Github, I wrote a post explaining how typeclasses could solve a use case.
I wrote:
On the issue of output (associated) types, let’s keep in mind this motivating example:
I don’t see that point being considered in the discussion about whether there is a need for generics in Go.
EDIT: One of the creators of Go, Russ Cox spoke about his perspective on generics recently.
@shelby3 I think go does support recursive resolution, although I have not tried it. For example if types A and B are printable, you can define print on structure{A, B} by calling print on A and B, which would be recursive.
The problem for Go is that it would fundamentally change the language to implement multiple dispatch. The language has heavily bought into the concept of a method having a single 'reciever' which is the only part that takes part in dispatch. Adding a feature to a language is one thing, but changing fundamental design choices is another.
The other thing to note is that Go's simplification of existential types like []Animal doesn't help the expression problem much, as you still can only call methods from the Animal interface on the contents of that heterogeneous slice, unless you type cast on the contents using runtime reflection.
This brings me onto a small advantage of associated types over functional dependencies, and that is we can restrict the associated type to a typeclass in the definition:
typeclass Animal [a] {
type Legs[a] = Legs(forall b . b requires Integer[b])
}
I am not sure about the details, I would hope to be able to omit the existential, by automatically deriving it from the typeclass name, but I wanted to make clear here that Integer is the typeclass of all integer types.
This means that given any instance we know we can call functions on Integers on the 'Legs' of any Animal, avoiding types depending on runtime values in a runtime polymorphic collection. This would be a clear advantage over functional dependencies.
@keean wrote:
This brings me onto a small advantage of associated types over functional dependencies, and that is we can restrict the associated type to a typeclass in the definition:
typeclass Animal [a] { type Legs[a] = Legs(forall b . b requires Integer[b]) }I am not sure about the details, I would hope to be able to omit the existential, by automatically deriving it from the typeclass name, but I wanted to make clear here that Integer is the typeclass of all integer types.
This means that given any instance we know we can call functions on Integers on the 'Legs' of any Animal, avoiding types depending on runtime values in a runtime polymorphic collection. This would be a clear advantage over functional dependencies.
Interesting insight.
Apparently the anti-modularity of associated types is only in the presence of [unbounded] existential quantification?
The problem for Go is that it would fundamentally change the language to implement multiple dispatch. The language has heavily bought into the concept of a method having a single 'reciever' which is the only part that takes part in dispatch.
Multiple parameter typeclasses are not often necessary if there’s associated types. I guess they’re needed for Equals between two different types.
But another way to handle [EDIT 2020: in retrospect that idea looks like it flamed out]Equals is to have recursive model where the function of the typeclass takes a typeclass for the second data type, which must conform to an interface which is compatible for testing equality to the first data type. Multiple parameter typeclasses make no sense for [unbounded] existentially quantified objects, but could work with this alternative form of Equals. So maybe we should not allow multiple parameter typeclasses?
EDIT: I have posted an example of what I writing about in the prior paragraph above.
Tikhon Jelvis, Lead Data Scientist at Target (2016-present) wrote at Quora:
⁴ It’s hard to underrate just how big an impact typeclasses have on day-to-day programming. They have some unavoidable limitations—they’re rather anti-modular—but they make so many things so much nicer and easier I’m amazed more languages haven’t followed suit. OCaml modules, like the name implies, have a better story for actual modularity, but I’ve used OCaml enough to tell you it doesn’t really matter. Typeclasses are just so much handier in Haskell! The difference is massive, to the point where OCaml programmers naturally write significantly less flexible libraries because the syntactic and conceptual overhead of needing to instantiate multiple functors is simply too large. Typeclasses allow Haskell code to be effortlessly flexible in a truly amazing way.
The design of type classes that we came up with for Genus is lighter-weight than Haskell's (declaring instances is mostly not needed), in some ways more powerful (multiple instances can exist to satisfy a single type class, and the type system distinguishes them), and also modular.
@andrewcmyers I will read the Genus paper more thoroughly, and see whether it's something we can apply here. The biggest problem we are discussing is that of existentials. For example how can I define a heterogeneous equality over the contents of an existential collection. There are obviously problems because you would need to pass a dictionary for each possible type pairing in the collection. Adding a single type to the collection may require adding several entries to the dictionary for the collection.
If we are to keep multi-parameter type-classes, it seems we don't want existentials. On the other hand if we keep existentials we might want to restrict ourselves to single parameter typeclasses (but we can have associated types in the type-classes).
Genus has both multiparameter type classes and existentials. It does not have associated types.
Familia adds associated types and unifies them with nested inheritance.
@andrewcmyers, what we mean is that the existential quantification can’t be used with multiparameter typeclasses even if you have both features in the language. Did you find some way to make existentials (i.e. unbounded dynamic polymorphism) work with multiparameter interfaces? I think it’s impossible. I would be very interested and surprised if there’s a way to make the two paradigms work together.
I am not positive that I understand what use case you have in mind, but my guess is that you could get what you want if the language supported type-level products; that is, it had a kind of the form type * type.
I’m attempting to make a summary in the other thread where we’re been discussing the interaction of multiparameter and existentials in more detail.
@andrewcmyers Here's an example, we have a heterogeneous queue with runtime polymorphism (say an event queue) this is implemented as an existential list. We now wish to define a heterogeneous binary operator like == that allows any two event types to be compared together. If we implement the equality as a multi-parameter type-class then there is way to make it range over the collection. We can easily do this with single parameter typeclasses by having a LeftEquality class assign a 'value' to its type and passing this to a RightEquality which compares the 'value' to the right type. In other words if every type in the collection provides both LeftEquality and RightEquality we can solve this. In general an existential is defined data E = forall a . E a there is only a single type variable, so we can only use single parameter typeclasses in the case that we are talking about binary operators or any function that needs to access more than one member of a heterogeneous collection at one time.
Infact I think you cannot statically bound a multi-parameter typeclass like this under any circumstances, because the condition for adding dictionaries is that you must add an Equality dictionary for the type you are adding against every type already in the list, which is not statically knowable, so it cannot be done.
So when it comes to things like sorting heterogeneous lists, existentials don't work with multi-parameter type-classes.
Being interested in generic programming and DRY, we don't want to define two versions of Equals, one multi-parameter and one single-parameter. Even the single parameter reduces to a fixed list of cases (closes the bounds on the heterogeneous collection), so I the end it is no better than a 'Sum' type.
This leads me to the conclusion that for generic programming existentials are an anti-pattern, and we should probably exclude them from the language.
@keean wrote:
Infact I think you cannot statically bound a multi-parameter typeclass like this under any circumstances, because the condition for adding dictionaries is that you must add an Equality dictionary for the type you are adding against every type already in the list, which is not statically knowable, so it cannot be done.
I'm not sure if I understand you correctly, so I will try to show it by an example.
You have a function fun which compares events and removes redundancies of the event list:
fun(l:List[E]):Nothing where E = (exists e1, e2. e1 requires Equals[e1,e2] && Equals[e1,e1])
for e1 in l:
for e2 in l && ref(e2)!=ref(e1):
if(equals(e1,e2)) l.remove(e2)
return nothingThe function equals is here a function provided by the typeclass Equals. Assume further, that function fun is located in library X. Note that only e1 is used for E is no restriction as Equals is symmetric and also transitiv. You should think about how to force this for users using Equals (with attributes?).
Because function fun is located in a library, it doesn't get monomorphized. It is stored as AST as every generic function or is partial monomorphized.
In application A1 function fun is used and the members of typeclass Equals are now bounded to Int and Float. Then function fun gets monomorphized to:
fun(l:List[Int | Float]):Nothing
for e1 in l:
for e2 in l && ref(e2)!=ref(e1):
switch((e1,e2)):
case (Int,Float):
if(equalsIntFloat(e1,e2)) l.remove(e2)
case (Int,Int):
...
case (Float,Int):
...
case (Float,Float):
...
return nothingAnd if instead application A2 is using function fun with typeclass members Float, Int, String, then function fun gets monomorphized to all permutations of it (2^8 typeswitches).
Is this the problem you try to solve?
@keean wrote:
I will read the Genus paper more thoroughly, and see whether it's something we can apply here.
@andrewcmyers I hope @keean has time to really study the Genus and Familia designs, because when I tried to read those papers last year when you first mentioned them on the Go issues thread, I was lacking sufficient knowledge to really digest them entirely. I have learned a lot since then (and by now 60-80% recovered from the multiple years of Tuberculosis I was ill with at that time), but now I am really lacking time to go on many tangents. @keean is more expert than me on programming language and type theory, so I’m interested to read his summaries and comparisons to ideas we’re discussing. If you end up articulating comparisons for us, I’m also be interested to read your perspective.
I’m too ignorant of Genus to comment on it. My vague recollection of (my vague original understanding of) the Genus paper is that it combines many different paradigms including subtyping, some facets of OOP, and some means of selecting interface injection somewhat analogous to typeclasses. I remember also the paper contains a feature/paradigm table comparison to C++ and other PLs. At the time I tried to read it, I didn’t know what all the items in the table mean. I have since come to believe that C++ is a mess and we may not want all of the features of C++. Some people argue it’s what we leave out of the PL that can make it a good PL.
So I’ve proposed anonymous, non-disjoint, structural unions (i.e. not disjoint sum types, yet both are tagged) that we only subsume by explicit casting so that unification is sound. I proposed we could simulate subtyping of records at typeclass bounds. I suppose we could also offer subsumption of records by explicit casting. I also proposed that we avoid variance annotations and instead only allow (explicitly casted) subsumption and supersumption that avoid the need for variance annotations, such as subsuming the element type of a List. This will be accomplished via read-only, write-only, and immutability annotations.
We’re also discussing how to selectively inject typeclass bounds so as to accommodate more modularity.
@keean proposed Actors for parallelism and I have developed that idea into a rough sketch for a proposal that I posit could achieve near to zero-cost resource safety without the overhead of mark-and-sweep GC nor the lifetimes tsuris of Rust. IMO, that is a very tantalizing epiphany, which you may want to read. Perhaps that could potentially shake up the entire computer industry, if it proves to be the correct vision for the future.
Btw, I mentioned my grammar work-around your point from the Go issues thread about which form of brackets are best for type parameters.
So my caution about Genus is my excessively vague notion that maybe you’re trying to put too many paradigms in one PL. I mentioned today about “kitchen sink” PL design. That’s not a criticism of Genus because again I’m ignorant of the finer details of Genus. I’m just throwing that out there, to stimulate if someone more knowledgeable than me (about Genus such as compared to our ideas in this Github) wants to discuss whether it is true or not. I’m coming from being an assembly, then C, then C++, then some Java, JavaScript, and PHP programmer in the 1980s and 1990s, before I started to learn functional programming languages circa 2009. So I look back on OOP with disdain.
(Obviously my avatar photo was taken when I was in my 20s which I am not now, lol)
P.S. Note I met @keean in 2015 in the Rust discussion forums. Since 2009, I been trying to find a PL that could meet my needs as programmer, and the closest I found was Haskell and Scala, but they both lack something I want. Rust and Go (and I guess for @keean also ADA) also stimulated the thought process about what we need for resource safety, concurrency, and parallelism. So I was forced to get involved in creating a PL, because it seems like no one else is going to create what I’m contemplating to fit my needs. I seem to need features from all those PLs, yet I need to discard warts from all of those. Or maybe I’m just fooling myself and I should just adopt Haskell? @keean’s design preferences seem to most align with mine of anyone I’ve met so far. And I’m lucky to find someone with his knowledge willing to discuss ideas. I hope we’re getting close to finalizing a design. I don’t think we’re quite there yet, especially final decisions w.r.t. to purity, non-strictness, and imperative effects.
Jon Harrop replied to me on Quora:
“And typeclasses could be added to an eager programming language (e.g. Rust) so thus no reason to use Haskell in that case (although the right language does not exist yet)”
Type classes are a side show, IMO. They work well for operator overloading but little else and there are simpler and more predictable solutions for operator overloading.
@keean I find that to be a very shocking claim. Do you have a reaction to that? Note I will probably be linking this discussion in a reply to Jon. You may know Jon is a former OCaml fanboy and I guess currently favors F#. He seems to not like Haskell much. I think you and I actually agree with most of his conclusions about Haskell, but the above claim about typeclasses seems to be more arguable.
Does he not fully appreciate the benefits of abstraction? He apparently must think that for example implementing a polymorphic sort abstraction is as elegantly refactored with functor modules? What about HKT which MLs can’t do (at least not without boilerplate hell)? What about abstraction in general? I think we discussed in the Modularity issues thread #39 why we think typeclasses are superior to ML modules. We even recently discussed in that thread parametrising modules with typeclasses. Maybe he’s not aware of our discussions about adding modularity to typeclass instances.
Does he not fully appreciate the benefits of abstraction?
He doesn't seem to talk about abstraction much. I would respond with two questions:
-
What simpler and more predictable solutions for operator overloading was he referring to specifically, just a list of technique names or paper references we can google would do, he doesn't need to go into a great deal of explanation about each one.
-
He says typeclasses only work well for "little else", what does he think of the abstractive power of typeclasses? Parametric polymorphism constrained by typeclasses permits generic algorithms to be written with very little clutter, the typeclasses being implicitly passed to the function. What forms of abstraction does he recommend instead?
Hi both,
Please don't take my flippant remark about type classes too seriously. The point I was trying to make is that the presence or absence of type classes in a language is far less important to me as a pragmatist than aspects like non-strict evaluation by default, i.e. I'm not going to drop MLs and go with Haskell despite all of its problems just because it has type classes. A (very) few people have told me that they regard type classes as an indispensable feature of a language. Having never programmed too seriously in any language with type classes I probably don't know what I am missing out on but I have asked for genuine case studies showcasing their practical benefits and have never been convinced.
So I certainly don't have any objection to seeing type classes in a strictly-evaluated language and, in fact, I'd probably rather have type classes than a higher-order module system. Incidentally, I regard type classes and higher-order modules as quite distinct: although their theoretical capabilities overlap to a large extent their strengths in practice appear to be completely different.
Furthermore, I haven't had the chance to study your proposals around type classes on steroids so I am completely unqualified to comment.
I am actually working on a language myself and my current intent is to hack special cases (for all combinations of polymorphic equality, comparison, hashing, pretty printing and serialization with only a handful of built-in collections: list, array, set, dictionary) because I consider those use cases to cover >>99% of all practical applications. I haven't decided if type classes would be a good way for me to solve that problem: a primary goal of mine is to keep the language implementation as small and simple as possible so I'd need to implement type classes in less code than it requires to solve those special cases, which may or may not be possible.
"What about abstraction in general?"
"What forms of abstraction does he recommend instead?"
I think these questions hit the nail on the head. My opinion is increasingly that minimal abstraction capabilities are usually preferable in industry. And by minimal I mean roughly the HM type system. For example, after 14 years of using OCaml in industry I can only recall seeing one higher-order module in the wild: used at XenSource to mock a library so it could be tested more easily. Most added forms of abstraction above HM are in diminishing returns because they make the language harder to learn (and adoption has been a critical problem with all such languages historically) and make code harder to maintain because such features are often abused by junior developers.
EDIT: To give a concrete example, I consider IDE support like Intellisense for F# to be vastly more practically useful than any type system extension beyond HM. Hence my language is designed from the ground up to be used with a completely graphical IDE. I am even considering making all of the non-trivial syntactic constructs (e.g. let, match) graphical rather than ASCII.
Please don't take my flippant remark about type classes too seriously.
Sure, I don't think module systems are that bad either :-)
I'd probably rather have type classes than a higher-order module system.
That's our general consensus too.
My opinion is increasingly that minimal abstraction capabilities are usually preferable in industry.
I think this is a very interesting point, and one that is worth discussing, so lets come back to that.
And by minimal I mean roughly the HM type system
I think something around HM is optimal. I have my own variant, which I don't think is totally original, but that has both principal typings and principal types. I played with intersection type systems, but found they admitted too many programs to really be useful as a type system, and of course undecidability makes inference tricky.
To give a concrete example, I consider IDE support like Intellisense for F# to be vastly more practically useful than any type system extension beyond HM.
We are probably just going to differ on this one. I don't use an IDE, and I program mostly in VIM. I do use syntax highlighting, but I find in general programming is best done away from the keyboard, thinking about the data-structures and the algorithms. The way I normally program is half my screen with the language/library docs in a web-browser and half VIM.
So lets come back to the main point, abstraction. I think that abstraction is absolutely vital, and the most effective way to manage complexity in programming is by getting the right abstraction. When I am thinking about an application I would start with what is the right data model, and how can this be abstracted. For example a "file" is a common abstraction, you can open, close, delete, stream data to, and stream data from, and reposition. This abstraction is very useful for managing secondary storage. I don't think anyone would want to do this by sending IDE/SATA commands directly to the drive. Windows and the UI components are another abstraction. Nobody wants to manage the display by using 'plot' and 'draw' commands to draw the UI, its the wrong level of abstraction (unless you are writing a drawing program. So we can see that for operating system services abstraction is vital to make the complexity manageable. The same applies to the application domain. For any application domain there exists good abstractions, for example in 3D graphics we have the 4-vector and matrix. Trying to code such an application without the appropriate abstraction would quickly expend your complexity budget, but with the right abstraction all the domain operations become trivial. For business applications you need an abstraction that makes the processes of the business trival, and you can then make writing business domain applications trivial, maybe something like MS Dynamics is a good starting point, but I think we can have better business process abstractions.
Now I am not saying finding the right abstractions are easy, it often takes many attempts, operating systems have evolved and so have many other domains of abstraction in computer science. To paraphrase Stepanov, you don't start with abstractions, you start with algorithms. Only once you have a bunch of similar algorithms or a set of algorithms for a domain do you start to abstract an interface for that. In a similar way, Knuth talks about losing the source code to TeX and staring again resulted in much better code. This is because he had a better idea about what abstraction would work well and which would not.
@keean wrote:
[Microsoft] Windo
ws[ze]and the UI components are[is] anotherab[di]straction.
F.t.f.y..
@jdh30 wrote:
And by minimal I mean roughly the HM type system
I think something around HM is optimal.
I want non-disjoint, structural unions a la Scala 3 and Ceylon [but not their intersection types] which we have discussed in the Subtyping issues thread #8. I think they’re essential for static expression combined with runtime polymorphism and modularity, especially when combined with my idea for “chunking” the bounded union existential typeclass bound quantification (which is a strange concept that I still need to explain better as nobody seems to fully grok my descriptions on that yet). I have proposed that subsumption and supersumption must be explicitly cast, so that I presume type inference will remain decidable.
I agree with @keean, no intersections because it make inference undecidable.
To give a concrete example, I consider IDE support like Intellisense for F# to be vastly more practically useful than any type system extension beyond HM.
We are probably just going to differ on this one. I don't use an IDE, and I program mostly in VIM. I do use syntax highlighting, but I find in general programming is best done away from the keyboard, thinking about the data-structures and the algorithms. The way I normally program is half my screen with the language/library docs in a web-browser and half VIM.
@analyticbastard was relating to me yesterday that VS Code has less IDE integration with Scala then IntelliJ. I replied that I head that IntelliJ is using a customized Scalac compiler and thus I was concerned about it tracking Scala changes given they not have their competing language Kotlin which was recently officially approved by Google for Android development. He replied that all those neato IDE features are worthy. I replied that I originated from old school where we had paper books only. [EDIT: Scala 3 is adopting the compiler interoperability standard employed by VS Code so should have full IDE integration]
I think a powerful IDE is useful though. For example with type inference, the IDE can display the inferred types for you. I want both powerful abstraction and a well integrated IDE. But I do want to be able to turn off those bells and whistles on the IDE, because often they just annoy me and slow me down.
I agreed with @keean that we need a flexible and powerful paradigm for expressing abstraction. So far, I like what I have seen of abstraction with typeclasses. Until I see otherwise or something better, then that is the best I know of at the moment.
We are attempting to add some modularity to typeclasses without destroying the goal to remain as close as possible to an abstract algebra. The modularity we’re proposing should be used cautiously.
I do agree though that when the abstractions become a layered thicket of opaque mechanisms a la monad transformers in Haskell, that sort of abstraction is too abstruse and I would not like to promote that style of programming. That is abstraction complexity that is required just to shoehorn a separation of effects such as IO into a limited, low-level paradigm of monolithic purity. As I stated in my comments about Haskell, that imperative effects are inherently highly granular; and thus trying to get all those ducks into a row is (borrowing from Robert Harper’s comments about modules versus typeclasses) like trying to turn the Titanic. I have instead agreed with @keean that maybe we should look at Actors instead and designing our abstractions to be indifferent to message arrival order (i.e. inherently design for disorder) rather than having that abstraction enforced at such a low-level as purity. Purity seems to be a useful concept for expressing an invariant but it appears to be too low-level for dealing with effects in general.
I wrote:
I wrote:
and we want to control effects with an effects system. With actors only the actors implement interfaces, and side effects are implicit in the actors.
I don’t want to control effects with your idea of typeclasses. I want to eliminate effects by making actors function independent of ordering of messages. For other types of effects, we need some model that works at the actor level.
By effects we mean management of a state machine.
What I am saying is that the state machines of Actors should be independent from each other. They should function correctly regardless of the state machines outside.
So this means abandoning determinism and accepting statistical progress.
Determinism is the wrong assumption. We can’t move forward with computing until we abandon it.
In fact, my decentralization solution to the centralization problem of blockchains which nobody else has solved, is basically at the generative essence about abandoning determinism.
So I think the entire concept of algebraic effects, purity, and monads is far too low-level to deal with the problem they are actually purporting to deal with.
I want to refute every point in favor of Scala’s OOP in In Defense of OOFP.
-
His introduction offers some links to what sucked about the v2.8 Scala Collections. Here follows the relevant information I extracted from those links.
I wrote in the WD-40 issues thread #35:
Indeed that is the problem with Scala. Some of the complexity probably derives by trying to model class inheritance along with FP (instead of typeclasses!, c.f. also)
I wrote in the Modularity issues thread #41 comparing a typeclass solution with associated types to an OOP equivalent:
For example to implement the
mapinterface on aStringrequires knowing that the callback function takes a typeCharbutStringhas no type parameterChar. Compare how clumsy this is to implement in an OOP paradigm without typeclasses (and thus without associated types). -
His section 2.3. Seq or Other Super-types claims that OOP isn’t root cause of Daniel Spiewak’s problem. Even though Daniel didn’t explicit mention typeclasses as his advised solution, his problem is indeed with
Seq[B]supertype masquerading as the actual data type of the algorithm. In fact, @djspiewak tweeted that typeclasses are better than“sub typing”[subclassing]:But even if you do just want these operations, you’re far better off with a typeclass rather than sub typing.
-
His section 2.4. Not Using Type-classes admits the advantage of typeclasses but then begin his apparently ignorance of existential quantification and/or union bounds for expressing runtime polymorphism:
Also, not clearly visible here is that type-classes such as
FoldableorTraversable, while more generic are also strictly less capable thanIterable.He continues this myopia in section 3. OOP vs Constrained Parametric Polymorphism (Type-classes):
CON: this method […] is parametric polymorphism and it moves the dispatch cost at compile time, with both the good and the bad
In section 4.2. Liskov Substitution Principle: OOP Strikes Back his problem of expression is because Scala 2 doesn’t support a non-disjoint structural (i.e. not nominal) union type
Array[A] | List[A]. -
In section 4.3. Iterator vs Foldable and Traverse he seems to not realize iterators can be typeclasses. As well in section 5.1. Seq on Return he seems to not realize that what he wants can also be achieved with typeclasses, which avoid the badness of
Seqsubclassing.
Hi @shelby3,
First of all, some credentials are in order — I'm a contributor to Typelevel Cats, lead developer of Cats-Effect and author of Monix. In order words I have designed type class hierarchies and I actually love type classes.
For example to implement the map interface on a String requires knowing that the callback function takes a type Char but String has no type parameter Char. Compare how clumsy this is to implement in an OOP paradigm without typeclasses (and thus without associated types).
What happened in Scala's collections to implement map is not an OOP paradigm. CanBuildFrom is a type class providing a kind of flexibility that's hard to achieve in other languages. The issue being that the cost is too high and users are best served if we got rid of it.
Also that Char is not a type parameter for String is irrelevant. Indeed, if we are talking about writing generic code that makes use of Functors (e.g. map) or Monads (e.g. flatMap), you aren't served well by an OOP solution because an OOP solution would require F-bounded polymorphism, which Scala supports, but would get very clumsy for writing generic code. F-bounded polymorphism has been used in Scala's collections implementation for sharing implementation (not necessarily a good idea if not planned well, hopefully they got it right this time).
But as far as String is concerned, that String is not an M[Char] is actually more of a problem for type classes than it is for OOP. So not sure what your point was, but in particular in Typelevel Cats the only instance we implement is Show[String], see string.scala, because a Functor for Scala's String is not possible without converting it to an M[Char] first.
- You missed the point.
And btw, Daniel Spiewak is a collaborator on the projects I mentioned above and I know very well what he's thinking of OOP and type classes.
-
You missed the point again and no, a union type like
Array[A] | List[A]does not fix the problem. -
Nope, that's not true.
Btw, I don't like getting engaged in arguments using strong, aggressive language. Maybe that's how the big boys like it, but my life is too short for that.
Please don't involve me anymore and please don't send me any more emails.
Thanks 👍
@alexandru wrote:
First of all I want to thank you for responding. And I hope you will not be shy to fully complete your arguments, because nothing is learned if mistakes are not corrected (whether they end up mine or yours T.B.D.). If we expect life to be without strife, then maybe hide under our pillows instead? Anyway, I didn’t start it…
Btw, I don't like getting engaged in arguments using strong, aggressive language. Maybe that's how the big boys like it, but my life is too short for that.
Please don't involve me anymore and please don't send me any more emails.
Hey I tried to comment on your blog, but Disqus won’t let me in — actually non-functional will not even display a login prompt.
You make a very controversial claim that class inheritance form of OOP (aka subclassing) has some benefit and AFAICT, (@keean and) I strongly disagree with that. So you should be able to defend your claims. If you write a controversial blog making such claims, then be prepared to defend them. Because I don’t think we’re alone among well learnt programmers in thinking that subclassing is an anti-pattern.
Also please understand that I’m trying to make decisions right now on PL design and I had reviewed your blogs (when I was searching about Scala’s concurrency solutions) to see if I had any mistakes in my conceptualizations. Your assertion (which would imply) that our plan to abandon subclassing is somehow incorrect is a cause of great concern to me, because I don’t want to make a mistake in the design because I failed to understand why some paradigm is useful or important. So I was a bit disappointed (after being forced to expend the time to figure out what the heck your assumptions and points are in the blog) to find out that not only do you not seem to know enough to write that blog, but you were also (perhaps inadvertently) maligning someone who helped me learn in the past. Perhaps I did react a little bit too strongly, I dunno. I am just stating what I think to the facts and awaiting your rebuttals.
It doesn’t necessarily mean it has to be antagonistic, but I think you instigated animosity when you jumped the gun a bit as I will reiterate below. You called out Daniel and then didn’t even credit him for the fact that he was arguing against erasing the compile-time knowledge of the (for example result / return) data types with subclassing— which is what typeclasses can achieve. Daniel even mentioned in the Twitter feed that you linked to that typeclasses would be preferred over subtyping of Seq. But you made it appear that Daniel was not making that stance. I had to dig on his Twitter to see that you had AFAICT unfairly characterized his more nuanced thoughts. IOW, if you can dish it out, then you can also take it in. But if you want to backtrack a bit with a more fair in your characterization of Daniel’s input to the process, then I will also reciprocate my demeanor. Btw, I also wanted tell you that you forgot to change the “two” to “four” when you added more links to the Background section of your blog post we’re discussing here. I will also make you aware of a comment I posted in the Concurrency thread today about something on your Monix Task website.
If you can convince me I am incorrect, then I will of course thank you for that.
-
For example to implement the
mapinterface on aStringrequires knowing that the callback function takes a typeCharbutStringhas no type parameterChar. Compare how clumsy this is to implement in an OOP paradigm without typeclasses (and thus without associated types).But as far as
Stringis concerned, thatStringis not anM[Char]is actually more of a problem for type classes than it is for OOP. So not sure what your point was, but in particular in Typelevel Cats the only instance we implement isShow[String], see string.scala, because aFunctorfor Scala'sStringis not possible without converting it to anM[Char]first.You’re ostensibly conceptually conflating the encapsulation of the internal representation of the (perhaps Unicode encoded)
Stringobject with the API of aChartype provided to the callback function of an applicative functormap. IOW, the said callback function of amapdoesn’t iterate an array ofChar— rather encapsulated implementation ofmapinvokes the callback with the unencodedChardata and walks the characters of the encodedStringdata. (Readers don’t conflate applicative functors with ML module functors)A typeclass can have an associated type (there are abstract types in Scala). So if we make the element type of collections an abstract type instead of a type parameter, then the typeclass instance call output the element type rather than receiving it as a input type parameter. So AFAICT, I am sorry but you’re incorrect. Please learn Haskell’s typeclasses. If you designed your typeclasses in Cats to not use abstract types for the element type, then AFAIK (what I believe I’ve learned from @keean) then you’re doing it wrong.
EDIT: So appears Cats is doing this wrong, should be instead for example:
def foldRight[B](fa: F, z: B)(f: (F.A, B) => B): B
What happened in Scala's collections to implement
mapis not an OOP paradigm.CanBuildFromis a type class providing a kind of flexibility that's hard to achieve in other languages. The issue being that the cost is too high and users are best served if we got rid of it.My understanding is the
CanBuildFromwas necessary to overcome a type inference weakness (that has since been improved) that discouraged creating two separate overloads (and I believe perhaps also separate OOP hierarchies although I didn’t check it). And most definitely that issue would not even exist in the first place had typeclasses been employed instead as I suggested 7 years ago on 9/13/2011, although I seem to be under the impression (such as from some Tony Morris blogs) that Scala’s limited type inference has always been a thorn with Scalaz.It seems absurd though to claim that Scala’s 2.8 Collections weren’t using subclassing and imply that there wasn’t an issue with trying to shoehorn everything into subclasses that made it very convoluted.
Also that
Charis not a type parameter forStringis irrelevant.I just explained above why it is relevant.
Indeed, if we are talking about writing generic code that makes use of
Functors(e.g.map) orMonads(e.g.flatMap), you aren't served well by an OOP solution because an OOP solution would require F-bounded polymorphism, which Scala supports, but would get very clumsy for writing generic code. F-bounded polymorphism has been used in Scala's collections implementation for sharing implementation (not necessarily a good idea if not planned well, hopefully they got it right this time).Correct. I am arguing against subclassing form of OOP and you are telling me we can’t implement Functors (well) with subclassing. So it seems you’re agreeing with me.
-
You missed the point.
I don’t think so. I explained it again above. What point do you think I missed?
And btw, Daniel Spiewak is a collaborator on the projects I mentioned above and I know very well what he's thinking of OOP and type classes.
That’s great. I’m all for great teamwork. But in that particular blog I interpret what you wrote on your blog as putting Daniel unfairly in a bad light. He was clearly arguing against subtyping because it erases the compile-time knowledge about the data types. Although he didn’t full argue for typeclasses in his github gist proposal, he did even mention typeclasses as superior to the subtyping choice in the Twitter feed you linked to. You made a claim as quoted below which is incorrect and counter to the point he was making:
OOP interfaces might expose leaky operations that are booby traps (e.g. Seq indexing), but this is not specific to OOP interfaces, vigilance in design is always needed!
That problem is specifically due to subclassing. And the fix is typeclasses. Period.
-
You missed the point again and no, a union type like
Array[A] | List[A]does not fix the problem.There goes your unbridled arrogance again, which I detected in way you handled Daniel in your blog. I did not miss the point in first two listed items— you did. Now let’s determine whether I missed the point or you did on this third listed item.
I do instead claim that a union type can fix the issue you have in this item. But that of course also requires incorporating associated types as I explained for the first listed item which you apparently did not understand. The fix will look something like
case class NextBatch[F, T]( batch: T … extends Iterant[F, T], whereTcan take on the type of the union. This is going to require some reorganization of your example. Thus what we appear have here are compounded errors on your part, because the first error leads to you not understanding how to do this item correctly. If you claim otherwise, then please explain why?EDIT: if I am incorrect about being able to resolve the issue with a non-disjoint, structural union, there is also a typeclass paradigm to address what you describe quoted below that can be accomplished with existential quantification and typeclasses, so that is the other option that I had mentioned that you seem to be ignoring in the following quote from your blog:
Wrong - if you're passing that type-class instance around, that's effectively a vtable so congratulations, you have an OOP encoding
-
Nope, that's not true.
Dude on this point, you’re simply wrong. @keean has shown us how to do iterators with typeclasses. If you were little bit less arrogant / flippant, I would say that as, “Sorry that’s not quite right.”
As I said, if I am shown I am incorrect, then I will apologize. I am simply defending what I believe to be correct.
I do not think my original post pointing out errors in your blog was overly harsh. I merely pointed out that I thought you were being unfair to Daniel and also the technical errors I think you made in your blog. I have also now learned from this discussion that Scala folks seem to be not using associated types in their typeclasses for some reason that I am not yet aware of. It seems to be the wrong way at least from the standpoint of wanting to be able to implement typeclass instances for non-parameterised types such as String.
@shelby3 you might have missed this lesson in your first 7 years of life, but when somebody explicitly asks to not be bothered, by email or GitHub mentions, it's the social norm to comply.
Cheers,
Is it just me, or is it rude to come here and post a reply and then say, "don't talk to me again"? Why even post here at all if you don't want to engage in discussion?
I think his post starts with a bit of an appeal to authority... maybe we should compare how many citations our respective research papers have had? (Just in case you cannot tell I am being sarcastic, it should not matter who you are, your arguments should speak for themselves as @Ichoran pointed out.)
Also he didn't explicitly mention "GitHub" mentions in his above reply:
Please don't involve me anymore and please don't send me any more emails.
So I guess he will never see this response, and won't be able to have a discussion about how we can implement iterators using typeclasses. Whilst I find @shelby3 challenging to deal with at times, I feel I have benefited from our discussions, either by having to refine my argument so it can be understood, discovering my unstated assumption, of just plain being wrong. It is hard enough to find anyone interested in this level of language design, that enthusiasm is something not to be ignored. I think I understand how Oleg Kiselyov might have felt when we first worked together :-)
Is it just me, or is it rude to come here and post a reply and then say, "don't talk to me again"? Why even post here at all if you don't want to engage in discussion?
Because I keep getting emails, both from GitHub and personal emails from @shelby3, telling me I'm wrong.
And the way I was engaged was rude in the first place. I don't want to block GitHub in my Inbox and I can't unsubscribe from this conversation if I keep being mentioned, as GitHub resubscribes me each time that happens. And I need my GitHub subscription and my Inbox to be free of crap.
And yes, I mentioned my background in order to make it clear that I love type classes and I have a background in using type classes successfully.
But then I don't have an interest in engaging in such discussions.
So yes, I am rude. Thanks, bye.
Btw, I think I'm unsubscribing for the 4-th time. Maybe now my wish will be respected.
Cheers,
Notice I did not include you with an '@' in my response, but I understand not wanting to receive too many emails.
It's a shame you don't want to discuss these issues, but I guess you are just not that interested in language design. You are wasting all our time with this approach, and instead of discussing how rude your response was, I could have explained how to implement iterators with type-classes and you might have learned something. Even if you don't want to hear it from me you could try reading (re-reading) Alexander Stepanov's "Elements of Programming" which explains it using C++ type-classes, which I am sure you already know are closely related. In fact Stepanov's C++ "Concepts" really are type-classes.
But then I don't have an interest in engaging in such discussions.
It seems strange to post an opinion on this subject online if you are not interested in discussing it. Why bother posting it in the first palace, just keep your opinion to yourself and you don't have to worry about people challenging it.
Notice I did not include you with an '@' in my response, so I don't see why you are being so rude?
Let me re-quote what I said: "because I keep getting emails, both from GitHub and personal emails from @shelby3, telling me I'm wrong.". This is how I found out about this conversation in the first place. As to why I keep getting notifications, including yours, I don't know, maybe it's that GitHub's broken.
I could have explained how to implement iterators with type-classes
I know that Iterators can already be implemented with type-classes. When you know how OOP works under the hood and how type classes work under a hood, you can see that there's an obvious equivalence.
A well versed C++ developer will never have doubts about what OOP is ... a vtable attached to objects for doing method dispatches at runtime, whereas type classes are nothing more than a specification for a global dictionary of such functions passed around, per type instead of per instance.
The issue with type classes in actual usage however are that:
- the instance can be and is often defined in another package/project than the original type definition, which makes them desirable from an "expression problem" perspective, however this means access to internals;
- giving access to internals for pure data structures is often not problematic due to encapsulation not being so crucial for pure stuff
- however it is often the case in the design of persistent collections that developers do dirty tricks for performance that are then hidden away; for example the
Vectorfrom Scala is a HAMT with some dirty optimizations and the internals needed to implement an efficientIteratorfor it are not exposed; trying to implement your ownIteratorfor it after the fact is an ugly business and this diminishes the utility of such a type class - if the
Iteratorprotocol is implemented for a type likeVectorwithout declaring it as a type class instance and then the type class instance gets implemented after the fact, because we already have the needed functions (so no internals access needed), then theIteratoras an OOP interface cannot hurt ... this is how we can define theTraversabletype class for Scala's collections efficiently
- as argued in my post, OOP is great at hiding implementation details, which often works at the type level as well; this is what it was made for, this is where it shines, this is a fact
- an
Iteratoras an OOP interface is often desirable precisely so you can have a heterogenous collection of things, because a type class design leads to homogeneity - the critique of my article missed this point entirely because via an OOP
Iterablethe user can use any collection in Scala's standard library and it doesn't need to care about it because the actual type being used is entirely irrelevant at the call site - the virtue of hiding type details is also important for languages that are less potent at working with generic types than Haskell; even in Haskell there's a limit to how many type-level tricks you can pull, people often ending up working with types so complex they don't understand, being amazed at how the compiler can sort it out, but then feeling despair when it doesn't; hence the general advice to keep it simple
Also, it is not my interest to waist your time.
I'm just not interested in conversations where people obviously have a tendency to talk past each other, due to aggressive language. From where I'm coming from, being accused of myopia is an offense and not exactly a conversation starter.
So if you wasted your time, it's not my doing.
This is my last message here, I promise.
Let me re-quote what I said: "because I keep getting emails, both from GitHub and personal emails from @shelby3, telling me I'm wrong.". This is how I found out about this conversation in the first place. As to why I keep getting notifications, including yours, I don't know, maybe it's that GitHub's broken.
I don't know why you keep getting emails? Notice I again have not '@' referenced you. I get large numbers of emails from recruitment people I don't want, I just ignore them. If you respond you encourage them :-)
I know that Iterators can already be implemented with type-classes. When you know how OOP works under the hood and how type classes work under a hood, you can see that there's an obvious equivalence.
Sure you might like to read the OOHaskell paper https://arxiv.org/pdf/cs/0509027 (Kiselyov et al) that I contributed to and was a follow up to the highly cited paper on heterogeneous collections that I co-wrote with Oleg, and Ralf Lemmel. The paper goes into some depth about type system soundness for subtyping etc.
A well versed C++ developer will never have doubts about what OOP is ... a vtable attached to objects for doing method dispatches at runtime, whereas type classes are nothing more than a specification for a global dictionary of such functions passed around, per type instead of per instance.
In C++ you never define functions per-instance, even a virtual function is per-type. The vtable is not on the object, it is on the class (at least it is in C++, languages like JavaScript with prototypical inheritance are different). Yes you can get a function into the instance by passing a closure to the constructor (if it is supported by the language) and then storing the closure to a property, but this is highly unusual.
the instance can be and is often defined in another package/project than the original type definition, which makes them desirable from an "expression problem" perspective, however this means access to internals;
This is no worse than defining a subclass in a different package from the superclass? Why would this be worse? Remember a typeclass is really just an interface and you can implement interfaces defined in other packages on classes to. In fact it would be a bit useless if you could not. The whole point is you do not need to know anything about the implementation because the type-class defines the interface. Consider a global property like "Printable" you definitely want this defined in a different package, so that everyone can declare they types instances of Printable. Then when I write any program I can just specify the requirement, "accept any Printable type" and I know that I can print it. This is what I want from generics.
giving access to internals for pure data structures is often not problematic due to encapsulation not being so crucial for pure stuff
Typeclasses separate concerns so encapsulation is not their responsibility. Classes conflate encapsulation with interface definition. You can have modules as an orthogonal concept to typeclasses to provide encapsulation and data-hiding.
however it is often the case in the design of persistent collections that developers do dirty tricks for performance that are then hidden away; for example the Vector from Scala is a HAMT with some dirty optimizations and the internals needed to implement an efficient Iterator for it are not exposed; trying to implement your own Iterator for it after the fact is an ugly business and this diminishes the utility of such a type class
Anything below the level of the "interface" should not be visible from outside. Dirty tricks for performance are fine, as long as you conform to the interface (type-class) which forms the contract between the implementer of the vector and the user. If you break those invariants then bugs will abound. So with a well designed type system this is not a problem, and is in fact to be encouraged, so typeclass users get a nice simple consistent interface, but also get the performance.
if the Iterator protocol is implemented for a type like Vector without declaring it as a type class instance and then the type class instance gets implemented after the fact, because we already have the needed functions (so no internals access needed), then the Iterator as an OOP interface cannot hurt ... this is how we can define the Traversable type class for Scala's collections efficiently
Typeclasses can always be implemented after the fact, that's one of their advantages. Typeclasses can depend on other typeclasses too. Typeclasses are interfaces, so you want to make sure your 'Vector' methods conform to the interface, or you write something that cannot be used generically.
as argued in my post, OOP is great at hiding implementation details, which often works at the type level as well; this is what it was made for, this is where it shines, this is a fact
Typeclasses, being interfaces hide implementation details too, and by avoiding subclassing and inheritance avoid the pitfalls of OOP.
an Iterator as an OOP interface is often desirable precisely so you can have a heterogenous collection of things, because a type class design leads to homogeneity
This is what existential types are for. You can simply define a collection:
data E = forall a . Printable a => E aWhich is a heterogeneous collection of things that implement the Printable interface. You should read the HList paper: http://okmij.org/ftp/Haskell/HList-ext.pdf which goes into a lot of detail on this topic. Ultimately open or closed anonymous unions may provide a better solution here than existentials, but I think the reasons are too complex to get into when you don't want to really engage.
the critique of my article missed this point entirely because via an OOP Iterable the user can use any collection in Scala's standard library and it doesn't need to care about it because the actual type being used is entirely irrelevant at the call site
This is true for typeclasses too, so I don't see why it missed the point. Typeclasses are interfaces (just with some more formal type system theory behind them).
the virtue of hiding type details is also important for languages that are less potent at working with generic types than Haskell; even in Haskell there's a limit to how many type-level tricks you can pull, people often ending up working with types so complex they don't understand, being amazed at how the compiler can sort it out, but then feeling despair when it doesn't; hence the general advice to keep it simple
Unlike your other points which seem disprovable, this is an opinion. In my experience hiding the type is almost always a bad idea. Existentials provide type-hiding as an orthogonal feature to type-classes and modules. I prefer this separation of concerns rather than mashing it all together into OOP classes.
If we adopt dependent types we can eliminate type-hiding altogether, as we push types into the runtime as well. I am not convinced this is a great idea, because like you I prefer to keep things simple. I know the type gymnastics we did in the HList paper will be beyond a lot of programmers. Part of what I have been working on since that paper is to preserve the style and flexibility but get rid of the type complexity. I think union types and typeclasses can do this, also compile time reflective typeclasses help allowing things like iterating over a tuple etc.
@alexandru wrote:
@shelby3 you might have missed this lesson in your first 7 years of life, but when somebody explicitly asks to not be bothered, by email or GitHub mentions, it's the social norm to comply.
Cheers,
Copy of both emails sent:
---------------------------- Original Message ----------------------------
Subject: In Defense of OOFP is incorrect
From: "Shelby Moore" <s…@coolpage.com>
Date: Thu, September 13, 2018 10:07 am
To: website@temp18.alexn.org
--------------------------------------------------------------------------
https://github.com/keean/zenscript/issues/30#issuecomment-421019593
---------------------------- Original Message ----------------------------
Subject: hey man you’re just incorrect okay
From: "Shelby Moore" <s…@coolpage.com>
Date: Thu, September 13, 2018 1:59 pm
To: website@temp18.alexn.org
--------------------------------------------------------------------------
https://github.com/keean/zenscript/issues/30#issuecomment-421093789
Maybe you should be more appreciative of those who correct your mistakes.
First email was sent because your blog comment feature was not functioning for me. Second email was sent because I thought it was rude that you showed me no appreciation for correcting (what I believe to your) mistakes (note I have not yet read your other replies). Every programmer should want to correct their mistakes. I thought it was also ridiculous that you would start a discussion and then not defend it or admit his mistakes. I judge the quality of a programmer not only by his mistakes, but how he learns. I certainly have a hell of a lot of mistakes. For example, see how I butchered Daniel’s Monads Are Not Metaphors blog. Good grief. Highly embarrassing for me. But I’m not going to blame others for my learning process. I appreciate Daniel for helping me learn.
@alexandru you’re ostensibly a very capable programmer. I don’t think anybody here has a goal of making you look bad in front of your peers. All good programmers value the correctness of the technical work. It’s not personal. I just wish you would not scapegoat others, such as Daniel when making your technical points, unless it really is necessary. And I hope you will be more circumspect when others challenge you, because they might just know something you don’t know yet. This can be a wonderful synergy of sharing. We don’t need to be snobbish to each other. We can work together and teamwork. As I wrote in my prior reply to, I admire teamwork. Perhaps it is just a personality difference between us? Maybe oil and water don’t mix?
As I said, I am more than willing to back off the animosity if you will also. What’s this snobbish crap about hurling allegations and then saying you will not engage anymore? It’s like analogous to the absurdity of thinking one can be partially pregnant.
I am not trying to gang up on you. I hate groupthink. It is not the three of us against you. Believe me. We will certainly recognize you for any correct point you make. There is no such corrupt practices here of ganging up on a new commentator.
@shelby3 you might have missed this lesson in your first 7 years of life, but when somebody explicitly asks to not be bothered, by email or GitHub mentions, it's the social norm to comply.
Btw, I learned this from my mother. When she doesn’t want to discuss the facts, she shuts down. Period. So reminding me of that trait of my mother (who I love dearly) is not going to ring the bell of a trait I respect.
I understand boundaries. But if you want absolute boundaries then don’t write that blog where you are acting as though you are an authority on why OOFP is not an anti-pattern. Or just ignore my first email. How many people will read this Github? Maybe a dozen.
Why write controversial blogs if you do not like controversy?
It’s not like I was stalking you where you live. Or really truly invading your space. I respected your boundaries. I sent you only two very terse emails. Direct to the point.
I do understand the feeling of being challenged and losing a technical argument. It is somewhat discomforting. But we learn from the experience. We become more diligent, more circumspect, etc..
I don’t know your culture there in your (birth)place. But typically I have found that males (at least West of the Hajnal line) tend to butt horns and then come away respecting each other more, unless one of the parties simply isn’t worthy.
Let me add that I don’t disrespect you (yet). I hope that was clear from my reply herein.
And the way I was engaged was rude in the first place.
How could I make the first email any less “rude”? All I wrote was the word “is incorrect”.
You must be referring to my first post in this thread challenging your blog. Well I guess maybe it was worded a bit strongly. But remember I was already a bit put off by you being rude to someone I respect. I took your blog post as being like “I know it all and these other people are wrong” but then you IMO incorrectly maligned or mis-characterized what Daniel’s point really is/was. So yeah I was sort of responding to what I thought was some guy trying to put others down while telling others they are wrong while making numerous errors. So I guess I worded my post (at 2am) perhaps a bit more strongly than I could have if I had taken more time to think it over. Note the times displayed on my emails are I think the server time in the USA, not my local time (I am in Asia).
But then I don't have an interest in engaging in such discussions.
Is this a protection mechanism to avoid any stress in life? Or to avoid wasting time?
I understand that. But is the discussion here with @keean who is an expert really a waste? I need to read your follow-ups to see what transpired since I was gone.
I'm just not interested in conversations where people obviously have a tendency to talk past each other, due to aggressive language. From where I'm coming from, being accused of myopia is an offense and not exactly a conversation starter.
I don’t know @sighoya’s age, but @keean and I are pushing 50+, so we don’t have the hypersensitivities of the younger generation who seems to think life is bed of roses. I think we’re just interested in getting into the meat of the technical discussion and letting the mistakes in communication style slide off our back over time. Forgive and forget sort of thing. I mean I get offended also, but I just fight back (or ignore of whatever is the most logical action). I don’t try to delude myself into thinking life can be devoid of conflict.
Btw, I recently wrote about the destruction that being caused by this “fairness” doctrine of the youth. (both the linked post and my follow-up reply are on point)
So that means I think y’all are very wrong on that point. The Millennials (and you may be a bit older than that?) really need to back away from that crap now before you turn the world into a political totalitarianism.
So if you wasted your time, it's not my doing.
Okay I wasn’t really trying to blame you for my consumption of my time with your blog. Actually obviously I want to test my knowledge to be sure I am not incorrect. I was just lamenting that I thought you had something important to say, but then I realized it seemed to be the same myopia I see over and over again. Again I want to review your follow-up with @keean before I decide if I was incorrect.
@keean wrote:
Notice I did not include you with an '@' in my response, but I understand not wanting to receive too many emails.
Well I do. Because I don’t want to multifurcate the consistent appearance of my Github posts in order to appease the whims of numerous Prima Donnas. Why should my consistent use of Github be handcuffed and shattered by feature request complaints which should instead be directed at Github’s programming and design team? And moreover I don’t think it is my fault that Github doesn’t give the user the control they need to ignore what they want to not see. I think I should use the @ as that is the standard way of addressing people on Github when replying to them. I have stopped using the @ when referring to people who are not party to the discussion, e.g. I have been using ‘Daniel’ instead of his Github username after the first time. So if Daniel does not want to be pinged, he is not.
@alexandru wrote:
A well versed C++ developer will never have doubts about what OOP is ... a vtable attached to objects for doing method dispatches at runtime, whereas type classes are nothing more than a specification for a global dictionary of such functions passed around, per type instead of per instance.
I think this is wrong. OOP is not just about a vtable, because typeclass bounds can also be expressed with a vtable for runtime polymorphism by employing existential quantification. This is the 3rd time I am mentioning that to you. I mentioned it in my first post challenging your blog.
Rather the anti-pattern of OOP is the subclassing, i.e. discarding the data types earlier than necessary and replacing them with some very inflexible rigor mortis of superclasses. It is not even subtyping that is an anti-pattern per se (as we have discussed in our Subtyping issue thread #8 because non-disjoint structural unions and polymorphic records expressed subtyping relations) but rather very specifically the premature binding of interface to data types and the premature discarding of the data types for superclasses. Although existentials can also do the former, they aren’t always necessary with typeclasses and I am exposing new ideas for existentials in the Subtyping thread.
Also we recently discussed whether OOP in general is not really subclassing. The general form of OOP appears to be Actors, is really not an anti-pattern as I argued in OOP Is Not Class Inheritance. But that is a tangential point.
as argued in my post, OOP is great at hiding implementation details
Actors can do that! Even they require you to not even rely on any ordering details of the global state machine.
an Iterator as an OOP interface is often desirable precisely so you can have a heterogenous collection of things, because a type class design leads to homogeneity
This can be accomplished with typeclasses by employing either unions or existential quantification of the typeclass bounds. The former remains open to adding new interfaces (which subclassing OOP can’t do), the later is open to new data types the same as subclassing OOP can do. By defining typeclasses, we get to employ both techniques. With subclassing OOP you’re locked into one way.
the critique of my article missed this point entirely because via an OOP
Iterablethe user can use any collection in Scala's standard library and it doesn't need to care about it because the actual type being used is entirely irrelevant at the call site
My critique did not miss this point. This is 3rd or 4th time I am repeating to you that unions and existentials can do what subsclassing OOP can do and more.
@keean responded to @alexandru:
the instance can be and is often defined in another package/project than the original type definition, which makes them desirable from an "expression problem" perspective, however this means access to internals;
This is no worse than defining a subclass in a different package from the superclass? Why would this be worse? Remember a typeclass is really just an interface and you can implement interfaces defined in other packages on classes to. In fact it would be a bit useless if you could not. The whole point is you do not need to know anything about the implementation because the type-class defines the interface. Consider a global property like "Printable" you definitely want this defined in a different package, so that everyone can declare they types instances of Printable. Then when I write any program I can just specify the requirement, "accept any Printable type" and I know that I can print it. This is what I want from generics.
I must concur. The entire point is the function or existential which is consuming the typeclass instances with a typeclass interface bound (e.g. requires) is that the said function or existential needs to know nothing about the implementation.
Seems @alexandru is making some point about encapsulation as in hiding a data type internals in a module and not allowing any other module to implement a typeclass instance for that data type. Indeed that is possible and we plan to support that feature. So I see no encapsulation advantage for subclassing OOP.
giving access to internals for pure data structures is often not problematic due to encapsulation not being so crucial for pure stuff
Typeclasses separate concerns so encapsulation is not their responsibility. Classes conflate encapsulation with interface definition. You can have modules as an orthogonal concept to typeclasses to provide encapsulation and data-hiding.
Good point. That is yet another way that typeclasses more correctly separate concerns compared to the rigor mortis of subclassing OOP. Encapsulation is the responsibility of the module.
the virtue of hiding type details is also important for languages that are less potent at working with generic types than Haskell; even in Haskell there's a limit to how many type-level tricks you can pull, people often ending up working with types so complex they don't understand, being amazed at how the compiler can sort it out, but then feeling despair when it doesn't; hence the general advice to keep it simple
Unlike your other points which seem disprovable, this is an opinion. In my experience hiding the type is almost always a bad idea. Existentials provide type-hiding as an orthogonal feature to type-classes and modules. I prefer this separation of concerns rather than mashing it all together into OOP classes.
If we adopt dependent types we can eliminate type-hiding altogether, as we push types into the runtime as well. I am not convinced this is a great idea, because like you I prefer to keep things simple. I know the type gymnastics we did in the HList paper will be beyond a lot of programmers. Part of what I have been working on since that paper is to preserve the style and flexibility but get rid of the type complexity. I think union types and typeclasses can do this, also compile time reflective typeclasses help allowing things like iterating over a tuple etc.
EDIT 2020: the issue of type inference with underconstrained generics and type classes has been recently brought to my attention. I think you may have a point here that by attaching the interface to the data type and eliding the data type by replacing it with its subclass in function arguments instead of type class bounded type parameters for function arguments, there’s less opportunities for accidentally stumbling into such complexity sinkholes. As usual that systems with more expressive power violate the Principle of Least Power but perhaps justifiably so.
@shelby3 this is unbelievable, but maybe we don't speak the same language.
What I write on my blog is my own business. And your whole rebuttal is flawed due to logical fallacies and misunderstandings of the subject being discussed. And if you would have approached me on a friendly manner, I might have responded in kind and maybe we would have learned something. As it is however, I'm not interested.
So let me leave clear instructions that you can hopefully parse:
- Stop sending me emails
- Stop mentioning me on GitHub issues
Or otherwise you're going to find out how rude I can be.
@shelby3 - When busy and capable people don't have time to discuss your views with you, the custom is to respect it. I happen to agree with "In Defense of OOFP", and would even strengthen it in places, but whether I agree or disagree I don't expect Alexandru or anyone else who writes a blog to entertain my alternate views for very long, and I certainly don't expect that they should appreciate it if I repeatedly involve them when I ask them to stop.
Civility is incredibly enabling. It allows people with different perspectives to cooperate or at least not bother each other and get stuff done. Very often, getting some stuff done is more important than who is technically more correct about some point, and very often, all parties who believe themselves to be correct have inadequate basis for their beliefs and thus cannot even in principle convince each other rationally. Indeed, in such cases, maintaining a diversity of beliefs is probably an advantage since then alternate paths can be explored and if one really is vastly better, the evidence for this will be found more rapidly.
You seem to spend a great deal of time discussing things. That's fine. Other people may wish to spend more time building, or with other life activities. It is civil to let them, even if they express irritation or disagreement before disengaging.
@keean - I think a lot of what is being missed here is what OOP makes easy. I don't really care if there's an equivalent complicated way to encode things with some other framework when it comes time to writing actual code. (I do care theoretically; it's nice to know which systems are strictly more powerful than which others, and so on. But then I have to write code to actually get things done.)
In certain contexts, OOP enables complex code reuse with minuscule effort. These contexts tend to be where there is a hierarchy of capability, and somewhere up towards the root of that hierarchy the capability depends on a particular data structure (especially when the data is mutable). OOP allows the data and the methods that require it to be conveniently moved around in concert, without the syntactic and computational overhead of composition that this entails. (For example, consider the case where you have a function that can pick Ps, and you have Qs that contain Ps; you use your function to pick a P from your Qs, but now how do you get the Q back? Headache! But if Q extends P you are done! Yes, you could make everything more complicated such that you could pick from any A as long as you supply a A => P, or somesuch, but that's a bunch more work).
(Aside regarding data hiding: in Scala, Vector has a number of really finicky operations that everyone is shielded from via a combination of inheritance and composition. Having a typeclass able to see any of those details is a recipe for disaster--it is very easy to overlook something and introduce bugs when working with the data structures. OOP methodology makes the hiding of the data and implementation pretty straightforward, and yet you can still override for performance (and Vector does so liberally). You intrinsically can't add a typeclass for basic collections operations to Vector after the fact because you need to see the internals to be able to do so, but if you see the internals you're going to do it wrong; the only way is to have people who really understand it and test carefully implement a predefined set of behaviors. I.e. exactly what OOP encourages you to do.)
Typeclasses are also great, but they are--in every language I'm familiar enough with to judge--great at different things. They're sufficiently powerful and general that you can, if you want, encode a vtable in them; and if you have union types, you can glue together vtables in a manual inheritance-like scheme. And you can even, if you want, create for yourself many of the problems of OOP (gluing together things that syntactically work at the type level, but are semantic nonsense)! But that's irrelevant. The question I have is not about the capability, but the facility with which you can do things. I can write sqrt(plus(times(a, a), times(b, b))) instead of (a*a + b*b).sqrt, but that doesn't mean I'd ever willingly do the first if the second was an option.
I view "In Defense of OOFP" in this light. It's not that typeclass-based solutions are impossible to use in cases where OOP ones are touted, it's that they're awkward. Maybe more awkward than they need to be in Scala, but even in an ideal setting they involve more machinery.
If you want your "toolbox" to consist of exactly one hammer, I'd agree that OOP isn't the right hammer. (Just try to implement a sane equals!) But with a more diverse set of tools, I like having OOP capabilities around, mostly for the reasons stated in "In Defense of OOFP".
@alexandru wrote:
Stop mentioning me on GitHub issues
Or otherwise you're going to find out how rude I can be.
I refuse! You are not going to dictate to me what I can write and not write. As long as you post in the Github where I am participating, I will reply to you in the normal way. Please feel free to demonstrate your rudeness if you are unable to participate as a normal human being does. In my first message in this thread today, I tried to be conciliatory to you.
As if @ is a crime. It is the normal way of replying on Github when you made a post in this issues thread. If you do not like the features of Github then perhaps you can ask Microsoft to improve the product.
You’re acting like a spoiled prima donna. As if we are supposed to change the way we use Github just because you want to express some personal power over me.
What I write on my blog is my own business.
And what I write here is my own business, unless you choose to engage. You chose to engage. Nobody put a gun to your head.
And your whole rebuttal is flawed due to logical fallacies and misunderstandings of the subject being discussed.
Yet you have failed to explain that allegation. I think @keean and I have adequately explained it.
And if you would have approached me on a friendly manner, I might have responded in kind and maybe we would have learned something. As it is however, I'm not interested.
Ah I know this is how tribal cultures are. I experience this in the Philippines. Best friends here will hack each other with machetes to death because one of them said one wrong word while they were drinking. Very primitive behavior. I guess we all have that in us to some extent from prehistoric times.
Video: Fighting over "Honey": Thai men battle it out with a sword in broad daylight
Stop sending me emails
I have not sent you any emails today, other than the two listed from yesterday. I do not plan to send you any more emails. I explained to you that I sent you a very terse email because I could not post to your blog. That is absolute truth. Then at like 2am or what ever it was when I had been working like 16 hours nonstop, I sent you second very concise email out of frustration for you making allegations which are false and you continue to make false allegations.
but when somebody explicitly asks to not be bothered, by email or GitHub mentions, it's the social norm to comply
It’s a social norm to not to a “drive-by-shooting” and then make demands on others. You could have ignored my original terse email informing you that I had rebutted you here. I didn’t even name you nor ping you with @ that first post here. You instead decided to engage. Once you made the decision to engage here, you can’t dictate to me that I can’t reply or that I can’t use the Github feature of attributing a response with @. Your ignorance of the features of Github which enable you to block me, if you don’t want Github to ping you, is not an excuse for you to threaten me and attempt to dictate how I can use Github. Your hubris is equally “unbelievable”.
As it is however, I'm not interested.
So then GTFO then. And don’t let the door hit you in the ass on the way out. I tried being conciliatory but it seems you do not appreciate anyone here. So why are you still here?
For the record, GitHub has a way to deal with unwanted @ - https://help.github.com/articles/blocking-a-user-from-your-personal-account/
@Ichoran wrote:
When busy and capable people don't have time to discuss your views with you, the custom is to respect it.
Hey I do not want to get in a heated (non-technical) argument with you. I appreciate all of your input. It has been very helpful. But I did not force him to do anything. I sent an initial email letting him know I thought he was incorrect. It was the most terse possible email I could possibly write.
He chose to engage and he made what I have explained were incorrect and false allegations alleging that I missed all the points. I am so far convinced that he missed most of the points. Regardless of who is correct on that, (yesterday before this blew up as much as it has) I simply expressed my frustration to him with a terse email letting him know I think he should be more appreciative of those who correct his mistakes. Because I genuinely think we should. I know I do. If he showed me a mistake of mine, I would be appreciative.
I did not email him after that. I did not continue badgering him. I did not stalk him. I came back here in this thread to respond to the discussion that has transpired since. I responded normally.
I think his only valid claim about my behavior is that my initial post was worded quite strongly. And I said I think he deserves it for maligning Daniel. That was my opinion at the time I wrote the post. I hence tried to be conciliatory but he wants no part of that. So he is welcome to leave since he is so busy and capable and leave the rest of us incapable fools who are not busy to drool all over ourselves.
That being said, I feel absolutely no animosity towards you. And my next reply will consider your much appreciated technical feedback.
Also I thank you for defending @alexandru. I wanted him to see that we tolerate that here. We not totalitarians.
Civility is incredibly enabling. It allows people with different perspectives to cooperate or at least not bother each other and get stuff done. Very often, getting some stuff done is more important than who is technically more correct about some point, and very often, all parties who believe themselves to be correct have inadequate basis for their beliefs and thus cannot even in principle convince each other rationally. Indeed, in such cases, maintaining a diversity of beliefs is probably an advantage since then alternate paths can be explored and if one really is vastly better, the evidence for this will be found more rapidly.
I agree with this. At 2am, I allowed myself to be put off by his what I perceived to be incivility towards Daniel and I let that feeling creep into the wording of my initial post. I tried to backtrack from that with a more conciliatory post today but he has expressed no interest in that. Thus I think I correctly judged his personality. I am pretty good at detecting personality early on. His behavior has proven he is exactly the person that picked him to be when I read how he could be very careless when referring to a long-time Scala expert (Daniel) that AFAICT was around a long-time before @alexandru arrived on the Scala scene in 2012.
It is civil to let them, even if they express irritation or disagreement before disengaging.
Again I don’t see how I forced him to engage. Or how I prevented him from doing what he wants to do with his own time. I challenged him. He has every right to ignore me. I did not send him long, endless emails filling up his Inbox. His ignorance of the Github block feature is not my fault.
I already tried to backtrack a bit and be conciliatory [and it obviously failed to be a mutual olive branch that he could embrace].
Having said that, I appreciate you making these points and giving me the opportunity to respond to them. Your effort was not in vain. I do understand I should just avoid that sort of thing entirely in the future. It is not worth the potential insight gained. I need to trust my initial judgement of people and steer clear instead to avoid losing time on useless fighting.
You seem to spend a great deal of time discussing things. That's fine. Other people may wish to spend more time building, or with other life activities.
Actually most of my life as a programmer I spent all my time alone coding and not discussing. I would actually like to get back to that.
But what I learned when I first starting trying to figure out what PL I needed (back in 2008 or so) was how much I didn’t know about PL theory and design. And so I am here discussing a lot because I want to make sure I do not go off into my own world and miss very important things like I had done until 2009 or so. I remember when I was made a laughing stock of the Lambda-the-Ultimate because I didn’t agree and know why everyone there was telling me subclassing is an anti-pattern. I felt like a fool. Because I was a fool. I was lacking knowledge.
You should not assume I am not a prolific coder. I had coded probably a million LOC by myself before I got sick with TB in 2011. I did not know the illness I had was TB until 2017. I had no cough. I suspected Multiple Sclerosis because my symptoms were like M.S.. The doctors only found NAFLD. Because no doctor thought of testing for TB! Apparently the TB can also infect the gut and it wrecked my liver, spleen, etc pretty bad over those years.
I am just getting back on my feet healthwise. Really I am disappointed that you think I was not a prolific coder. You don’t know my history.
@shelby3 - Your continued use of @ is hard to interpret in any way but as intentional badgering. It is way more normal to not disturb people who don't want to be disturbed than it is to use @ when mentioning them. GitHub is set up for @ to be helpful, trusting that its users will be responsible.
Also, I didn't mean to imply that you weren't a prolific coder, just that the priorities you have right now, for whatever reason you have them, do not necessarily align with the priorities others have, for whatever reasons they have them. If they also are spending a lot of time discussing issues online, then one might rightly frown upon a stated wish to be left alone. But if they're not (and I see little evidence that Alexandru is), the polite thing to do is to let them get on with whatever it is they're doing.
Even if they're wrong.
Even if they're arrogant.
Even if they "started it".
You can always correct the record here to your satisfaction without employing @.
@Ichoran I appreciate your detailed response.
I think a lot of what is being missed here is what OOP makes easy.
Sure things have to be easy, we need to be careful about the complexity budget. However we also need to be careful not to mistake familiarity for ease. If we cannot offer simple solutions to the common programming problems, then it's simply not going to work. However I very much see design-patterns as making obvious the flaws of OOP. Every time you reach for a design pattern you are coding something that has a non-obvious solution in OOP. Of course over time you become familiar with these design patterns, but we must not mistake this for ease. If they were obvious they would not need documenting in the first place.
In certain contexts, OOP enables complex code reuse with minuscule effort.
As do generics.
These contexts tend to be where there is a hierarchy of capability, and somewhere up towards the root of that hierarchy the capability depends on a particular data structure (especially when the data is mutable).
In my opinion code does not belong in objects. Algorithms are always the same, they do not depend on the object on which they operate. A sort can sort any object, why would it be on an object. So we should not be finding any complex algorithms embedded in objects.
For example, consider the case where you have a function that can pick Ps, and you have Qs that contain Ps; you use your function to pick a P from your Qs, but now how do you get the Q back? Headache! But if Q extends P you are done! Yes, you could make everything more complicated such that you could pick from any A as long as you supply a A => P, or somesuch, but that's a bunch more work).
You see by having a specific function that can pick 'Ps' you have already gone wrong :-) you have a 'pick' function that can pick whatever (polymorphic), then you supply a projection from Q to the part of Q that you want that is used by pick that returns the Q. So pick would be:
pick : c -> (a -> Bool) -> a requires Collection(c), ValueType(c) == a
This pick is far more reusable, as it can be used on any collection (array, list, map whatever) containing any type, and with any projection, so it can be more complex that just a property of Q, but can be a deeply nested property, a combination of properties etc. A function like pick is so widely applicable it would probably be a standard library function, so you would not ever have to implement it.
(Aside regarding data hiding: in Scala, Vector has a number of really finicky operations that everyone is shielded from via a combination of inheritance and composition. Having a typeclass able to see any of those details is a recipe for disaster--it is very easy to overlook something and introduce bugs when working with the data structures. OOP methodology makes the hiding of the data and implementation pretty straightforward, and yet you can still override for performance (and Vector does so liberally). You intrinsically can't add a typeclass for basic collections operations to Vector after the fact because you need to see the internals to be able to do so, but if you see the internals you're going to do it wrong; the only way is to have people who really understand it and test carefully implement a predefined set of behaviors. I.e. exactly what OOP encourages you to do.)
This seems bad to me. You can easily have a vector with all the performance optimisations, that still complies with your interface (typeclass) after all this interface is the contract between developer of the vector and the user of the vector.
Typeclasses are also great, but they are--in every language I'm familiar enough with to judge--great at different things. They're sufficiently powerful and general that you can, if you want, encode a vtable in them; and if you have union types, you can glue together vtables in a manual inheritance-like scheme.
Why would you want to do that? That is completely missing the point. Of course trying to emulate the OOP idiom using type-classes is going to be more complex. Inheritance and generics are two different ways to avoid repeating yourself. Use generics like generics :-)
The question I have is not about the capability, but the facility with which you can do things. I can write sqrt(plus(times(a, a), times(b, b))) instead of (aa + bb).sqrt, but that doesn't mean I'd ever willingly do the first if the second was an option.
What about:
sqrt(a * a + b * b)
Operator overloading is one of the things that type-classes do really well.
I view "In Defense of OOFP" in this light. It's not that typeclass-based solutions are impossible to use in cases where OOP ones are touted, it's that they're awkward. Maybe more awkward than they need to be in Scala, but even in an ideal setting they involve more machinery.
I think the opposite, that OOP makes things more awkward. You cannot generically reuse algorithms, so you end up having to implement them yourself and making mistakes, introducing bugs. Stepanov envisions a worldwide repository for algorithms as they are generic there only ever needs to be one implementation of quicksort, and everyone can contribute to improving it. You cannot really get better code reuse than that.
You should not really be implementing your own data structures, you are much better off using compositions of generic containers with well known properties, like a List of Maps, a Vector of BTrees, a Trie of Priority Queues, a Deque of DAGs etc. You should not be implementing your own algorithms, you are better off composing well known generic algorithms like sort, rotate then partition etc.
If you want your "toolbox" to consist of exactly one hammer, I'd agree that OOP isn't the right hammer. (Just try to implement a sane equals!)
Here we agree, equals is trivial using typeclasses. Also I want a simple language, not a multi-paradigm mess like C++. So it is true that I am looking for a simple single paradigm that doesn't make anything that should be simple into something complex. Of course there is no avoiding the complexity of the problem domain, but you should not have to fight with the language as well.
I like having OOP capabilities around, mostly for the reasons stated in "In Defense of OOFP".
I think that there is a simpler and better way. There is a saying that extraordinary claims require extraordinary evidence. There is no denying the popularity of OOP, and I was an OOP advocate before they started teaching this stuff in undergraduate courses, so I understand the attraction. It's only when programming large complex systems that have to be maintained over time that the problems become more obvious - with process workflows split between multiple objects making it hard to work out what is going on without a lot of effort. Design patterns like DCI (the process equivalent of MVC) help a lot, and this is how I would build an application in an OOP language, but it would be a better if the language directly supported such abstractions. It's like you can write object oriented code in 'C', but the language does not make it easy, I find it is possible to write good abstractions in OOP but the languages do not make it easy.
Look at the C++ standard library, this started out as an OOP project, but it didn't really work, then Stepanov et al produced a generic alternative that was so much better that modern C++ has abandoned the old approach and the whole standard library is implemented using generic techniques. You have to think if reimplementing C++ from scratch why would you include the OOP stuff?
The kind of thing I am advocating uses parametric polymorphism constrained by typeclasses (with associated types) for abstraction, modules for data-hiding, union types for runtime polymorphism.
@Ichoran wrote:
Your continued use of
@is hard to interpret in any way but as intentional badgering.
[EDIT 2020: frankly coming back to this 2 years hence, I find it highly irrational to conflate some imagined intention on my part with my continued use of the discussion features of Github as I always have consistently used them and as they are designed to be used to reference the person I am replying to so that the reader will know which post to look upthread for context. Frankly you’re too smart to make that mistake, so in retrospect with a calm rational mind I can only presume a Freudian slip. Others may be more careless about how their comments will be followed by the reader but that preference shouldn’t be equated with an intention on my part when I am consistently doing that for past years without any intention to badger with my comment posts. Ugh.
Did I initially intend to challenge him? Yes. Did I intend to continue dragging him in, no. I was continuing the discussion here in this thread. He can and did eventually choose to ignore this thread. I presume he eventually found a Github setting to put this thread on ignore which he could have done from the get go instead of trying to dictate to me that I have to change the way I have always referenced comment posts (just for him because he’s so special) when replying to them.]
I don’t see it that way. I see it as where ever I refer to someone by name, I should use that feature because it links to their Github profile, so that the reader will know who I am addressing. Sorry I am just using the feature of Github how it is intended. And he can block me if he does not want to be pinged. And I am doing that only because he participated here in this thread. Nevertheless I have started referring to him as “him” as you can see.
I think what you are getting at is you think I am purposefully trying to resist his demands. Well yes, nobody will tell me to stop using the Github interface the way I have always used it. I have often used @ more than once in a single post. I have not started increasing that to badger him. So I do hope you will not judge me incorrectly.
Also I think there is an important principle here at stake and you may not realize how close we are in the West to burning all the books again: http://esr.ibiblio.org/?p=8120
Very important to read that link!
If they also are spending a lot of time discussing issues online, then one might rightly frown upon a stated wish to be left alone. But if they're not (and I see little evidence that Alexandru is), the polite thing to do is to let them get on with whatever it is they're doing.
He had every right to not ever come over here and engage. Then he would have indeed been left entirely alone (other than the initial extremely terse email letting him know I had posted a rebuttal to his blog, which would have been a blog comment if his blog was accepting comments from me but the Disqus would not load for me to allow me to make a blog comment). I did not even use @ or refer to him by name in my first post! Please get your facts straight because you know that mistakes have a way of being repeated as truth.
Well guys look on the bright side. We were able to elicit more good feedback from @(&#$^&^#Ichoran.
There’s usually an upside when people are sincere.