Introduction
I recently attended the Scalar Conf in Warsaw which featured several interesting speakers and presentations, many of which pertained to hot new features of Scala 3 (which I will probably never use due to overwhelming tech debt) as well as a 'surprise' presentation from Mr. Martin Odersky, where he discussed opportunities to shift away from reliance on monadic semantics in favor of direct styling allowed by the suspended continuation/boundary paradigms.
I sheathed my sneakers at this remark.
The goal of this post is to capture my learnings from Nicolas Rinaudo's wonderfully silly presentation titled Things that are things, but not other things1 which approached the daunting challenge of grokking category theory abstractions in my preferred format of counter example: ⚔️.
The problem, of course, is that I now have to find something interesting to say about monads. And one thing that I often found frustrating when learning about that family of abstractions is how just about everything is a monad - why even have all these intermediate abstractions if everything is all the things, all the time?
This ties in to one of the ways I learn things: when given a definition of a thing, it helps me just as much to have examples of things that are not the thing as of things that are the thing. I really wished I'd had such examples of things that are things, but not other things, while learning abstractions.
Taxonomy of the Typelevel Heirarchy
First of all – just look at this mess:2
A common issue I encounter is failure to lean on the minimum-necessary type class. Implicit instances of Applicative, Monad, Apply, etc. crop up in the codebase, and transient/impartial recollection of rote definitions of these terms flutter through my head as I fail to intuit why the author of any given method signature chose one type class and not the other. Additionally, I know that I am not alone in this confusion as I overwhelmingly see one of the stronger type classes (i.e. Monad), implying that many of my fellow contributors also don't know what the tightest bound on their use case is, just that Applicative or Monad will probably suffice. (Certainly, my observations are biased since much of my work takes place at the level of abstraction where these are in fact the correct type classes to rely upon, but perhaps not all the time...)
Nicolas' presentation helped make at least a portion of the above diagram less scary by breaking each element of the following hierarchy down:
Type Constructor
The first and lowliest category in the hierarchy is the Type Constructor. The narrowest "higher kinded type" –if you could even call it that– which encapsulates the behavior of parameterization e.g.
List[A] which alone is not worth breaking.
Functor
The Functor is the first category on the list we'll break by demonstrating what it is and what it is not. A Functor is a mapping between categories, usually for us it's a mapping between the category of types. Referencing the overall hierarchy diagram above, we can see that all we need to transform our type constructor into a Functor is an implementation of map:
defmap[A,B](f:A⟹B)(fa:F[A]):F[B] We have some F[A]:
defmap[A,B](f:A⟹B)(fa: F[A]):F[B] which we want to map to an F[B]:
defmap[A,B](f:A⟹B)(fa:F[A]):F[B] which we achieve via the first argument of map which is a function f from A to B:
defmap[A,B]( f: A ⟹ B )(fa:F[A]):F[B] which is then reinserted into our type constructor F which is the definition of the map itself:
defmap[A,B](f:A⟹B)(fa:F[A]):F[B] So, how can we break a Functor?
Function
Let's examine a practical example where our type constructor maps some X to an A; this is the definition of a function
typeF[A]=X ⟹ Adefmap[A,B](f:A⟹B)(fa:F[A]):F[B]=??? We can verify that this fits the pattern we need to be a Functor by checking it has all the constituent parts including an fa
typeF[A]=X⟹Adefmap[A,B](f:A⟹B)( fa: F[A]):F[B]=??? an f:
typeF[A]=X⟹Adefmap[A,B](f: A ⟹ B)(fa:F[A]):F[B]=??? and the desired resultant F[B]:
typeF[A]=X⟹Adefmap[A,B](f:A⟹B)(fa:F[A]):F[B]=??? but we're missing the implementation of map to take us from X to B:
typeF[A]=X⟹Adefmap[A,B](f:A⟹B)(fa:F[A]):F[B]=??? well, map has all the pieces we need to just compose:
typeF[A]=X⟹Adefmap[A,B](f:A⟹B)(fa:F[A]):F[B]=fa andThen f yielding the correct result:
Breaking Function
To break a function so that it is no longer a Functor, we can simply reverse the mapping of our type constructor:
typeF[A]=A ⟹ Xdefmap[A,B](f:A⟹B)(fa:F[A]):F[B]=faandThenf which then breaks our implementation of map: since fa and f no longer compose.
typeF[A]=A⟹Xdefmap[A,B](f:A⟹B)(fa:F[A]):F[B]=fa andThen f Concretely, we might have a function that goes from A⟹Boolean:
typeF[A]=A⟹Boolean such a function is called a Predicate
typePredicate[A]=A⟹Boolean which is not a Functor :).
Key takeaways
A function parameterised on its:
- output type is a Functor
- input type is not a Functor
Interlude: Product Types
To break the rest of the higher kinded types, we'll rely on the slightly-more-complex composite type called a Product. Glossing over many tedious laws of Category Theory, a Product is simply when you cram two types into a box like so:
caseclass×[A,B](fst:A,snd:B) Given any combination of A and B, we can get a Product via "Product Introduction":
defbuild[A,B](a: A,b: B):A×B=a×b Similarly, from any instance of a Product, we can extract the individual elements comprising it via "Product Elimination":
deffst[A,B](ab:A×B):A=vala×b=aba Products are Functors
Revisiting our previous diagram of map:
typeF[A]=A × Xdefmap[A,B](f:A⟹B)(fa:F[A]):F[B]=??? we still have all the necessary parts to complete the mapping, we must simply perform product elimination in the implementation:
typeF[A]=A×Xdefmap[A,B](f:A⟹B)(fa:F[A]):F[B]= (???: B) × (???: X) We can get the product A×X by definition of fa
typeF[A]=A×Xdefmap[A,B](f:A⟹B)(fa:F[A]):F[B]= val (a × x) = fa (???:B)×(???:X) from which we can get a B from the provided f:
typeF[A]=A×Xdefmap[A,B](f:A⟹B)(fa:F[A]):F[B]=val(a×x)=fa val b = f(a) (???:B)×(???:X) and now we have an X and a B with which we can construct our F[B]
typeF[A]=A×Xdefmap[A,B](f:A⟹B)(fa:F[A]):F[B]=val(a×x)=favalb=f(a)(???:B)×(???:X) All we have to do is introduce the product between these two like so:
typeF[A]=A×Xdefmap[A,B](f:A⟹B)(fa:F[A]):F[B]=val(a×x)=favalb=f(a) b × x Key takeaways
A binary parametric product type is:
- a Functor
Functor but not Apply
Advancing along the hierarchy diagram, we can now use our knowledge of Product types to break the Apply type class. An Apply.3 An Apply is anything that lets us take the product of three types in any order isomorphically. This necessitates another map which takes two arguments in the second list rather than one – map2:
defmap2[A,B,C](f:(A,B)⟹C)(fa:F[A], fb:F[B]):F[C] Breaking the signature down, observe that we start with a product of two type constructors over A and B:
defmap2[A,B,C](f:(A,B)⟹C)( fa: F[A], fb : F[B] ):F[C] We want an F[C]
defmap2[A,B,C](f:(A,B)⟹C)(fa:F[A], fb:F[B]):F[C] and we will produce this F[C] by way of the provided mapping f:
defmap2[A,B,C]( f: (A, B) ⟹ C)(fa:F[A], fb:F[B]):F[C] and all together this gives us the definition of map2:
Implementing map2 for Product
Let's take a look at how we would go about implementing map2. For starters, we have our (for the time being) trusty Product and the method signature:
typeF[A]=A × Xdefmap2[A,B,C](f:(A,B)⟹C)(fa:F[A], fb:F[B]):F[C]=??? We're also given fa and fb so by product elimination we get F[A] and F[B] individually from our initial product:
We know we want an F[C], so we can pencil that in as our target type:
typeF[A]=A×Xdefmap2[A,B,C](f:(A,B)⟹C)(fa:F[A], fb:F[B]):F[C]=(???: C) × (???: X) Next, we can perform product introduction via fa and fb to yield X×X and A×B:
typeF[A]=A×Xdefmap2[A,B,C](f:(A,B)⟹C)(fa:F[A], fb:F[B]):F[C]= val (a × x1) = fa val (b × x2) = fb (???:C)×(???:X) and we can use the provided f on our new product of A×B to get a C:
typeF[A]=A×Xdefmap2[A,B,C](f:(A,B)⟹C)(fa:F[A], fb:F[B]):F[C]=val(a×x1)=faval(b×x2)=fbval c = f(a, b)(???:C)×(???:X) Lastly, we just need to combine our product of Xs into a single X, and drop it into our type constructor to get an F[C]:
typeF[A]=A×Xdefmap2[A,B,C](f:(A,B)⟹C)(fa:F[A], fb:F[B]):F[C]=val(a×x1)=faval(b×x2)=fbvalc=f(a,b)(???: C)×(???: X) To combine our x1 and x2 we do literally need a combine method, but its implementation is entirely arbitrary at this abstract level. If the types are numeric it could arithmetically combine them, or perhaps concatenate them if they're strings or some binary bytestream, we could also just select the first, or just the second, flip a coin to pick which one to select, etc – it does not matter:
typeF[A]=A×Xdefmap2[A,B,C](f:(A,B)⟹C)(fa:F[A], fb:F[B]):F[C]=val(a×x1)=faval(b×x2)=fbvalc=f(a,b)val x3 = combine(x1, x2)(???:C)×(???:X)defcombine[X](lhs:X,rhs:X):X=??? and now we have a single X as well as a C to yield our desired F[C], thus completing the implementation of map2
typeF[A]=A×Xdefmap2[A,B,C](f:(A,B)⟹C)(fa:F[A], fb:F[B]):F[C]=val(a×x1)=faval(b×x2)=fbvalc=f(a,b)valx3=combine(x1,x2)c × x3defcombine[X](lhs:X,rhs:X):X=??? Breaking Product
The simplest way for us to illustrate something that is a Product and therefore a Functor but is not an Apply is to semantically constrain our Product.
The humorous example Nicolas provided was working with Jira labels. We can define our Product to be an Enum which is perfectly valid:
enumLabel:caseFrontEndcaseBackEndcaseBugcaseFeature and use this as our X:
typeF[A]=A×Labeldefmap2[A,B,C](f:(A,B)⟹C)(fa:F[A], fb:F[B]):F[C]=val(a×x1)=faval(b×x2)=fbvalc=f(a,b)valx3=combine(x1,x2)c×x3defcombine[Label]( lhs: Label, rhs: Label ):Label=??? However, there is no longer a valid abstract definition of combine since Label is closed... That is – there is no valid definition of a ticket which pertains to both the FrontEnd and the BackEnd for example, and so we can no longer guarantee that this kind of product can be combined:
typeF[A]=A×Labeldefmap2[A,B,C](f:(A,B)⟹C)(fa:F[A], fb:F[B]):F[C]=val(a×x1)=faval(b×x2)=fbvalc=f(a,b)valx3=combine(x1,x2)c×x3 def combine[Label] ( lhs: Label, rhs: Label ): Label = ??? Thus, the whole implementation falls apart as we cannot introduce a product between our c and x3 since we cannot get an x3 without somehow combining our x1 and x2
typeF[A]=A×Labeldefmap2[A,B,C](f:(A,B)⟹C)(fa:F[A], fb:F[B]):F[C]=val(a×x1)=faval(b×x2)=fbvalc=f(a,b)val x3 = combine(x1, x2) c × x3 def combine[Label] ( lhs: Label, rhs: Label ): Label = ??? Key takeways
A binary parametric product type:
- is a Functor
- is not an Apply if the other member is not a Semigroup
- Label is not a Semigroup because the combine operation is not closed on its members
Apply but not Applicative
For something to be Applicative it must already be an Apply and have an implementation of pure. All pure does is wrap a value into the given (higher kinded) type constructor:
defpure[A](a:A):F[A] Let's take a look at how this might look for a Product:
typeF[A]=A×Xdefpure[A](a:A):F[A]=(???:A)×(???:X) We can see that pure only takes an A, so we need an aribtrary way to introduce an X. We do this by way of a method called empty
typeF[A]=A×Xdefpure[A](a:A):F[A]=a×emptydefempty:X=??? Breaking Product
Once again, the easiest way to break Applicative is by way of Product by defining it in such a way that there is no valid implementation of empty, e.g. replacing the arbitrary type X with PosInt:
typeF[A]=A×PosIntdefpure[A](a:A):F[A]=a×emptydefempty:PosInt=??? There is no reasonable notion of emptiness for the set of positive integers, so we cannot implement empty – which in turn breaks the rest of our implementation:
typeF[A]=A×PosIntdefpure[A](a:A):F[A]=a×empty def empty: PosInt = ??? Key takeways
A binary parametric product type:
- is a Functor
- is not an Apply if the other member is not a Semigroup
- is not an Applicative if the other member is not a Monoid
- a Monoid has an associative operation and an identity element
Apply but not Flatmap
FlatMap
Given an F[A] and wanting an F[B], a FlatMap takes a function f from A⟹F[B]:
defflatMap[A,B](f:A⟹F[B])(fa:F[A]):F[B] Deriving the implementation of flatMap for a Product should be a familiar process by now. We start with our target product F[B] and work backwards:
typeF[A]=A×XdefflatMap[A,B](f:A⟹F[B])(fa:F[A]):F[B]= (???: B) × (???: X) First, leveraging our fa, we introduce an X:
typeF[A]=A×XdefflatMap[A,B](f:A⟹F[B])(fa:F[A]):F[B]= val (a × x1) = fa (???:B)×(???:X) Then we map to our F[B] via f and perform product elimination to get a product of Xs:
typeF[A]=A×XdefflatMap[A,B](f:A⟹F[B])(fa:F[A]):F[B]=val(a×x1)=fa val fb = f(a) (???:B)×(???:X) typeF[A]=A×XdefflatMap[A,B](f:A⟹F[B])(fa:F[A]):F[B]=val(a×x1)=fa val (b × x2) = f(a) (???:B)×(???:X) We use a handy-dandy combine to go from our X×X product to a single X:
typeF[A]=A×XdefflatMap[A,B](f:A⟹F[B])(fa:F[A]):F[B]=val(a×x1)=faval(b×x2)=f(a) val x3 = combine(x1, x2) (???:B)×(???:X)defcombine[X](lhs:X,rhs:X):X=??? And finally, complete the implementation by returning the target product b×x3:
typeF[A]=A×XdefflatMap[A,B](f:A⟹F[B])(fa:F[A]):F[B]=val(a×x1)=faval(b×x2)=f(a)valx3=combine(x1,x2)b×x3defcombine[X](lhs:X,rhs:X):X=??? Breaking Product
Our point of entry here is the target type B
typeF[A]=A×XdefflatMap[A,B](f:A⟹F[B])(fa:F[A]):F[B]=val(a×x1)=faval(b×x2)=f(a)valx3=combine(x1,x2)b×x3defcombine[X](lhs:X,rhs:X):X=??? Working backwards from here, we see that we derive b from fa and fa from our type F[A]. Well, what if we did something dastardly and decided that our type F[A] was no longer a product and instead just
typeF[A]=X This feels like cheating, but such a type is legal and exists. They're called "Phantom Types." A practical example might again use the PosInt in place of X and thus our type F[A] might be thought of instead as Weight causing the FlatMap to break down accordingly:
typeWeight[A]=PosIntdefcombine[PosInt](lhs:PosInt,rhs:PosInt):PosInt=??? Key takeways
A binary parametric product type:
- is a Functor
- is not an Apply if the other member is not a Semigroup
- is not an Applicative if the other member is not a Monoid
- is not a FlatMap if the Apply is of a Phantom Type
Applicative but not Monad
We've introduced all the fundamental operations of a Monad at this point, so all that's left to do is devise clever scenarios for which these operations are invalid for our chosen type. For an Applicative to be a Monad, it must implement flatMap, so let's make that impossible.
A perfectly valid and even common pattern we might encounter would be the product of some data as well as a boolean flag to indicate some state:
typeFlagged[A]=A×Booleandefpure[A](a:A):Flagged[A]=a×false Our implementation of flatMap then becomes:
typeFlagged[A]=A×Booleandefpure[A](a:A):Flagged[A]=a×falsedefflatMap[A,B](f:A⟹Flagged[B])(fa:Flagged[A]):Flagged[B]=val(a×x1)=faval(b×x2)=f(a)valx3=x1 || x2b×x3 And once again abusing Phantom Types, suppose we stripped our type constructor of is Product nature, reducing it to just a Boolean:
typeFlagged[A]=Boolean Backtracking from B again, we get the following:
typeFlagged[A]=A ×Booleandefpure[A](a:A):Flagged[A]=a ×falsedefflatMap[A,B](f:A⟹Flagged[B])(fa:Flagged[A]):Flagged[B]=val(a×x1)=faval(b×x2)=f(a)valx3=x1∣∣x2b×x3 which breaks flatMap and leaves us with just our type which is now more of a Flag rather than a Flagged since it has nothing to flag anymore.
typeFlag[A]=Booleandefpure[A](a:A):Flag[A]=false Key takeways
A binary parametric product type:
- is a Functor
- is not an Apply if the other member is not a Semigroup
- is not an Applicative if the other member is not a Monoid
- is not a FlatMap if the Apply is of a Phantom Type
- is not a Monad if the Applicative is of a Phantom Type
FlatMap but not Monad
And finally, the last leg of the journey requires illustrating how a thing which is pretty much everything else cannot be a Monad.
We've broken pure already, so we'll just do that again here by swapping out our Flagged type with something we know cannot be combined since it has no empty such as Weighted:
typeWeighted[A]=A×PosIntdefpure[A](a:A):Weighted[A]=a× ???defflatMap[A,B](f:A⟹Weighted[B])(fa:Weighted[A]):Weighted[B]=val(b×x1)=faval(b×x2)=f(a)valx3=x1+x2b×x3 Right away, this breaks pure and we have no way to lift an A into a Monoidal Weighted:
Key takeways
A binary parametric product type:
- is a Functor
- is not an Apply if the other member is not a Semigroup
- is not an Applicative if the other member is not a Monoid
- is not a FlatMap if the Apply is of a Phantom Type
- is not a Monad if the Applicative is of a Phantom Type
- is not a Monad if a FlatMap if the other member is not a Monoid.
Conclusion
And there we have it – a detailed breakdown of why some things are things but not other things – chock-full of hopefully-not-too-contrived counter examples!