In this blog post I’m presenting an implementation of a Wiki System in the spirit of the legendary C2Wiki  written in Haskell with the Yesod framework.
There will also be some nice addons like a graphical representation of the page links.
The WikiWikiWeb is the first wiki, or usereditable website. It was launched on 25 March 1995 by its inventor, programmer Ward Cunningham, to accompany the Portland Pattern Repository website discussing software design patterns.
The WikiWikiWeb was the earliest incarnation of a collaborative hypertext platform on the internet. It started with a small set of features which proved to provide the essential tools required to create a large content base with a dense hyperlink structure. Editing and creating new pages was extremely simple which fostered free contributions and a high frequency of interactions between participants.
The most prominent features are:
In the following I’m going to explain how I implemented each of those features.
The original WikiWikiWeb markup language provided basic syntax for layouting text content. Modern markup languages like Markdown are a more convenient to use, provide much more features and are already widely used. So I’m going to use Markdown instead of the original markup language.
Yesod comes with a set of templating mechanisms that ease the generation of HTML, CSS and Javascript for dynamic web content. The HTML templating is backed by the Blaze Html generator. Thus Yesod is optimized to use Blaze for HTML content. If, for example, the Blaze Html
data type is returned from routehandlers, Yesod will automatically set the ContentType to text/html
.
So my basic idea is to use a Markdown renderer that can output Blaze Html
data and let Yesod do all the heavy lifting.
I’m using the cmarkgfm library to render (GitHub flavoured) Markdown content to HTML. In order to output Html
data, my renderMdToHtml
function has to look like follows:
import CMarkGFM (commonmarkToHtml)
import Data.Text (Text)
import Text.Blaze.Html (Html, preEscapedToHtml)
renderMdToHtml :: Text > Html
= preEscapedToHtml . commonmarkToHtml [] [] renderMdToHtml
In order to work with the wiki page names in a type safe manner we first introduce a newtype PageName
. In order to make sure that only proper WikiWords can be used as page names I’m using a smart constructor pageName
which only constructs a PageName
instance if the intented page name matches the wikiWordMatch
regular expression:
newtype PageName = Page Text deriving (Eq, Read, Show)
pageName :: Text > Maybe PageName
=
pageName name if isWikiWord name
then Just (Page name)
else Nothing
  checks if a given Text is a WikiWord
isWikiWord :: Text > Bool
=
isWikiWord pageName case find wikiWordMatch pageName of
Nothing > False
Just _ > True
  the magic WikiWord Regex
wikiWordMatch :: Regex
= "([AZ][az09]+){2,}" wikiWordMatch
The following PathPiece
instance declaration is required to use the PageName
as part of a Yesod route definition:
instance PathPiece PageName where
= asText page
toPathPiece page = pageName text
fromPathPiece text
asText :: PageName > Text
Page name) = name asText (
Again the usage of the pageName
smart constructor ensures that only proper WikiWord pagenames are constructed.
Here comes the Yesod route definition for displaying and editing of wiki pages:
newtype HsWiki = HsWiki
contentDir :: String
{
}
"HsWiki" [parseRoutes
mkYesod
/#PageName PageR GET  (1)
/edit/#PageName EditR GET POST  (2) ]
Definition (1) can be read as follows: for any PageName
that is accessed via a HTTP GET a route PageR is defined, which (according to the rules of the Yesod routing DSL) requires us to implement a function with the following signature:
getPageR :: PageName > Handler Html
This function will have to lookup an existing page, render its Markdown content to Html and return it a Handler Html
object. We’ll have a look at this function shortly.
The definition (2) states that for any route /edit/PageName
two functions must be defined, one for GET one for POST:
getEditR :: PageName > Handler Html
postEditR :: PageName > Handler Html
If you want to know how exactly handler function are invoked from the Yesod framework and how the route dispatching works, please have a look at the excellent Yesod documentation which features a complete walkthrough with a HelloWorld application.
Now let’s study the implementation of these two function step by step, first the GET handler:
  handler for GET /edit/#PageName
getEditR :: PageName > Handler Html
= do
getEditR pageName < getDocumentRoot  obtain path to document root
path let fileName = fileNameFor path pageName  construct a file from the page name
< liftIO $ doesFileExist fileName  check whether file already exists
exists <
markdown if exists
then liftIO $ TIO.readFile fileName  if file exists, assign markdown with file content
else return newPage  else assign markdown with default content
return $ buildEditorFor pageName markdown  return Html for an Editor page
  retrieve the name of the HsWiki {contentDir} attribute, defaults to 'content'
getDocumentRoot :: Handler String
= getsYesod contentDir
getDocumentRoot
  construct the proper file name for a PageName
fileNameFor :: FilePath > PageName > FilePath
= path ++ "/" ++ asString pageName ++ ".md"
fileNameFor path pageName
  create default content for a new page
newPage :: Text
=
newPage "Use WikiWords in PascalCase for Links. \n\n"
<> "Use [Markdown](https://github.com/adamp/markdownhere/wiki/MarkdownCheatsheet) to format page content"
As we can see from the reading of markdown content from files, the idea is to just keep all pages as static content files in the filesystem. By default these files reside in the local folder content (this folder can be configured by a commandline argument).
Next we’ll have a look at the buildEditorFor
function that will generate the actual Html content of the editor page:
buildEditorFor :: PageName > Text > Html
=
buildEditorFor pageName markdown
toHtmlFalse,
[ pageHeader "",
menuBar $ "# " <> page <> " \n",
renderMdToHtml $
preEscapedToHtml "<form action=\""
<> page
<> "\" method=\"POST\">"
<> "<textarea style=\"height: auto;\" name=\"content\" cols=\"120\" rows=\"25\">"
<> markdown
<> "</textarea>"
<> "<input type=\"submit\" name=\"save\" value=\"save\" /> "
<> "<input class=\"button buttonoutline\" type=\"button\" name=\"cancel\" value=\"cancel\" onClick=\"window.history.back()\" /> "
<> "</form>",
pageFooter
]where page = asText pageName
The most important element here is the creation of an Html <form ...>...</form> element. The action for that form is just the same page but with a
POSTmethod (we'll come to the respective handler function
postEditR` shortly).
Now imagine we point our browser to http://localhost:3000/edit/BrandNewPage
. Yesod will do the routing to getEditR (Page "BrandNewPage")
and the generated Html for editing a new page ‘BrandNewPage’ will be sent back to the browser. The page will look like this:
As we can see, I’ve applied some basic CSS styling (using Milligram CSS). This is done in the pageHeader
function.
The editor has two buttons, SAVE and CANCEL. On cancel we just navigate back to the previous page in the browser history. On save the browser sends the form data via the POST
method to the server. To handle this incoming POSTrequest we’ll the postEditR
handler function:
postEditR :: PageName > Handler Html
= do
postEditR pageName < getDocumentRoot  obtain path to document root
path let fileName = fileNameFor path pageName  construct a file from the page name
< lookupPostParam "content"  retrieve POST data
maybeContent < remoteHost <$> waiRequest  retrieve info on remote client from request
client case maybeContent of
Just content > liftIO $ do
 if content exists write it to disk
TIO.writeFile fileName content  also write a log entry to file RecentChanges
writeLogEntry path pageName client Nothing > return ()  no content: do nothing
$ PageR pageName  redirect to GET Page route (display content) redirect
So essentially we are just writing the markdown content into a file. After that we redirect to the PageR
route. This will result in redirecting the browser to http://localhost:3000/BrandNewPage
. As you can see in the following screenshot the markdown content that was entered in the editor form is now rendered as HTML:
As promised above we’ll now have a closer look at the getPageR
route handler function:
  Handler for GET /#PageName
getPageR :: PageName > Handler Html
= do
getPageR pageName < getDocumentRoot  obtain path to document root
path < lookupGetParam "showBackrefs"  check whether URL ends with '?showBackrefs'
maybeShowRefs < liftIO $  if showBackrefs was set, Just [PageName]
maybeBackrefs  else Nothing
computeMaybeBackrefs path pageName maybeShowRefs let fileName = fileNameFor path pageName  compute proper filename from pageName
< liftIO $ doesFileExist fileName  check whether such a file exists
exists if exists
then do
< liftIO $ TIO.readFile fileName  file exists, read its content
content return $ buildViewFor
 build HTML for content and return it
pageName content maybeBackrefs else do
$ EditR pageName  file does not exist, redirect to EditR redirect
Let’s ignore the lines with maybeShowRefs
and maybeBackrefs
for a moment. We just assume that maybeBackrefs == Nothing
. So we first check whether a file exists for the given pageName
. If yes, the filecontent is read and bound to content
; next we build a HTML view for the page with buildViewFor
and return it. If no file was found matching pageName
we redirect directly to the EditR
which will in turn open up an editor for an empty page as already shown in the previous section.
Let’s have a closer look at buildViewFor
. It will first evaluate the maybeBackrefs
arguments. For the moment let’s assume equals Nothing
, so that hasBackref
is bound to True
and backrefEntry
to ""
.
Then the actual HTML for the page is constructed from a set of template functions:  pageHeader
creates the HTML head with css definitions,  menuBar
creates the menu line on top of the page,  pageTitle
creates a headline from the pageName
,  backrefEntry
is just empty text in this scenario  renderMdToHtml (wikiWordToMdLink content)
first replaces all ocurrences of WikiWords with proper Markdown hyperlinks of the form [WikiWord](WikiWord)
the result is then rendered to HTML (this is the single place where we convert from WikiWords to hyperlinks and thus make the Wiki magic happen…),  finally pageFooter
closes all open html tags:
buildViewFor :: PageName > Text > Maybe [PageName] > Html
=
buildViewFor pageName content maybeBackrefs let (hasBackref, backrefEntry) = case maybeBackrefs of
Nothing > (False, text "")
Just backrefs > (True, renderedBackrefs)
where
concatMap :: (a > Text) > [a] > Text
concatMap = (T.intercalate "" .) . map
= renderMdToHtml $ concatMap ((\b > " [" <> b <> "](/" <> b <> ") \n") . asText) backrefs
renderedBackrefs in toHtml [pageHeader False,
menuBar (asText pageName),
pageTitle pageName hasBackref,
backrefEntry,
renderMdToHtml (wikiWordToMdLink content),
pageFooter]
  converts a WikiWord into a Markdown link: [WikiWord](WikiWord)
wikiWordToMdLink :: Text > Text
=
wikiWordToMdLink text let match = wikiWordMatch
= "[$0]($0)"
replace in replaceAll match replace text
Another important feature of the original WikiWiki was the seamless integration of back links:
If page A links to page B, then a ‘back link’ would be a link which goes from page B back to page A.
On this wiki, the title of each page works as a back link. Clicking on the title of any page finds all the pages referring to that page. It works for any wiki page. E.g. to find all pages that link to this page, click the title at the top of this page.
This feature can best be demonstrated with an example. First we lookup up page http://localhost:3000/CategoryMammal
, a page meant to represent the class of all mammaĺ animals:
The headline of this page is a hyperlink which references http://localhost:3000/CategoryMammal?showBackrefs
. Following the link results in the following page:
Now we see a bullet point list of all pages linking to CategoryMammal above the normal page content. Following one of these links, e.g. http://localhost:3000/SpeciesCat
, leads to the following page:
At the bottom of this page we see the WikiWord CategoryMammal. This is interpreted as a link from SpeciesCat to CategoryMammal. And as a result the backlink display on page CategoryMammal contains a link to SpeciesCat.
Let’s see how this works on the code level. In fact we already came across this mechanism but had skipped over it for the time being. Now it’s time to revisit. We start with the getPageR
function.
In our scenario a click on the link http://localhost:3000/CategoryMammal?showBackrefs
leads to a call to getPageR
. But this time lookupGetParam "showBackrefs"
will succeed and thus now maybeShowRefs
is bound to Just ""
. This will lead to a different execution path in the call to computeMaybeBackrefs
:
  Handler for GET /#PageName
getPageR :: PageName > Handler Html
= do
getPageR pageName < getDocumentRoot  obtain path to document root
path < lookupGetParam "showBackrefs"  check whether URL ends with '?showBackrefs'
maybeShowRefs < liftIO $  if showBackrefs was set, Just [PageName]
maybeBackrefs  else Nothing
computeMaybeBackrefs path pageName maybeShowRefs let fileName = fileNameFor path pageName  compute proper filename from pageName
< liftIO $ doesFileExist fileName  check whether such a file exists
exists if exists
then do
< liftIO $ TIO.readFile fileName  file exists, read its content
content return $ buildViewFor
 build HTML for content and return it
pageName content maybeBackrefs else do
$ EditR pageName  file does not exist, redirect to EditR
redirect
  if maybeShowRefs isJust then a list of a pages referencing pageName is computed
computeMaybeBackrefs :: FilePath > PageName > Maybe Text > IO (Maybe [PageName])
=
computeMaybeBackrefs path pageName maybeShowRefs case maybeShowRefs of
Nothing > return Nothing  if maybeShowRefs == Nothing, return Nothing
Just _ > do  else compute list of all references to page by
< computeIndex path  computing list of all pages in wiki
allPages < computeBackRefs path pageName allPages  compute all back references
backrefs return $ Just backrefs  return this list wrapped as a Maybe
  compute a list of all pages that contain references to pageName
computeBackRefs :: FilePath > PageName > [PageName] > IO [PageName]
= do
computeBackRefs path pageName allPages let filteredPages = delete pageName allPages  filter pagename from list of pages
< mapM  create a list of bools: True if a page contains
markRefs fmap containsBackref . TIO.readFile . fileNameFor path)  a reference, else False
(
filteredPageslet pageBoolPairs = zip filteredPages markRefs  create a zipped list of (pageName, Bool) pairs
return $ map fst (filter snd pageBoolPairs)  return only pages marked True
where
=  returns True if content contains pageName
containsBackref content `T.isInfixOf` content asText pageName
Next we revisit buildViewFor
. Here we see a case match on maybeBackrefs
. In our current scenario it will match to Just backrefs
. Thus renderedBackrefs
will be bound to Html generated by rendering a Markdown list of hyperlinks that is constructed from the backrefs
list of PageNames.
This generated Html is then included as backrefEntry
into the overall page layout:
buildViewFor :: PageName > Text > Maybe [PageName] > Html
=
buildViewFor pageName content maybeBackrefs let (hasBackref, backrefEntry) = case maybeBackrefs of
Nothing > (False, text "")
Just backrefs > (True, renderedBackrefs)
where
concatMap :: (a > Text) > [a] > Text
concatMap = (T.intercalate "" .) . map
=
renderedBackrefs $ concatMap
renderMdToHtml > " [" <> b <> "](/" <> b <> ") \n") . asText)
((\b
backrefsin toHtml [pageHeader False,
menuBar (asText pageName),
pageTitle pageName hasBackref,
backrefEntry,
renderMdToHtml (wikiWordToMdLink content), pageFooter]
I already covered the postEditR
function, but I did not explain the writeLogEntry
function which traces each change to pagecontent. So here comes the full picture:
postEditR :: PageName > Handler Html
= do
postEditR pageName < getDocumentRoot  obtain path to document root
path let fileName = fileNameFor path pageName  construct a file from the page name
< lookupPostParam "content"  retrieve POST data
maybeContent < remoteHost <$> waiRequest  retrieve info on remote client from request
client case maybeContent of
Just content > liftIO $ do
 if content exists write it to disk
TIO.writeFile fileName content  also write a log entry to file RecentChanges
writeLogEntry path pageName client Nothing > return ()  no content: do nothing
$ PageR pageName  redirect to GET Page route (display content)
redirect
  write a log entry to the RecentChanges page
writeLogEntry :: FilePath > PageName > SockAddr > IO ()
= do
writeLogEntry path pageName client let fileName = fileNameFor path recentChanges  path to RecentChanges page
< getCurrentTime  create timestamp
now let logEntry = toStrict $  create a log entry consisting of:
" " % string % " " % string % " from " % string % "\n")
format ( page edited/created
(asString pageName) takeWhile (/= '.') (show now))  current timestamp
(takeWhile (/= ':') (show client))  IP address of client
( add log entry at end of log file
TIO.appendFile fileName logEntry
  the RecentChanges PageName
recentChanges :: PageName
= Page "RecentChanges" recentChanges
And here comes a sample screen shot of the RecentChanges page:
For the full text search Iǜe provided a specific route /actions/find
to avoid overlap with ordinary content pages:
"HsWiki" [parseRoutes
mkYesod
/actions/find/ FindR GET ]
The corresponding handler function getFindR
is defined as follows:
  handler for GET /actions/find
getFindR :: Handler Html
= do
getFindR < getDocumentRoot  obtain path to document root
path < liftIO $ computeIndex path  compute a list of all page names in wiki
allPages < lookupGetParam "search"  check whether query param 'search' is set
maybeSearch case maybeSearch of
Nothing > return $ buildFindPage "" []  if maybeSearch == Nothing or Just ""
Just "" > return $ buildFindPage "" []  then return empty find page
Just search > do
< liftIO $  else create a list of Bools by
markMatches mapM  returning True for each file that matches
> fmap containsSearchText $  search, else False
(\p return (asText p) <> TIO.readFile (fileNameFor path p))
allPageslet pageBoolPairs = zip allPages markMatches  create a zipped list [(PageName, Bool)]
let matchingPages = map fst (filter snd pageBoolPairs)  filter for all matching pages
return $ buildFindPage search matchingPages  build find page with search term and
where  list of matching pages
= T.toLower search `T.isInfixOf` T.toLower content containsSearchText content
The buildFindPage
function is responsible for assembling the Html view of this page.
buildFindPage :: Text > [PageName] > Html
= toHtml
buildFindPage search pages True,
[ pageHeader "",
menuBar "# FindPage ",
renderMdToHtml
searchBox search,$ T.pack $ concatMap (\p > " [" ++ asString p ++ "](/" ++ asString p ++ ") \n") pages,
renderMdToHtml
pageFooter
]
searchBox :: Text > Html
=
searchBox search $
preEscapedToHtml "<script type=\"text/javascript\">"
++ "function init()"
++ "{"
++ " document.getElementById(\"search\").focus();"
++ "}"
++ "</script>"
++
"<form action=\"/actions/find\""
++ " method=\"GET\">"
++ "<input type=\"text\" id=\"search\" name=\"search\" value=\"" ++ T.unpack search ++ "\" "
++ "onfocus=\"this.setSelectionRange(9999, 9999)\" "
++ "onkeyup=\"this.form.submit()\" /> "
++ "<input type=\"submit\" value=\"find\" />"
++ "</form>"
The only interesting thing here is that I’ve include a bit of JavaScript to enable page updates while typing into the find box. See the FindPage in action below:
So far I’ve just reimplemented stuff that was already there in the original WikiWiki. While toying around with my HsWiki I thought it might be a nice addition to have a graph representation of the site content.
As always I try to code as little as possible myself and get the hard work done by the experts. In this case I’m relying on my alltime favourite Graph rendering library GraphViz. This time in it’s web assembly incarnation d3graphviz.
So again we’ll have a specific route:
"HsWiki" [parseRoutes
mkYesod
/actions/graph GraphR GET ]
And a corresponding route handler function:
  handler for GET /actions/graph
getGraphR :: Handler Html
= do
getGraphR < getDocumentRoot  obtain document root folder
path < liftIO $ computeIndex path  compute list of all wiki pages
allPages < liftIO $ mapM  compute list of all back references
allRefs > computeBackRefs path p allPages)
(\p  for each file in allPages
allPages return $ buildGraphView $ zip allRefs allPages  return Html view for [([PageName], PageName)] graph
Please note that this implementation has \(O(n^2)\). This is caused by relying on computeBackrefs
this function traverses all files and is called once for each file by mapM
. Improving this is left as an exercise for the interested reader (all pull requests are welcome!
The actual Html rendering is a bit more involved as I have to integrate the JS code for d3graphviz and also to render the GraphViz DOT graph representation:
  build view for GraphViz representation of wiki page structure
buildGraphView :: [([PageName], PageName)] > Html
=
buildGraphView graph
toHtmlFalse,
[ pageHeader "",
menuBar "# Site Map \n",
renderMdToHtml "[View as List](/actions/toc) \n",
renderMdToHtml  load wasm scripts, begin JS script
preGraph, $ renderNodes $ allNodes graph,  build list of all PageName nodes
preEscapedToHtml $ renderGraph graph,  build link structure as directed graph
preEscapedToHtml  render DOT digraph
postGraph,
pageFooter
]
  render graph in DOT syntax (from > to;)
renderGraph :: [([PageName], PageName)] > String
=
renderGraph graph foldr
> ((str ++ ",\n") ++))
(\str ""
concatMap (\(sources, target) >
(map
> "'\"" ++ asString s ++ "\" > \"" ++ asString target ++ "\";'")
(\s
sources)
graph)
  extract list of unique PageNames from graph
allNodes :: [([PageName], PageName)] > [PageName]
= nub . (uncurry (flip (:)) =<<)
allNodes
  render list of PageNames as DOT list of nodes with some nice formatting
renderNodes :: [PageName ] > String
=
renderNodes concatMap
>
( \n "'\"" ++ asString n
++ "\" [shape=\"rect\", style=\"rounded,filled\", fillcolor=\"#f4f5f6\", fontcolor=\"#9b4dca\", fontname=\"Roboto\", URL=\"/"
++ asString n
++ "\"];', \n"
)
  Html with script code for loading d3graphviz and opening the DOT digraph
preGraph :: Html
=
preGraph $
preEscapedToHtml "<script src=\"//d3js.org/d3.v5.min.js\"></script>"
++ "<script src=\"https://unpkg.com/@hpccjs/wasm@0.3.11/dist/index.min.js\"></script>"
++ "<script src=\"https://unpkg.com/d3graphviz@3.0.5/build/d3graphviz.js\"></script>"
++ "<div id=\"graph\" ></div>"
++ "<script>"
++ "var dot =\n"
++ " [\n"
++ " 'digraph {',\n"
  Html with script code for rendering the DOT digraph
postGraph :: Html
=
postGraph $
preEscapedToHtml " '}'\n"
++ " ];\n"
++ " \n"
++ " d3.select(\"#graph\").graphviz()\n"
++ " .renderDot(dot.join(''));\n"
++ " \n"
++ " </script>\n"
You can see this in action in the following screen shot:
stack init
stack install
HsWiki
Under Windows you will have to install the ICU library. I used the latest win64 version from https://github.com/unicodeorg/icu/releases/tag/release701. You’ll have to manually copy .ddl and .h files to the following locations:
C:\Users\<username>\AppData\Local\Programs\stack\x86_64windows\msys2<installdate>\mingw64\lib
Don’t forget to strip version number from the .dll files (so icuuc70.dll becomes icuuc.dll)C:\Users\<username>\AppData\Local\Programs\stack\x86_64windows\msys2<installdate>\mingw64\include\unicode
Implementing a small functional language with a classic combinator based graphreduction machine in Haskell.
The implementation is structured into three parts:
A λcalculus parser from A Combinatory Compiler which was extended to cover a tiny functional language based on the untyped λcalculus.
A compiler from λcalculus to combinatory logic combinators (S,K,I,B,C and Y) which is based on bracketabstraction and some optimization rules.
A graphreducer. Combinator terms are allocated into a graph datastructure. Which is then reduced by applying combinator graphreduction. The destructive inplace reduction of the graph is made possible by using STRef
mutable references.
In my last blog post I presented two ways of transforming λterms into variable free representations:  bracket abstraction to combinatory logic terms (SKI) and  bracket abstraction to terms of closed cartesian categories (CCC).
I demonstrated that both representations are equivalent as they imply the same reduction rules.
My original intention was to extend an existing Haskell CCC implementation to a proofofconcept implementation of a small functional language. I even promised to cover this in my next blog post.
I invested a lot of time in this idea but I failed to get it off the ground. At least the code of these experiments has been preserved.
So I came back to writing a SKI graphreduction as the backend of my language implementation. This is a wellworn path. I took the basic ideas from the classic compiling functional languages which dates back to 1988.
Fortunately, I did not fail this time! In the following I’m explaining my implementation approach. I’ll also share some of my insights and talk about possible future extensions.
I’m aiming at a very rudimentary language that is basically just pure λcalculus plus integers. Here is an example:
Y = λf . (λx . x x)(λx . f(x x))
= Y(\f n > if (is0 n) 1 (* n (f (sub1 n))))
fact = fact 10 main
As you can see it’s possible to use classical λcalculus notation λx . x x
as well as Haskell syntax: \x > x x
. It’s also possible to freely mix both styles.
λexpressions can be assigned to names in a toplevel environment by using the =
sign. those names may be referred to in other λexpressions. As of now recursive (also mutually recursive) references are not supported.
The main
expression has a special meaning; it is interpreted as the entry point to a program.
With this knowledge at hand you will immediately recognize that the above program will compute the factorial of 10. Where fact
is defined in a nonrecursive way by means of the fixedpoint combinator Y
.
Expressions of this language are represented by the data type Expr
:
infixl 5 :@
data Expr
= Expr :@ Expr
 Var String
 Int Integer
 Lam String Expr
deriving (Eq, Show)
The toplevel environment which maps names to λExpressions is represented by the following type:
type Environment = [(String, Expr)]
There is not much to see in this area. It’s just a simple Parsec based parser. Most of the code was taken from A Combinatory Compiler. I just added the parsing of Integers.
The parser module exports to function parseEnvironmentEither
and parseEnvironment
. The former is a total function returning an Either: parseEnvironmentEither :: String > Either ParseError Environment
, whereas the latter simply returns an Environment
but may throw runtime errors.
The following snippet demonstrates how a program is parsed into an Environment:
testSource :: String
=
testSource "Y = λf > (λx > x x)(λx > f(x x)) \n"
++ "fact = Y(λf n > if (is0 n) 1 (* n (f (sub1 n)))) \n"
++ "main = fact 10 \n"
= do
main let env = parseEnvironment testSource
mapM_ print env
putStrLn ""
This code results in the following output, which shows all (String, Expr)
tuples in the environment:
"Y", Lam "f" (Lam "x" (Var "x" :@ Var "x") :@ Lam "x" (Var "f" :@ (Var "x" :@ Var "x"))))
("fact",Var "Y" :@ Lam "f" (Lam "n" (((Var "if" :@ (Var "is0" :@ Var "n")) :@ Int 1) :@
(Var "*" :@ Var "n") :@ (Var "f" :@ (Var "sub1" :@ Var "n"))))))
(("main",Var "fact" :@ Int 10) (
Of course it is possible to write interpreters that evaluate these λexpression to normalform. This is what any Lisp or Scheme eval/apply interpreter does at its core (See a tiny example here).
One of the most problematic areas of these interpreters is the handling of variables. In order to provide static binding you will need closures that captures the current environment of variable bindings and thread them through the whole interpreter execution.
Language implemetors have thus experimented with many ways to tackle this issue. One of the most influential ideas was to completely get rid of variables by abstracting them.
The earliest version of this approach was the SKI combinator calculus invented by Haskell Curry and Moses Schönfinkel.
A λterm that does not contain any free variables is said to be closed. Closed lambda terms are also called combinators.
Schönfinkel and Curry found out that any closed λterm can be rewritten in terms of three basic combinators I, K and S (in fact only K and S are essential, as I can be expressed as SKK):
In Haskell these combinators can simply be defined as:
= x
i x = x
k x y = f x (g x) s f g x
The idea of bracket abstraction is to rewrite any closed λterm in terms of I, K and S. This recursive transformation is defined by the following equations:
This can be implemented in Haskell as follows:
  most basic bracket abstraction (plus resolution of free variables in the environment).
babs0 :: Environment > Expr > Expr
Lam x e)  this clause implements the three basic equations for bracket abstraction
babs0 env ( Var y < t, x == y = Var "i"
 x `notElem` fv [] t = Var "k" :@ t
 m :@ n < t = Var "s" :@ babs0 env (Lam x m) :@ babs0 env (Lam x n)
where t = babs0 env e
Var s)  this clause resolves free variables by looking them up in the environment env
babs0 env ( Just t < lookup s env = babs0 env t
 otherwise = Var s
:@ n) = babs0 env m :@ babs0 env n  this clause recurses into applications
babs0 env (m = x  returns anything else unchanged
babs0 _env x
  compute the list of free variables of a lambda expression
fv :: [String] > Expr > [String]
Var s)  s `elem` vs = []
fv vs ( otherwise = [s]
:@ y) = fv vs x `union` fv vs y
fv vs (x Lam s f) = fv (s:vs) f
fv vs (= vs fv vs _
Let’s have a look at a simple example. first we parse a simple expression into a lambdaterm:
> env = parseEnvironment "main = (λx > + 4 x) 5\n"
ghci> env
ghci"main",Lam "x" ((Var "+" :@ Int 4) :@ Var "x") :@ Int 5)] [(
Next we apply bracket abstraction:
ghci> skiExpr = babs env (snd . head $ env)
ghci> skiExpr
((Var "s" :@ (Var "k" :@ (Var "+" :@ Int 4))) :@ Var "i") :@ Int 5
The result of bracket abstraction is still a lambdaterm, but one where all Lam
expression have been eliminated.
Even from this simple example it is obvious that the SKIcombinator terms become larger than the original expressions. This will be an impediment to efficient implementation. So many different approaches have been conceived to mitigate this issue.
The earliest solution, already suggested by Schönfinkel, is to introduce additional combinators B and C that cover specific patterns in the source code. Here are the reduction rules for B and C.
C f g x = ((f x) g)
B f g x = (f (g x))
We could extend babs
to cover B and C. But the most common way is to run a second optimization pass over the SKIexpression.
Here is is a simple example of such an optimization:
opt :: Expr > Expr
Var "i" :@ n@(Int _n)) = n
opt (Var "s" :@ (Var "k" :@ e1)) :@ (Var "k" :@ e2)) = Var "k" :@ (e1 :@ e2)
opt ((Var "s" :@ e1) :@ (Var "k" :@ e2)) = (Var "c" :@ e1) :@ e2
opt ((Var "s" :@ (Var "k" :@ e1)) :@ e2) = (Var "b" :@ e1) :@ e2
opt ((:@ y) = opt x :@ opt y
opt (x = x
opt x
ropt :: Expr > Expr
=
ropt expr let expr' = opt expr
in if expr' == expr
then expr
else case expr' of
:@ y) > ropt $ ropt x :@ ropt y
(x > ropt expr' _
Let’s try this out:
> optExpr = ropt skiExpr
ghci> optEpr
ghciVar "b" :@ (Var "+" :@ Int 4)) :@ Var "i") :@ Int 5 ((
This looks much better than before. See this project for a more in depth coverage of optimization techniques. I’m also planning to write a separate blog post on this subtopic.
The sourcecode for this section can be found here
So now that we have eliminated lambda abstractions from our lambda terms it should be straight forward to evaluate these expressions with a simple interpreter.
Let’s have a look at a simple example:
= λx > * x x
sqr = sqr (+ 3 2) main
When we implement a strict interpreter with applicativeorder semantics, (+ 3 2)
will be computed first and the result bound to the variable x
in the local environment and then the body of sqr
will be evaluated in this environment. That’s fine. but it’s not normalorder reduction.
When implementing a lazy interpreter with normalorder semantics, we can not compute (+ 3 2)
before binding it to x
. Thus we will have to bind an unevaluated thunk to x
. We will also have to make sure that x
is only evaluated when needed and only once, even when it is used at several places in the body of sqr
. (See these lecture notes for all the intricacies of this approach)
Graphreduction on the other hand, has some very interesting features:  It maintains normalorder reduction (that is lazy evaluation)  double evaluations of terms is avoided  dealing with local environments, variable scope, etc. at runtime is avoided  copying of argument data is significantly reduced as compared to eval/apply interpreters
Let’s see this in action with our toy example. The above program can be transformed into the following SKI combinator term:
Var "s" :@ Var "*") :@ Var "i") :@ ((Var "+" :@ Int 3) :@ Int 2) ((
This term can be represented as a binary graph, where each application :@
is represented as an @
node, all combinators like (Var "s")
are represented with Constructors like S
, I
, MUL
, ADD
and integer values like Int 2
are just shown as numeric values like 2
:
@
/ \
/ \
/ @
/ / \
/ @ 2
@ / \
/ \ ADD 3
@ I
/ \
S MUL
In the following diagram we follow the reduction of this graph:
@ @ @ @ 25
/ \ / \ / \ / \
/ \ / \ / \ / 
/ @ / @ / @ / /
/ / \ @ / \ @ / \ @ /
/ @ 2 / \ I  / \ I  / \ /
@ / \ / @ ––––/ / 5 ––––/ / 5
/ \ ADD 3 / / \ / /
@ I / @ 2 / /
/ \ / / \ / /
S MUL MUL ADD 3 MUL MUL
Step 0 Step 1 Step 2 Step 3 Step 4
Step 0: This is just the initial state of the graph as explained above. Please note that in this state the S
is our redex (i.e. the leftmost ancestor of the root node) and saturated (i.e all three arguments of the combinator) are populated, so according to the reduction rule s f g x = f x (g x)
we expect to see a reduction S MUL I (ADD 3 2) = MUL (ADD 3 2) (I (ADD 3 2))
in step 1.
Step 1: As expected the first reduction step mutates the graph to represent MUL (ADD 3 2) (I (ADD 3 2))
. Please note that both occurrences of (ADD 3 2)
are represented by references to one and the same node.
Step 2: Now MUL
has become the redex (short for reducible expression). But this time both arguments (ADD 3 2)
and I (ADD 3 2)
are not in normalform and thus have to be reduced first before MUL
can be executed. So first (ADD 3 2)
is reduced to 5
. Please note that both references to the former (ADD 3 2)
node now point to 5
. So in effect the I (ADD 3 2)
node has changed to I 5
as (ADD 3 2)
was a shared node.
Step 3: next the I 5
node is reduced according to the equation i x = x
. That is, the reference to the application node I @ 5
is modified to directly point to 5
instead. Please note that both arguments point to one and the same numeric value 5
.
Step 4: As a result of the transformation in step 3 both arguments of MUL
are in normalform. So now MUL 5 5
can be performed: Accordingly the root node is now changed to 25
.
Now that we have a basic understanding of the ideas behind graphreduction we will have a closer look at the actual implementation in the following sections.
As we have seen in the last section we will have to deal with mutable references in order to implement things like node sharing and inplace mutation of nodes.
I will use the Haskell datatype Data.STRef
which provides mutable references in the ST
monad.
Here comes a basic example that demonstrates the basic functionality of STRef
. A list of numbers is summed up by adding each of them to an accumulator. The accumulator is implemented by a reference acc
pointing to an initial value of 0
. Then we iterate over the list of numbers and update the value of the accumulator by adding each number x
to it. Finally the result is read out from the accumulator and extracted from the ST Monad by runST. From this example we can see that STRef
s work much like pointers in imperative languages:
import Data.STRef (STRef, modifySTRef, newSTRef, readSTRef writeSTRef)
import Control.Monad.ST (runST)
  sum up a list of numerical values
sumST :: Num a => [a] > a
= runST $ do  runST takes stateful ST code and makes it pure.
sumST numbers < newSTRef 0  Create an STRef (a mutable variable) to an accumulator
acc $ \x >  iterate over all numbers
forM_ numbers + x)  add each number to what we have in acc.
modifySTRef acc ( read the value of acc, which will be returned by the runST above. readSTRef acc
This looks promising. So now lets implement a binary graph for our compiled combinator terms with it:
infixl 5 :@:
data Graph s
= (STRef s (Graph s)) :@: (STRef s (Graph s))
 Comb Combinator
 Num Integer
deriving (Eq)
data Combinator = I  K  S  B  C  Y  P  ADD  SUB  MUL  DIV  REM  SUB1  EQL  ZEROP  IF
deriving (Eq, Show)
So we basically mimic the Expr
data type used to encode λexpression but without variables and lambdaabstractions. The data type Combinator
contains constructors for combinators that we intend to implement in the graphreduction engine.
Next we define a function allocate
that allows to allocate a ‘lambdaabstracted’ λexpression (of type Expr
) into a reference to a Graph
:
  allocate a 'lambdaabstracted' Expr into a referenced Graph
allocate :: Expr > ST s (STRef s (Graph s))
Var name) = newSTRef $ Comb $ fromString name
allocate (Int val) = newSTRef $ Num val
allocate (:@ r) = do
allocate (l < allocate l
lg < allocate r
rg $ lg :@: rg
newSTRef Lam _ _) = error "lambdas must already be abstracted away!"
allocate (
  lookup Combinator constructors by their names
fromString :: String > Combinator
"i" = I
fromString "k" = K
fromString "s" = S
fromString "b" = B
fromString "c" = C
fromString "y" = Y
fromString "p" = P
fromString "+" = ADD
fromString "sub" = SUB
fromString "div" = DIV
fromString "rem" = REM
fromString "*" = MUL
fromString "sub1" = SUB1
fromString "eq" = EQL
fromString "is0" = ZEROP
fromString "if" = IF
fromString = error $ "unknown combinator " ++ _c fromString _c
So let’s see this in action:
> optExpr = ((Var "s" :@ Var "*") :@ Var "i") :@ ((Var "+" :@ Int 3) :@ Int 2)
ghci> graph = allocate optExpr
ghci> runST $ mToString graph
ghci"(((S :@: MUL) :@: I) :@: ((ADD :@: 3) :@: 2))"
I’m using the mToString
helper function to render ST s (STRef s (Graph s))
instances:
mToString :: ST s (STRef s (Graph s)) > ST s String
= toString =<< g
mToString g
toString :: STRef s (Graph s) > ST s String
= do
toString graph < readSTRef graph
g where
toString' g Comb c) = return $ show c
toString' (Num i) = return $ show i
toString' (:@: rP) = do
toString' (lP < readSTRef lP
lG < readSTRef rP
rG < toString' lG
lStr < toString' rG
rStr return $ "(" ++ lStr ++ " :@: " ++ rStr ++ ")"
Now that we have allocated our expression as an ST s (STRef s (Graph s))
the next step will be to perform graph reduction on it.
First we have to compute the stack of left ancestors  or spine  of a graph for an efficient reduction.
In the following diagram I have marked the members of this stack with >
arrows:
> @
/ \
/ \
/ @
/ / \
/ @ 2
> @ / \
/ \ ADD 3
> @ I
/ \
> S MUL
The following function spine
computes this left ancestors’ stack by traversing all application nodes to the left:
 we simply represent the stack as a list of references to graph nodes
type LeftAncestorsStack s = [STRef s (Graph s)]
spine :: STRef s (Graph s) > ST s (LeftAncestorsStack s)
= spine' graph []
spine graph where
spine' :: STRef s (Graph s) > LeftAncestorsStack s > ST s (LeftAncestorsStack s)
= do
spine' graph stack < readSTRef graph
g case g of
:@: _r) > spine' l (graph : stack)
(l > return (graph : stack) _
Using this spine
function we can implement a function step
that performs a single reduction step on a Graph
node:
step :: STRef s (Graph s) > ST s ()
= do
step graph :stack) < spine graph
(top< readSTRef top
node case node of
Comb k) > reduce k stack
(> return () _
If a combinator is found in redex position, reduce
is called to perform the actual reduction work according to the combinator specific reduction rules.
Let’s study this for some of the combinators, starting with the most simple one, I x = x
:

> @
p / \
> I x
reduce :: Combinator > LeftAncestorsStack s > ST s ()
I (p : _) = do
reduce :@: xP) < readSTRef p
(_I < readSTRef xP
xVal writeSTRef p xVal
In this case a reference p
to (I :@: xP )
is on top of the stack. The actual value of x is read from xP
with readSTRef
and than p
is made to point to this value by using writeSTRef
.
The reduction of S f g x = f x (g x)
is already a bit more involved:

> @
p3 / \
> @ x
p2 / \
> @ g
p1 / \
> S f
S (p1 : p2 : p3 : _) = do
reduce :@: fP) < readSTRef p1
(_S :@: gP) < readSTRef p2
(_ :@: xP) < readSTRef p3
(_ < newSTRef $ fP :@: xP
node1 < newSTRef $ gP :@: xP
node2 :@: node2) writeSTRef p3 (node1
In this case reference to f (fP
), g (gP
) and x (xP
) are obtained. Then a new application node is created that represents ((f @ x) @ (g @ x))
. Then p3
is made to point to this new node.
Binary arithmentic combinators like ADD
and MUL
are implemented as follows:
ADD (p1 : p2 : _) = binaryMathOp (+) p1 p2
reduce MUL (p1 : p2 : _) = binaryMathOp (*) p1 p2
reduce
binaryMathOp ::
Integer > Integer > Integer) >  ^ a binary arithmetic function on Integers like (+)
(STRef s (Graph s) >  ^ first node on the spine stack
STRef s (Graph s) >  ^ second node on spine stack
ST s ()
= do
binaryMathOp op p1 p2 :@: xP) < readSTRef p1
(_ :@: yP) < readSTRef p2
(_ Num xVal) < (readSTRef <=< normalForm) xP  reduce xP to normal form and obtain its value as xVal
(Num yVal) < (readSTRef <=< normalForm) yP  reduce yP to normal form and obtain its value as yVal
(Num $ xVal `op` yVal)  apply op on xVal and yVal, modify p2 to point to the resulting value writeSTRef p2 (
The interesting bit here is that the arithmetic combinators are strict, that is they require their arguments to be in normalform. (Please note that S
, I
, K
, etc. don’t have this requirement. They are nonstrict or lazy).
normalForm
just applies step
in a loop while the graph has not been reduced to a combinator or an integer:
normalForm :: STRef s (Graph s) > ST s (STRef s (Graph s))
= do
normalForm graph
step graph< readSTRef graph
g case g of
:@: _rP > normalForm graph
_lP Comb _com > return graph
Num _n > return graph
Using a helper function reduceGraph
that computes the normalform of a graph while staying entirely in the ST
Monad, we can finally reduce our tiny toy graph:
reduceGraph :: ST s (STRef s (Graph s)) > ST s (STRef s (Graph s))
= do
reduceGraph graph < graph
gP
normalForm gP
> runST $ mToString graph
ghci"(((S :@: MUL) :@: I) :@: ((ADD :@: 3) :@: 2))"
> runST $ mToString $ reduceGraph graph
ghci"25"
λcalculus does not directly support recursion using selfreferential functions (see this nice exposition). That’s why we need a fixedpoint combinator to realize recursive operation. Here once again the definition of the factorial function that makes use of the Y
Combinator to implement recursive behaviour:
Y = λf . (λx . x x)(λx . f(x x))
= Y(\f n > if (is0 n) 1 (* n (f (sub1 n))))
fact = fact 10 main
With only a few lines of equational reasoning we can demonstrate the special property of the Y
combinator when applied to any function g
:
Y g = (λf.(λx.x x)(λx.f(x x))) g  (1) by definition of Y
= (λx.g (x x))(λx.g (x x))  (2) by function application of λf
= g((λx.g (x x))(λx.g (x x)))  (3) by function application of λx.g(x x) to λx.g(x x)
= g(Y g)  (4) by equation (2)
Applying equation (4)
repeatedly will lead to:
Y g = g(g(Y g))  (5) by equation (4)
= g(...g(Y g) ...)  (6) by repeatedly applying (4)
In this way the Y
combinator achieves recursion by reproducing a (selfreproducing) copy of the function’s selfapplication with each application of (4)
.
This selfreproducing pattern becomes even more visible when looking at the graphstructure of the reduction of (Y g)
:
__@ ==> @ ==> @ ==> ... @ \
/ \ / \ / \ / \__/
Y g g @ g @ g
/ \ / \
Y g g @
/ \
Y g
One can see how at each application of (4)
another copy of (Y g) is generated and incorporated into the graph as an argument of g.
The last step of the diagram shows that  in the graph  selfreproduction can be achieved by simply bending the argument pointer back to the application node.
This realization leads us to the following implementation of the Ycombinator:
Y (p1 : _) = do
reduce :@: gP) < readSTRef p1
(_YP :@: p1) writeSTRef p1 (gP
Using this implementation of the Ycombinator instead of the source level defined version Y = λf.(λx.x x)(λx.f(x x))
reduces the execution time for fact 10000
by a factor of about 250.
The sourcecode for this section can be found here.
Here are some ideas for possible future extensions and improvements.
letrec
) for global function definitionsapply
and (△)
.P
combinator)Recently I read the very interesting Compiling to Categories paper by Conal Elliot.
He presents the idea to compile haskell programs into expressions of cartesian closed categories by λelimination. These expressions can then be used for different purposes like alternative program evaluation, graphic representation of program graphs, designing hardware layouts for algorithms, etc.
The λelimination process applied reminded me a lot of the bracket abstraction used when compiling λterms to SKICombinators.
In the following I’m having a closer look at the parallels between compiling lambda to CCC and compiling lambda to SKIcombinators
I assume at least a rough familiarity with the λcalculus. If you need a refresher I recommend the chapter on λcalculus in Stephen Diels excellent Write You a Haskell.
Instead of the classical notation of lambda terms I’ll use the equivalent Haskell notation throughout this post. So instead of writing λx.x a
I’ll write \x > x a
.
The SKI combinator calculus is a combinatory logic, a computational system that may be perceived as a reduced version of the untyped lambda calculus. It can be thought of as a computer programming language […] because it is an extremely simple Turing complete language. It was introduced by Moses Schönfinkel and Haskell Curry.
Quoted from Wikipedia
λterms can be converted to variable free SKI combinator terms with a process called bracket abstraction. Bracket abstraction absCL
is defined by the following equations (given in pseudo Haskell notation, as pattern matching on functions is not possible in Haskell):
> x) = i
absCL (\x > y) = k y
absCL (\x > p q) = s (\x > p) (\x > q) absCL (\x
where the combinators i
, k
and s
are defined as follows (these are valid haskell definitions):
i :: a > a
= x
i x
k :: a > b > a
= x
k x y
s :: (a > b > c) > (a > b) > a > c
= p x (q x) s p q x
Please note that i
is identical to id
and k
is identical to const
from the Haskell Prelude.
Once the λterms are compiled to combinator terms, these terms can be interpreted quite efficiently as they don’t contain any variables and so no environmenthandling is needed.
Combinator terms also allow to apply several more advanced interpretation techniques like graphreduction, nodesharing, parallel reduction, etc.
For a very cool demo have a look at the web assembly based graph reduction engine by Ben Lynn.
In his famous paper Compiling to Categories Conal Elliot describes a way to compile from simply typed lambdacalculus terms to cartesian closed categories(CCC).
At the core of his approach sits a transformation from lambdaterms to CCC expressions that are done by eliminating variables by an abstraction function absCCC
(again in pseudoHaskell):
> x) = id
absCCC (\x > y) = const y
absCCC (\x > p q) = apply . ((\x > p) △ (\x > q)) absCCC (\x
Where (△)
is introduced by the Cartesian
category:
class Category k => Cartesian k where
:: (a `k` c) > (a `k` d) > (a `k` (c, d)) (△)
In the (>)
instance of Cartesian
(△)
is defined as:
:: (t > a) > (t > b) > t > (a, b)
(△)= (f x, g x) (f △ g) x
And where apply
is introduced by the Closed
category:
class Cartesian k => Closed k where
apply :: ((a > b), a) `k` b
In the (>)
instance of Closed
apply
is defined as
apply :: (a > b, a) > b
= f x apply (f, x)
The function absCCC
looks surprisingly similar to the absCL
function defined above. The first two pattern matches are obviously equivalent as i
and id
are identical as well as k y
and const y
.
But what about the third clause? We have:
 on the one hand: abstracting lambdaterms to combinator expresssions:
> p q) = s (\x > p) (\x > q)
absCL (\x
 and on the other: abstracting lambdaterms to CCC expressions:
> p q) = apply . ((\x > p) △ (\x > q)) absCCC (\x
Are these two definitions equal?
By eliminating all variables from the term apply . ((\x > p) △ (\x > q))
we can write it as a combinator s'
with variables p, q, x
:
= (apply . (p △ q)) x s' p q x
Now we can apply equational reasoning:
= (apply . (p △ q)) x
s' p q x = apply ((p △ q) x)  by definition of (.)
= apply (p x, q x)  by definition of (△)
= (p x) (q x)  by definition of apply
This equals the definition of the s
combinator:
= (p x) (q x) s p q x
So we can conclude that the transformations from λcalculus to SKIcombinators and CCC are equivalent.
For me this was a new insight. But it seems that I was not the first to discover this: P.L. Curien presents a much more elaborate proof of this correspondence in his classic paper Categorical Combinators. See also Cartesian Closed Categories and LambdaCalculus.
In my next blog post I will have a closer look at a CCC based execution model for a subset of Haskell.
]]>Quite a while back I wrote a larger article on the algebraic foundation of software patterns which also covered the MapReduce algorithm.
During the research digged out a paper on algebraic properties of distributed big data analytics, which explained that a MapReduce will always work correctly when the intermediate data structure resulting from the map
phase is a Monoid under the reduce
operation.
For some reason, I was not convinced that this Monoidcondition was enough, because all the typical examples like wordfrequency maps are even commutative Monoids under the respective reduce operation.
So I came up with the following personal theory:
Only if the intermediate data structure resulting from the
map
phase is a commutative Monoid under thereduce
operation, then a parallel MapReduce will produce correct results.
I tried to validate this property using the QuickCheck test framework.
Interestingly the QuickCheck tests failed! This finally convinced me that my theory was wrong, and after a little deeper thought, I could understand why.
I was impressed with the power of QuickCheck, so I thought it would be a good idea to share this lesson in falsification.
The code shown in this blog is also available on GitHub
In abstract algebra, a monoid is a set equipped with an associative binary operation and an identity element.
The simplest example for a commutative Monoid is \((\mathbb{N}_0, +, 0)\): the natural numbers under addition with \(0\) as the identity (or neutral) element. We can use QuickCheck to verify that indeed the Monoid laws plus commutativity are maintained.
If we want to use GHC.Natural
type to represent natural numbers, we first have to make Natural
instantiate the Arbitrary
type class which is used by QuickCheck to automatically generate test data:
import Test.QuickCheck (Arbitrary, arbitrary, NonNegative (..))
import GHC.Natural (Natural, naturalFromInteger)
instance Arbitrary Natural where
= do
arbitrary NonNegative nonNegative < arbitrary
return $ naturalFromInteger nonNegative
Now we can start to write our property based tests. For algebraic structures it is straightforward to come up with properties: we just write the required laws (associativity, 0 is identity element and commutativity) as properties.
I am using Hspec as a wrapper around QuickCheck as it provides a very nice testing DSL which makes it easy to read the code and the output of the test suite:
import Test.Hspec
spec :: Spec
= do
spec "The Monoid 'Natural Numbers under Addition'" $ do
describe "is associative" $
it $ \x y z > ((x + y) + z) `shouldBe` ((x + (y + z)) :: Natural)
property
"has 0 as left and right identity element" $
it $ \x > (x + 0 `shouldBe` (x :: Natural)) .&&. (0 + x `shouldBe` x)
property
"is commutative" $
it $ \x y > x + y `shouldBe` (y + x :: Natural) property
The output of these tests will be as follows:
Monoid
The Monoid 'Natural Numbers under Addition'
is associative
+++ OK, passed 100 tests.
has 0 as identity (or neutral) element
+++ OK, passed 100 tests.
is commutative
+++ OK, passed 100 tests.
So behind the scenes, QuickCheck has generated test data for 100 tests for each property under test. For all these data the test cases passed.
This is definitely not a proof. But it gives us some confidence that our math textbooks are correct when giving Natural Numbers under addition as an example for a commutative Monoid.
OK, that was easy! Now let’s move to noncommutative Monoids.
Strings (or any other Lists) under concatenation are a typical example. It’s easy to see that "hello" ++ ("dear" ++ "people")
equals "(hello" ++ "dear") ++ "people"
, but that "hello" ++ "world"
differs from "world" ++ "hello"
.
Now let’s try to formalize these intuitions as QuickCheck property based tests again.
First I’m introducing an alias for (++)
, as it is defined on any list type, it would be required to have type signatures in all properties (as we had all those :: Natural
signatures in the examples above). So I define an operation (⊕)
which is only defined on String
instances:
:: String > String > String
(⊕)= a ++ b (⊕) a b
Now we can extend our test suite with the following test cases:
"The Monoid 'Strings under concatenation'" $ do
describe
"is associative" $
it $ \x y z > ((x ⊕ y) ⊕ z) `shouldBe` (x ⊕ (y ⊕ z))
property
"has \"\" as left and right identity element" $
it $ \x > (x ⊕ "" `shouldBe` x) .&&. ("" ⊕ x `shouldBe` x) property
The output looks promising:
The Monoid 'Strings under concatenation'
is associative
+++ OK, passed 100 tests.
has "" as left and right identity element
+++ OK, passed 100 tests.
Now let’s try to test the noncommutativity:
"is NOT commutative" $
it $ \x y > x ⊕ y `shouldNotBe` y ⊕ x property
But unfortunately the output tells us that this is not true:
is NOT commutative FAILED [1]
1) Monoid, The Monoid 'Strings under concatenation', is NOT commutative
Falsifiable (after 1 test):
""
""
not expected: ""
We formulated the property in the wrong way. The (⊕)
may be commutative for some edge cases, e.g. when one or both of the arguments are ""
. But it is not commutative in general – that is for all possible arguments.
We could rephrase this property as “There exists at least one pair of arguments \((x, y)\) for which \(\oplus\) is not commutative”:
\[\exists (x,y) \left [ x \oplus y \neq y \oplus x \right ]\]
QuickCheck does not come with a mechanism for existential quantification. But as is has forAll
, that is universal quantification. So we can try to make use of the following equivalence:
\[\exists (x,y) \left [ x \oplus y \neq y \oplus x \right ] \equiv \neg \forall (x,y) \left [ x \oplus y = y \oplus x \right ]\]
Unfortunately we can not write this simply as not forAll
, as forAll
returns a Property
but not
expects a Bool
. But as explained in this discussion on Stackoverflow it is still posible to implement our own exists
:
exists :: (Show a, Arbitrary a) => (a > Bool) > Property
= forSome $ resize 1000 arbitrary
exists
forSome :: (Show a, Testable prop) => Gen a > (a > prop) > Property
=
forSome gen prop > r {P.reason = "No witness found.", P.callbacks = []}) $
mapResult (\r $ disjoin $ replicate 1000 $ forAll gen prop once
Now we can rewrite the property \(\exists (x,y) \left [ x \oplus y \neq y \oplus x \right ]\) as follows:
"is not commutative (via exists)" $
it $ \(x,y) > x ⊕ y /= y ⊕ x exists
I like how close the Haskell code stays to the concise mathematical formulation! The output of this test fits much better into our intuitive understanding:
is not commutative (via exists)
+++ OK, passed 1 test.
MapReduce is a programming model and an associated implementation for processing and generating large data sets. Users specify a map function that processes a key/value pair to generate a set of intermediate key/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key.
[This] abstraction is inspired by the map and reduce primitives present in Lisp and many other functional languages. Quoted from Google Research
I’m not going into more details here, as You’ll find detailed information on this approach and a working example in my original article.
Here is the definition of a sequential MapReduce:
simpleMapReduce :: (a > b)  map function
> ([b] > c)  reduce function
> [a]  list to map over
> c  result
= reduceFunc . map mapFunc simpleMapReduce mapFunc reduceFunc
We can test the sequential MapReduce algorithm with the following property based test:
"works correctly with a sequential mapreduce" $
it $ \a b c d > (simpleMapReduce reverse (foldr (⊕) "") [a,b,c,d])
property `shouldBe` (reverse a) ⊕ (reverse b) ⊕ (reverse c) ⊕ (reverse d)
What I have shown so far just demonstrates the general mechanism of chaining map
and reduce
functions without implying any parallel execution. Essentially we are chaining a map
with a fold
(i.e. reduction) function. In the Haskell base library there is a higher order function foldMap
that covers exactly this pattern of chaining. Please note that foldMap
does only a single traversal of the foldable data structure. It fuses the map
and reduce
phase into a single one by function composition of mappend
and the mapping function f
:
  Map each element of the structure to a monoid,
 and combine the results.
foldMap :: (Foldable t, Monoid m) => (a > m) > t a > m
foldMap f = foldr (mappend . f) mempty
Now we come to the tricky part that kicked off this whole discussion: parallelism.
As an example we consider a simple sequential MapReduce, taking an input list of Int
s, computing their squares and computing the sum of these squares:
> simpleMapReduce (^2) (foldr (+) 0) [1,2,3,4]
λ30
Let’s try to design this as a massively parallelized algorithm:
Mapping of (^2)
over the inputlist [1,2,3,4]
would be started in parallel to the reduction of the intermediary list of squares by (foldr (+) 0)
.
The mapping phase will be executed as a set of parallel computations (one for each element of the input list).
The reduction phase will also be executed as a set of parallel computations (one for each addition).
Of course the reduction phase can begin only when at least one list element is squared. So in effect the mapping process would have to start first. The parallel computation of squares will result in a nondeterministic sequence of computations. In particular it is not guaranteed that all elements of the input list are processed in the original list order. So it might for example happen that 3
is squared first. Now the reduction phase would receive it’s first input 9
, and would start reduction, that is compute 9 + 0
.
Let’s assume the following random sequence of mapping steps: Next the first element of the input 1
, then the fourth 4
and finally the second element 2
would be squared, resulting in a reduction sequence of 4 + 16 + 1 + 9 + 0
. As this sums up to 30
everything is fine. Addition is commutative, so changing the sequence of reduction steps does not affect the overall result.
But now imagine we would parallelize:
> simpleMapReduce reverse (foldr (⊕) "") [" olleh"," ym"," raed"," sklof"]
λ"hello my dear folks "
If we assume the same sequence as above, the third element of the input list would be reversed first, resulting in a first reduction step "dear " ⊕ ""
. Next the first, the fourth and finally the second element would be reversed, resulting in a reduction sequence of "my " ⊕ "folks " ⊕ "hello " ⊕ "dear " ⊕ "" = "my folks hello dear "
. As string concatenation is not commutative it does not really come as a surprise that random changes to the reduction sequence will eventually result in wrong computations.
So our conclusion is:
If the MapReduce algorithm is parallelized in the way that I outlined above — which may result in random changes of the order of list elements in the reduction phase — it will only work correct if the intermediary data structure is a commutative Monoid under the reduce operation.
In the following section we will implement a parallel MapReduce in Haskell in order to validate our theory with property based testing.
We can define a parallel MapReduce implementation as follows (for more details see Real World Haskell, Chapter 24):
import Control.Parallel (par)
import Control.Parallel.Strategies (using, parMap, rpar)
parMapReduce :: (a > b)  map function
> ([b] > c)  reduce function
> [a]  list to map over
> c  result
=
parMapReduce mapFunc reduceFunc input `par` reduceResult
mapResult where mapResult = parMap rpar mapFunc input
= reduceFunc mapResult `using` rpar reduceResult
This implementation will start computing mapResult
and reduceResult
in parallel and finally return reduceResult
. The mapResult
is computed with a parallelized map
function parMap
. The reduceResult
is computed by applying a parallel reduction strategy rpar
.
Next we will write a property based test to valdate our theory:
"has some cases where parallel reduction deviates from sequential reduction" $
it $ \text > parMapReduce reverse (foldr (⊕) "") text
exists /= simpleMapReduce reverse (foldr (⊕) "") text
But it turns out that QuickCheck does not find any evidence for this assumption:
has some cases where parallel reduction deviates from sequential reduction FAILED [1]
Failures:
test\MonoidSpec.hs:69:5:
1) Monoid, The Monoid 'Strings under concatenation', has some cases where parallel reduction deviates from sequential reduction
*** Failed! No witness found. (after 1 test):
After seeing this result I had to deal with some growing cognitive dissonance, much like a flat earther confronted with experimental evidence…
I began verifying my setup. I made sure that the package.yaml
contains the right GHC options to provide parallel execution of the test suite:
ghcoptions:
 O2
 threaded
 rtsopts
 eventlog
 withrtsopts=N
I also made sure that all cores of my CPU were actually running at 100% utilization during the parallel tests.
I also inspected the runtime behaviour with the amazing ThreadScope tool (A ThreadScope eventlog can be produced by adding the runtime flags +RTS ls N
when executing an application).
I also increased the number of test executions to give better chances to hit any rare cases.
But to no avail.
Since QuickCheck kept telling me, “You’re wrong,” I finally started to admit, “Well, maybe I am indeed wrong and should take a closer look at the problem.”
Taking a closer look at the definition of the parallel MapReduce will allow us to better understand what’s actually going on:
import Control.Parallel (par)
import Control.Parallel.Strategies (using, parMap, rpar)
parMapReduce :: (a > b)  map function
> ([b] > c)  reduce function
> [a]  list to map over
> c  result
=
parMapReduce mapFunc reduceFunc input `par` reduceResult
mapResult where mapResult = parMap rpar mapFunc input
= reduceFunc mapResult `using` rpar
reduceResult
 and now an actual example usage:
= parMapReduce reverse (foldr (⊕) "") [" olleh"," ym"," raed"," sklof"] x
In this concrete example mapResult
will be:
= parMap rpar reverse [" olleh"," ym"," raed"," sklof"] mapResult
parMap is defined as follows:
parMap :: Strategy b > (a > b) > [a] > [b]
= (`using` parList strat) . map f parMap strat f
The parMap
evaluation strategy will spark a parallel evaluation for each element of the input
list. Nevertheless the actual order of elements will not be changed as internally the classical sequential map
function is used which will not change the order of elements. So the reduce phase will never receive a changed order of elements from the map phase, even if map
computations for the individual list elements might be executed in random order!
mapResult
will always be ["hello", "my ", "dear ", "folks"]
.
Thus reduceResult
will be:
= (foldr (⊕) "") ["hello", "my ", "dear ", "folks"] `using` rpar reduceResult
Again the traditional semantics of foldr
is maintained — in particular the order of arguments of all (⊕)
operations, only we allow for parallel evaluation of those (⊕)
operations during the reduction phase.
So the final output will always be "hello my dear folks"
. The parallelism introduced by the Control.Parallel
package does not in any way change the semantics of pure functional programs as our example. This is exactly what the failed test cased kept telling me:
There do not exist any cases where sequential and parallel MapReduce result in deviating results!
We can again evaluate our improved theory with a QuickCheck test:
"parallel reduction always equals sequential reduction" $
it $ \l > simpleMapReduce reverse (foldr (⊕) "") l
property `shouldBe` parMapReduce reverse (foldr (⊕) "") l
And — not so surprisingly — this test succeeds!
If you want to know more about parallel evaluation in Haskell I highly recommend the exellent Parallel and Concurrent Programming in Haskell by Simon Marlow.
The parallelism as provided by the Haskell Control.Parallel
package maintains the semantics of pure functional code and thus a parallel MapReduce maintains the same properties as its sequential counterpart. So a parallel MapReduce will still work correctly if the intermediate data structure resulting from the map
phase is just a Monoid — not necessarily a commutative Monoid.
Nevertheless there may be implementations that do not strictly maintain the original order of the input data during the map
 and reduce
phases. With such implementations the intermediate data structure resulting from the map
phase must be a commutative Monoid under the reduce
operation to produce correct results.
Property based testing with QuickCheck is a powerful tool to verify assumptions about a given codebase. I really like using it as intended by Karl Poppers Theory of Falsifiability:
Some weeks ago I gave an introductory talk on functional programming at an inhouse miniconference where I tried to explain basic concepts of functional programming to a crowd of developers with their backgrounds mostly in imperative and OO languages.
Initially I had planned to present the contents of my Why Haskell Matters article but that turned out impractical because of the time constraints of my talk.
So I prepared a condensed slide deck focussing on the more elementary sections, which worked quite well.
Recently I stumbled across IHaskell, a Haskell kernel for the Jupyter Notebook platform. As I like the interactive approach of Jupyter notebooks quite a lot I transferred my presentation into the Jupyter notebook format.
As I’m quite happy with the result I’d like to make it available to a wider audience. I have prepared three different versions:
The notebook as rendered by nbviewer.jupyter.org. Unfortunately it’s not interactive. But you won’t need a local Jupyter installation.
A reveal.js presentation which I generated from the notebook. This version just needs a webbrowser.
An Interactive version of the reveal.js slides hosted on Binder: (Press ALTr to start the presentation.) This version uses RISE to allow interactive notebook cells while still being in presentation mode. This version is based on a Dockerfile that adds the RISE extension to an ihaskellnotebook docker image. This Dockerfile is then built and served by Binder.
If you intend to use IHaskell notebooks locally Please follow this installation guide.
Sourcecode of my IHaskell notebook and the dockerfile are hosted in this github repo.
]]>This post is part of the Advent of Haskell 2020 series. Hence, I tried to keep the content easy and enjoyable but still present some food for thought!
Some time ago I came across an interesting post on the CleanCoderBlog, which kept me busy for weeks until I finally decided to write this article.
In his blogpost Uncle Bob tries to reconcile concepts from both Functional Programming and Object Oriented Programming by explaining that both approaches are not mutually exclusive but both provide useful principles that go very well together and in fact are complementary:
In this blog I will make the case that while OO and FP are orthogonal, they are not mutually exclusive. That a good functional program can (and should) be object oriented. And that a good object oriented program can (and should) be functional.
He begins his argument by reducing FP and OOP each to a single central guiding principle in order to contrast the essential features of these two approaches as clearly as possible:
He gives the following characterisation of OOP:
The technique of using dynamic polymorphism to call functions without the source code of the caller depending upon the source code of the callee.
With this short statement Uncle Bob points to the core of object orientation since its first incarnation in the Smalltalk language:
In an OO language a call of methods on a target object is dispatched based on the target object’s type, its class
. So a method call shape.draw()
may invoke different code based on the class
of the actual shape object:
The code of the draw
method of class Rectangle
may be different from the code in Circle.draw()
.
Client code will just call shape.draw()
, not even knowing which actual Shape
subclass it’s working on. This kind of polymorphism provides a very useful decoupling of clients from the target objects by using the methods of the baseclass Shape
as the API for all Objects inheriting Shape
.
This mechanism allows to build elegant design like the ModelViewController (MVC) pattern which is at the core of Smalltalks GUI and which influenced many similar designs in other OOlanguages.
MVC is the seminal insight of the whole field of graphical user interfaces. I believe the MVC work was the first to describe and implement software constructs in terms of their responsibilities. I also believe that MVC was the first significant use of protocols to define components instead of using concrete implementations — each controller class had a certain set of messages it had to respond to, as did each view class, but otherwise there were no constraints on what they did and how they did it.
This quote conveys two major achievements of OOP:
It’s interesting to note that Uncle Bob does not consider Inheritance or Encapsulation to be the most important and central concepts in OOP.
Next he gives a very brief characterization of functional programming:
Referential Transparency – no reassignment of values.
Referential transparency is implying purity as explained in the following definition from Wikipedia:
An expression is called referentially transparent if it can be replaced with its corresponding value (and viceversa) without changing the program’s behavior. This requires that the expression be pure, that is to say the expression value must be the same for the same inputs and its evaluation must have no side effects.
The second part of Uncle Bob’s statement may be implied by this definition, but I prefer to see it as separate yet closely related principle, namely immutability:
In objectoriented and functional programming, an immutable object (unchangeable object) is an object whose state cannot be modified after it is created. […]
After this dense characterization of the two programming paradigms Uncle Bob continues his arguments like follows:
The concepts of Polymorphism and Referential Transparency are orthogonal. You can have Polymorphism without Referential Transparency – and vice versa.
But orthogonality does not imply that both concepts are mutually exclusive. It is possible to have languages that support both Dynamic Polymorphism and Referential Transparency. It is not only possible, but even desirable to combine both concepts:
Dynamic Polymorphism is desirable as it allows building strongly decoupled designs:
Dependencies can be inverted across architectural boundaries. They are testable using Mocks and Fakes and other kinds of Test Doubles. Modules can be modified without forcing changes to other modules. This makes such systems much easier to change and improve.
Uncle Bob
Referential Transparency is desirable as it allows designs that are much easier to understand, to reason about, to change and to improve. It also allows designs that are much better suited for scalability and concurrency as the chances of race conditions etc. are drastically reduced.
Uncle Bob concludes that Dynamic Polymorphism and Referential Transparency are both desirable as part of software systems:
A system that is built on both OO and FP principles will maximize flexibility, maintainability, testability, simplicity, and robustness.
Uncle Bob
In the following sections I will have a look at the Haskell language to see how the principles of Adhoc Polymorphism and Referential Transparency are covered in our favourite language.
Referential Transparency
Haskell is one of the rare incarnations of a purely functional language. So it goes without saying that Referential Transparency, Purity and Immutability are a given in Haskell. Yes, there are things like unsafePerformIO
or IORef
but overall it’s very easy to write clean code in Haskell due to the strict separation of pure and impure code by making side effects directly visibly in functions type signatures.
Referential Transparency in Haskell is so much a given that it’s quite possible to apply equational reasoning to proof certain properties of Haskell programs. See for example the following Proof of Functor laws for the Maybe type. What’s remarkable here is that you can use the same language to write your code and to reason about it. This is not possible in languages that do not provide Referential Transparency and Immutability. To reason about programs in such languages you have to use external models like an abstract stack + register machine.
Adhoc Polymorphism
Being able to overload functions and operators with different implementations depending on the type of its arguments is called Adhoc Polymorphism. For example, the +
operator does something entirely different when applied to floatingpoint values as compared to when applied to integers. In Haskell, this kind of polymorphism is achieved with type classes and class instances.
Haskell’s type classes are quite different from the classes in OOP languages. They have more in common with interfaces in that they specify a set of functions with their respective type signatures to be implemented by instance declarations.
In this section I’m showcasing how these two concepts are supported in Haskell and how they can be combined without sacrificing FP principles.
Let’s have a look at a simple example that is frequently used in introductions to OOP: a class hierarchy representing geometrical shapes. In a typical OO language, we would have an abstract base class Shape
which specifies a set of methods, and concrete classes Rect
, Circle
, Triangle
, etc. which would implement specific behaviour.
This simple class hierarchy is shown in the following UML diagram:
In Haskell there is no inheritance between types. But with type classes we can specify an interface which must be implemented by concrete types that wish to instantiate the type class. So we start with a Shape
type class:
  The Shape type class. It defines four functions that all concrete Shape types must implement.
class Shape a where
  render a Shape
draw :: a > IO ()
  move a Shape by an x and y amount
move :: (Double,Double) > a > a
  compute the area of a Shape
area :: a > Double
  compute the circumference of a Shape
circum :: a > Double
Any concrete type a
instantiating Shape
must implement the four functions draw
, move
, area
and circum
.
We start with a Circle
type:
  a circle defined by the centre point and a radius
data Circle = Circle Point Double deriving (Show)
  a point in the twodimensional plane
data Point = Point Double Double
  making Circle an instance of Shape
instance Shape Circle where
Circle centre radius) = putStrLn $ "Circle [" ++ show centre ++ ", " ++ show radius ++ "]"
draw (Circle centre radius) = Circle (movePoint x y centre) radius
move (x,y) (Circle _ r) = r ^ 2 * pi
area (Circle _ r) = 2 * r * pi
circum (
  move a Point by an x and y amount
movePoint :: Double > Double > Point > Point
Point x_a y_a) = Point (x_a + x) (y_a + y) movePoint x y (
As you can see, I’m not going to implement any real graphical rendering in draw
but simply printing out the coordinates of the centre point and the radius. But at least area
and circum
implement the wellknown geometrical properties of a circle.
Following this approach it’s straightforward to implement data types Rect
and Triangle
. Let’s start with Rect
:
  a rectangle defined by to points (bottom left and top right corners)
data Rect = Rect Point Point deriving (Show)
  making Rect an instance of Shape
instance Shape Rect where
Rect a b) = putStrLn $ "Rectangle [" ++ show a ++ ", " ++ show b ++ "]"
draw (Rect a b) = Rect a' b'
move (x,y) (where
= movePoint x y a
a' = movePoint x y b
b' = width * height
area rect where
= widthAndHeight rect
(width, height) = 2 * (width + height)
circum rect where
= widthAndHeight rect
(width, height)
  computes the width and height of a rectangle, returns them as a tuple
widthAndHeight :: Rect > (Double, Double)
Rect (Point x_a y_a) (Point x_b y_b)) = (abs (x_b  x_a), abs (y_b  y_a)) widthAndHeight (
There is nothing special here, we are just implementing the functions specified by the Shape
type class in a most simple way.
On to Triangle
:
  a triangle defined by three points
data Triangle = Triangle Point Point Point deriving (Show)
  making Triangle an instance of Shape
instance Shape Triangle where
Triangle a b c) = putStrLn $ "Triangle [" ++ show a ++ ", " ++ show b ++ ", " ++ show c ++ "]"
draw (Triangle a b c) = Triangle a' b' c'
move (x,y) (where
= movePoint x y a
a' = movePoint x y b
b' = movePoint x y c
c' = sqrt (s * (s  a) * (s  b) * (s  c))  using Heron's formula
area triangle where
= 0.5 * (a + b + c)
s = sides triangle
(a, b, c) = a + b + c
circum triangle where
= sides triangle
(a, b, c)
  computing the length of all sides of a triangle, returns them as a triple
sides :: Triangle > (Double, Double, Double)
Triangle x y z) = (distance x y, distance y z, distance x z)
sides (
  compute the distance between two points
distance :: Point > Point > Double
Point x_a y_a) (Point x_b y_b) = sqrt ((x_b  x_a) ^ 2 + (y_b  y_a) ^ 2)
distance (
  provide a more dense representation of a point
instance Show Point where
show (Point x y) = "(" ++ show x ++ "," ++ show y ++ ")"
Let’s create three sample instances:
rect :: Rect
= Rect (Point 0 0) (Point 5 4)
rect
circle :: Circle
= Circle (Point 4 5) 4
circle
triangle :: Triangle
= Triangle (Point 0 0) (Point 4 0) (Point 4 3) triangle
Now we have all ingredients at hand for a little demo.
The type class Shape
specifies a function draw :: Shape a => a > IO ()
. This function is polymorphic in its argument: it will take an argument of any type a
instantiating Shape
and will perform an IO ()
action, rendering the shape to the console in our case.
Let’s try it in GHCi:
> draw circle
Circle [(4.0,5.0), 4.0]
> draw rect
Rectangle [(0.0,0.0), (5.0,4.0)]
> draw triangle
Triangle [(0.0,0.0), (4.0,0.0), (4.0,3.0)]
This code makes use of Haskell’s Adhoc polymorphism and elegantly fulfils the requirements given for Dynamic Polymorphism in Uncle Bob’s blog post: “call functions without the source code of the caller depending upon the source code of the callee”. On the call site, we just rely on the function draw :: (Shape a) => a > IO ()
. This type signature assures us that it will work on any concrete type a
that instantiates the Shape
type class.
By making use of the reversed application operator (&)
we can create a more OOP lookandfeel to our code. Depending on the context it may be more convenient to write and read code using (&)
even when you are not after an OOP lookandfeel.
> import Data.Function ((&))
> circle & draw
Circle [(4.0,5.0), 4.0]
We can use the (&)
operator to even work in a fluent api style:
main :: IO ()
= do
main
rect& move (4,2)
& draw
rect& draw
circle& move (4,2)
& draw
circle& draw
 and then in GHCi:
> main
Rectangle [(4.0,2.0), (9.0,6.0)]
Rectangle [(0.0,0.0), (5.0,4.0)]
Circle [(8.0,7.0), 4.0]
Circle [(4.0,5.0), 4.0]
In Haskell all values are immutable: printing the original shapes a second time demonstrates that operations like move
are not destructive.
With this little setup we have shown that Haskell allows us to have both: Referential Transparency plus adhoc polymorphism. That is, we can use the essential elements of OOP and FP in one language. And: we are doing it all the time, as it’s quite common to use class types in this way.
In Haskell, container types like lists are polymorphic, but it is not possible to define a list like this:
shapes :: [Shape]
= [circle,rect,triangle] shapes
because type classes are not types, but constraints on types.
So in haskell a list like [circle,rect,triangle]
is considered to be heterogeneous, as the concrete types of all the elements differ.
There are several ways to have heterogeneous collections in Haskell. I will demonstrate just one of them, which is based on existential types. (I have chosen this approach as it keeps the code easier to read and allows to add more Shape
types whenever needed.
There is also a recent blog post on Existential Haskell which demonstrates some interesting use cases for existential types.
However, the sourcecode for this example also demonstrates a solution based on a simple sum type.)
Once we activate the ExistentialQuantification
language extension, we can define a data type ShapeType
with a single constructor MkShape
that will take any instance of a concrete type instantiating the Shape
type class:
{# LANGUAGE ExistentialQuantification #}
data ShapeType = forall a . (Show a, Shape a) => MkShape a
Now we can make ShapeType
an instance of Shape
which will delegate all function calls to the wrapped types:
instance Shape ShapeType where
MkShape s) = area s
area (MkShape s) = circum s
circum (MkShape s) = draw s
draw (MkShape s) = MkShape (move vec s)
move vec (
 we also have to manually derive a Show instance as auto deriving is not possible on the existential type
instance Show ShapeType where
show (MkShape s) = show s
With this setup we can define a list of shapes as follows:
shapes :: [ShapeType]
= [MkShape rect, MkShape circle, MkShape triangle] shapes
Finally, we are able to use this list just as any other:
main :: IO ()
= do
main print $ map area shapes
print $ map circum shapes
print $ map (move (4,10)) shapes
putStrLn ""
mapM_ draw shapes
 and then in GHCi:
> main
20.0,50.26548245743669,6.0]
[18.0,25.132741228718345,12.0]
[Rect (4.0,10.0) (9.0,14.0),Circle (8.0,15.0) 4.0,Triangle (4.0,10.0) (8.0,10.0) (8.0,13.0)]
[
Rectangle [(0.0,0.0), (5.0,4.0)]
Circle [(4.0,5.0), 4.0]
Triangle [(0.0,0.0), (4.0,0.0), (4.0,3.0)]
In our short Demo we have seen that Haskell supports both Referential Transparency and Polymorphism. We have also seen that the reversed application operator (&)
allows us to structure code in a way that even has some kind of OOP lookandfeel while remaining purely functional.
If we follow Uncle Bob’s argumentation to view Polymorphism to be the central concept of OOP (and in consequence regard other important OO features like Inheritance or Encapsulation as not so distinctive), we can conclude that Haskell is already well prepared to implement programs in the hybrid way proposed by him.
In fact, the benefits associated with this approach (flexibility, maintainability, testability, simplicity, and robustness) are typical key features of systems implemented in Haskell.
Thanks to David Feuer for helping me with a stupid error in the existential type code!
Thanks to the Advent Of Haskell 2020 team for having this blog post in their advents calendar!
]]>Over the weekend I’ve set up a hakyll powered blog on github pages (actually the one you are reading right now).
I’d like to share my findings, as I found an easier way to integrate Hakyll with GitHub Pages.
I followed the tutorial Using Hakyll with GitHub Pages. This tutorial assumes that the GitHub pages must always be served from the root folder of a github repository.
It then describes a way to achieve this by using a develop
branch to do all the Hakyll work and finally writing the contents to the default _site
folder. This folder is excluded from version control by entries in the .gitignore
file both in the develop
and master
branches.
So to finally publish the generated site you’ll have to switch to the master
branch and copy the contents of the _site
folder to the root folder of your project.
I tried this approach and it works nicely.
But then I found out that GitHub pages also allows to use a docs
folder as the document root of your GitHub Pages site.
This makes things significantly easier, as you can do all the necessary hakyll tasks and the final publishing on the same master
branch.
/docs
folder from the master
branch should be used as documentroot of your site. Please refer to the documentation in case of problems._cache/
.stackwork/
If required, create a new Hakyll project. If you’re a stack user, there is a Hakyll template available that makes this step easy:
stack new myblog hakylltemplate
Create a .gitignore file in your blog’s directory with at a minimum, the same directories listed as in the GitHub repository. Use the following git commands to setup your local repository:
git init
# track all the source files for our blog.
git add .
# make our first commit
git commit m "initial commit."
# and add the GitHub repository as a remote.
git remote add origin <URL to your GitHub pages repository>
In order to make Hakyll generate the site into the docs
folder we have to edit the Hakyll Main module (site.hs
if you use the stack template) slightly:
config :: Configuration
= defaultConfiguration
config = "docs"
{ destinationDirectory
}
main :: IO ()
= do
main $ do
hakyllWith config ...
So everything’s all setup, and we’re ready to deploy.
We need to be able to run the executable that generates the website, so we need to compile it first. If you are using stack, this can be done using:
stack build
Next we get a clean build of our site:
stack exec myblog clean
stack exec myblog build
After this step you should see a folder docs
under your projects root folder, which contains the generated Hakyll site.
Now we commit our changes:
git add A
git commit m "Publish."
And send them to GitHub:
git push origin master:master
That’s all.
Now your Hakyll site should be visible under your GitHub Pages URL!
The source code for this blog post lies under my thma.github.io GitHub project. I’ve adopted the composeconference css from Katy Chuangs great Hakyll CSS Garden and tweaked it a little bit to look more like GitHub markdown style and to provide some basic responsive design.
The Hakyll project has just accepted my pull request for the Using Hakyll with GitHub Pages tutorial, which updates the text according to the findings presented in this post.
The amended version will thus be online shortly, rendering this blog entry kind of obsolete…
]]>This article shows how algebraic effect systems can be used to maintain a clear separation of concerns between different parts of software systems. From a practical programming perspective this improves composability and testability of software components.
I’m demonstrating this idea by using the Polysemy library to implement a multilayered REST application conforming to the guidelines of the Clean Architecture model.
While writing Why Haskell Matters I prepared a little demo application that was meant to showcase a cleanly designed REST application in Haskell. In particular, I wanted to demonstrate how the clear separation of pure and impure code helps to provide strict separation of concerns and stateoftheart testability of all application layers.
I failed!
I was able to write the domain logic in pure code consisting only of total functions. It was a great pleasure to write unit tests for them!
However, as soon as I started writing controllers that coordinate access to the domain logic as well as to a persistence layer to retrieve and store data, I was stuck in the IO Monad. That is, in test cases I was not able to test the controllers independently of the concrete backend.
Then I tried to apply the final tagless pattern for the persistence layer. This allowed abstracting out the concrete persistence layer and writing controller tests with a mocked persistence backend. But when it came to testing the REST API handlers (written with Servant) I was again stuck in the IO Monad as the Handler type is defined as newtype Handler a = Handler { runHandler' :: ExceptT ServerError IO a }
. Maybe it’s not a principle issue but just my brain being too small…
I was desperately looking for something that allowed me to combine different types of effects (like persistence, logging, configuration, http handlers, error handling, etc.) in controllers and handlers but still to be able to write tests that allow using mocks or stubs to test components in isolation.
As I reached a dead end, I had a look at some of the algebraic effect systems available in Haskell, like eff, extensibleeffects, fusedeffects, freersimple and Polysemy.
In algebraic effect systems, effectful programs are split into two separate parts: the specification of the effects to be performed, and the interpretation (or semantics) given to them.
So my idea was to provide special effect interpretations that would allow building mocked effects for my test suite.
After seeing a presentation on maintainable software architecture with Polysemy which answered many of my questions I rewrote my application based on Polysemy powered algebraic effects.
I’m pretty satisfied with the result, and of course I’m eager to share my approach with you!
A very small boutique restaurant (serving excellent vietnamese food) is looking for a reservation system that allows managing reservations. The restaurant has only twenty seats, they also take only a maximum of twenty reservations per day. (So guests can stay the whole evening and don’t have to leave after some time.) (I adopted this scenario from a inspiring talk by Mark Seemann)
They have asked us to write the REST backend for their reservation system.
The chef insists on a scrupulously clean kitchen and is also a lover of clean code. He has read about clean architecture and wants his new software to be a perfect example!
So we cannot just hack away but first have to understand what is expected from us when we are to deliver a clean architecture.
I’m following the introduction to clean architecture by Robert C. Martin on his Clean Code blog. He states that his concept builds up on several earlier approaches like hexagonal architecture, ports and adapters or Onion Architecture.
According to him all these approaches share a similar objective: achieve separation of concerns by dividing a software system into different layers. All approaches result in system designs that share a common set of features:
The architecture does not depend on any specific software libraries or frameworks. This allows to freely choose such tools according to the actual needs. This avoids “vendor lock in”.
High testability. The business logic can be tested without any external element like UI, DB, Web Server, etc.
The UI is loosely coupled to the core system. So it can be easily changed or replaced without affecting the rest of the system.
The Database is also “external” to the core system. It can be easily changed (even from an RDBMS to NonSQL DB) without affecting the business logic.
The Business logic is agnostic of the outside world. It has no dependencies to any external systems like DB, ESB, etc.
The architecture consists of four layers, each of which contains components with a specific scope and a limited set of responsibilities.
At the centre sits the Domain layer consisting of entities and core business logic.
Next comes the Use Cases layer where all resources are coordinated that are required to fulfill a given use case. In particular, it uses entities and logic from the domain layer to implement use cases. But typically it must also interface to a persistent storage to retrieve and store entities.
The Interface Adapters layer holds code for UI controllers and presenters as well as adapters to external resources like databases, message queues, configuration, Logging, etc.
The External Interfaces layer contains the technical implementation of external interfaces. For example, a concrete REST service assembly, Web and UI infrastructure, databases, etc.
The overriding rule that makes this architecture work is The Dependency Rule. This rule says that source code dependencies can only point inwards. Nothing in an inner circle can know anything at all about something in an outer circle. In particular, the name of something declared in an outer circle must not be mentioned by the code in the an inner circle. That includes, functions, classes. variables, or any other named software entity.
Quoted from Clean Architecture blog post
This dependency rule leads to a very interesting consequence: If a use case interactor needs to access a component from an outer circle, e.g. retrieve data from a database, this must be done in a specific way in order to avoid breaking the dependency rule: In the use case layer we don’t have any knowledge about the components of the outer circles. If we require access to a database (or any other external resources), the call interface, as well as the data transfer protocol must be specified in the use case layer.
The components in the outer circles will then implement this interface. Using this kind of interfaces, it is possible to communicate accross the layer boundaries, but still maintain a strict separation of concerns.
If you want to dive deeper into clean architecture I recommend the Clean Architecture blog post as an entry point. Robert C. Martin later also published a whole book Clean Architecture: A Craftsman’s Guide to Software Structure and Design on this concept.
In the following sections I’ll explain how the clean architecture guidelines can be implemented in a Haskell REST API application by making use of the algebraic effect library Polysemy.
The ReservationDomain module implements the business logic for seat reservations in a very small boutique restaurant. The restaurant has only one big table with 20 seats. Each day the restaurants accepts only 20 reservations. (There is no limited timeslot for each guest.)
Please note:  all functions in this module are pure (they don’t do any IO) and total (they produce defined results for all possible input values).
At the core of our Domain lies the Reservation
data type:
  a data type representing a reservation
data Reservation = Reservation
date :: Day  ^ the date of the reservation
{ name :: String  ^ the name of the guest placing the reservation
, email :: String  ^ the email address of the guest
, quantity :: Natural  ^ how many seats are requested
,
}deriving (Eq, Generic, Read, Show)
This type can be used to express facts like Mr. Miller reserved two seats on 20200601, he can be reached via his email address: manfred@miller.com:
= Reservation {name = "Mr. Miller", quantity = 2, date = read "20200601", email = "manfred@miller.com"} reservation
All reservations of a specific day are represented as a list of reservations: [Reservation]
.
A ReservationMap
is a map from Day
to [Reservation]
:
  a key value map holding a list of reservations for any given day
type ReservationMap = Map Day [Reservation]
That is, we can keep track of all reservations by maintaining them in such a map:
fromList
[
(20200601,
[Reservation {date = 20200601, name = "Mr. Miller", email = "manfred@miller.com", quantity = 2},
Reservation {date = 20200601, name = "Andrew M. Jones", email = "amjones@example.com", quantity = 4}
]
) ]
Based on these data types we can define domain logic like computing the used capacity of a list of reservations:
  computes the number of reserved seats for a list of reservations
usedCapacity :: [Reservation] > Natural
= 0
usedCapacity [] Reservation _ _ _ quantity : rest) = quantity + usedCapacity rest usedCapacity (
Based on this we can compute the number of available seats (given a maximum capacity and a list of reservations):
  computes the number of available seats from a maximum capacity and a list of reservations.
availableSeats :: Natural> [Reservation] > Natural
= maxCapacity  usedCapacity reservations availableSeats maxCapacity reservations
The Reservation
data type and some of the domain logic functions are depicted in the in the following diagram:
As already mentioned: this layer has no knowledge of the world and it’s all pure code. Testing domain logic in isolation therefore is straight forward, as you can see from the DomainSpec code.
The data types and functions of the domain layer can be used directly, without any mocking of components:
= fromGregorian 2020 1 29
day = Reservation day "Andrew M. Jones" "amjones@example.com" 4
res1 = Reservation day "Thomas Miller" "tm@example.com" 3
res2 = [res1, res2]
reservations = 20
totalCapacity
spec :: Spec
=
spec "Domain Logic" $ do
describe "computes the used capacity for an empty list of reservations" $
it `shouldBe` 0
usedCapacity []
"computes the used capacity for a list of reservations" $
it `shouldBe` 7
usedCapacity [res1, res2]
"computes the available seats for a list of reservations" $
it `shouldBe` 13 availableSeats totalCapacity [res1, res2]
The software in this layer contains application specific business rules. It encapsulates and implements all of the use cases of the system. These use cases orchestrate the flow of data to and from the entities, and direct those entities to use their enterprise wide business rules to achieve the goals of the use case.
Quoted from the Clean Architecture blog post
The module ReservationUseCase specifies the available use cases for the reservation system. It coordinates access to Effects and the actual domain logic. The module exposes service functions that will be used by the REST API in the ExternalInterfaces layer.
Implemented Use Cases:
Display the number of available seats for a given day
Enter a reservation for a given day and keep it persistent. If the reservation can not be served as all seats are occupies provide a functional error message stating the issue.
Display the list of reservations for a given day.
Delete a given reservation from the system in case of a cancellation. NO functional error is required if the reservation is not present in the system.
Display a List of all reservation in the system.
In the Use Case layer we have left the garden Eden of world agnostic code:
In order to compute the number of available seats for a given day, we will have to first look up the actual reservations for that day from a persistent storage, and only then can we call the domain function availableSeats
. In addition we also will have to write a Log message when calling the functions to provide an audit trail.
However, the dependency rule of clean architecture bans all direct access to a database or a logginginfrastructure from the use case layer!
Algebraic Effect systems offer a consistent answer: 1. We declare effects in the use case layer by defining them as an abstract interface.
We also specify the actual usage of effects in the use case layer by having calls against the abstract interface.
We provide an interpretation of these effects only in the outer layers. This also allows us to provide different implementations. So we can easily swap backends, e.g. migrating from MySQL to PostgreSQL, and it can be used to provide mock implementations for testing purposes.
Let’s see how all this looks like when using Polysemy.
  compute the number of available seats for a given day. the result must be a natural number, incl. 0
availableSeats :: (Member Persistence r, Member Trace r) => Day > Sem r Natural
= do
availableSeats day $ "compute available seats for " ++ show day
trace < fetch day
todaysReservations return $ Dom.availableSeats maxCapacity todaysReservations
  fetch the list of reservations for a given day from the key value store.
 If no match is found, an empty list is returned.
fetch :: (Member Persistence r, Member Trace r) => Day > Sem r [Dom.Reservation]
= do
fetch day $ "fetch reservations for " ++ show day
trace < getKvs day
maybeList return $ fromMaybe [] maybeList
  the maximum capacity of the restaurant.
maxCapacity :: Natural
= 20 maxCapacity
The type signature of availableSeats
contains two constraints on the effect stack type r
: (Member Persistence r, Member Trace r)
This means that the function may perform two different effects: persistence via the Persistence
effect and Logging via the Trace
effect.
The type signature also specifies that we need an input of type Day
and will return the Natural
result wrapped in the Sem r
monad.
The Sem
monad handles computations of arbitrary extensible effects. A value of type Sem r
describes a program with the capabilities of the effect stack r
.
The first step of the function body of availableSeats
specifies a Log action based on the (Polysemy builtin) Trace
effect:
$ "compute available seats for " ++ show day trace
I repeat: trace
does not directly do any logging. The actual logging action  the effect interpretation  will be defined in the application assembly or in a test setup.
The next line specifies a lookup of the reservation list for day
from the persistence layer:
< fetch day todaysReservations
where fetch is defined as:
fetch :: (Member Persistence r, Member Trace r) => Day > Sem r [Dom.Reservation]
= do
fetch day $ "fetch reservations for " ++ show day
trace < getKvs day
maybeList return $ fromMaybe [] maybeList
To understand the fetch
function, in particular the expression maybeList < getKvs day
we first have to know the definition of the Persistence
effect:
type Persistence = KVS Day [Dom.Reservation]
Where KVS (standing for Key/Value Store) is a type that is also defined in the use case layer (KVS.hs):
  a key value store specified as a GADT
data KVS k v m a where
ListAllKvs :: KVS k v m [(k, v)]
GetKvs :: k > KVS k v m (Maybe v)
InsertKvs :: k > v > KVS k v m ()
DeleteKvs :: k > KVS k v m ()
'KVS makeSem '
The four operations of the key value store are defined in the GADT as type constructors. makeSem ''KVS
then uses TemplateHaskell to generate effect functions (or smart Constructors) from the GADT definition. This call results in the definition of the following four functions that represent the specific operations of the key value store:
listAllKvs :: Member (KVS k v) r => Sem r [(k, v)]
getKvs :: Member (KVS k v) r => k > Sem r (Maybe v)
insertKvs :: Member (KVS k v) r => k > v > Sem r ()
deleteKvs :: Member (KVS k v) r => k > Sem r ()
These functions can be used in the Sem
Monad. So now we understand much better what is going on in fetch
:
fetch :: (Member Persistence r, Member Trace r) => Day > Sem r [Dom.Reservation]
= do
fetch day $ "fetch reservations for " ++ show day
trace < getKvs day
maybeList return $ fromMaybe [] maybeList
As fetch
operates in the Sem
monad, maybeList
is bound to a Maybe [Dom.Reservation]
value, which results from the getKVs day
action. The function finally uses fromMaybe
to return a list of reservations that were retrieved (or []
in case Nothing
was found for day
).
Then, back in availableSeats
we call the domain logic function Dom.availableSeats
to compute the number of available seats. The resulting Natural
value is lifted into the Sem r
monad, thus matching the signature of the return type Sem r Natural
.
In the next diagram I’m depicting the layers Use Cases and Domain. The arrow from Use Cases to Domain represents the dependency rule: use case code may only reference domain logic but the domain logic may not reference anything from the use case layer.
On the left side of the diagram we see the use case controllers (aka use case interactors) like availableSeats
that coordinate all activities and resources to fulfill a specific use case.
On the right we see the gateway (or interface) code like the KVS
abstraction of a keyvalue store or the fetch
operation that wraps the access to the keyvalue store.
The key value store functions like getKvs
don’t perform any concrete operation. They just declare
access to an abstract keyvalue store interface.
The concrete interpretation of these calls will be specified in the application assembly (typically in Main.hs
) or in the setup code of test cases. If we provide a pure interpretation then the resulting code will also be pure. This allows writing tests in the same pure way as for the domain logic.
As an example, in UseCasePureSpec I’m providing pure interpretations for all effects.
The runPure
function takes a program with effects and handles each effect till it gets reduced to Either ReservationError (ReservationMap‚ a)
:
runPure :: ReservationMap
> Sem '[UC.Persistence, State ReservationMap, Error UC.ReservationError, Trace] a
> Either UC.ReservationError (ReservationMap, a)
=
runPure kvsMap program
program& runKvsPure kvsMap  run the keyvalue store on a simple ReservationMap
& runError @UC.ReservationError  run error handling to produce an Either UC.ReservationError (ReservationMap, a)
& ignoreTrace  run Trace by simply ignoring all messages
& run  run a 'Sem' containing no effects as a pure value
In addition to that I’m providing wrapping functions like runAvailableSeats
that use runPure
to interprete the effects of the use case functions (eg. UC.availableSeats
) and extract the actual result from the [Either UC.ReservationError (ReservationMap, a)]
return value:
runAvailableSeats :: ReservationMap > Day > Natural
= do
runAvailableSeats kvsMap day case runPure kvsMap (UC.availableSeats day) of
Right (_, numSeats) > numSeats
Left err > error "availableSeats failed"
This is all that it takes to abstract away persistence layer, logging facility and exception handling. We can now write tests in pure code:
 setting up test fixtures
initReservations :: ReservationMap
= M.singleton day res
initReservations
= read "20200502"
day = [Reservation day "Andrew M. Jones" "amjones@example.com" 4]
res
spec :: Spec
=
spec "Reservation Use Case (only pure code)" $ do
describe
"computes the number of available seats for a given day" $ do
it `shouldBe` 16 (runAvailableSeats initReservations day)
This layer holds code for adapters to external resources like databases, message queues, configuration, Logging, etc.
The Logging effect Trace
ships with Polysemy, so we don’t have to implement anything here. (Of course we could overzealously implement our own Graylog adapter here, Hingegen hat unser reservationServer
eine Typensignatur but I leave this as an exercise for the reader… )
However, as the KVS
type is our own invention we’ll have to provide our own implementations. (We could have used the KVStore
type from polysemyzoo, but for didactic purposes we will roll our own.)
The following code is the inmemory implementation from the KVSInMemory module. It defines a keyvalue store in terms of State (Map k v)
that is a Map k v
in a State
effect context:
runKvsOnMapState :: ( Member (State (M.Map k v)) r, Ord k)
=> Sem (KVS k v : r) a
> Sem r a
= interpret $ \case
runKvsOnMapState ListAllKvs > fmap M.toList get
GetKvs k > fmap (M.lookup k) get
InsertKvs k v > modify $ M.insert k v
DeleteKvs k > modify $ M.delete k
So whenever the interpret
functions detects a GetKvs k
value, that was constructed by a call to getKvs k
in the use case layer, it patternmatches it to a Map
lookup of k
that is executed against state retrieved by get
.
Interestingly get
is a smart constructor of the State
effect. This means that by interpreting the KVS
we have created new effects that in turn have to be interpreted.
The runKvsPure
functions (which we already have seen in the use case testing) chains interpretation of the effects KVS
and State
and thus allows us to work with pure Maps as mocks for a keyvalue store:
runKvsPure :: Ord k
=> M.Map k v
> Sem (KVS k v : State (M.Map k v) : r) a
> Sem r (M.Map k v, a)
map = runState map . runKvsOnMapState runKvsPure
As we are in the interface adapters layer, we are allowed to get our hands dirty with real world code, like database access. As an example I have provided a SQLite based interpretation of the KVS
effect in KVSSqllite.hs.
The effect interpreting function is runKvsAsSQLite
:
  Run a KVStore effect against a SQLite backend. Requires a Config object as input.
runKvsAsSQLite :: (Member (Embed IO) r, Member (Input Config) r, Member Trace r, Show k, Read k, ToJSON v, FromJSON v)
=> Sem (KVS k v : r) a
> Sem r a
= interpret $ \case
runKvsAsSQLite GetKvs k > getAction k
ListAllKvs > listAction
InsertKvs k v > insertAction k v
DeleteKvs k > deleteAction k
The function’s type signature introduces a two more constraints on the effect stack type r
: Member (Embed IO) r
and Member (Input Config) r
. (Embed IO)
is needed as accessing SQLite will require IO, which can be lifted into the Sem r
monad with Embed IO
.
SQLite always needs a file name to create a database connection. As we want to be able to keep this name configurable, we use the (Input Config)
effect. Config
is a data type that I created to represent global application configuration, including the database file name. Input
is a Polysemy builtin effect which can provide input to an application, quite similar to a Reader
monad.
These effects are introduced by the actual implementations of the KVS
constructors, like getAction k
, which retrieves a value from the database by looking up the key k
:
getAction :: (Member (Input Config) r, Member (Embed IO) r, Member Trace r, Show k, Read k, ToJSON v, FromJSON v) => k > Sem r (Maybe v)
= do
getAction key < connectionFrom input
conn < embed (SQL.queryNamed conn
rows "SELECT key, value FROM store WHERE key = :key"
":key" := show key] :: IO [KeyValueRow])
[$ "get: " ++ show rows
trace case rows of
> return Nothing
[] KeyValueRow _key value):xs > return $ (decode . encodeUtf8) value
(
  create a connection based on configuration data, make sure table "store" exists.
connectionFrom :: (Member (Embed IO) r) => Sem r Config > Sem r SQL.Connection
= do
connectionFrom c < c
config
embed (getConnection (dbPath config))where
getConnection :: FilePath > IO SQL.Connection
= do
getConnection dbFile < SQL.open dbFile
conn "CREATE TABLE IF NOT EXISTS store (key TEXT PRIMARY KEY, value TEXT)"
SQL.execute_ conn return conn
Let’s have a closer look at what is going on in getAction
:
First connectionFrom input
is used to create a database connection based on the Config
object obtained by input
(the smart Constructor of the Input
effect). The Config
type contains a field dbPath
which is read and used to create the connection with getConnection
. As this is an IO operation we have to use embed
to lift it into the Sem r
monad.
In the second step SQL.queryNamed
is used to perform the actual select statement against the db connection. Again embed
must be used to lift this IO operation.
Finally the resulting [KeyValueRow]
list is pattern matched: if the list is empty Nothing
is returned. Otherwise Aeson.decode
is called to unmarshal a result value from the JSON data retrieved from the database.
The JSON encoding and decoding to and from the DB is the reason for the ToJSON v, FromJSON v
constraints on the value type v
.
This implementation is inspired by keyvalue store of a password manager in Polysemy.
Our task was to build the backend for the reservation system. We will have to implement a REST API to allow access to the business logic that we defined in the use case layer.
The overall idea is to provide a REST route for all exposed functions of the ReservationUseCase
. The following table shows the mapping of those functions to the REST routes that we want to achieve:
listAll GET /reservations
fetch GET /reservations/YYYYMMDD
tryReservation POST /reservations
cancel DELETE /reservations
availableSeats GET /seats/YYYYMMDD
I’m using Servant to define our REST API. The great thing about Servant is that it allows us to define REST APIs in a typesafe manner by using a type level DSL.
Here comes the declaration of our API (please note that we declare our routes to accept and emit data in JSON format):
  in order to allow JSON serialization for the Dom.Reservation type, it must instantiate FromJSON and ToJSON.
instance ToJSON Dom.Reservation
instance FromJSON Dom.Reservation
  Declaring the routes of the REST API for Restaurant Reservations
type ReservationAPI =
"reservations" :> Summary "retrieve a map of all reservations (Day > [Reservation])"
:> Get '[ JSON] Dom.ReservationMap  GET /reservations
:<> "reservations" :> Summary "retrieve list of reservations for a given day"
:> Capture "day" Day
:> Get '[ JSON] [Dom.Reservation]  GET /reservations/YYYYMMDD
:<> "reservations" :> Summary "place a new reservation"
:> ReqBody '[ JSON] Dom.Reservation
:> Post '[ JSON] ()  POST /reservations
:<> "reservations" :> Summary "cancel a reservation"
:> ReqBody '[ JSON] Dom.Reservation
:> Delete '[ JSON] ()  DELETE /reservations
:<> "seats" :> Summary "retrieve number of free seats for a given day"
:> Capture "day" Day
:> Get '[ JSON] Natural  GET /seats/YYYYMMDD
Next we have to create the connection between the declared routes and the actual business logic. This will be our REST service implementation. In our case we simply delegate to the use case controller functions. Off course, we might also implement additional functionality here like validation:
import qualified UseCases.ReservationUseCase as UC
  implements the ReservationAPI
reservationServer :: (Member UC.Persistence r, Member (Error UC.ReservationError) r,
Member Trace r, Member (Input Config) r) => ServerT ReservationAPI (Sem r)
=
reservationServer  GET /reservations
UC.listAll :<> UC.fetch  GET /reservations/YYYYMMDD
:<> UC.tryReservation  POST /reservations
:<> UC.cancel  DELETE /reservations
:<> UC.availableSeats  GET /seats/YYYYMMDD
I really love how declarative this code is. We don’t have to tell how to exchange data between the REST server and the use case controllers.
We just tell what we want: a mapping from the routes to the controller functions. That’s all!
In the following diagram, we now see the third layer. Again, the arrow symbolises the dependency rule, which prohibits access from domain or use case layer to the interface adapters layer. To the right we see the ReservationAPI
and its reservationServer
implementation, which we just explored. They interact with the use case controller functions like availableSeats
, listAll
, etc.
To the left we see the interpretations of the KVS
effect (which was defined in the use case layer): KVSInMemory
, KVSSqlite
(and a third one KVSFileServer
, a file based implementation which you could explore on your own).
We’ll have a closer look at the test of the SQLite implementation of the KVS
effect.
As Polysemy effects are involded we will need to provide an interpretation to actually perform the SQLLite operation.
The test setup looks quite similar to the tests in the use case layer.
We want our test to evaluate the KVS implementation independently of the domain logic and the use case layer. Therefore, we first define an example use case, featuring a data type Memo
and a set of typical CRUD operations. The CRUD operations are using the KVS
smart constructors and thus exhibit the typical Polysemy effect signatures:
  a key value table mapping Natural to a list of Strings
type KeyValueTable = KVS Int [String]
data Memo = Memo Int [String]
deriving (Show)
persistMemo :: (Member KeyValueTable r) => Memo > Sem r ()
Memo id lines ) = insertKvs id lines
persistMemo (
fetchMemo :: (Member KeyValueTable r) => Int > Sem r (Maybe [String])
= getKvs
fetchMemo
fetchAll :: (Member KeyValueTable r) => Sem r (M.Map Int [String])
= fmap M.fromList listAllKvs
fetchAll
deleteMemo :: (Member KeyValueTable r) => Int > Sem r ()
= deleteKvs deleteMemo
Next we define a set of helper functions that allow us to execute the CRUD operations as ordinary IO ()
actions, which we can use in our test code:
 Helper functions for interpreting all effects in IO
runPersist :: Memo > IO ()
= runAllEffects (persistMemo memo)
runPersist memo
runFetch :: Int > IO (Maybe [String])
= runAllEffects (fetchMemo k)
runFetch k
runFetchAll :: IO (M.Map Int [String])
= runAllEffects fetchAll
runFetchAll
runDelete :: Int > IO ()
= runAllEffects (deleteMemo k) runDelete k
These wrapper function make use of the runAllEffects
function that takes a program with effects and handles each effect till it gets reduced to IO a
:
runAllEffects :: Sem '[KeyValueTable, Input Config, Trace, Embed IO] a > IO a
=
runAllEffects program
program& runKvsAsSQLite  use SQLite based interpretation of the (KVS Int [String]) effect
& runInputConst config  use the variable config as source for (Input Config) effect
& ignoreTrace  ignore all traces
& runM  reduce Sem r (Embed IO a) to IO a
where config = Config {port = 8080, dbPath = "kvstest.db", backend = SQLite, verbose = False}
 errors are rethrown as Runtime errors, which can be verified by HSpec.
handleErrors :: IO (Either err a) > IO a
= do
handleErrors e either < e
case either of
Right v > return v
Left _ > error "something bad happend"
With these preliminaries at hand we can now write our test cases:
= 4711
key = ["In the morning", "I don't drink coffee", "But lots of curcuma chai."]
text = Memo key text
memo
spec :: Spec
=
spec "The KV Store SQLite Implementation" $ do
describe "returns Nothing if nothing can be found for a given id" $ do
it < runFetch key
maybeMatch `shouldBe` Nothing
maybeMatch
"persists a keyvalue pair to the SQLite database" $ do
it
runPersist memo< runFetch key
maybeMatch `shouldBe` Just text
maybeMatch
"fetches a Map of all keyvalue entries from the KV store" $ do
it map < runFetchAll
map `shouldBe` 1
M.size
"deletes an entry from the key value store" $ do
it
runDelete key< runFetch key
maybeMatch `shouldBe` Nothing maybeMatch
The actual code for testing the REST API looks pretty straightforward. We create a WAI Application
instance with createApp
and execute REST operations like get
and postJSON
against it:
reservationData :: LB.ByteString
= "{\"email\":\"amjones@example.com\",\"quantity\":10,\"date\":\"20200502\",\"name\":\"Amelia Jones\"}"
reservationData
= request methodPost path [(hContentType, "application/json")]
postJSON path = request methodDelete path [(hContentType, "application/json")]
deleteJSON path
spec :: Spec
=
spec $
with (createApp) "Rest Service" $ do
describe "responds with 200 for a call GET /reservations " $
it "/reservations" `shouldRespondWith` "{\"20200502\":[{\"email\":\"amjones@example.com\",\"quantity\":4,\"date\":\"20200502\",\"name\":\"Andrew M. Jones\"}]}"
get "responds with 200 for a valid POST /reservations" $
it "/reservations" reservationData `shouldRespondWith` 200
postJSON "responds with 412 if a reservation can not be done on a given day" $
it "/reservations" reservationData >> postJSON "/reservations" reservationData) `shouldRespondWith` 412
(postJSON "responds with 200 for a valid DELETE /reservations" $
it "/reservations" reservationData `shouldRespondWith` 200 deleteJSON
Please note that these tests don’t need a deployment of the WAI application to a web server. ALl testing can be done within a single process. We stick to the dependency rule not to use anything from a more outward layer.
The interesting part is the creation of the Application
instance.
If we had a simple implementation myServer
of a REST API myApi
, not using any Polysemy effects, we could create an Application
instance like so:
createSimpleApp :: Application
::= serve myApi myServer createSimpleApp
In contrast, our reservationServer
has a type signature that contains Polysemy effects:
reservationServer :: (Member UC.Persistence r, Member (Error UC.ReservationError) r,
Member Trace r, Member (Input Config) r) => ServerT ReservationAPI (Sem r)
Instead of building the Application
instance directly, as in the simple example, we use liftServer
to lift reservationServer
into the required ServerT ReservationAPI Handler
type by running all effects and by lifting the business logic exception ReservationNotPossible
into a Servant ServerError
. This time we also use the SQLite based interpretation of the KVS
effect:
createApp :: Config > IO Application
= return $ serve reservationAPI (liftServer config)
createApp config
liftServer :: Config > ServerT ReservationAPI Handler
= hoistServer reservationAPI (interpretServer config) reservationServer
liftServer config where
=
interpretServer config sem
sem& runKvsAsSQLite
& runInputConst config
& runError @ReservationError
& ignoreTrace
& runM
& liftToHandler
= Handler . ExceptT . (fmap handleErrors)
liftToHandler Left (ReservationNotPossible msg)) = Left err412 {errBody = pack msg}
handleErrors (Right value) = Right value handleErrors (
The outermost layer is generally composed of frameworks and tools such as the Database, the Web Framework, etc. Generally you don’t write much code in this layer other than glue code that communicates to the next circle inwards.
This layer is where all the details go. The Web is a detail. The database is a detail. We keep these things on the outside where they can do little harm.
Quoted from Clean Architecture blog post
For the database we are already finished as the SQliteSimple library includes the SQLLite C runtime library and is thus selfcontained.
We will use WARP as our Web Server, which can be used as a library within our Main
program. What we still have to do though, is to assemble a Servant web Application
so that it can be executed on the warp server.
We have done this step already for the testing of the REST service. The createApp
function that we define in the ApplicationAssembly module will look quite familiar, it just provides some more bells and whistles to integrate all the features that we have developed so far.
createApp
accepts a Config
parameter which is used to configure application settings.selectKvsBackend
selects the concrete KVS
interpretation.selectTraceVerbosity
selects the Trace
interpretation:  creates the WAI Application that can be executed by Warp.run.
createApp :: Config > IO Application
= do
createApp config return (serve reservationAPI $ hoistServer reservationAPI (interpretServer config) reservationServer)
where
= sem
interpretServer config sem & selectKvsBackend config
& runInputConst config
& runError @ReservationError
& selectTraceVerbosity config
& runM
& liftToHandler
= Handler . ExceptT . (fmap handleErrors)
liftToHandler Left (ReservationNotPossible msg)) = Left err412 { errBody = pack msg}
handleErrors (Right value) = Right value
handleErrors (
  can select between SQLite or FileServer persistence backends.
selectKvsBackend :: (Member (Input Config) r, Member (Embed IO) r, Member Trace r, Show k, Read k, ToJSON v, FromJSON v)
=> Config > Sem (KVS k v : r) a > Sem r a
= case backend config of
selectKvsBackend config SQLite > runKvsAsSQLite
FileServer > runKvsAsFileServer
InMemory > error "not supported"
  if the config flag verbose is set to True, trace to Console, else ignore all trace messages
selectTraceVerbosity :: (Member (Embed IO) r) => Config > (Sem (Trace : r) a > Sem r a)
=
selectTraceVerbosity config if verbose config
then traceToIO
else ignoreTrace
The application assembly also features a function to load a Config
instance. Typically, this would involve loading a configuration file or reading command line arguments. We take a shortcut here and just provide a static instance:
  load application config. In real life, this would load a config file or read commandline args.
loadConfig :: IO Config
= return Config {port = 8080, backend = SQLite, dbPath = "kvs.db", verbose = True} loadConfig
With the whole application assembly written as library code, there is not much left to do in the Main
module:
import ExternalInterfaces.ApplicationAssembly (createApp, loadConfig)
import InterfaceAdapters.Config
import Network.Wai.Handler.Warp (run)
main :: IO ()
= do
main < loadConfig
config < createApp config
app putStrLn $ "Starting server on port " ++ show (port config)
run (port config) app
The following diagram shows the elements added by the External Interface layer:  On the left we have application assembly code like createApp
used by the Warp
server or some of the different runPure
functions that we used in HSpec tests.  On the right we have the SQLite runtime library that provides access to the SQLite database and the Haskell runtime in general, which provides access to the filesystem and the OS in general.
Testing the application assembly is quite straightforward and resembles the testing of the REST service:
loadConfig :: IO Config
= return Config {port = 8080, backend = SQLite, dbPath = "kvsassembly.db", verbose = False}
loadConfig
spec :: Spec
=
spec >>= createApp) $
with (loadConfig "Rest Service" $ do
describe
"responds with 20 for a first call to GET /seats/YYYYMMDD" $
it "/seats/20200502" `shouldRespondWith` "20"
get
"responds with 200 for a valid POST /reservations" $
it "/reservations" reservationData `shouldRespondWith` 200
postJSON
"responds with 200 for a call GET /reservations " $
it "/reservations" `shouldRespondWith` "{\"20200502\":[{\"email\":\"amjones@example.com\",\"quantity\":12,\"date\":\"20200502\",\"name\":\"Amelia Jones\"}]}"
get
"responds with 412 if a reservation can not be done on a given day" $
it "/reservations" reservationData >> postJSON "/reservations" reservationData) `shouldRespondWith` 412
(postJSON
"responds with 20 for a first call to GET /seats/YYYYMMDD" $
it "/seats/20200502" `shouldRespondWith` "8"
get
"responds with 200 for a valid DELETE /reservations" $
it "/reservations" reservationData `shouldRespondWith` 200 deleteJSON
For all those who have been patient enough to stay with me until here, I now have a little bonus.
There is a servantswaggerui addon available which allows to serve a SwaggerDoc UI for any Servant API. This UI renders an automatically generated documentation of our Reservation API and even allows to test all API operations directly.
You can launch it by executing stack build exec PolysemyCleanArchitecture
in the root folder of the project.
This will launch the REST service and open up the Swagger UI in your Web browser:
The code for this goody can be found in the SwaggerUI module.
Robert C. Martin concludes his blog post with a brief summary:
Conforming to these simple rules is not hard, and will save you a lot of headaches going forward. By separating the software into layers, and conforming to The Dependency Rule, you will create a system that is intrinsically testable, with all the benefits that implies. When any of the external parts of the system become obsolete, like the database, or the web framework, you can replace those obsolete elements with a minimum of fuss.
Quoted from the Clean Architecture blog post
I have emphasized the testability aspect quite a lot in this article. However, this approach allows switching freely between alternative backends in production environments as well.
As we have seen Polysemy — or algebraic effect systems in general — make this possible by the separation of effect declaration, effect usage and effect interpretation.
Furthermore, Polysemy also allows you to freely combine several effects. This is a huge gain in software composability.
]]>In this post I want to give a short example of how equational reasoning can be used to proof certain properties of a given piece of code in Haskell.
So without further ado let’s begin:
The Functor
instance declaration of the type Maybe
is defined as:
instance Functor Maybe where
fmap _ Nothing = Nothing  (1)
fmap f (Just a) = Just (f a)  (2)
The composition operator (.)
is defined as:
(.) :: (b > c) > (a > b) > a > c
. g x = f (g x)  (3) f
The Identity function id
is defined as:
id :: a > a
id x = x  (4)
The claim is that Maybe
fulfils the two functor laws:
1.: fmap id = id
2.: fmap (f . g) = (fmap f . fmap g)
Claim: fmap id m = id m
, for any m
of type Maybe a
.
Proof. On cases of m
.
Case 1: m = Nothing
.
fmap id m = fmap id Nothing  by expansion of m
= Nothing  by applying equation (1)
= id m  by definition m, by applying equation (4)
Case 2: m = Just a
.
fmap id m = fmap id (Just a)  by expansion of m
= Just (id a)  by applying equation (2)
= Just a  by expansion of id (equation (4))
= m  by definition of m
= id m  by applying equation (4)
Therefore, fmap id m = id m
in all cases.∎
Claim: fmap (f . g) m = (fmap f . fmap g) m
, for any m
of type Maybe a
.
Proof. On cases of m
.
Case 1: m = Nothing
.
fmap (f . g) m = fmap (f . g) Nothing  by expansion of m
= Nothing  by applying equation (1)
fmap f . fmap g) m = fmap f (fmap g Nothing)  by applying equation (4) and expanding m
(= fmap f Nothing  by applying equation (1)
= Nothing  by applying equation (1)
Case 2: m = Just a
.
fmap (f . g) m = fmap (f . g) (Just a)  by expansion of m
= Just ((f . g) a)  by applying equation (2)
fmap f . fmap g) m = fmap f (fmap g (Just a))  by applying equation (4) and expanding m
(= fmap f (Just (g a))  by applying equation (2)
= Just (f (g a)  by applying equation (2)
= Just ((f . g) a)  by applying equation (3)
Therefore, fmap (f . g) m = (fmap f . fmap g) m
in all cases. ∎
You’ll see this kind of reasoning quite a lot in Haskell documentation and online discussions. The simple reason is: if you can prove something you don’t have to test it.
]]>With Haskell, you don’t solve different problems. But you solve them differently.
In this article I try to explain why Haskell keeps being such an important language by presenting some of its most important and distinguishing features and detailing them with working code examples.
The presentation aims to be selfcontained and does not require any previous knowledge of the language.
The target audience are Haskell freshmen and developers with a background in nonfunctional languages who are eager to learn about concepts of functional programming and Haskell in particular.
Exactly thirty years ago, on April 1st 1990, a small group of researchers in the field of nonstrict functional programming published the original Haskell language report.
Haskell never became one of the most popular languages in the software industry or part of the mainstream, but it has been and still is quite influential in the software development community.
In this article I try to explain why Haskell keeps being such an important language by presenting some of its most distinguishing features and detailing them with working code examples.
The presentation aims to be selfcontained and does not require any previous knowledge of the language. I will also try to keep the learning curve moderate and to limit the scope of the presentation; nevertheless this article is by no means a complete introduction to the language.
(If you are looking for thorough tutorials have a look at Haskell Wikibook or Learn You a Haskell
Before diving directly into the technical details I’d like to first have a closer look on the reception of Haskell in the software developers community:
In a talk in 2017 on the Haskell journey since its beginnings in the 1980ies Simon Peyton Jones speaks about the rather unusual life story of Haskell.
First he talks about the typical life cycle of research languages. They are often created by a single researcher (who is also the single user), and most of them will be abandoned after just a few years.
A more successful research language might gain some interest in a larger community but will still not escape the ivory tower and typically will be given up within ten years.
On the other hand we have all those popular programming languages that are quickly adopted by large numbers of developers and thus reach “the threshold of immortality”. That is the base of existing code will grow so large that the language will be in use for decades.
A little jokingly he then depicts the sad fate of languages designed by committees by flat line through zero: They simply never take off.
Finally, he presents a chart showing the Haskell timeline:
The development shown in this chart seems rather unexpected: Haskell started as a research language and was even designed by a committee; so in all probability it should have been abandoned long before the millennium!
Instead, it gained some momentum in its early years followed by a rather quiet phase during the decade of OO hype (Java being released in 1995). And then again we see a continuous growth of interest since about 2005. I’m writing this in early 2020, and we still see this trend!
Then Simon Peyton Jones points out another interesting characteristic of the reception of Haskell in recent years: In statistics that rank programming languages by actual usage Haskell is typically not under the 30 most active languages. But in statistics that instead rank languages by the volume of discussions on the internet Haskell typically scores much better (often in the top ten).
A very short answer might be: Haskell has a number of features that are clearly different from those of most other programming languages. Many of these features have proven to be powerful tools to solve basic problems of software development elegantly.
Therefore, over time other programming languages have adopted parts of these concepts (e.g. pattern matching or type classes). In discussions about such concepts the Haskell heritage is mentioned and differences between the original Haskell concepts and those of other languages are discussed. Sometimes people feel encouraged to have a closer look at the source of these concepts to get a deeper understanding of their original intentions. That’s why we see a growing number of developers working in Python, Typescript, Scala, Rust, C++, C# or Java starting to dive into Haskell.
A further essential point is that Haskell is still an experimental laboratory for research in areas such as compiler construction, programming language design, theoremprovers, type systems etc. So inevitably Haskell will be a topic in the discussion about these approaches.
In the following sections we will try to find the longer answer by studying some of the most distinguishing features of Haskell.
In computer science, a programming language is said to have firstclass functions if it treats functions as firstclass citizens. This means the language supports passing functions as arguments to other functions, returning them as the values from other functions, and assigning them to variables or storing them in data structures.[1] Some programming language theorists require support for anonymous functions (function literals) as well.[2] In languages with firstclass functions, the names of functions do not have any special status; they are treated like ordinary variables with a function type.
quoted from Wikipedia
We’ll go through this one by one:
Let’s have a look how this looks like in Haskell. First we define some simple values:
 define constant `aNumber` with a value of 42.
aNumber :: Integer
= 42
aNumber
 define constant `aString` with a value of "hello world"
aString :: String
= "Hello World" aString
In the first line we see a type signature that defines the constant aNumber
to be of type Integer
. In the second line we define the value of aNumber
to be 42
. In the same way we define the constant aString
to be of type String
.
Haskell is a statically typed language: all type checks happen at compile time. Static typing has the advantage that type errors don’t happen at runtime. This is especially useful if a function signature is changed and this change affects many dependent parts of a project: the compiler will detect the breaking changes at all affected places.
The Haskell Compiler also provides type inference, which allows the compiler to deduce the concrete data type of an expression from the context. Thus, it is usually not required to provide type declarations. Nevertheless, using explicit type signatures is considered good style as they are an important element of a comprehensive documentation.
Next we define a function square
that takes an integer argument and returns the square value of the argument:
square :: Integer > Integer
= x * x square x
Definition of a function works exactly in the same way as the definition of any other value. The only thing special is that we declare the type to be a function type by using the >
notation. So :: Integer > Integer
represents a function from Integer
to Integer
. In the second line we define function square
to compute x * x
for any Integer
argument x
.
Ok, seems not too difficult, so let’s define another function double
that doubles its input value:
double :: Integer > Integer
= 2 * n double n
Anonymous functions, also known as lambda expressions, can be defined in Haskell like this:
> x * x \x
This expression denotes an anonymous function that takes a single argument x and returns the square of that argument. The backslash is read as λ (the greek letter lambda).
You can use such as expressions everywhere where you would use any other function. For example you could apply the anonymous function \x > x * x
to a number just like the named function square
:
 use named function:
= square 5
result
 use anonymous function:
= (\x > x * x) 5 result'
We will see more useful applications of anonymous functions in the following section.
Do you remember function composition from your highschool math classes? Function composition is an operation that takes two functions f
and g
and produces a function h
such that h(x) = g(f(x))
The resulting composite function is denoted h = g ∘ f
where (g ∘ f )(x) = g(f(x))
. Intuitively, composing functions is a chaining process in which the output of function f
is used as input of function g
.
So looking from a programmers perspective the ∘
operator is a function that takes two functions as arguments and returns a new composite function.
In Haskell this operator is represented as the dot operator .
:
(.) :: (b > c) > (a > b) > a > c
.) f g x = f (g x) (
The brackets around the dot are required as we want to use a nonalphabetical symbol as an identifier. In Haskell such identifiers can be used as infix operators (as we will see below). Otherwise (.)
is defined as any other function. Please also note how close the syntax is to the original mathematical definition.
Using this operator we can easily create a composite function that first doubles a number and then computes the square of that doubled number:
squareAfterDouble :: Integer > Integer
= square . double squareAfterDouble
In this section we look at another interesting example of functions producing other functions as return values. We start by defining a function add
that takes two Integer
arguments and computes their sum:
 function adding two numbers
add :: Integer > Integer > Integer
= x + y add x y
This look quite straightforward. But still there is one interesting detail to note: the type signature of add
is not something like
add :: (Integer, Integer) > Integer
Instead it is:
add :: Integer > Integer > Integer
What does this signature actually mean? It can be read as “A function taking an Integer argument and returning a function of type Integer > Integer
”. Sounds weird? But that’s exactly what Haskell does internally. So if we call add 2 3
first add
is applied to 2
which return a new function of type Integer > Integer
which is then applied to 3
.
This technique is called Currying
Currying is widely used in Haskell as it allows another cool thing: partial application.
In the next code snippet we define a function add5
by partially applying the function add
to only one argument:
 partial application: applying add to 5 returns a function of type Integer > Integer
add5 :: Integer > Integer
= add 5 add5
The trick is as follows: add 5
returns a function of type Integer > Integer
which will add 5
to any Integer argument.
Partial application thus allows us to write functions that return functions as result values. This technique is frequently used to provide functions with configuration data.
I could keep this section short by telling you that we have already seen an example for this: the function composition operator (.)
. It accepts two functions as arguments and returns a new one as in:
squareAfterDouble :: Integer > Integer
= square . double squareAfterDouble
But I have another instructive example at hand.
Let’s imagine we have to implement a function that doubles any odd Integer:
ifOddDouble :: Integer > Integer
=
ifOddDouble n if odd n
then double n
else n
The Haskell code is straightforward: new ingredients are the if ... then ... else ...
and the odd odd
which is a predicate from the Haskell standard library that returns True
if an integral number is odd.
Now let’s assume that we also need another function that computes the square for any odd number:
ifOddSquare :: Integer > Integer
=
ifOddSquare n if odd n
then square n
else n
As vigilant developers we immediately detect a violation of the Don’t repeat yourself principle as both functions only vary in the usage of a different growth functions double
versus square
.
So we are looking for a way to refactor this code by a solution that keeps the original structure but allows to vary the used growth function.
What we need is a function that takes a growth function (of type (Integer > Integer)
) as first argument, an Integer
as second argument and returns an Integer
. The specified growth function will be applied in the then
clause:
ifOdd :: (Integer > Integer) > Integer > Integer
=
ifOdd growthFunction n if odd n
then growthFunction n
else n
With this approach we can refactor ifOddDouble
and ifOddSquare
as follows:
ifOddDouble :: Integer > Integer
= ifOdd double n
ifOddDouble n
ifOddSquare :: Integer > Integer
= ifOdd square n ifOddSquare n
Now imagine that we have to implement new function ifEvenDouble
and ifEvenSquare
, that will work only on even numbers. Instead of repeating ourselves we come up with a function ifPredGrow
that takes a predicate function of type (Integer > Bool)
as first argument, a growth function of type (Integer > Integer)
as second argument and an Integer as third argument, returning an Integer
.
The predicate function will be used to determine whether the growth function has to be applied:
ifPredGrow :: (Integer > Bool) > (Integer > Integer) > Integer > Integer
=
ifPredGrow predicate growthFunction n if predicate n
then growthFunction n
else n
Using this higher order function that even takes two functions as arguments we can write the two new functions and further refactor the existing ones without breaking the DRY principle:
ifEvenDouble :: Integer > Integer
= ifPredGrow even double n
ifEvenDouble n
ifEvenSquare :: Integer > Integer
= ifPredGrow even square n
ifEvenSquare n
ifOddDouble'' :: Integer > Integer
= ifPredGrow odd double n
ifOddDouble'' n
ifOddSquare'' :: Integer > Integer
= ifPredGrow odd square n ifOddSquare'' n
With the things that we have learnt so far, we can now start to implement some more interesting functions. So what about implementing the recursive factorial function?
The factorial function can be defined as follows:
For all n ∈ ℕ_{0}:
0! = 1 n! = n * (n1)!
With our current knowledge of Haskell we can implement this as follows:
factorial :: Natural > Natural
=
factorial n if n == 0
then 1
else n * factorial (n  1)
We are using the Haskell data type Natural
to denote the set of nonnegative integers ℕ_{0}. Using the literal factorial
within the definition of the function factorial
works as expected and denotes a recursive function call.
As these kind of recursive definition of functions are typical for functional programming, the language designers have added a useful feature called pattern matching that allows to define functions by a set of equations:
fac :: Natural > Natural
0 = 1
fac = n * fac (n  1) fac n
This style comes much closer to the mathematical definition and is typically more readable, as it helps to avoid nested if ... then ... else ...
constructs.
Pattern matching can not only be used for numeric values but for any other data types. We’ll see some more examples shortly.
Haskell supports userdefined data types by making use of a well thought out concept. Let’s start with a simple example:
data Status = Green  Yellow  Red
This declares a data type Status
which has exactly three different instances. For each instance a data constructor is defined that allows to create a new instance of the data type.
Each of those data constructors is a function (in this simple case a constant) that returns a Status
instance.
The type Status
is a so called sum type as it is represents the set defined by the sum of all three instances Green
, Yellow
, Red
. In Java this corresponds to Enumerations.
Let’s assume we have to create a converter that maps our Status
values to Severity
values representing severity levels in some other system. This converter can be written using the pattern matching syntax that we already have seen above:
 another sum type representing severity:
data Severity = Low  Middle  High deriving (Eq, Show)
severity :: Status > Severity
Green = Low
severity Yellow = Middle
severity Red = High severity
The compiler will tell us when we did not cover all instances of the Status
type (by making use of the fwarnincompletepatterns
pragma).
Now we look at data types that combine multiple different elements, like pairs ntuples, etc. Let’s start with a PairStatusSeverity
type that combines two different elements:
data PairStatusSeverity = P Status Severity
This can be understood as: data type PairStatusSeverity
can be constructed from a data constructor P
that takes a value of type Status
and a value of type Severity
and returns a Pair
instance.
So for example P Green High
returns a PairStatusSeverity
instance (the data constructor P
has the signature P :: Status > Severity > PairStatusSeverity
).
The type PairStatusSeverity
can be interpreted as the set of all possible ordered pairs of Status and Severity values, that is the cartesian product of Status
and Severity
.
That’s why such a data type is called product type.
Haskell allows you to create arbitrary data types by combining sum types and product types. The complete range of data types that can be constructed in this way is called algebraic data types or ADT in short.
Using algebraic data types has several advantages:
Status
to Severity
there is no need to use if..then..else..
constructs.We will cover the interesting combination of ADTs and pattern matching in the following sections.
Forming pairs or more generally ntuples is a very common task in programming. Therefore it would be inconvenient and repetitive if we were forced to create new Pair or Tuple types for each concrete usage. consider the following example:
data PairStatusSeverity = P Status Severity
data PairStatusString = P' Status String
data PairSeverityStatus = P'' Severity Status
Luckily data type declarations allow to use type variables to avoid this kind of cluttered code. So we can define a generic data type Pair
that allows us to freely combine different kinds of arguments:
 a simple polymorphic type
data Pair a b = P a b
This can be understood as: data type Pair
uses two elements of (potentially) different types a
and b
; the data constructor P
takes a value of type a
and a value of type b
and returns a Pair a b
instance (the data constructor P
has the signature P :: a > b > Pair a b
). The type Pair
can now be used to create many different concrete data types it is thus called a polymorphic data type. As the Polymorphism is defined by type variables, i.e. parameters to the type declarations, this mechanism is called parametric polymorphism.
As pairs and ntuples are so frequently used, the Haskell language designers have added some syntactic sugar to work effortlessly with them.
So you can simply write tuples like this:
tuple :: (Status, Severity, String)
= (Green, Low, "All green") tuple
Another very useful polymorphic type is the List
.
A list can either be the empty list (denoted by the data constructor []
) or some element of a data type a
followed by a list with elements of type a
, denoted by [a]
.
This intuition is reflected in the following data type definition:
data [a] = []  a : [a]
The cons operator (:)
(which is an infix operator like (.)
from the previous section) is declared as a data constructor to construct a list from a single element of type a
and a list of type [a]
.
So a list containing only a single element 1
is constructed by:
1 : []
A list containing the three numbers 1, 2, 3 is constructed like this:
1 : 2 : 3 : []
Luckily the Haskell language designers have been so kind to offer some syntactic sugar for this. So the first list can simply be written as [1]
and the second as [1,2,3]
.
Polymorphic type expressions describe families of types. For example, (forall a)[a]
is the family of types consisting of, for every type a
, the type of lists of a
. Lists of integers (e.g. [1,2,3]
), lists of characters (['a','b','c']
), even lists of lists of integers, etc., are all members of this family.
Function that work on lists can use pattern matching to select behaviour for the []
and the a:[a]
case.
Take for instance the definition of the function length
that computes the length of a list:
length :: [a] > Integer
length [] = 0
length (x:xs) = 1 + length xs
We can read these equations as: The length of the empty list is 0, and the length of a list whose first element is x and remainder is xs is 1 plus the length of xs.
In our next example we want to work with a of some random integers:
someNumbers :: [Integer]
= [49,64,97,54,19,90,934,22,215,6,68,325,720,8082,1,33,31] someNumbers
Now we want to select all even or all odd numbers from this list. We are looking for a function filter
that takes two arguments: first a predicate function that will be used to check each element and second the actual list of elements. The function will return a list with all matching elements. And of course our solution should work not only for Integers but for any other types as well. Here is the type signature of such a filter function:
filter :: (a > Bool) > [a] > [a]
In the implementation we will use pattern matching to provide different behaviour for the []
and the (x:xs)
case:
filter :: (a > Bool) > [a] > [a]
filter pred [] = []
filter pred (x:xs)
 pred x = x : filter pred xs
 otherwise = filter pred xs
The []
case is obvious. To understand the (x:xs)
case we have to know that in addition to simple matching of the type constructors we can also use pattern guards to perform additional testing on the input data. In this case we compute pred x
if it evaluates to True
, x
is a match and will be cons’ed with the result of filter pred xs
. If it does not evaluate to True
, we will not add x
to the result list and thus simply call filter recursively on the remainder of the list.
Now we can use filter
to select elements from our sample list:
someEvenNumbers :: [Integer]
= filter even someNumbers
someEvenNumbers
 predicates may also be lambdaexpresssions
someOddNumbers :: [Integer]
= filter (\n > n `rem` 2 /= 0) someNumbers someOddNumbers
Of course we don’t have to invent functions like filter
on our own but can rely on the extensive set of predefined functions working on lists in the Haskell base library.
There is a nice feature that often comes in handy when dealing with lists of numbers. It’s called arithmetic sequences and allows you to define lists of numbers with a concise syntax:
upToHundred :: [Integer]
= [1..100] upToHundred
As expected this assigns upToHundred
with a list of integers from 1 to 100.
It’s also possible to define a step width that determines the increment between the subsequent numbers. If we want only the odd numbers we can construct them like this:
oddsUpToHundred :: [Integer]
= [1,3..100] oddsUpToHundred
Arithmetic sequences can also be used in more dynamic cases. For example we can define the factorial
function like this:
n! = 1 * 2 * 3 ... (n2) * (n1) * n, for integers > 0
In Haskell we can use an arithmetic sequence to define this function:
= prod [1..n] fac' n
In objectoriented and functional programming, an immutable object is an object whose state cannot be modified after it is created. This is in contrast to a mutable object (changeable object), which can be modified after it is created.
Quoted from Wikipedia
This is going to be a very short section. In Haskell all data is immutable. Period.
Let’s look at some interactions with the Haskell GHCi REPL (whenever you see the λ>
prompt in this article it is from a GHCi session):
> a = [1,2,3]
λ> a
λ1,2,3]
[> reverse a
λ3,2,1]
[> a
λ1,2,3] [
In Haskell there is no way to change the value of a
after its initial creation. There are no destructive operations available unlike some other functional languages such as Lisp, Scheme or ML.
The huge benefit of this is that refactoring becomes much simpler than in languages where every function or method might mutate data. Thus it will also be easier to reason about a given piece of code.
Of course this also makes programming of concurrent operations much easier. With a shared nothing approach, Haskell programs are automatically threadsafe.
In this section I want to explain how programming with higher order functions can be used to factor out many basic control structures and algorithms from the user code.
This will result in a more declarative programming style where the developer can simply declare what she wants to achieve but is not required to write down how it is to be achieved.
Code that applies this style will be much denser, and it will be more concerned with the actual elements of the problem domain than with the technical implementation details.
We’ll demonstrate this with some examples working on lists. First we get the task to write a function that doubles all elements of a [Integer]
list. We want to reuse the double
function we have already defined above.
With all that we have learnt so far, writing a function doubleAll
isn’t that hard:
 compute the double value for all list elements
doubleAll :: [Integer] > [Integer]
= []
doubleAll [] :rest) = double n : doubleAll rest doubleAll (n
Next we are asked to implement a similar function squareAll
that will use square
to compute the square of all elements in a list. The naive way would be to implement it in the WET (We Enjoy Typing) approach:
 compute squares for all list elements
squareAll :: [Integer] > [Integer]
= []
squareAll [] :rest) = square n : squareAll rest squareAll (n
Of course this is very ugly: both function use the same pattern matching and apply the same recursive iteration strategy. They only differ in the function applied to each element.
As role model developers we don’t want to repeat ourselves. We are thus looking for something that captures the essence of mapping a given function over a list of elements:
map :: (a > b) > [a] > [b]
map f [] = []
map f (x:xs) = f x : map f xs
This function abstracts away the implementation details of iterating over a list and allows to provide a user defined mapping function as well.
Now we can use map
to simply declare our intention (the ‘what’) and don’t have to detail the ‘how’:
doubleAll' :: [Integer] > [Integer]
= map double
doubleAll'
squareAll' :: [Integer] > [Integer]
= map square squareAll'
Now let’s have a look at some related problem. Our first task is to add up all elements of a [Integer]
list. First the naive approach which uses the already familiar mix of pattern matching plus recursion:
sumUp :: [Integer] > Integer
= 0
sumUp [] :rest) = n + sumUp rest sumUp (n
By looking at the code for a function that computes the product of all elements of a [Integer]
list we can again see that we are repeating ourselves:
prod :: [Integer] > Integer
= 1
prod [] :rest) = n * prod rest prod (n
So what is the essence of both algorithms? At the core of both algorithms we have a recursive function which
(+)
or (*)
in our case),[]
is reached, where the neutral element is returned.This essence is contained in the higher order function foldr
which again is part of the Haskell standard library:
foldr :: (a > b > b) > b > [a] > b
foldr f acc [] = acc
foldr f acc (x:xs) = f x (foldr f acc xs)
Now we can use foldr
to simply declare our intention (the ‘what’) and don’t have to detail the ‘how’:
sumUp' :: [Integer] > Integer
= foldr (+) 0
sumUp'
prod' :: [Integer] > Integer
= foldr (*) 1 prod'
With the functions map
and foldr
(or reduce
) we have now two very powerful tools at hand that can be used in many situation where list data has to be processed.
Both functions can even be composed to form yet another very important programming concept: Map/Reduce. In Haskell this operation is provided by the function foldMap
.
I won’t go into details here as it would go beyond the scope of this article, but I’ll invite you to read my introduction to Map/Reduce in Haskell.
Now we come to topic that was one of the main drivers for the Haskell designers: they wanted to get away from the then standard model of strict evaluation.
NonStrict Evaluation (aka. normal order reduction) has one very important property.
If a lambda expression has a normal form, then normal order reduction will terminate and find that normal form.
ChurchRosser Theorem II
This property does not hold true for other reduction strategies (like applicative order or callbyvalue reduction).
This result from mathematical research on the lambda calculus is important as Haskell maintains the semantics of normal order reduction.
The realworld benefits of lazy evaluation include:
So let’s have a closer look at those benefits:
Consider the following example function:
ignoreY :: Integer > Integer > Integer
= x ignoreY x y
It takes two integer arguments and returns the first one unmodified. The second argument is simply ignored.
In most programming languages both arguments will be evaluated before the function body is executed: they use applicative order reduction aka. eager evaluation or callbyvalue semantics.
In Haskell on the other hand it is safe to call the function with a nonterminating expression in the second argument. First we create a nonterminating expression viciousCircle
. Any attempt to evaluate it will result in an endless loop:
 it's possible to define nonterminating expressions like
viciousCircle :: a
= viciousCircle viciousCircle
But if we use viciousCircle
as second argument to the function ignoreY
it will simply be ignored and the first argument is returned:
 trying it in GHCi:
> ignoreY 42 viciousCircle
λ42
In the section on lists we have already met arithmetic sequences like [1..10]
.
Arithmetic sequences can also be used to define infinite lists of numbers. Here are a few examples:
 all natural numbers
= [1..]
naturalNumbers
 all even numbers
= [2,4..]
evens
 all odd numbers
= [1,3..] odds
Defining those infinite lists is rather easy. But what can we do with them? Are they useful for any purpose? In the viciousCircle
example above we have learnt that defining that expression is fine but any attempt to evaluate it will result in an infinite loop.
If we try to print naturalNumbers
we will also end up in an infinite loop of integers printed to the screen.
But if we are bit less greedy than asking for all natural numbers everything will be OK.
> take 10 naturalNumbers
λ1,2,3,4,5,6,7,8,9,10]
[
> take 10 evens
λ2,4,6,8,10,12,14,16,18,20]
[
> take 10 odds
λ1,3,5,7,9,11,13,15,17,19] [
We can also peek at a specific position in such an infinite list, using the (!!)
operator:
> odds !! 5000
λ10001
> evens !! 10000
λ20002
Do you remember set comprehension notation from your math classes?
As simple example would be the definition of the set of even numbers:
Evens = {i  i = 2n ∧ n ∊ ℕ}
Which can be read as: Evens is defined as the set of all i
where i = 2*n
and n
is an element of the set of natural numbers.
The Haskell list comprehension allows us to define  potentially infinite  lists with a similar syntax:
= [2*n  n < [1..]] evens'
Again we can avoid infinite loops by evaluating only a finite subset of evens'
:
> take 10 evens'
λ2,4,6,8,10,12,14,16,18,20] [
List comprehension can be very useful for defining numerical sets and series in a (mostly) declarative way that comes close to the original mathematical definitions.
Take for example the set PT
of all pythagorean triples
PT = { (a,b,c)  a,b,c ∊ ℕ ∧ a² + b² = c² }
The Haskell definition looks like this:
pt :: [(Natural,Natural,Natural)]
= [(a,b,c)  c < [1..],
pt < [1..c],
b < [1..b],
a ^2 + b^2 == c^2] a
In most languages it is not possible to define new conditional operations, e.g. your own myIf
statement. A conditional operation will evaluate some of its arguments only if certain conditions are met. This is very hard  if not impossible  to implement in language with callbyvalue semantics which evaluates all function arguments before actually evaluating the function body.
As Haskell implements callbyneed semantics, it is possible to define new conditional operations. In fact this is quite helpful when writing domain specific languages.
Here comes a very simple version of myIf
:
myIf :: Bool > b > b > b
= if p then x else y
myIf p x y
> myIf (4 > 2) "true" viciousCircle
λ"true"
A somewhat more useful controlstructure is the cond
(for conditional) function that stems from LISP and Scheme languages. It allows you to define a more tablelike decision structure, somewhat resembling a switch
statement from Cstyle languages:
cond :: [(Bool, a)] > a
= error "make sure that at least one condition is true"
cond [] True, v):rest) = v
cond ((False, _):rest) = cond rest cond ((
With this function we can implement a signum function sign
as follows:
sign :: (Ord a, Num a) => a > a
= cond [(x > 0 , 1 )
sign x < 0 , 1)
,(x otherwise , 0 )]
,(
> sign 5
λ1
> sign 0
λ0
> sign (4)
λ1
Now we come to one of the most distinguishing features of Haskell: type classes.
In the section Polymorphic Data Types we have seen that type variables (or parameters) allow type declarations to be polymorphic like in:
data [a] = []  a : [a]
This approach is called parametric polymorphism and is used in several programming languages.
Type classes on the other hand address ad hoc polymorphism of data types. This approach is also known as overloading.
To get a first intuition let’s start with a simple example.
We would like to be able to use characters (represented by the data type Char
) as if they were numbers. E.g. we would like to be able to things like:
> 'A' + 25
λ'Z'
 please note that in Haskell a string is List of characters: type String = [Char]
> map (+ 5) "hello world"
λ"mjqqt%twqi"
> map (\c > c  5) "mjqqt%twqi"
λ"hello world"
To enable this we will have to overload the infix operators (+)
and ()
to work not only on numbers but also on characters. Now, let’s have a look at the type signature of the (+)
operator:
> :type (+)
λ(+) :: Num a => a > a > a
So (+)
is not just declared to be of type (+) :: a > a > a
but it contains a constraint on the type variable a
, namely Num a =>
. The whole type signature of (+)
can be read as: for all types a
that are members of the type class Num
the operator (+)
has the type a > a > a
.
Next we obtain more information on the type class Num
:
> :info Num
λclass Num a where
(+) :: a > a > a
() :: a > a > a
(*) :: a > a > a
negate :: a > a
abs :: a > a
signum :: a > a
fromInteger :: Integer > a
{# MINIMAL (+), (*), abs, signum, fromInteger, (negate  ()) #}
 Defined in `GHC.Num'
instance Num Word  Defined in `GHC.Num'
instance Num Integer  Defined in `GHC.Num'
instance Num Int  Defined in `GHC.Num'
instance Num Float  Defined in `GHC.Float'
instance Num Double  Defined in `GHC.Float'
This information details what functions a type a
has to implement to be used as an instance of the Num
type class. The line {# MINIMAL (+), (*), abs, signum, fromInteger, (negate  ()) #}
tells us what a minimal complete implementation has to provide. It also tells us that the types Word
, Integer
, Int
, Float
and Double
are instances of the Num
type class.
This is all we need to know to make the type Char
an instance of the Num
type class, so without further ado we dive into the implementation (please note that fromEnum
converts a Char
into an Int
and toEnum
converts an Int
into an Char
):
instance Num Char where
+ b = toEnum (fromEnum a + fromEnum b)
a  b = toEnum (fromEnum a  fromEnum b)
a * b = toEnum (fromEnum a * fromEnum b)
a abs c = c
signum = toEnum . signum . fromEnum
fromInteger = toEnum . fromInteger
negate c = c
This piece of code makes the type Char
an instance of the Num
type class. We can then use (+)
and `() as demonstrated above.
Originally the idea for type classes came up to provide overloading of arithmetic operators in order to use the same operators across all numeric types.
But the type classes concept proved to be useful in a variety of other cases as well. This has lead to a rich sets of type classes provided by the Haskell base library and a wealth of programming techniques that make use of this powerful concept.
Here comes a graphic overview of some of the most important type classes in the Haskell base library:
I won’t go over all of these but I’ll cover some of the most important ones.
Let’s start with Eq:
class Eq a where
==), (/=) :: a > a > Bool
(
 Minimal complete definition:
 (==) or (/=)
/= y = not (x == y)
x == y = not (x /= y) x
This definition states two things:
a
is to be made an instance of the class Eq
it must support the functions (==)
and (/=)
both of them having type a > a > Bool
.Eq
provides default definitions for (==)
and (/=)
in terms of each other. As a consequence, there is no need for a type in Eq
to provide both definitions  given one of them, the other will work automatically.Now we can turn some of the data types that we defined in the section on Algebraic Data Types into instances of the Eq
type class.
Here the type declarations as a recap:
data Status = Green  Yellow  Red
data Severity = Low  Middle  High
data PairStatusSeverity = PSS Status Severity
First, we create Eq instances for the simple types Status
and Severity
by defining the (==)
operator for each of them:
instance Eq Status where
Green == Green = True
Yellow == Yellow = True
Red == Red = True
== _ = False
_
instance Eq Severity where
Low == Low = True
Middle == Middle = True
High == High = True
== _ = False _
Next, we create an Eq
instance for PairStatusSeverity
by defining the (==)
operator:
instance Eq PairStatusSeverity where
PSS sta1 sev1) == (PSS sta2 sev2) = (sta1 == sta2) && (sev1 == sev2) (
With these definitions it is now possible to use the (==)
and (/=)
on our three types.
As you will have noticed, the code for implementing Eq
is quite boring. Even a machine could do it!
That’s why the language designers have provided a deriving
mechanism to let the compiler automatically implement type class instances if it’s automatically derivable as in the Eq
case.
With this syntax it much easier to let a type implement the Eq
type class:
data Status = Green  Yellow  Red deriving (Eq)
data Severity = Low  Middle  High deriving (Eq)
data PairStatusSeverity = PSS Status Severity deriving (Eq)
This automatic deriving of type class instances works for many cases and reduces a lof of repetitive code.
For example, its possible to automatically derive instances of the Ord
type class, which provides ordering functionality:
class (Eq a) => Ord a where
compare :: a > a > Ordering
<), (<=), (>), (>=) :: a > a > Bool
(max, min :: a > a > a
...
If you are using deriving
for the Status
and Severity
types, the Compiler will implement the ordering according to the ordering of the constructors in the type declaration. That is Green < Yellow < Red
and Low < Middle < High
:
data Status = Green  Yellow  Red deriving (Eq, Ord)
data Severity = Low  Middle  High deriving (Eq, Ord)
Two other quite useful type classes are Read
and Show
that also support automatic deriving.
Show
provides a function show
with the following type signature:
show :: Show a => a > String
This means that any type implementing Show
can be converted (or marshalled) into a String
representation. Creation of a Show
instance can be achieved by adding a deriving (Show)
clause to the type declaration.
data PairStatusSeverity = PSS Status Severity deriving (Show)
> show (PSS Green Low)
λ"PSS Green Low"
The Read
type class is used to do the opposite: unmarshalling data from a String with the function read
:
read :: Read a => String > a
This signature says that for any type a
implementing the Read
type class the function read
can reconstruct an instance of a
from its String representation:
data PairStatusSeverity = PSS Status Severity deriving (Show, Read)
data Status = Green  Yellow  Red deriving (Show, Read)
data Severity = Low  Middle  High deriving (Show, Read)
> marshalled = show (PSS Green Low)
λ
> read marshalled :: PairStatusSeverity
λPSS Green Low
Please note that it is required to specify the expected target type with the :: PairStatusSeverity
clause. Haskell uses static compile time typing. At compile time there is no way to determine which type an expression read "some string content"
will return. Thus the expected type must be specified at compile time. Either by an implicit declaration given by some function type signature, or as in the example above, by an explicit declaration.
Together show
and read
provide a convenient way to serialize (marshal) and deserialize (unmarshal) Haskell data structures. This mechanism does not provide any optimized binary representation, but it is still good enough for many practical purposes, the format is more compact than JSON, and it does not require a parser library.
The most interesting type classes are those derived from abstract algebra or category theory. Studying them is a very rewarding process that I highly recommend. However, it is definitely beyond the scope of this article. Thus, I’m only pointing to two resources covering this part of the Haskell type class hierarchy. The first one is the legendary Typeclassopedia by Brent Yorgey. The second one is Lambda the ultimate Pattern Factory by myself. This text relates the algebraic type classes to software design patterns, and therefore we will only cover some of these type classes.
In the section on declarative programming we came across two very useful concepts:
map :: (a > b) > [a] > [b]
)foldr :: (a > b > b) > b > [a] > b
)These concepts are not only useful for lists, but also for many other data structures. So it doesn’t come as a surprise that there are type classes that abstract these concepts.
The Functor
type class generalizes the functionality of applying a function to a value in a context without altering the context, (e.g. mapping a function over a list [a]
which returns a new list [b]
of the same length):
class Functor f where
fmap :: (a > b) > f a > f b
Let’s take a closer look at this idea by playing with a simple binary tree:
data Tree a = Leaf a  Node (Tree a) (Tree a) deriving (Show)
 a simple instance binary tree:
statusTree :: Tree Status
= Node (Leaf Green) (Node (Leaf Red) (Leaf Yellow))
statusTree
 a function mapping Status to Severity
toSeverity :: Status > Severity
Green = Low
toSeverity Yellow = Middle
toSeverity Red = High toSeverity
We want to use the function toSeverity :: Status > Severity
to convert all Status
elements of the statusTree
into Severity
instances.
Therefore, we let Tree
instantiate the Functor
class:
instance Functor Tree where
fmap f (Leaf a) = Leaf (f a)
fmap f (Node a b) = Node (fmap f a) (fmap f b)
We can now use fmap
on Tree
data structures:
> fmap toSeverity statusTree
λNode (Leaf Low) (Node (Leaf High) (Leaf Middle))
> :type it
λit :: Tree Severity
As already described above, fmap maintains the tree structure unchanged but converts the type of each Leaf
element, which effectively changes the type of the tree to Tree Severity
.
As derivation of Functor
instances is a boring task, it is again possible to use the deriving
clause to let data types instantiate Functor
:
{# LANGUAGE DeriveFunctor #}  this pragma allows automatic deriving of Functor instances
data Tree a = Leaf a  Node (Tree a) (Tree a) deriving (Show, Functor)
As already mentioned, Foldable
provides the ability to perform folding operations on any data type instantiating the Foldable
type class:
class Foldable t where
fold :: Monoid m => t m > m
foldMap :: Monoid m => (a > m) > t a > m
foldr :: (a > b > b) > b > t a > b
foldr' :: (a > b > b) > b > t a > b
foldl :: (b > a > b) > b > t a > b
foldl' :: (b > a > b) > b > t a > b
foldr1 :: (a > a > a) > t a > a
foldl1 :: (a > a > a) > t a > a
toList :: t a > [a]
null :: t a > Bool
length :: t a > Int
elem :: Eq a => a > t a > Bool
maximum :: Ord a => t a > a
minimum :: Ord a => t a > a
sum :: Num a => t a > a
product :: Num a => t a > a
besides the abstraction of the foldr
function, Foldable
provides several other useful operations when dealing with containerlike structures.
Because of the regular structure algebraic data types it is again possible to automatically derive Foldable
instances by using the deriving
clause:
{# LANGUAGE DeriveFunctor, DeriveFoldable #}  allows automatic deriving of Functor and Foldable
data Tree a = Leaf a  Node (Tree a) (Tree a) deriving (Eq, Show, Read, Functor, Foldable)
Of course, we can also implement the foldr
function on our own:
instance Foldable Tree where
foldr f acc (Leaf a) = f a acc
foldr f acc (Node a b) = foldr f (foldr f acc b) a
We can now use foldr
and other class methods of Foldable
:
statusTree :: Tree Status
= Node (Leaf Green) (Node (Leaf Red) (Leaf Yellow))
statusTree
= foldr max Green statusTree
maxStatus = maximum statusTree
maxStatus'
 using length from Foldable type class
= length statusTree
treeSize
 in GHCi:
> :t max
λmax :: Ord a => a > a > a
> foldr max Green statusTree
λRed
 using maximum from Foldable type class:
> maximum statusTree
λRed
> treeSize
λ3
 using toList from Foldable type class:
> toList statusTree
λGreen,Red,Yellow] [
Now we will take the data type Maybe
as an example to dive deeper into the more complex parts of the Haskell type class system.
The Maybe
type is quite simple, it can be either a null value, called Nothing
or a value of type a
constructed by Just a
:
data Maybe a = Nothing  Just a deriving (Eq, Ord)
The Maybe type is helpful in situations where certain operation may return a valid result. Take for instance the function lookup
from the Haskell base library. It looks up a key in a list of keyvalue pairs. If it finds the key, the associated value val
is returned  but wrapped in a Maybe: Just val
. If it doesn’t find the key, Nothing
is returned:
lookup :: (Eq a) => a > [(a,b)] > Maybe b
lookup _key [] = Nothing
lookup key ((k,val):rest)
 key == k = Just val
 otherwise = lookup key rest
The Maybe
type is a simple way to avoid NullPointer errors or similar issues with undefined results. Thus, many languages have adopted it under different names. In Java for instance, it is called Optional
.
In Haskell, it is considered good practise to use total functions  that is functions that have defined return values for all possible input values  where ever possible to avoid runtime errors.
Typical examples for partial (i.e. nontotal) functions are division and square roots. We can use Maybe
to make them total:
safeDiv :: (Eq a, Fractional a) => a > a > Maybe a
0 = Nothing
safeDiv _ = Just (x / y)
safeDiv x y
safeRoot :: (Ord a, Floating a) => a > Maybe a
safeRoot x x < 0 = Nothing
 otherwise = Just (sqrt x)
In fact, there are alternative base libraries that don’t provide any partial functions.
Now let’s consider a situation where we want to combine several of those functions. Say for example we first want to lookup the divisor from a keyvalue table, then perform a division with it and finally compute the square root of the quotient:
findDivRoot :: Double > String > [(String, Double)] > Maybe Double
map =
findDivRoot x key case lookup key map of
Nothing > Nothing
Just y > case safeDiv x y of
Nothing > Nothing
Just d > case safeRoot d of
Nothing > Nothing
Just r > Just r
 and then in GHCi:
> findDivRoot 27 "val" [("val", 3)]
λJust 3.0
> findDivRoot 27 "val" [("val", 0)]
λNothing
> findDivRoot 27 "val" [("val", 3)]
λNothing
The resulting control flow is depicted in the following diagram, which was inspired by the Railroad Oriented Programming presentation:
In each single step we have to check for Nothing
, in that case we directly short circuit to an overall Nothing
result value. In the Just
case we proceed to the next processing step.
This kind of handling is repetitive and buries the actual intention under a lot of boilerplate. As Haskell uses layout (i.e. indentation) instead of curly brackets to separate blocks the code will end up in what is called the dreaded staircase: it marches to the right of the screen.
So we are looking for a way to improve the code by abstracting away the chaining of functions that return Maybe
values and providing a way to short circuit the Nothing
cases.
We need an operator andThen
that takes the Maybe
result of a first function application as first argument, and a function as second argument that will be used in the Just x
case and again returns a Maybe
result. In case that the input is Nothing
the operator will directly return Nothing
without any further processing. In case that the input is Just x
the operator will apply the argument function fun
to x
and return its result:
andThen :: Maybe a > (a > Maybe b) > Maybe b
Nothing _fun = Nothing
andThen Just x) fun = fun x andThen (
We can then rewrite findDivRoot
as follows:
map =
findDivRoot'''' x key lookup key map `andThen` \y >
`andThen` \d >
safeDiv x y safeRoot d
(Side note: In Java the Optional
type has a corresponding method: Optional.flatmap)
This kind of chaining of functions in the context of a specific data type is quite common. So, it doesn’t surprise us that there exists an even more abstract andThen
operator that works for arbitrary parameterized data types:
(>>=) :: Monad m => m a > (a > m b) > m b
When we compare this bind operator with the type signature of the andThen
operator:
andThen :: Maybe a > (a > Maybe b) > Maybe b
We can see that both operators bear the same structure. The only difference is that instead of the concrete type Maybe
the signature of (>>=)
uses a type variable m
with a Monad
type class constraint. We can read this type signature as:
For any type m
of the type class Monad
the operator (>>=)
is defined as m a > (a > m b) > m b
Based on (>>=)
we can rewrite the findDivRoot
function as follows:
map =
findDivRoot' x key lookup key map >>= \y >
>>= \d >
safeDiv x y safeRoot d
Monads are a central element of the Haskell type class ecosystem. In fact the monadic composition based on (>>=)
is so frequently used that there exists some specific syntactic sugar for it. It’s called the doNotation. Using doNotation findDivRoot
looks like this:
map = do
findDivRoot''' x key < lookup key map
y < safeDiv x y
d safeRoot d
This looks quite like a sequence of statements (including variable assignments) in an imperative language. Due to this similarity Monads have been aptly called programmable semicolons. But as we have seen: below the syntactic sugar it’s a purely functional composition!
A function is called pure if it corresponds to a function in the mathematical sense: it associates each possible input value with an output value, and does nothing else. In particular,
Purity makes it easy to reason about code, as it is so close to mathematical calculus. The properties of a Haskell program can thus often be determined with equational reasoning. (As an example I have provided an example for equational reasoning in Haskell).
Purity also improves testability: It is much easier to set up tests without worrying about mocks or stubs to factor out access to backend layers.
All the functions that we have seen so far are all pure code that is free from side effects.
So how can we achieve side effects like writing to a database or serving HTTP requests in Haskell?
The Haskell language designers came up with a solution that distinguishes Haskell from most other languages: Side effects are always explicitly declared in the function type signature. In the next section we will learn how exactly this works.
Monadic I/O is a clever trick for encapsulating sequential, imperative computation, so that it can “do no evil” to the part that really does have precise semantics and good compositional properties.
The most prominent Haskell Monad is the IO
monad. It is used to compose operations that perform I/O. We’ll study this with a simple example.
In an imperative language, reading a String from the console simply returns a String value (e.g. BufferedReader.readline()
in Java: public String readLine() throws IOException
).
In Haskell the function getLine
does not return a String
value but an IO String
:
getLine :: IO String
This could be interpreted as: getLine
returns a String in an IO context. In Haskell, it is not possible to extract the String value from its IO context (In Java on the other hand you could always catch away the IOException
).
So how can we use the result of getLine
in a function that takes a String
value as input parameter?
We need the monadic bind operation (>>=)
to do this in the same as we already saw in the Maybe
monad:
 convert a string to upper case
strToUpper :: String > String
= map toUpper
strToUpper
up :: IO ()
=
up getLine >>= \str >
print (strToUpper str)
 and then in GHCi:
> :t print
λprint :: Show a => a > IO ()
> up
λ
hello world"HELLO WORLD"
or with doNotation:
up' :: IO ()
= do
up' < getLine
str print (strToUpper str)
Making side effects explicit in function type signatures is one of the most outstanding achievements of Haskell. This feature will lead to a very rigid distinction between code that is free of side effects (aka pure code) and code that has side effects (aka impure code).
Keeping domain logic pure  particularly when working only with total functions  will dramatically improve reliability and testability as tests can be run without setting up mocks or stubbed backends.
It’s not possible to introduce side effects without making them explicit in type signatures. There is nothing like the invisible Java RuntimeExceptions
. So you can rely on the compiler to detect any violations of a rule like “No impure code in domain logic”.
I’ve written a simple Restaurant Booking REST Service API that explains how Haskell helps you to keep domain logic pure by organizing your code according to the ports and adapters pattern.
The section on type classes (and on Monads in particular) have been quite lengthy. Yet, they have hardly shown more than the tip of the iceberg. If you want to dive deeper into type classes, I recommend The Typeclassopedia.
We have covered quite a bit of terrain in the course of this article.
It may seem that Haskell has invented an intimidating mass of programming concepts. But in fact, Haskell inherits much from earlier functional programming languages.
Features like first class functions, comprehensive list APIs or declarative programming had already been introduced with Lisp and Scheme.
Several others, like pattern matching, nonstrict evaluation, immutability, purity, static and strong typing, type inference, algebraic data types and polymorphic data types have been invented in languages like Hope, Miranda and ML.
Only a few features like type classes and explicit side effects / monadic I/O were first introduced in Haskell.
So if you already know some functional language concepts, Haskell shouldn’t seem too alien to you. For developers with a background in OO languages, the conceptual gap will be much larger.
I hope that this article helped to bridge that gap a bit and to better explain why functional programming  and Haskell in particular  matters.
Using functional programming languages  or applying some of their techniques  will help to create designs that are closer to the problem domain (as intented by domain driven design), more readable (due to their declarative character), allow equational reasoning, will provide more rigid separation of business logic and side effects, are more flexible for future changes or extensions, provide better testability (supporting BDD, TDD and property based testing), will need much less debugging, are better to maintain and, last but not least, will be more fun to write.
]]>