There’s a saying in business that no organization operates in a vacuum. The same applies to Clojure. For all the cool tools and techniques Clojure offers, there are still a number of activities and techniques that for whatever reason aren’t always on the direct path to shipping software. Some might call them academic, or incidental complexity, but for the time being, we call them life.
This chapter covers some of the topics about Clojure development that don’t quite fill chapters on their own. Topics like:
Obtain the Clojure Java archive (JAR) file by downloading and unzipping a release from http://clojure.org/downloads. Using a terminal, navigate to where you extracted the JAR, and start a Clojure REPL:
$ java -cp "clojure-1.5.1.jar" clojure.main
You are now running an interactive Clojure REPL (read-eval-print loop). Type an expression and hit Enter to evaluate it. Press Ctrl-D to exit.
The fact that Clojure on the JVM is encapsulated in a simple JAR file has some great benefits. For one, it means that Clojure is never really installed. It’s just a dependency, like any other Java library. You can easily swap out one version of Clojure for another by replacing a single file.
Let’s dissect the java
invocation here a bit. First, we set the Java classpath to include Clojure (and only Clojure, in this example):
-cp "clojure-1.5.1.jar"
A full explanation of the classpath is beyond the scope
of this recipe, but suffice it to say thatit is a list of places where Java
should look to load classes. A full discussion of classpaths on the
JVM can be found at http://bit.ly/docs-classpaths. In the final part of the invocation, we specify the class that Java should load and execute the main
method:
clojure.main
Yes, clojure.main
is really a Java class. The
reason this doesn’t look like a typical Java invocation is because
Clojure namespaces, which are compiled to classes, do not
conventionally use capitalized names like Java classes do.
This is the absolute bare-minimum Clojure environment and is all you need to run Clojure code on any system with Java installed. Of course, for regular use and development, you will most certainly want a more feature-rich solution like Leiningen.
In some cases, however, hand-tuning a Java invocation may be the best way to integrate Clojure into your environment. This is particularly useful on servers where deploying a simple JAR file is trivial compared to installing more complex packages.
Print the documentation for a function at the REPL with the doc
macro:
user=> (doc conj)
-------------------------
clojure.core/conj
([coll x] [coll x & xs])
conj[oin]. Returns a new collection with the xs
'added'. (conj nil item) returns (item). The 'addition' may
happen at different 'places' depending on the concrete type.
Print the source code for a function at the REPL with the source
macro:
user=> (source reverse)
(defn reverse
"Returns a seq of the items in coll in reverse order. Not lazy."
{:added "1.0"
:static true}
[coll]
(reduce1 conj () coll))
Find functions with documentation matching a given regular expression using find-doc
:
user=> (find-doc #"defmacro")
-------------------------
clojure.core/definline
([name & decl])
Macro
Experimental - like defmacro, except defines a named function whose
body is the expansion, calls to which may be expanded inline as if
it were a macro. Cannot be used with variadic (&) args.
-------------------------
clojure.core/defmacro
([name doc-string? attr-map? [params*] body]
[name doc-string? attr-map? ([params*] body) + attr-map?])
Macro
Like defn, but the resulting function name is declared as a
macro and will be used as a macro by the compiler when it is
called.
Clojure supports inline documentation of functions (more about that
later), along with other metadata, which allows you to introspect
things like documentation any time you want. The doc
and source
macros are just convenience functions for the REPL.
You can peek under the hood at almost everything in Clojure at any time. The next example may be a bit mind-expanding if you’re not used to this level of introspection at runtime:
user=> (source source)
(defmacro source
"Prints the source code for the given symbol, if it can find it.
This requires that the symbol resolve to a Var defined in a
namespace for which the .clj is in the classpath.
Example: (source filter)"
[n]
`(println (or (source-fn '~n) (str "Source not found"))))
Keeping in mind that source
was defined in the clojure.repl
namespace, we can peek at how exactly it retrieves the source by
evaluating (source clojure.repl/source-fn)
.
In most REPL implementations, clojure.repl
macros like source
and doc
are
only referred into the user
namespace. This means as soon as you switch into
another namespace, the unqualified clojure.repl
macros will no longer be
available. You can get around this by namespacing the macros
(clojure.repl/doc
instead of doc
,) or, for extended use, by use
-ing the
namespace:
user=> (ns foo)
foo=> (doc +)
CompilerException java.lang.RuntimeException: Unable to resolve symbol: doc
in this context, compiling:(NO_SOURCE_PATH:1:1)
foo=> (use 'clojure.repl)
nil
foo=> (doc +)
-------------------------
clojure.core/+
([] [x] [x y] [x y & more])
Returns the sum of nums. (+) returns 0. Does not auto-promote
longs, will throw on overflow. See also: +'
Exploring Clojure in this way is a great way to learn about core
functions and advanced Clojure programming techniques. The
clojure.core
namespace is chock-full of high-quality and
high-performance code at your fingertips.
clojure.repl
API documentation
Use loaded-libs
to obtain the set of currently loaded namespaces. For example, from a REPL:
user=> (pprint (loaded-libs))
#{clojure.core.protocols clojure.instant clojure.java.browse
clojure.java.io clojure.java.javadoc clojure.java.shell clojure.main
clojure.pprint clojure.repl clojure.string clojure.uuid clojure.walk}
Use dir
from a REPL to print the public vars in a namespace:
user=> (dir clojure.instant)
parse-timestamp
read-instant-calendar
read-instant-date
read-instant-timestamp
validated
Use ns-publics
to obtain a mapping of symbols to public vars in a namespace:
(
ns-publics
'clojure.instant
)
;; -> {read-instant-calendar #'clojure.instant/read-instant-calendar,
;; read-instant-timestamp #'clojure.instant/read-instant-timestamp,
;; validated #'clojure.instant/validated,
;; read-instant-date #'clojure.instant/read-instant-date,
;; parse-timestamp #'clojure.instant/parse-timestamp}
Namespaces in Clojure are dynamic mappings of symbols to vars. A
namespace is not available until it is required by something else;
for example, when starting a REPL or as a dependency in an ns
declaration. Nothing is known about available Clojure libraries and
namespaces until runtime, which is in contrast to typical Java
development (where most everything about a package is known at compile
time).
The downside of this dynamic nature is that you need to at least know which namespaces to load in order to explore them.
clojure.repl
API documentation
You want to try a library in the REPL without having to modify your project’s dependencies or create a new project.
Use Ryan Neufeld’s lein-try
to launch the REPL. Library dependencies
will be met automatically.
To gain this capability, first make sure you are using Leiningen 2.1.3 or later. Then edit your ~/.lein/profiles.clj file, adding [lein-try "0.4.1"]
to the :plugins
vector of the :user
profile:
{
:user
{
:plugins
[[
lein-try
"0.4.1"
]]}}
Now you can experience nearly instant gratification with the library of your choice:
$ lein try clj-time
Retrieving clj-time/clj-time/0.6.0/clj-time-0.6.0.pom from clojars
Retrieving clj-time/clj-time/0.6.0/clj-time-0.6.0.jar from clojars
nREPL server started on port 58981 on host 127.0.0.1
REPL-y 0.2.1
Clojure 1.5.1
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Source: (source function-name-here)
Javadoc: (javadoc java-object-or-class-here)
Exit: Control+D or (exit) or (quit)
user=>
Notice that we did not have to give a version number for the library
in the example. lein-try
will automatically grab the most recent
released version.
Of course, you can specify a library version if you like. Just add the version number after the library name:
$ lein try clj-time 0.5.1
#...
user=>
For a quick view of usage options, invoke lein help try
:
$ lein help try
Launch REPL with specified dependencies available.
Usage:
lein try [io.rkn/conformity "0.2.1"] [com.datomic/datomic-free "0.8.4020.26"]
lein try io.rkn/conformity 0.2.1
lein try io.rkn/conformity # This uses the most recent version
NOTE: lein-try does not require []
Arguments: ([& args])
As befits a Clojure tool, lein-try
is an elegant way to make a task
less laborious. Use it to summon powerful libraries from the Net at
your whim, without having to set them up, and enjoy a wizardly
satisfaction from the sudden confluence of new abilities.
Run a file full of Clojure expressions by passing the filename as an
argument to clojure.main
.
To follow along with this recipe, you can download a version of clojure.jar at http://clojure.org/downloads.
For example, given a file my_clojure_program.clj with the contents:
(
println
"Hi."
)
invoke the java
command with my_clojure_program.clj as the final argument:
$ java -cp clojure.jar clojure.main my_clojure_program.clj
Hi.
In a more structured project, you’ll probably have files organized in a src/ folder. For example, given a file src/com/example/my_program.clj:
(
ns
com.example.my-program
)
(
defn
-main
[
&
args
]
(
println
"Hey!"
))
to load and run the -main
function, specify the desired namespace
with the -m
/--main
option and add src to the classpath list (via
-cp
):
$ java -cp clojure.jar:src clojure.main --main com.example.my-program
Hey!
Although you will spend most of your time evaluating Clojure code in a
REPL, it is sometimes useful to be able to either run a simple “script” full of
Clojure expressions or run a more structured Clojure application
with a -main
entry point.
In either case, you have access to any extra command-line arguments passed after the script name or the main namespace name.
For example, let’s say you have written the following program, in a file called hello.clj:
(
defn
greet
[
name
]
(
str
"Hello, "
name
"!"
))
(
doseq
[
name
*command-line-args*
]
(
println
(
greet
name
)))
Invoking this Clojure program directly will yield predictable output:
$ java -cp clojure.jar clojure.main hello.clj Alice Bob
Hello, Alice!
Hello, Bob!
This simple script has the side effect of printing output when it is loaded. Most Clojure code is not organized this way.
As you will typically want to keep your code in well-organized
namespaces, you can provide an entry point through a namespace with a
-main
function. This allows you to avoid side effects while loading,
and you can even tweak and invoke your -main
function from the REPL
just like any other function during interactive development.
Let’s say you’ve moved your greet
function into a foo.util
namespace, and your project is structured something like this:
./src/foo/util.clj ./src/foo.clj
Your foo
namespace requires the foo.util
namespace and
provides a -main
function, like so:
(
ns
foo
(
:require
foo.util
))
(
defn
-main
[
&
args
]
(
doseq
[
name
args
]
(
println
(
foo.util/greet
name
))))
When you invoke Clojure with foo.core
as the “main” namespace, it
calls the -main
function with the provided command-line arguments:
$ java -cp clojure.jar:src clojure.main --main foo Alice Bob
Hello, Alice!
Hello, Bob!
You’ll also note the addition of :src
to the -cp
option. This
indicates to Java that the classpath for execution not only includes
clojure.jar, but also the contents of the src/ directory on disk.
In any Leiningen project, use the lein run
command to invoke your
application from the command line. To follow along with this recipe, create a new Leiningen project:
$ lein new my-cli
Configure which namespace will be the entry point to your application
by adding a :main
key to the project’s project.clj file:
(
defproject
my-cli
"0.1.0-SNAPSHOT"
:description
"FIXME: write description"
:url
"http://example.com/FIXME"
:license
{
:name
"Eclipse Public License"
:url
"http://www.eclipse.org/legal/epl-v10.html"
}
:dependencies
[[
org.clojure/clojure
"1.5.1"
]]
:main
my-cli.core
)
Finally, add a -main
function to the namespace configured in
project.clj:
(
ns
my-cli.core
)
(
defn
-main
[
&
args
]
(
println
"My CLI received arguments:"
args
))
Now, invoke lein run
to run your application:
$ lein run
My CLI received arguments: nil
$ lein run 1 :foo "bar"
My CLI received arguments: (1 :foo bar)
As it turns out, invoking your application from the command line
couldn’t be easier. Leiningen’s run
command quickly and easily
connects your application to the command line with little fuss. In its base form, lein run
will invoke the -main
function of
whatever namespace you have specified as :main
in your project’s
project.clj file. For example, setting :main my-cli.core
will
invoke my-cli.core/-main
. Alternatively, you may omit implementing
-main
and provide :main
with the fully qualified name of a
function (e.g., my.cli.core/alt-main
); this function will be invoked
instead of -main
.
While the printed arguments in the preceding solution look like Clojure
data, they are in fact regular strings. For simple arguments, you may
choose to parse these strings yourself; otherwise, we suggest using the
tools.cli
library. See
Recipe 3.7, “Parsing Command-Line Arguments”, for more information on
tools.cli
.
Although a project can only have one default :main
entry point, you
can invoke other functions from the command line by setting the -m
option to a namespace or function. If you set -m
to a namespace
(e.g., my-cli.core
), the -main
function of that namespace will be
invoked. If you provide -m
with the fully qualified name of a function
(e.g., my-cli.core/alt-main
), that function will be invoked. There’s
no requirement that this function be prefixed with a -
(indicating it is
a Java method); it simply must accept a variable number of arguments
(as -main
normally does).
For example, you can add a function add-main
to my.cli/core
:
(
ns
my-cli.core
)
(
defn
-main
[
&
args
]
(
println
"My CLI received arguments:"
args
))
(
defn
add-main
[
&
args
]
(
->>
(
map
#
(
Integer/parseInt
%
)
args
)
(
reduce +
0
)
(
println
"The sum is:"
)))
then invoke it from the command line with the command lein run -m
my-cli.core/add-main
:
$ lein run -m my-cli.core/add-main 1 2 3
The sum is: 6
java
tools.cli
Use the tools.cli
library.
Before starting, add [org.clojure/tools.cli "0.2.4"]
to your project’s
dependencies, or start a REPL using lein-try
:
$ lein try org.clojure/tools.cli
Use the clojure.tools.cli/cli
function in your project’s -main
function entry point to parse command-line arguments:[6]
(
require
'
[
clojure.tools.cli
:refer
[
cli
]])
(
defn
-main
[
&
args
]
(
let
[[
opts
args
banner
]
(
cli
args
[
"-h"
"--help"
"Print this help"
:default
false
:flag
true
])]
(
when
(
:help
opts
)
(
println
banner
))))
;; Simulate entry into -main at the command line
(
-main
"-h"
)
;; *out*
;; Usage:
;;
;; Switches Default Desc
;; -------- ------- ----
;; -h, --no-help, --help false Print this help
Clojure’s tools.cli
is a simple library, with only one function,
cli
, and a slim data-oriented API for specifying how arguments
should be parsed. Handily enough, there isn’t much special about this
function: an arguments vector and specifications go in, and a map of parsed
options, variadic arguments, and a help banner come out. It’s really the
epitome of good, composable functional programming.
To configure how options are parsed, pass any number of spec vectors
after the args
list. To specify a :port
parameter, for example,
you would provide the spec ["-p" "--port"]
. The "-p"
isn’t
strictly necessary, but it is customary to provide a single-letter
shortcut for command-line options (especially long ones). In the
returned opts
map, the text of the last option name will be interned
to a keyword (less the --
). For example, "--port"
would become
:port
, and "--super-long-option"
would become :super-long-option
.
If you’re a polite command-line application developer, you’ll also include a description for each of your options. Specify this as an optional string following the final argument name:
[
"-p"
"--port"
"The incoming port the application will listen on."
]
Everything after the argument name and description will be interpreted
as options in key/value pairs. tools.cli
provides the following
options:
:default
:default
is nil
.
:flag
false
or nil
), indicates an argument
behaves like a flag or switch. This argument will not take any
value as its input.
:parse-fn
:assoc-fn
Here’s a complete example:
(
def
app-specs
[[
"-n"
"--count"
:default
5
:parse-fn
#
(
Integer.
%
)
:assoc-fn
max
]
[
"-v"
"--verbose"
:flag
true
:default
true
]])
(
first
(
apply
cli
[
"-n"
"2"
"-n"
"50"
]
app-specs
))
;; -> {:count 50, :verbose true}
(
first
(
apply
cli
[
"--no-verbose"
]
app-specs
))
;; -> {:count 5, :verbose false}
When writing flag options, a useful shortcut is to omit the :flag
option and add a "[no-]
" prefix to the argument’s name. cli
will
interpret this argument spec as including :flag true
without you having
to specify it as such:
[
"-v"
"--[no-]verbose"
:default
true
]
One thing the tools.cli
library doesn’t provide is a hook into the
application container’s launch life cycle. It is your responsibility to
add a cli
call to your -main
function and know when to print the
help banner. A general pattern for use is to capture the results of
cli
in a let
block and determine if help needs to be printed. This
is also useful for ensuring the validity of arguments (especially since
there is no :required
option):
(
def
required-opts
#
{
:port
})
(
defn
missing-required?
"Returns true if opts is missing any of the required-opts"
[
opts
]
(
not-every?
opts
required-opts
))
(
defn
-main
[
&
args
]
(
let
[[
opts
args
banner
]
(
cli
args
[
"-h"
"--help"
"Print this help"
:default
false
:flag
true
]
[
"-p"
"--port"
:parse-fn
#
(
Integer.
%
)])]
(
when
(
or
(
:help
opts
)
(
missing-required?
opts
))
(
println
banner
))))
As with many applications, you may want to accept a variable number of
arguments; for example, a list of filenames.
In most cases, you don’t need to do anything special to capture these
arguments—just supply them after any other options. These variadic
arguments will be returned as the second item in cli
’s returned vector:
(
second
(
apply
cli
[
"-n"
"5"
"foo.txt"
"bar.txt"
]
app-specs
))
;; -> ["foo.txt" "bar.txt"]
If your variadic arguments look like flags, however, you’ll need
another trick. Use --
as an argument to indicate to cli
that
everything that follows is a variadic argument. This is useful if
you’re invoking another program with the options originally passed to
your program:
(
second
(
apply
cli
[
"-n"
"5"
"--port"
"80"
]
app-specs
))
;; -> Exception '--port' is not a valid argument ...
(
second
(
apply
cli
[
"-n"
"5"
"--"
"--port"
"80"
]
app-specs
))
;; -> ["--port" "80"]
Once you’ve finished toying with your application’s option parsing at
the REPL, you’ll probably want to try invoking options via lein run
.
Just like your application needs to use --
to indicate arguments to
pass on to subsequent programs, so too must you use --
to indicate to
lein run
which arguments are for your program and which are for it:
# If app-specs were rigged up to a project...
$ lein run -- -n 5 --no-verbose
clojure-lanterna
, a wrapper
around the Lanterna terminal output library
You regularly create new, similar projects and want an easy way to generate customized boilerplate. Or, you work on an open source project and want to give users an easy way to get started with your software.
Leiningen templates give Clojure programmers an easy way to automatically generate customized project boilerplate with a single shell command. We’ll explore them by creating a template for a simple web service.
First, generate a new template with lein new template
cookbook-sample-template-<github_user>
. Replace <github_user>
with
your own GitHub username—you’ll be publishing this
template to Clojars, and it will need a unique name. In the examples,
we’ll use clojure-cookbook
as our GitHub username:
$ lein new template cookbook-sample-template-clojure-cookbook
Generating fresh 'lein new' template project.
$ cd cookbook-sample-template-clojure-cookbook
Create a new project file template with the following contents in src/leiningen/new/<project-name>/project.clj.
(
defproject
{{
ns-name
}}
"0.1.0"
:description
"FIXME: write description"
:url
"http://example.com/FIXME"
:license
{
:name
"Eclipse Public License"
:url
"http://www.eclipse.org/legal/epl-v10.html"
}
:dependencies
[[
org.clojure/clojure
"1.5.1"
]])
Since you are creating a template for a web service and you’ll want
Clojure’s ring
and ring-jetty-adapter
to be available by default,
add them to the :dependencies
section:
:dependencies
[[
org.clojure/clojure
"1.5.1"
]
[
ring
"1.1.8"
]
[
ring/ring-jetty-adapter
"1.2.0"
]]
Next, open the template definition
(src/leiningen/new/<project-name>.clj) and add project.clj
to the
list of files to be generated. Add sanitize-ns
to the namespace’s
:require
directive to expose a sanitized namespace string:
(
ns
leiningen.new.cookbook-sample-template-clojure-cookbook
(
:require
[
leiningen.new.templates
:refer
[
renderer
name-to-path
->files
sanitize-ns
]]
[
leiningen.core.main
:as
main
]))
;
(
def
render
(
renderer
"cookbook-sample-template-clojure-cookbook"
))
(
defn
cookbook-sample-template-clojure-cookbook
"FIXME: write documentation"
[
name
]
(
let
[
data
{
:name
name
:ns-name
(
sanitize-ns
name
)
;
:sanitized
(
name-to-path
name
)}]
(
->files
data
[
"project.clj"
(
render
"project.clj"
data
)]
;
[
"src/{{sanitized}}/foo.clj"
(
render
"foo.clj"
data
)])))
A good template gives users a basic skeleton on which to build. Create a new file at src/leiningen/new/<project-name>/site.clj with some bare-bones web server logic:
(
ns
{{
ns-name
}}
.site
"My website! It will rock!"
(
:require
[
ring.adapter.jetty
:refer
[
run-jetty
]]))
(
defn
handler
[
request
]
{
:status
200
:headers
{
"Content-Type"
"text/html"
}
:body
"Hello World"
})
(
defn
-main
[]
(
run-jetty
handler
{
:port
3000
}))
Back in the template’s project.clj file, add a key/value for the
:main
option to indicate my-website.site
is the core runnable
namespace for the project:
:main
{{
ns-name
}}
.site
Go back to your template definition (<project-name>.clj) and change
both foo.clj
references to site.clj
. Delete the
src/leiningen/new/<project-name>/foo.clj file as well:
;; ...
[
"src/{{sanitized}}/site.clj"
(
render
"site.clj"
data
)])))
To test the template locally, change directories to the root of your template project and run:
$ lein install
$ lein new cookbook-sample-template-clojure-cookbook my-first-website --snapshot
$ cd my-first-website
$ lein run
# ... Leiningen noisily fetching dependencies ...
2013-08-22 16:41:43.337:INFO:oejs.Server:jetty-7.6.8.v20121106
2013-08-22 16:41:43.379:
INFO:oejs.AbstractConnector:Started [email protected]:3000
If lein
prints an error about not being able to find your template, you should
make sure you’re using the latest version with lein upgrade
.
To make the template available to other users, you’ll need to publish it
to Clojars. First, open up the template project’s project.clj and change
the version to a release version—by default lein
will only use non-SNAPSHOT
templates:
(
defproject
cookbook-sample-template-clojure-cookbook/lein-template
"0.1.0"
;; ...
Next, visit clojars.org to create a Clojars account and then deploy from the template project root:
$ lein deploy clojars
Other users can now create projects using your template name as the
first argument to lein new
. Leiningen will automatically fetch your
project template from Clojars:
$ lein new cookbook-sample-template-clojure-cookbook my-second-website
Leiningen uses Clojars as a well-known source of templates. When you
pass a template name to lein new
, it first looks for that template by
name in the local Maven repository. If it doesn’t find it there, it
will look for an appropriately named template on http://clojars.org. If it
finds one, it will download the template and use it to create the new
project. The result is an almost magic-seeming project creation
interface that lends itself extremely well to getting Clojure
programmers going with new technology very quickly.
Once a project template has been downloaded, Leiningen will use src/leiningen/new/<project-name>.clj to create a new project. This file can be customized extensively to create sophisticated templates that match your needs. We’ll review this file and talk about some of the tools available to the template developer.
We first declare a namespace that matches the template name and
require some useful functions provided by Leiningen for template
development: leiningen.new.templates
contains a variety of other
functions you may find useful and is worth reviewing before you
develop your own templates—problems you encounter during development
may already be solved by the library. In this case, name-to-path
and
sanitize-ns
will help us create strings that we’ll substitute into
file templates in a number of places:
(
ns
leiningen.new.cookbook-sample-template-clojure-cookbook
(
:require
[
leiningen.new.templates
:refer
[
renderer
name-to-path
->files
sanitize-ns
]]))
A new project is generated by loading a set of
mustache template files and rendering them
in the context of a named set of strings. The renderer
function
creates a function that looks for mustache templates in a place
determined by the name of your template. In this case it will look for
templates in
src/leiningen/new/cookbook_sample_template_clojure_cookbook/:
(
def
render
(
renderer
"cookbook-sample-template-clojure-cookbook"
))
Continuing the spirit of convention over configuration, Leiningen will search this namespace for a function with the same name as your template. You may execute arbitrary Clojure code in this function, which means you can make project generation arbitrarily sophisticated:
(
defn
cookbook-sample-template-clojure-cookbook
"FIXME: write documentation"
[
name
]
This is the data our renderer will use to create your new project files from the templates your provide. In this case, we give our templates access to the project name, the namespace that will result from that name, and a sanitized path based on that name:
(
let
[
data
{
:name
name
:ns-name
(
sanitize-ns
name
)
:sanitized
(
name-to-path
name
)}]
Finally, we pass the ->files
(pronounced “to files”) function a list
of filename/content tuples. The filename determines where in the new
project a file will end up. Content is generated using the render
function we defined earlier. render
accepts a relative path to the
template file and the key/value map we created:
(
->files
data
[
"project.clj"
(
render
"project.clj"
data
)]
[
"src/{{sanitized}}/site.clj"
(
render
"site.clj"
data
)])))
Mustache templates are very simple, implementing nothing more than
simple key substitution. For example, the following snippet is used to
generate the ns
statement for our new project’s main file,
site.clj:
(
ns
{{
ns-name
}}
.site
"My website! It will rock!"
(
:require
[
ring.adapter.jetty
:refer
[
run-jetty
]]))
Leiningen templates are a powerful tool for saving Clojure developers from the drudgery of project setup. More importantly, they are an invaluable tool for open source developers to showcase their projects and make it incredibly easy for potential users to get started with an unfamiliar piece of software. If you’ve been developing Clojure for a while, or even if you’ve just started, it’s well worth your time to take templates for a spin today!”
leiningen.new.templates
namespace
mustache
templates
You want to create functions whose behavior varies based upon the arguments passed to them. For example, you want to develop a set of flexible geometry functions.
The easiest way to implement runtime polymorphism is via hand-rolled,
map-based dispatch using functions like cond
or condp
:
(
defn
area
"Calculate the area of a shape"
[
shape
]
(
condp
=
(
:type
shape
)
:triangle
(
*
(
:base
shape
)
(
:height
shape
)
(
/
1
2
))
:rectangle
(
*
(
:length
shape
)
(
:width
shape
))))
(
area
{
:type
:triangle
:base
2
:height
4
})
;; -> 4N
(
area
{
:type
:rectangle
:length
2
:width
4
})
;; -> 8
This approach is a little raw, though: area
ties together dispatch
and multiple shapes’ area implementations, all under one function. Use
the defmulti
and defmethod
macros to define a multimethod, which will separate dispatch from implementation and introduce a measure
of extensibility:
(
defmulti
area
"Calculate the area of a shape"
:type
)
(
defmethod
area
:rectangle
[
shape
]
(
*
(
:length
shape
)
(
:width
shape
)))
(
area
{
:type
:rectangle
:length
2
:width
4
})
;; -> 8
;; Trying to get the area of a new shape...
(
area
{
:type
:circle
:radius
1
})
;; -> IllegalArgumentException No method in multimethod 'area' for
;; dispatch value: :circle ...
(
defmethod
area
:circle
[
shape
]
(
*
(
.
Math
PI
)
(
:radius
shape
)
(
:radius
shape
)))
(
area
{
:type
:circle
:radius
1
})
;; -> 3.141592653589793
Better, but things start to fall apart if you want to add new
geometric functions like perimeter
. With multimethods you’ll need to
repeat dispatch logic for each function and write a combinatorial
explosion of implementations to suit. It would be better if these
functions and their implementations could be grouped and written
together.
Use Clojure’s protocol facilities to define a protocol interface and extend it with concrete implementations:
;; Define the "shape" of a Shape object
(
defprotocol
Shape
(
area
[
s
]
"Calculate the area of a shape"
)
(
perimeter
[
s
]
"Calculate the perimeter of a shape"
))
;; Define a concrete Shape, the Rectangle
(
defrecord
Rectangle
[
length
width
]
Shape
(
area
[
this
]
(
*
length
width
))
(
perimeter
[
this
]
(
+
(
*
2
length
)
(
*
2
width
))))
(
->Rectangle
2
4
)
;; -> #user.Rectangle{:length 2, :width 4}
(
area
(
->Rectangle
2
4
))
;; -> 8
As you’ve seen in this recipe, there are a multitude of different ways to implement polymorphism in Clojure. While the preceding example settled on protocols as a method for implementing polymorphism, there are no hard and fast rules about which technique to use. Each approach has its own unique set of trade-offs that need to be considered when introducing polymorphism.
The first approach considered was simple map-based polymorphism using
condp
. In retrospect, it’s not the right choice for building a
geometry library in Clojure, but that is not to say it is without its
uses. This approach is best used in the small: you could use cond
to
prototype early iterations of a protocol at the REPL, or in places
where you aren’t defining new types.
It’s important to note that there are techniques beyond cond
for
implementing map-based dispatch. One such technique is a dispatch
map, generally implemented as a map of keys to functions.
Next up are multimethods. Unlike cond
-based polymorphism,
multimethods separate dispatch from implementation. On account of
this, they can be extended after their creation. Multimethods are
defined using the defmulti
macro, which behaves similarly to defn
but specifies a dispatch function instead of an implementation.
Let’s break down the defmulti
declaration for a rather simple
multimethod, the area
function:
(
defmulti
area
;
"Calculate the area of a shape"
;
:type
)
;
Using the keyword :type
as a dispatch function doesn’t do justice to
the flexibility of multimethods: they’re capable of much more.
Multimethods allow you to perform arbitrarily complex introspection of
the arguments they are invoked with.
When choosing a map lookup like :type
for a dispatch function, you
also imply the arity of the function (the number of arguments it accepts).
Since keywords act as a function on one argument (a map), area
is a
single-arity function. Other functions will imply different arities. A
common pattern with multimethods is to use an anonymous function to
make the intended arity of a multimethod more explicit:
(
defmulti
ingest-message
"Ingest a message into an application"
(
fn
[
app
message
]
;
(
:priority
message
))
;
:default
:low
)
;
(
defmethod
ingest-message
:low
[
app
message
]
(
println
(
str
"Ingesting message "
message
", eventually..."
)))
(
defmethod
ingest-message
:high
[
app
message
]
(
println
(
str
"Ingesting message "
message
", now."
)))
(
ingest-message
{}
{
:type
:stats
:value
[
1
2
3
]})
;; *out*
;; Ingesting message {:type :stats :value [1 2 3]}, eventually...
(
ingest-message
{}
{
:type
:heartbeat
:priority
:high
})
;; *out*
;; Ingesting message {:type :heartbeat, :priority :high}, now.
In all of the examples so far, we’ve always dispatched on a single value. Multimethods also support something called multiple dispatch, whereby a function can be dispatched upon any number of factors.
By returning a vector rather than a single value in our dispatch, we can make more dynamic decisions:
(
defmulti
convert
"Convert a thing from one type to another"
(
fn
[
request
thing
]
[(
:input-format
request
)
(
:output-format
request
)]))
;
(
require
'clojure.edn
)
(
defmethod
convert
[
:edn-string
:clojure
]
;
[
_
str
]
(
clojure.edn/read-string
str
))
(
require
'clojure.data.json
)
(
defmethod
convert
[
:clojure
:json
]
;
[
_
thing
]
(
clojure.data.json/write-str
thing
))
(
convert
{
:input-format
:edn-string
:output-format
:clojure
}
"{:foo :bar}"
)
;; -> {:foo :bar}
(
convert
{
:input-format
:clojure
:output-format
:json
}
{
:foo
[
:bar
:baz
]})
;; -> "{"foo":["bar","baz"]}"
All this power comes at a cost, however; because multimethods are so dynamic, they can be quite slow. Further, there is no good way to group sets of related multimethods into an all-or-nothing package.[7] If speed or implementing a complete interface is among your chief concerns, then you will likely be better served by protocols.
Clojure’s protocol feature provides extensible polymorphism with fast dispatch akin to Java’s interfaces, with one notable difference from multimethods: protocols can only perform single dispatch (based on type).
Protocols are defined using the defprotocol
macro, which accepts a name,
an optional docstring, and any number of named method signatures. A
method signature is made up of a few parts: the name, at least one
type signature, and an optional docstring. The first argument of any
type signature is always the object itself—Clojure dispatches on the
type of this argument. Perhaps an example would be the easiest way to
dig into defprotocol
’s syntax:
(
defprotocol
Frobnozzle
"Basic methods for any Frobnozzle"
(
blint
[
this
x
]
"Blint the frobnozzle with x"
)
;
(
crand
[
this
f
]
[
this
f
x
]
(
str
"Crand a frobnozzle with another "
;
"optionally incorporating x"
)))
Once a protocol is defined, there are numerous ways to provide an
implementation for it. deftype
, defrecord
, and reify
all define a
protocol implementation while creating an object. The deftype
and
defrecord
forms create new named types, while reify
creates an
anonymous type. Each form is used by indicating the protocol being
extended, followed by concrete implementations of each of that
protocol’s methods:
;; deftype has a similar syntax, but is not really applicable for an
;; immutable shape
(
defrecord
Square
[
length
]
Shape
;
(
area
[
this
]
(
*
length
length
))
;
(
perimeter
[
this
]
(
*
4
length
))
;
)
(
perimeter
(
->Square
1
))
;; -> 4
;; Calculate the area of a parallelogram without defining a record
(
area
(
let
[
b
2
h
3
]
(
reify
Shape
(
area
[
this
]
(
*
b
h
))
(
perimeter
[
this
]
(
*
2
(
+
b
h
))))))
;; -> 6
For implementing protocols on existing types, you will want to use the
extend
family of built-in functions (extend
, extend-type
, and
extend-protocol
). Instead of creating a new type, these functions
define implementations for existing types.
extend
and its
convenience macros extend-type
and extend-protocol
.
Suppose you would like to add domain-specific functions to
the core java.lang.String
type. In this example, you will add a
first-name
and a last-name
function to String
.
Define a protocol with the functions you need. The protocol declares the signature of the functions:
(
defprotocol
Person
"Represents the name of a person."
(
first-name
[
person
])
(
last-name
[
person
]))
Extend the type to the java.lang.String
class:
(
extend-type
String
Person
(
first-name
[
s
]
(
first
(
clojure.string/split
s
#
" "
)))
(
last-name
[
s
]
(
second
(
clojure.string/split
s
#
" "
))))
Now you can invoke your functions on strings:
(
first-name
"john"
)
;; -> "john"
(
last-name
"john smith"
)
;; -> "smith"
Why use protocols when multimethods already exist? For one, speed: protocols dispatch only on the type of their first parameter. Further, protocols allow you to group and name an extension. This makes it much easier to reason about what a group of functions confer about a type and ensures a proper, full implementation.
It is good practice to only extend a protocol to a type if you are the author of either the protocol or the type. This will avoid cases where you violate the assumptions of the original author(s).
If you already had functions to use, then it would make sense to use
extend
instead of the extend-type
form:
(
defn
first-word
[
s
]
(
first
(
clojure.string/split
s
#
" "
)))
(
defn
second-word
[
s
]
(
second
(
clojure.string/split
s
#
" "
)))
(
extend
String
Person
{
:first-name
first-word
:last-name
second-word
})
You want to decouple your program’s consumers and producers by introducing explicit queues between them.
For example, if you are building a web dashboard that fetches Twitter messages, this application must both persist these events to a database and publish them via server-sent events (SSE) to a browser.
Introducing explicit queues between components allows them to communicate asynchronously, making them simpler to manage independently and freeing up computational resources.
Use the core.async
library to introduce and coordinate asynchronous channels.
To follow along with this recipe, start a REPL using lein-try
:
$ lein try org.clojure/core.async
Consider the following passage illustrating a synchronous approach:
(
defn
database-consumer
"Accept messages and persist them to a database."
[
msg
]
(
println
(
format
"database-consumer received message %s"
msg
)))
(
defn
sse-consumer
"Accept messages and pass them to web browsers via SSE."
[
msg
]
(
println
(
format
"sse-consumer received message %s"
msg
)))
(
defn
messages
"Fetch messages from Twitter."
[]
(
range
4
))
(
defn
message-producer
"Produce messages and deliver them to consumers."
[
&
consumers
]
(
doseq
[
msg
(
messages
)
consumer
consumers
]
(
consumer
msg
)))
(
message-producer
database-consumer
sse-consumer
)
;; *out*
;; database-consumer received message 0
;; sse-consumer received message 0
;; database-consumer received message 1
;; sse-consumer received message 1
;; database-consumer received message 2
;; sse-consumer received message 2
;; database-consumer received message 3
;; sse-consumer received message 3
Each message received is passed directly to each consumer of
message-producer
. As implemented, this approach is rather brittle;
any slow consumer could cause the entire pipeline to grind to a halt.
To make processing asynchronous, introduce explicit queues with
clojure.core.async/chan
. Perform work asynchronously by wrapping
it in one of core.async
’s clojure.core.async/go
forms:
(
require
'
[
clojure.core.async
:refer
[
chan
sliding-buffer
go
go-loop
timeout
>!
<!
]])
(
defn
database-consumer
"Accept messages and persist them to a database."
[]
(
let
[
in
(
chan
(
sliding-buffer
64
))]
(
go-loop
[
data
(
<!
in
)]
(
when
data
(
println
(
format
"database-consumer received data %s"
data
))
(
recur
(
<!
in
))))
in
))
(
defn
sse-consumer
"Accept messages and pass them to web browsers via SSE."
[]
(
let
[
in
(
chan
(
sliding-buffer
64
))]
(
go-loop
[
data
(
<!
in
)]
(
when
data
(
println
(
format
"sse-consumer received data %s"
data
))
(
recur
(
<!
in
))))
in
))
(
defn
messages
"Fetch messages from Twitter."
[]
(
range
4
))
(
defn
producer
"Produce messages and deliver them to consumers."
[
&
channels
]
(
go
(
doseq
[
msg
(
messages
)
out
channels
]
(
<!
(
timeout
100
))
(
>!
out
msg
))))
(
producer
(
database-consumer
)
(
sse-consumer
))
;; *out*
;; database-consumer received data 0
;; sse-consumer received data 0
;; database-consumer received data 1
;; sse-consumer received data 1
;; database-consumer received data 2
;; sse-consumer received data 2
;; database-consumer received data 3
;; sse-consumer received data 3
There comes a time in all good programs when components or subsystems must stop communicating directly with one another.
— Rich Hickey Clojure core.async Channels
This code is larger than the original implementation. What has this afforded us?
The original approach was rigid. It offered no control over consumer latency and was therefore extremely vulnerable to lag. By buffering communication over channels and doing work asynchronously, we’ve created service boundaries around producers and consumers, allowing them to operate as independently as possible.
Let’s examine one of the new consumers in depth to understand how it has changed.
Instead of receiving messages via function invocation,
consumers now draw messages from a buffered channel. Where a consumer
(e.g., database-consumer
) used to consume a single message at a time,
it now uses a go-loop
to continuously consume messages from its
producer.
In traditional callback-oriented code, accomplishing something like
this would require splitting logic out across numerous functions,
introducing “callback hell.” One of the benefits of core.async
is
that it lets you write code inline, in a more straightforward style:
(
defn
database-consumer
"Accept messages and persist them to a database."
[]
(
let
[
in
(
chan
(
sliding-buffer
64
))]
;
(
go-loop
[
data
(
<!
in
)]
;
(
when
data
;
(
println
(
format
"database-consumer received data %s"
data
))
(
recur
(
<!
in
))))
;
in
))
Here the channel is given a buffer of size 64. The
sliding-buffer
variant dictates that if this channel
accumulates more than 64 unread values, older values will start
“falling off” the end, trading off historical completeness in
favor of recency. Using dropping-buffer
instead would optimize
in the opposite direction.
go-loop
is the core.async
equivalent to looping via something
like while true
. This go-loop
reads its initial value by
“taking” (<!
) from the input channel (in
).
Because channels return nil
when closed, as long as we can read
data
from them, we know we have work to do.
To recur
the go-loop
to the beginning, take the next value
from the channel and invoke recur
with it.
Because the go-loop
block is asynchronous, the take call (<!
)
parks until a value is placed on the channel. The remainder of the
go-loop
block—here, the println
call—is pending. Since the
channel is returned as the database-consumer
function’s value, other
parts of the system—namely, the producer—are free to write to
the channel while the take waits. The first value written to the
channel will satisfy that read call, allowing the rest of the
go-loop
block to continue.
This consumer is now asynchronous, reading values until the channel closes. Since the channel is buffered, we now have some measure of control over the system’s resiliency. For example, buffers allow a consumer to lag behind a producer by a specified amount.
Fewer changes are required to make producer
asynchronous:
(
defn
producer
[
&
channels
]
(
go
(
doseq
[
msg
(
messages
)
out
channels
]
;
(
<!
(
timeout
100
))
;
(
>!
out
item
))))
;
Although the operations are asynchronous, they still occur serially. Using unbuffered consumer channels would mean that if one of the consumers took from the channel too slowly, the pipeline would stall; the producer would not be able to put further values onto the channels.
core.async
has more advanced facilities for layout and coordination
of channels. For more details, see the
core.async
overview.
core.async
to communicate
over ZeroMQ.
You want to parse Clojure expressions, say, from the input to a macro, into a different representation (like maps).
For this example, consider a heavily simplified version of Clojure that consists of the following expression types:
You can represent this language by the following grammar:
Expr = var | (fn [var] Expr) | (Expr Expr)
Use core.match
to pattern match over the input and return the
expression represented as maps of maps.
Before starting, add [org.clojure/core.match "0.2.0"]
to your
project’s dependencies, or start a REPL using lein-try
:
$ lein try org.clojure/core.match
Now, codify the language’s grammar using clojure.core.match/match
:
(
require
'
[
clojure.core.match
:refer
(
match
)])
(
defn
simple-clojure-parser
[
expr
]
(
match
[
expr
]
[(
var
:guard
symbol?
)]
{
:variable
var
}
[([
'fn
[
arg
]
body
]
:seq
)]
{
:closure
{
:arg
arg
:body
(
simple-clojure-parser
body
)}}
[([
operator
operand
]
:seq
)]
{
:application
{
:operator
(
simple-clojure-parser
operator
)
:operand
(
simple-clojure-parser
operand
)}}
:else
(
throw
(
Exception.
(
str
"invalid expression: "
expr
)))))
(
simple-clojure-parser
'a
)
;; -> {:variable a}
(
simple-clojure-parser
'
(
fn
[
x
]
x
))
;; -> {:closure {:arg x, :body {:variable x}}}
(
simple-clojure-parser
'
((
fn
[
x
]
x
)
a
))
;; -> {:application
;; {:operator {:closure {:arg x, :body {:variable x}}}
;; :operand {:variable a}}}
;; fn expression can only have one argument!
(
simple-clojure-parser
'
(
fn
[
x
y
]
x
))
;; -> Exception invalid expression: (fn [x y] x) ...
A match
statement in core.match
is made up of two basic parts. The
first part is a vector of vars to be matched. In our example, this
is [expr]
. This vector isn’t limited to a single entry—it can
contain as many items to match as you would like. The next part is a
variable list of question/answer pairs. A question is a vector
representing the shape the vars vector must take. As with cond
, an
answer is what will be returned should a var satisfy a question.
Questions take a variety of forms in core.match
. Here are explanations
of the preceding samples:
[(var :guard symbol?)]
, matches the
variable case of our syntax, binding the matched expression to var
.
The special :guard
form applies the predicate symbol?
to var
,
only returning the answer if symbol?
returns true
.
[(['fn [arg] body] :seq)]
, matches the fn
case.[8] Note the special ([...] :seq)
syntax for matching
over lists, used here to represent an fn
expression. Also notice
that to match on the literal fn
, it had to be quoted in the match
pattern. Interestingly, since the body
expression should also be
accepted by this parser, it makes a self-recursive call,
(simple-clojure-parser body)
, in the righthand side of the match
pattern.
For the third :application
pattern, the parser again
matches on a list using the ([...] :seq)
syntax. As in the body of the
fn
expression, both the operator
and operand
expressions should
be accepted by the parser, so it makes a recursive call for each one.
Finally, the parser throws an exception if the given expression doesn’t match any of the three accepted patterns. This gives a somewhat more helpful error message if you accidentally hand the parser a malformed expression.
Writing your parser this way gives you succinct code that closely
resembles the target input. Alternatively, you could write it
using conditional expressions (if
or cond
) and explicitly
destructure the input. To illustrate the difference in length and
clarity of the code, consider this function that only parses the fn
expressions of the Clojure subset:
(
defn
parse-fn
[
expr
]
(
if
(
and
(
list?
expr
)
(
=
(
count
expr
)
3
)
(
=
(
nth
expr
0
)
'fn
)
(
vector?
(
nth
expr
1
))
(
=
(
count
(
nth
expr
1
))
1
))
{
:closure
{
:arg
(
nth
(
nth
expr
1
)
0
)
:body
(
simple-clojure-parser
(
nth
expr
2
))}}
(
throw
(
Exception.
(
str
"unexpected non-fn expression: "
expr
)))))
Notice how much more code this version needed in order to express the
same properties about an fn
expression? Not only did the non-match
version require more code, but the if
test doesn’t resemble the
structure of the expression the way the match
pattern does. Further,
match
binds the matched input to the variable names in the match
pattern automatically, saving you from having to let
-bind them
yourself or repeatedly write the same list access code (as shown with
(nth expr)
in parse-fn
above). Needless to say, the match
is
much easier to read and maintain.
core.match
wiki’s
Overview page for a
broader view over all of the library’s capabilities
You have a graph-like hierarchical data structure, serialized as a
flat list of nodes, that you want to query. For example, you have a
graph of movie metadata represented as entity-attribute-value triples.
Writing this code with the standard seq
functions has proven to be too
tedious and error prone.
The core.logic
library is a Clojure implementation of the
miniKanren domain-specific language (DSL) for logic programming. Its
declarative style is well suited for querying flattened hierarchical
data.
To follow along with this recipe, start a REPL using lein-try
:
$ lein try org.clojure/core.logic
The first thing you need is a dataset to query. Consider, for example, that you have represented a graph of movie metadata as a list of tuples:
(
def
movie-graph
[
;; The "Newmarket Films" studio
[
:a1
:type
:FilmStudio
]
[
:a1
:name
"Newmarket Films"
]
[
:a1
:filmsCollection
:a2
]
;; Collection of films made by Newmarket Films
[
:a2
:type
:FilmCollection
]
[
:a2
:film
:a3
]
[
:a2
:film
:a6
]
;; The movie "Memento"
[
:a3
:type
:Film
]
[
:a3
:name
"Memento"
]
[
:a3
:cast
:a4
]
;; Connects the film to its cast (actors/director/producer etc.)
[
:a4
:type
:FilmCast
]
[
:a4
:director
:a5
]
;; The director of "Memento"
[
:a5
:type
:Person
]
[
:a5
:name
"Christopher Nolan"
]
;; The movie "The Usual Suspects"
[
:a6
:type
:Film
]
[
:a6
:filmName
"The Usual Suspects"
]
[
:a6
:cast
:a7
]
;; Connects the film to its cast (actors/director/producer etc.)
[
:a7
:type
:FilmCast
]
[
:a7
:director
:a8
]
;; The director of "The Usual Suspects"
[
:a8
:type
:Person
]
[
:a8
:name
"Bryan Singer"
]])
With all of this data in hand, how would you go about querying it? In
an imperative model, you would likely arduously “connect the dots”
from node to node using filters, maps, and conditionals.[9] With core.logic
, however, it is possible to connect these dots
using declarative logic statements.
For example, to answer the question, “Which directors have made movies
at a given studio?” create a number of dots (logic variables) using
clojure.core.logic/fresh
and connect (ground) them using
clojure.core.logic/membero
. Finally, invoke
clojure.core.logic/run*
to obtain all of the possible solutions:
(
require
'
[
clojure.core.logic
:as
cl
])
(
defn
directors-at
"Find all of the directors that have directed at a given studio"
[
graph
studio-name
]
(
cl/run*
[
director-name
]
(
cl/fresh
[
studio
film-coll
film
cast
director
]
;; Relate the original studio-name to a film collection
(
cl/membero
[
studio
:name
studio-name
]
graph
)
(
cl/membero
[
studio
:type
:FilmStudio
]
graph
)
(
cl/membero
[
studio
:filmsCollection
film-coll
]
graph
)
;; Relate any film collections to their individual films
(
cl/membero
[
film-coll
:type
:FilmCollection
]
graph
)
(
cl/membero
[
film-coll
:film
film
]
graph
)
;; Then from film to cast members
(
cl/membero
[
film
:type
:Film
]
graph
)
(
cl/membero
[
film
:cast
cast
]
graph
)
;; Grounding to cast members of type :director
(
cl/membero
[
cast
:type
:FilmCast
]
graph
)
(
cl/membero
[
cast
:director
director
]
graph
)
;; Finally, attach to the director-name
(
cl/membero
[
director
:type
:Person
]
graph
)
(
cl/membero
[
director
:name
director-name
]
graph
))))
(
directors-at
movie-graph
"Newmarket Films"
)
;; -> ("Christopher Nolan" "Bryan Singer")
miniKanren is a domain-specific language written in Scheme, intended
to give many of the benefits of a logic programming language (such as
Prolog) from within Scheme. David Nolen created an implementation of miniKanren for Clojure, with a focus on performance. One
of the benefits of logic programming languages is their very
declarative style. By using core.logic
, we are able to say what we
are looking for in the graph without saying how core.logic
should go
about finding it.
In general, all core.logic
queries begin with one of the library’s
run
macros, with clojure.core.logic/run
returning a finite number of
solutions and clojure.core.logic/run*
returning all of the
solutions.
The first argument to the run
macro is the goal, a variable used
to store the result of the query. In the preceding solution, this was the
director-name
variable. The rest is the body of the core.logic
program. A program is made up of logic variables (created using
clojure.core.logic/fresh
) grounded to values or constrained by logic
statements.
run
is a clue that our programming paradigm is changing to logic
programming. In a core.logic
program, unification is used rather
than traditional variable assignment and seqential expression
evaluation. Unification uses substitution of values for variables in
an attempt to make two expressions syntactically identical. Statements
in a core.logic
program can appear in any order. For example, you can use
clojure.core.logic/==
to unify 1
and q
:
(
cl/run
1
[
q
]
(
cl/==
1
q
))
;; -> (1)
(
cl/run
1
[
q
]
(
cl/==
q
1
))
;; -> (1)
core.logic
is also able to unify the contents of lists and vectors,
finding the right substitution to make both expressions the same:
(
cl/run
1
[
q
]
(
cl/==
[
1
2
3
]
[
1
2
q
]))
;; -> (3)
(
cl/run
1
[
q
]
(
cl/==
[
"foo"
"bar"
"baz"
]
[
q
"bar"
"baz"
]))
;; -> ("foo")
Technically speaking, unification is a relation, relating the first
form with the second form. This is a kind of puzzle for core.logic
to
solve. In the previous example, q
is a logic variable, and core.logic
is
charged with binding a value to q
such that the left and the right
sides of the unification (the clojure.core.logic/==
relation) are
syntactically identical. When there is no binding that satisfies the
puzzle, no solution exists:
;; There is no way a single value is both 1 AND 2
(
cl/run
1
[
q
]
(
cl/==
1
q
)
(
cl/==
2
q
))
;; -> ()
fresh
is one way to create more logic variables:
(
cl/run
1
[
q
]
(
cl/fresh
[
x
y
z
]
(
cl/==
x
1
)
(
cl/==
y
2
)
(
cl/==
z
3
)
(
cl/==
q
[
x
y
z
])))
;; -> ([1 2 3])
Just as clojure.core.logic/==
is a relation between two forms,
clojure.core.logic/membero
is a relation between an element in a
list and the list itself:
(
cl/run
1
[
q
]
(
cl/membero
q
[
1
]))
;; -> (1)
(
cl/run
1
[
q
]
(
cl/membero
1
q
))
;; -> ((1 . _0))
The first example is asking for any member of the list [1]
, which
happens to only be 1
. The second example is the opposite, asking for any list
where 1
is a member. The dot notation indicates an improper tail
with _0
in it. This means 1
could be in a list by itself, or it
could be followed by any other sequence of numbers, strings, lists, etc.
_0
is an unbound variable, since there was no further restriction on
the list other than 1
being an element.
clojure.core.logic/run*
is a macro that asks for all possible
solutions. Asking for all of the lists that contain a 1
will not
terminate.
Unification can peek inside structures with
clojure.core.logic/membero
as well:
(
cl/run
1
[
q
]
(
cl/membero
[
1
q
3
]
[[
1
2
3
]
[
4
5
6
]
[
7
8
9
]]))
;; -> (2)
Logic variables live for the duration of the program, making it possible to use the same logic variable in multiple statements:
(
let
[
seq-a
[[
"foo"
1
2
]
[
"bar"
3
4
]
[
"baz"
5
6
]]
seq-b
[[
"foo"
9
8
]
[
"bar"
7
6
]
[
"baz"
5
4
]]]
(
cl/run
1
[
q
]
(
cl/fresh
[
first-item
middle-item
last-a
last-b
]
(
cl/membero
[
first-item
middle-item
last-a
]
seq-a
)
(
cl/membero
[
first-item
middle-item
last-b
]
seq-b
)
(
cl/==
q
[
last-a
last-b
]))))
;; -> ([6 4])
The previous example does not specify first-item
, only that it should
be the same for seq-a
and seq-b
. core.logic
uses the data provided
to bind values to the variable that satisfy the constraints. The same
is true with middle-item
.
Building up from this, we can traverse the graph described in the solution:
(
cl/run
1
[
director-name
]
(
cl/fresh
[
studio
film-coll
film
cast
director
]
(
cl/membero
[
studio
:name
"Newmarket Films"
]
graph
)
(
cl/membero
[
studio
:type
:FilmStudio
]
graph
)
(
cl/membero
[
studio
:filmsCollection
film-coll
]
graph
)
(
cl/membero
[
film-coll
:type
:FilmCollection
]
graph
)
(
cl/membero
[
film-coll
:film
film
]
graph
)
(
cl/membero
[
film
:type
:Film
]
graph
)
(
cl/membero
[
film
:cast
cast
]
graph
)
(
cl/membero
[
cast
:type
:FilmCast
]
graph
)
(
cl/membero
[
cast
:director
director
]
graph
)
(
cl/membero
[
director
:type
:Person
]
graph
)
(
cl/membero
[
director
:name
director-name
]
graph
)))
;; -> ("Christopher Nolan")
There is one minor difference between the preceding code and the original
solution: rather than using clojure.core.logic/run*
, asking for all
solutions, clojure.core.logic/run 1
was used. The program has
multiple answers to the query for a director at Newmarket Films.
Asking for more answers will return more with no other code change.
Slight modifications to the preceding query can significantly change the
results. Swapping "Newmarket Films"
for a new fresh variable will
return all directors, for all studios. A macro could also be created
to reduce some of the code duplication if desired.
One benefit of the relational solution to this problem is being able to generate a graph from the values:
(
first
(
cl/run
1
[
graph
]
(
cl/fresh
[
studio
film-coll
film
cast
director
]
(
cl/membero
[
studio
:name
"Newmarket Films"
]
graph
)
(
cl/membero
[
studio
:type
:FilmStudio
]
graph
)
(
cl/membero
[
studio
:filmsCollection
film-coll
]
graph
)
(
cl/membero
[
film-coll
:type
:FilmCollection
]
graph
)
(
cl/membero
[
film-coll
:film
film
]
graph
)
(
cl/membero
[
film
:type
:Film
]
graph
)
(
cl/membero
[
film
:cast
cast
]
graph
)
(
cl/membero
[
cast
:type
:FilmCast
]
graph
)
(
cl/membero
[
cast
:director
director
]
graph
)
(
cl/membero
[
director
:type
:Person
]
graph
)
(
cl/membero
[
director
:name
"Baz"
]
graph
))))
;; -> ([_0 :name "Newmarket Films"]
;; [_0 :type :FilmStudio]
;; [_0 :filmsCollection _1]
;; ...)
For small graphs, membero
is fast enough. Larger graphs will
experience performance problems as core.logic
will traverse the list
many times to find the elements. Using clojure.core.logic/to-stream
with some basic indexing can greatly improve the query performance.
core.logic
wiki
core.logic
repository for examples of using clojure.core.logic/to-stream
core.match
, a (nonunification) matching library with some similar ideas, described briefly in Recipe 3.12, “Making a Parser for Clojure Expressions Using core.match”
Use Overtone to bring the song to life.
Before starting, add [overtone "0.8.1"]
to your project’s
dependencies or start a REPL using lein-try
:[10]
$ lein try overtone
To start, define the melody for an old children’s song:
(
require
'
[
overtone.live
:as
overtone
])
(
defn
note
[
timing
pitch
]
{
:time
timing
:pitch
pitch
})
(
def
melody
(
let
[
pitches
[
0
0
0
1
2
; Row, row, row your boat,
2
1
2
3
4
; Gently down the stream,
7
7
7
4
4
4
2
2
2
0
0
0
; (take 4 (repeat "merrily"))
4
3
2
1
0
]
; Life is but a dream!
durations
[
1
1
2
/3
1
/3
1
2
/3
1
/3
2
/3
1
/3
2
1
/3
1
/3
1
/3
1
/3
1
/3
1
/3
1
/3
1
/3
1
/3
1
/3
1
/3
1
/3
2
/3
1
/3
2
/3
1
/3
2
]
times
(
reductions
+
0
durations
)]
(
map
note
times
pitches
)))
melody
;; -> ({:time 0, :pitch 0} ; Row,
;; {:time 1, :pitch 0} ; row,
;; {:time 2, :pitch 0} ; row
;; {:time 8/3, :pitch 1} ; your
;; {:time 3N, :pitch 2} ; boat
;; ...)
Convert the piece into a specific key by transforming each note’s pitch using a function that represents the key:
(
defn
where
[
k
f
notes
]
(
map
#
(
update-in
%
[
k
]
f
)
notes
))
(
defn
scale
[
intervals
]
(
fn
[
degree
]
(
apply +
(
take
degree
intervals
))))
(
def
major
(
scale
[
2
2
1
2
2
2
1
]))
(
defn
from
[
n
]
(
partial +
n
))
(
def
A
(
from
69
))
(
->>
melody
(
where
:pitch
(
comp
A
major
)))
;; -> ({:time 0, :pitch 69} ; Row,
;; {:time 1, :pitch 69} ; row,
;; ...)
Convert the piece into a specific tempo by transforming each note’s time using a function that represents the tempo:
(
defn
bpm
[
beats
]
(
fn
[
beat
]
(
/
(
*
beat
60
1000
)
beats
)))
(
->>
melody
(
where
:time
(
comp
(
from
(
overtone/now
))
(
bpm
90
))))
;; -> ({:time 1383316072169, :pitch 0}
;; {:time 4149948218507/3, :pitch 0}
;; ...)
Now, define an instrument and use it to play the melody. The following example synthesized instrument is a simple sine wave, whose amplitude and duration are controlled by an envelope:
(
require
'
[
overtone.live
:refer
[
definst
line
sin-osc
FREE
midi->hz
at
]])
(
definst
beep
[
freq
440
]
(
let
[
envelope
(
line
1
0
0.5
:action
FREE
)]
(
*
envelope
(
sin-osc
freq
))))
(
defn
play
[
notes
]
(
doseq
[{
ms
:time
midi
:pitch
}
notes
]
(
at
ms
(
beep
(
midi->hz
midi
)))))
;; Make sure your speakers are on...
(
->>
melody
(
where
:pitch
(
comp
A
major
))
(
where
:time
(
comp
(
from
(
overtone/now
))
(
bpm
90
)))
play
)
;; -> <music playing on your speakers>
If your nursery rhyme is a round, like “Row, Row, Row Your Boat,” you can use it to accompany itself:
(
defn
round
[
beats
notes
]
(
concat
notes
(
->>
notes
(
where
:time
(
from
beats
)))))
(
->>
melody
(
round
4
)
(
where
:pitch
(
comp
A
major
))
(
where
:time
(
comp
(
from
(
overtone/now
))
(
bpm
90
)))
play
)
A note is a sound of a particular pitch that occurs at a particular time. A song is a series of notes. We can therefore simply represent music in Clojure as a sequence of time/pitch pairs.
This representation is structurally very similar to Western music notation, where each dot on a stave has a time and a pitch determined by its horizontal and vertical position. But unlike traditional music notation, the Clojure representation can be manipulated by functional programming techniques.
Pieces of Western music, like “Row, Row, Row Your Boat,” aren’t composed of arbitrary pitches. Within a given melody, the notes are typically confined to a subset of all possible pitches called a scale.
The approach taken here is to express the pitches by integers
denoting where they appear in the scale, called degrees. So, for
example, degree 0
signifies the first pitch of the scale, and degree
4
signifies the fifth pitch of the scale.
This simplifies the description of the melody, because we don’t have to worry about inadvertently specifying pitches that are outside our chosen scale. It also allows us to vary our chosen scale without having to rewrite the melody.
To work with degrees, we need a function that translates a degree into the actual pitch. Since “Row, Row, Row Your Boat” is in a major scale, we need a function that represents such a scale.
We use the observation that in a major scale, there is a regular
pattern of double and single spaces between adjacent pitches (known to
musicians as tones and semitones). We define a function called
major
that accepts a degree and outputs the number of semitones it
represents.
Our pitches still aren’t quite right, because they’re relative to the lowest note of the piece. We need to establish a musical reference point that we will use to interpret our degrees.
Concert A is conventionally used as a reference point by orchestras,
so we’ll use it as our musical zero. In other words, we will put “Row, Row, Row Your Boat” into A major. Now a degree of 0
means A.
Note that we can simply compose together our functions for major and for A to arrive at a composite A major function.
We need to do a similar transformation for time. Each note’s time is expressed in beats, but we need it to be in milliseconds. We use the current system time as our temporal reference point, meaning that the piece will start from now (and not the start of the Unix epoch).
“Row, Row, Row Your Boat” is a round, meaning it harmonizes if sung as an accompaniment to itself, offset by a particular number of beats. As an extra flourish, we produce a second version of the melody that starts four beats after the first.
We encourage you to experiment with the tune, perhaps by varying the
speed or using a different key (as a hint, a minor key has the following
pattern of tones and semitones: [2 1 2 2 1 2 2]
).
We also encourage you to think about how this approach to modeling a series of events can be applied to other domains. The idea of expressing a time series as a sequence and then applying transformations across that series is a simple, flexible, and composable way of describing a problem.
Music is a wonderful and moving thing. It’s also incredibly well suited to being modeled in a functional programming language. We hope your children agree.
[6] Since
tools.cli
is so cool, this example can run entirely at the REPL.
[7] That is to say, you cannot force a multimethod to implement all of the required methods when extending behavior to its own type.
[8] The match pattern for fn
could (and should) include a guard on the arg
to ensure that it’s a symbol, but that’s elided here for brevity.
[9] Oh my!
[10] There are some additional installation concerns if you are running Overtone on Linux. See the Overtone wiki for more detailed installation instructions.