Getting a working build

With minimal hair on fire

So you've got a Fulcro site and you want to show it to the world! So you go to www.heroku.com, and why wouldn't you? They really help to make things nice, straightforward and easy.

You might want to have a read of some pitfalls.

Step 1 - deps deps deps

So you push your code to heroku's friendly git repo and watch the build log, 💥.

What happened?

deps.edn did!

Heroku's really focused on leiningen being the way you build stuff, I mean really focused! If you don't have a file called project.clj its clojure build process just falls over. (This may be fixed by the time you read this, or not...) 🤷

So let's create one:

touch project.clj

That's right, it doesn't even need to have anything in it, it just has to exist...

Now obviously that's not going to be enough, so let's take your deps.edn file and adjust it:

{:paths   ["src/main" "resources"]

 :deps    {...}

 :aliases {:clj-tests {:extra-paths ["src/test"]
           ...
            ;; v-- Add this!
           :depstar {:extra-deps {seancorfield/depstar {:mvn/version "0.5.2"}}}}}

and your shadow-cljs.edn file:

{:deps   {:aliases [:dev]}
 :nrepl  {:port 9000}
 :builds {:main       {:target  :browser
                       ...
                       ;; v-- and this!
                       :release {:compiler-options {:optimizations :advanced}}}

          :test       {...}

          :ci-tests   {...}

          :workspaces {...}

Next, we need to add a bin/build file, we're doing this because the presence of such a file will make heroku use that for our build.

#!/usr/bin/env bash
npm install
npx shadow-cljs release main
clojure -A:depstar -m hf.depstar.uberjar fulcro.jar

Don't forget to chmod u+x bin/build!

Finally, we need to add some buildpacks!

$ heroku buildpacks:clear
$ heroku buildpacks:add heroku/nodejs
$ heroku buildpacks:add heroku/clojure

When you're done they should look like this:

$ heroku buildpacks
=== <your machine name> Buildpack URLs
1. heroku/nodejs
2. heroku/clojure

Step 2 - Talking to Postgres

First let's adjust our config/prod.edn:

{...

 :org.httpkit.server/config {:port :env.edn/PORT}  ;; <-- Grab the port from heroku
 ...
 :database-spec {:jdbc-url :env/DATABASE_URL}}  ;; <-- Grab the DATABASE_URL as well

We'll need to make some adjustments to our postgres connection to parse heroku's DATABASE_URL, stick this in app.model.database:

(defn create-uri [url] (URI. url))

(defn parse-username-and-password [db-uri]
  (str/split (.getUserInfo db-uri) #":"))

(defn parse-db-name [db-uri]
  (str/replace (.getPath db-uri) "/" ""))

(defn parse-host [db-uri]
  (.getHost db-uri))

(defn parse-port [db-uri]
  (.getPort db-uri))

(defn hikari-connection-map
  "Converts Heroku's DATABASE_URL to a map that you can pass to hikari"
  [heroku-database-url]
  (let [db-uri (create-uri heroku-database-url)
        host    (parse-host db-uri)
        port    (parse-port db-uri)
        db-name (parse-db-name db-uri)
        [username password] (parse-username-and-password db-uri)]
    {:username      username
     :password      password
     :server-name   host
     :port-number   port
     :database-name db-name
     :sslmode       "require"}))

(defn process-database-spec [{:keys [jdbc-url] :as db-spec}]
  (if jdbc-url
    (hikari-connection-map jdbc-url)
    db-spec))

(defn datasource-options []
  (merge {:auto-commit        true
          :read-only          false
          :connection-timeout 30000
          :validation-timeout 5000
          :idle-timeout       600000
          :max-lifetime       1800000
          :minimum-idle       10
          :maximum-pool-size  10
          :pool-name          "db-pool"
          :adapter            "postgresql"
          :register-mbeans    false}
    (process-database-spec (:database-spec config))))

(defstate pool
  :start (pool/make-datasource (datasource-options))
  :stop (pool/close-datasource pool))

Step 3 - Something to Run

Now we just need a Procfile:

web: java -cp fulcro.jar clojure.main -m app.server-main

And we're good to go! 👍

PS: If you're getting:

Problem 1 - Connection Refused during deploy

[web.1]: Execution error (ConnectException) at java.net.PlainSocketImpl/socketConnect (PlainSocketImpl.java:-2).
[web.1]: Connection refused (Connection refused)

Then you many not be passing in src/main/config/prod.edn:

:sslmode "require"

Basically, your postgres database is refusing the connection, so you need to pass in that attribute (or something else if you aren't using hikari) =)...

Problem 2 - Just 404 everywhere!

You can't see anything and when you try and access your javascript files, they're 404ing. You may need to also wrap-files in src/main/config/default.edn.

:static    {:resources "public"
            :files     "resources/public"} ; <-- Add this!

This is to do with the build order, if you built your clojurescipt code with shadow-cljs first, then when you build your depstar uberjar, the jar file you make will have a copy of your code, so you won't need this.

On the other hand, if you're wanting to update your static files separate to what's in your jar, say because you're updating them at a different rate, you want to make sure your uberjaring doesn't see them.

So do the shadow-cljs after the uberjar. That way they'll get served off the file system. This is probably also what's wrong if you're asking yourself why the file you've just edited/changed isn't showing up on your server.

You probably don't want to do this on Heroku, but it's useful to know about :)...

Problem 3 - Some sort of CORS error?

You may have forgotten to add heroku to your list of legal origins in src/main/config/prod.edn:

:legal-origins #{"<some pretty heroku name>.herokuapp.com" "localhost"}

CORS is not a small topic, but well worth understanding.

Cliff notes, browsers don't let pages fetch content from other domain names, unless those other domain names say they already know them and are ok with it.

Depending on what you're doing, say serving content from s3 under a different domain and wanting to access this site you just deployed, you need to say that the domain serving the content is allowed to fetch. Not doing so leads to all kinds of annoying behaviour for you, caused by browsers trying to be secure.

Problem 4 - I just keep getting "Application Error" 🤯

First thing to do, check your logs!

heroku logs --tail

Look for stuff like this:

2020-04-12T12:15:00.000000+00:00 heroku[web.1]: State changed from crashed to starting
2020-04-12T12:15:01.000000+00:00 heroku[router]: at=error code=H10 desc="App crashed"

Your app's crashed, try restarting it!

heroku ps:restart

This can also happen with an R10 or boot timeout, if that's happening a lot, try and work out why your startup time is too slow?

Another one is:

2020-04-12T17:40:10.000000+00:00 app[worker.1]: Working
2020-04-12T17:40:10.000000+00:00 heroku[worker.1]: Process running mem=569MB(111.1%)
2020-04-12T17:40:11.000000+00:00 heroku[worker.1]: Error R14 (Memory quota exceeded)
2020-04-12T17:41:52.000000+00:00 app[worker.1]: Working

That R14 is you're going over the memory by a fair bit!

Maybe scale up your dyno?

heroku ps:scale web=1:standard-2x

Pay attention to that standard-2x bit, we're jumping up to a dyno that has 1GB memory, so your server cost is going to be $50 per month per dyno!

Problem 5 - What about all my out of memory errors?

If you're still getting out of memory and application crashes there are a few simple things you can try do:

1) Are you using the lastest jvm?

If you set your system.properties file to:

java.runtime.version=14

You'll get openjdk version 14, which has a bit of improved performance =)...

2) Try a different GC

You can try to see how your application runs under some of the newer garbage collectors, such as ZGC and Shenandoah.

You need to first pass -XX:+UnlockExperimentalVMOptions, then either -XX:+UseShenandoahGC for Shenandoah or -XX:+UseZGC for ZGC.

You do this by setting JAVA_OPTS in the heroku config:

heroku config:set JAVA_OPTS="-XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC"

Don't forget to edit your Procfile so that your web command takes those args:

web: java $JAVA_OPTS -cp fulcro.jar clojure.main -m app.server-main

I've had some success personally with Shenandoah and you should spend some time reading what they do, and what trade-offs they come with, but the options are there.

Just a note that, if you're already using openjdk do give G1 a good go, the developers have been doing some tweaks to it, and [here's mention that even if you do swap to a different GC, when version 15 comes out it will be even better.

3) Enhanced Metrics

If your dyno is at least on the hobby tier, you should be able to see metrics, under the options gear, there's a setting, Enhanced Language Metrics. Turn that on and take a look, do the numbers differ from your local profiling of the application?

Try running the application with reduced memory and see how it performs, if you're running on anything less than a standard-2x, heroku is going to set -Xmx300m limiting you max heap size to 300mb, try doing the same on your development machine and see how it does!

PPS: An Updated Example!

If you're having issues, I've updated my currently up-to-date version of the code here.

Acknowledgements

Thanks to Gert Goet for his enlightening blog post And the friendly folks in the #fulcro slack!