Core Async Returns
After fiddling about with `core.async`` in Trunk I decided it was time to reach out for some input. It was a good thing I did; I had reached a point where I was mostly poking at the functions I thought I understood in core.async, and wasn't really reading their API.
I posted over on clojureverse and quickly had a response that took me through usage of the promise-chan function that I was missing. It was what I was needed the whole time. I had misinterpreted somewhat the purpose of CSP and core-async; I was not in the need of building a series of processes that communicated through channels; I just wanted a way out of callback hell and to write code that looked a bit more synchronous.
To save you going back to my wondering in the previous post, here's a before and after:
;; definition
(defn <articles-get
[]
(<sql {:op :all :sql "SELECT * FROM articles ORDER BY date_created DESC"}))
;; use
(go (reply! event (s-ev :articles-received) (<! (db/<articles-get))))
(defn articles-get
[callback]
(let [sql "SELECT * FROM articles ORDER BY date_created DESC;"]
(.all db sql (fn [err rows]
;; do stuff here if necessary
(callback rows)))))
;; use
;;...
(s-ev :articles-get)
(fn [event data]
(db/articles-get (fn [data]
(reply! event (s-ev :articles-received) data))))
This is a pretty simple example and I wouldn't say that it really demonstrates a need for core.async. In fact, it's quite a bit more complicated as now, to properly use <articles-get I need the following a) The <sql function for abstracting the use of channels and the use of the go macro.
Ok, fine. But what about this mess of spaghetti? (I don't recommend trying to read this, just look at it and feel sad, perhaps?)
(defn- insert-article
"Creating an article involves taking a string, breaking it into a list and
then inserting each word into the words table, if it doesn't exist.
Once `insert-words` has happened, we return to create the article
linking the ids of every words in the article into the table's `word_ids` column.
Welcome to the callback swamp!
"
[data cb]
(let [{:keys [article title source]} data
words (u/split-article article)
word-ids []]
(letfn [;; 🔎 Insert the article fn. We do this once we have the word ids.
(insert-new-article [word-ids-vec]
(let [delimited-ids (str/join "$" word-ids-vec)
sql-new-article (str "INSERT INTO articles(original, word_ids, name, source, date_created)
VALUES (?, ?, ?, ?, ?)")
vals (apply array [article delimited-ids title source (js/Date.now)])]
;; once we have inserted the article, in our callback, get the article as well.
(.run db sql-new-article vals (fn [err]
(this-as this
(article-get (.-lastID ^js this) cb))))))
;; 🔎 recursively get the ids for all the words in the article.
(get-word-ids-recursive [words word-ids cb]
(if (empty? words)
(cb word-ids)
(let [[frst & rst] words
slug-word (u/slug-word frst)
query "SELECT word_id FROM words WHERE slug = ? AND name = ?"
vals (array slug-word frst)]
(.get db query vals
(fn [err res]
(get-word-ids-recursive rst (conj word-ids (.-word_id ^js res)) cb))))))]
;; Launch it off 🎯!
(get-word-ids-recursive words word-ids insert-new-article))))
Trunk is a bit of a tricky application in that, as far as I've been able to forsee, we have to store singular words in a separate table as articles. My SQL skills aren't quite high enough to make me feel confident regarding how to link an article's text with a series of words; so I went with the hacky way- when you create an article:
- Store all the words independantly
- Get the ids for all those words
- Create a delimited string of those words' ids
- Store that in the database.
Is this a good idea? It doesn't feel like it. But this is the fastest way I knew at the time (and still…don't…know) and I'm still prototyping.
Now then, back to core async. With the help of the <sql function which looks like this:
(defn <sql
"Creates a sql operation that returns a channel, allowsing for async/await like syntax.
Has been abstracted to handle variety of return types depending on sql op(eration)"
(let [out (promise-chan)
err-text (str "Failed to run async query of type " (name op))
params (apply array params) ;; TODO - if this is not a sequence, handle it?
cb (fn [err res]
(this-as this
(if err
(put! out (ex-info err-text {:error :sql-error :res res}))
;; TODO nil - nothing coming back.
(cond
(= :insert op) (put! out (.-lastID ^js this))
res (put! out (js->clj res :keywordize-keys true))
:else (put! out (js->clj this :keywordize-keys true))))))]
(case op
:all (.all db sql params cb)
:get (.get db sql params cb)
:insert (.run db sql params cb)
:run (.run db sql params cb))
out))
I can do steps 1 through 4 to insert an article in my handler like so:
(def ipcHandlers
{(s-ev :article-create)
(fn [event data]
(go (let [_ (<! (db/<insert-words (data :article)))
word-ids-str (<! (db/<get-word-ids (data :article)))
inserted-article (<! (db/<insert-article (merge data {:word_ids word-ids-str})))]
(reply! event (s-ev :article-created) inserted-article))))
Simple! Further, I have broken out th functions to be much simpler. Now, insert-article does just that. I can wrap it in a go block and it will insert an article, assuming you are giving it the right inputs. I could do that with the original callbacks (it hadn't occurred to me to simplify that side of things yet) but nonetheless, I'm happy about that:
(defn <insert-article
"Creates a new article. Requirements:
-> words for article are already in words table.
-> words from article have been re-queries
-> requiriesed words' ids have been made into a delimited string with $."
[{:keys [article title source word_ids]}]
(<sql {:op :insert
:params [article word_ids title source (js/Date.now)]
:sql "INSERT INTO articles(original, word_ids, name, source, date_created) VALUES (?, ?, ?, ?, ?)"}))
That's it! I hope this holds up for a while as I don't want to refactor again. I've heard people talk about ending up in a bit of a mess with core.async code (and I've also heard that the go macro expands in a way that makes stack traces a bit less traceable). With all that said, I feel like I'm more in line with the niceties of JavaScript async/await code (and isn't it all just syntactic sugar anyway?)