Refactoring Ecto queries with reduce

Posted on July 08, 2022

Oh, hello there.

Today I want to share a fun refactor that finally "clicked" for me (and it was right there in plain view for quite some time!). Let me set the stage.

Seen above is the search functionality for Galley. With it, you can search recipes by name, tag, and select filters. The current filters are:

All
My Recipes
Under an hour
Under 30 minutes
Recently posted
My favourites

I struggled with the original query because I did not yet grasp an elixir/ecto way of querying the data. I had yet to grasp how "compositional" an ecto query is (for a brief overview of Ecto queries specifically, visit these docs). In my previous post on this subject matter, I was beginning to grasp the idea of morphing a query over time, dynamically, but the code still wasn't quite right.

I was particularly challenged by the fact that I had to find a way to search, filter, and refine by tags, and this had to be done in a specific order; it doesn't make sense for a user to search by a recipe name and then try including recipes that have the tag spicy.

My original code looked something like this:

  def search_recipes(%{"filter" => filter, "query" => search_query, "tags" => tags}, user_id) do
    search_conditions =
      if search_query !== "",
        do: dynamic([r], like(r.title, ^"%#{search_query}%")),
        else: true

    filter_conditions =
      case filter do
        "My Recipes" ->
          dynamic([r], r.user_id == ^user_id)

        "Under an hour" ->
          dynamic(
            [r],
            fragment(~s|time->'hour' = '1' and time->'minute' = '0' or time->'hour' < '1'|)
          )

          #...

You can see the full diff of how things used to look here.

I was missing out on an opportunity to use pattern matching to handle for the conditional cases of a query. Originally I was thinking much more in a traditional language flow, where I would use if statements to determine if the query should change or not, and then I would string together the results of those queries into one single statement. This was a mess. I was checking if a user had any tags, which forked the function in two different directions, when, again, pattern matching waited just beyond my comprehension to clean things up (I should be clear, I'm not beating myself up, just documenting my journey from not-knowing to knowing :) )

And here's the thing - a nearly exact example of what I needed to do was in the docs; I just skimmed over it because it honestly looked more complex than I needed (And admittedly, i still sometimes get a bit turned around while reading code that uses reduce).

So here's what I ended up with (full diff here)

First off, I have a nice, clean, simple public function:

def search_recipes(params, user_id) do
    from(r in Recipe)
    |> filter_recipes(params, user_id)
    |> Repo.all()
    |> Repo.preload(:tags)
  end

# params look    like so:
# %{"query" => "", "tags" => "breakfast, vegan", "filter": "My favourites" }

The magic happens in filter_recipes. Let's see that now. It's a little long, but that's because it handles all the possible permutations of querying for recipes.

 defp filter_recipes(query, params, user_id) do
    ## params is still the same map as above
    Enum.reduce(params, query, fn
      {"tags", ""}, query ->
        query

      {"tags", val}, query ->
        split_tags =
          for tag <- String.split(val, ","),
              tag = tag |> String.trim() |> String.downcase(),
              tag != "",
              do: tag

        tagged_recipe_ids = get_by_tags(split_tags)
        query |> where([r], r.id in ^tagged_recipe_ids)

      {"query", ""}, query -> query

      {"query", val}, query ->
        like = "%#{val}%"
        query |> where([r], like(r.title, ^like))

      {"filter", "My Recipes"}, query ->
        query |> where([r], r.user_id == ^user_id)

      {"filter", "Under an hour"}, query ->
        query
        |> where([r], fragment(~s|time->'hour' = '1' and time->'minute' = '0' or time->'hour' < '1'|))

      {"filter", "Under 30 minutes"}, query ->
        query
        |> where([r], fragment(~s|time->'hour' < '1' and time->'minute' <= '30'|))

      {"filter", "Recently posted"}, query ->
        query |> where([r], r.inserted_at > ago(2, "week"))

      {"filter", "My favourites"}, query ->
        from(recipes in query,
          left_join: fav in Favourite,
          on: fav.recipe_id == recipes.id,
          preload: [favourites: fav],
          where: fav.user_id == ^user_id
        )

      {_, _}, query -> query
    end)

That's it! The important thing to remember when writing this code is that the Ecto query is moving through reduce, changing every time we match on a param. It's important to make sure that each anonymous function pattern match has the second param of "query" in order to take the most recently change query (aka, the accumulator), and not the original one that is passed in (maybe I should have named them differently). In sum, we are looping over a map, pattern matching on the key's and values, and then tweaking what the query is composed of (and then repeat).

This might not be particularly revolutionary for many, but I'm pleased how linear it feels to write this code. I don't have to pollute my code with if/else statements, which makes my mind have to fork every time I'd read the function. That's all for now. Do you see something I could be doing better? Let me know.

Thanks for reading!

PS. One thing I'm still not entirely sure of is when to use the dynamic query function. I thought I needed it (as detailed in the previous post) but instead I'm able to just move the query through the reduce. I'm probably misunderstanding something.