Dan Schultzer: Adding XML feeds to Phoenix

Today I was adding an Atom XML feed to my blog. Atom is used for feed syndication and is a more robust version of the well-known RSS feed.

My previous process for finding blog posts was to randomly discover them through social media feeds. When I moved over to Mastodon I realized how off that was. I was missing out on content just because the algorithm didn’t prioritize it! So now I’m using syndication with Feedly to catch up on the blogs I like.

Let’s add a dynamically generated Atom feed to this blog!

The XML EEx sigil

I could just use atomex to generate it in a plug or as a file in priv/static/. But what if we could have something that feels a little more like good ol’ Phoenix 1.7?

XML template defined in a sigil like HEEx
XML template with compile-time warnings/errors
XML-template generates the atom.xml file content or plug response body

Keeping it as generic XML is also nice since we can reuse it for other features like XML sitemaps or RSS. First, let’s start with how we want the Atom XML module to function:

# lib/my_blog_web/atom_xml.ex
defmodule MyBlogWeb.AtomXML do
  use MyBlogWeb, :xml

  def load(_) do
    # Atom feed MUST have ISO DateTime format
    [latest_post | _] = posts = Enum.map(MyBlog.Blog.list_posts(), &%{&1 | date: to_datetime(&1.date)})

    %{
      title: "Dan Schultzer",
      author: "Dan Schultzer",
      updated_at: latest_post.date,
      posts: posts
    }
  end

  def to_datetime(date) do
    DateTime.new!(date, ~T[00:00:00])
  end

  def render(assigns) do
    ~X"""
    <?xml version="1.0" encoding="utf-8"?>
    <feed xmlns="http://www.w3.org/2005/Atom">
      <title><%= @title %></title>
      <link href="<%= url(~p"/") %>" />
      <link href="<%= url(~p"/atom.xml") %>" rel="self" />
      <updated><%= @updated_at %></updated>
      <author>
        <name><%= @author %></name>
      </author>
      <id><%= url(~p"/") %></id>

      <%= for post <- @posts do %>
      <entry>
        <title><%= post.title %></title>
        <link href="<%= url(~p"/posts/#{post.id}") %>" />
        <id><%= url(~p"/posts/#{post.id}") %></id>
        <updated><%= post.date %></updated>
        <summary><%= post.description %></summary>
      </entry>
      <% end %>
    </feed>
    """
  end
end

Great, this looks very similar to Phoenix components!

The module has a render/1 function that contains the compiled template, and a load/1 function that’ll build the assigns map. Now we must tie this together with compiling the template.

First, we should add an xml/0 function to our web module so we can render routes:

# lib/my_blog_web/my_blog_web.ex
defmodule MyBlogWeb do
  # ...

  def xml do
    quote do
      @behaviour MyBlogWeb.XML.Engine

      import MyBlogWeb.XML.Engine
      import MyBlogWeb.Gettext

      # Routes generation with the ~p sigil
      unquote(verified_routes())
    end
  end

  # ...
end

Now, we get to the template engine!

# lib/my_blog_web/xml/engine.ex
defmodule MyBlogWeb.XML.Engine do
  @type document :: [any()]
  @type assigns :: map()

  @callback load(keyword()) :: assigns()
  @callback render(assigns()) :: document()

  defmacro sigil_X({:<<>>, meta, [expr]}, []) do
    unless Macro.Env.has_var?(__CALLER__, {:assigns, nil}) do
      raise "~H requires a variable named \"assigns\" to exist and be set to a map"
    end

    options = [
      engine: __MODULE__,
      file: __CALLER__.file,
      line: __CALLER__.line + 1,
      caller: __CALLER__,
      indentation: meta[:indentation] || 0,
      source: expr
    ]

    EEx.compile_string(expr, options)
  end

  @behaviour EEx.Engine

  @impl true
  def init(_opts) do
    %{
      iodata: [],
      dynamic: [],
      vars_count: 0
    }
  end

  @impl true
  def handle_begin(state) do
    %{state | iodata: [], dynamic: []}
  end

  @impl true
  def handle_end(quoted) do
    handle_body(quoted)
  end

  @impl true
  def handle_body(state) do
    %{iodata: iodata, dynamic: dynamic} = state
    iodata = Enum.reverse(iodata)
    {:__block__, [], Enum.reverse([iodata | dynamic])}
  end

  @impl true
  def handle_text(state, _meta, text) do
    %{iodata: iodata} = state
    %{state | iodata: [text | iodata]}
  end

  @impl true
  def handle_expr(state, "=", ast) do
    ast = Macro.prewalk(ast, &EEx.Engine.handle_assign/1)
    %{iodata: iodata, dynamic: dynamic, vars_count: vars_count} = state
    var = Macro.var(:"arg#{vars_count}", __MODULE__)
    ast = quote do: unquote(var) = unquote(ast)
    %{state | dynamic: [ast | dynamic], iodata: [var | iodata], vars_count: vars_count + 1}
  end

  def handle_expr(state, "", ast) do
    ast = Macro.prewalk(ast, &EEx.Engine.handle_assign/1)
    %{dynamic: dynamic} = state
    %{state | dynamic: [ast | dynamic]}
  end

  def handle_expr(state, marker, ast) do
    EEx.Engine.handle_expr(state, marker, ast)
  end

  def encode_to_iodata!(data), do: to_iodata(data)

  defp to_iodata([h | t]), do: [to_iodata(h) | to_iodata(t)]
  defp to_iodata(text) when is_binary(text), do: text
  defp to_iodata(%DateTime{} = date), do: DateTime.to_iso8601(date)
  defp to_iodata([]), do: ""
end

The template engine will transform the template string defined in MyBlogWeb.AtomXML into an AST. The above logic is similar to how the Phoenix.HTML.Engine works.

I did want to add XML document validation here as well, but that was a daunting task when looking at how Phoenix.LiveView.HTMLEngine deals with it. This is good enough as we still get a compile-time check of the embedded Elixir code!

Now that we can generate our XML document, we need to serve it as a file in Phoenix.

Option 1: Assets deploy pipeline

It makes sense that atom.xml only needs to be generated at compile time since all our blog posts are static. So we could add atom.xml file generation to the assets deploy pipeline.

As you will see below it feels messier than just generating it in a plug, due to how code-generated Phoenix thinks of URL generator options as runtime-only config.

First, we’ll add a mix alias to write to priv/static/atom.xml:

# mix.exs
defmodule MyBlog.MixProject do
  # ...

  defp aliases do
    [
      # ...
      "assets.gen.atom": &write_atom_feed/1,
      "assets.deploy": ["tailwind default --minify", "esbuild default --minify", "phx.digest", "assets.gen.atom"]
    ]
  end

  @atom_feed_path "priv/static/atom.xml"

  defp write_atom_feed(args) do
    Mix.Task.run("app.start", args)

    content =
      []
      |> MyBlogWeb.AtomXML.load()
      |> MyBlogWeb.AtomXML.render()
      |> MyBlogWeb.XML.Engine.encode_to_iodata!()
      |> IO.iodata_to_binary()

    case File.write(@atom_feed_path, content) do
      :ok -> Mix.shell().info([:green, "Generated file #{@atom_feed_path}"])
      {:error, posix} -> Mix.shell().info([:red, "Failed to write to #{@atom_feed_path}: #{inspect posix}"])
    end
  end
end

Now you can run mix assets.gen.atom to generate priv/static/atom.xml!

We add this alias to the assets.deploy alias after phx.digest because we need the URI to stay consistent for all deployments. It would break feed reader sync if the URI got an ever-changing cache-busting fingerprint.

To serve the file as static content in Phoenix, add atom.xml to statics_path/0 in the web module:

# lib/my_blog_web/my_blog_web.ex
defmodule MyBlogWeb do

  # ...
  
  def static_paths, do: ~w(assets fonts images favicon.ico robots.txt atom.xml)
  
  # ...
end

Now add the Atom feed link to <head> in our root template:

# lib/my_blog_web/components/layouts/root.html.heex
<link href={~p"/atom.xml"} type="application/atom+xml" rel="alternate" title="Atom feed" />

As the feed is autogenerated you may also want to ignore it in .gitignore so it doesn’t show up in the working tree:

# .gitignore

# Ignore assets that are produced by build tools.
/priv/static/atom.xml

When we build the docker image, we need to make sure that absolute URIs in our atom.xml are correct. In the code-generated Phoenix app, the URL options are set in config/runtime.exs with a PHX_HOST env var. Naturally, the config is not included when the code-generated Phoenix Dockerfile compiles the app.

So to have the URL options available at compile-time we first need to move the URL generation options over to config/prod.exs:

# config/prod.exs

config :dan, DanWeb.Endpoint,
  cache_static_manifest: "priv/static/cache_manifest.json",
  url: [host: System.fetch_env!("PHX_HOST"), port: 443, scheme: "https"]

Then we’ll need to make the PHX_HOST env var available in the Dockerfile before calling mix deps.compile:

# Dockerfile

# ...

# set PHX_HOST ENV
ARG PHX_HOST
ENV PHX_HOST $PHX_HOST

# ...

It’s all working now! But this doesn’t feel quite right. First, we had to prevent the cache-busting fingerprint, and then, ensure that we can generate absolute URIs at compile-time. Maybe it’s more Phoenix native to only generate absolute URIs at run-time?

Option 2: Plug

First, we’ll create a macro that sets up MyBlogWeb.AtomXML to be a plug:

# lib/my_blog_web/xml/plug.ex
defmodule MyBlogWeb.XML.Plug do
  defmacro __using__(_) do
    quote do
      @behaviour Plug

      import Plug.Conn

      @doc false
      def init(opts), do: opts

      @doc false
      def call(conn, opts) do
        data = __MODULE__.render(__MODULE__.load(opts))


        conn
        |> put_resp_header("content-type", "text/xml")
        |> resp(200, MyBlogWeb.XML.Engine.encode_to_iodata!(data))
      end
    end
  end
end

Now add use MyBlogWeb.XML.Plug to the xml/0 function in the web module:

# lib/my_blog_web/my_blog_web.ex
defmodule MyBlogWeb do
  # ...

  def xml do
    quote do
      @behaviour MyBlogWeb.XML.Engine

      use MyBlogWeb.XML.Plug

      import MyBlogWeb.XML.Engine
      import MyBlogWeb.Gettext

      # Routes generation with the ~p sigil
      unquote(verified_routes())
    end
  end

  # ...
end

Update the router to forward requests to the plug:

# lib/my_blog_web/router.ex
defmodule MyBlogWeb.Router do
  use MyBlogWeb, :router

  # ...

  forward "/atom.xml", MyBlogWeb.AtomXML

  # ...
end

Finally, make sure to link it in <head>:

# lib/my_blog_web/components/layouts/root.html.heex
<link href={~p"/atom.xml"} type="application/atom+xml" rel="alternate" title="Atom feed" />

Start the web server with mix phx.server, and you’ll see the feed generated at http://localhost:4000/atom.xml.

Nice, this option feels a lot more in line with how Phoenix works by default!