Write your own CRON with Elixir

Elixir is rising in popularity and I think this is mostly due to the amazing Phoenix Framework. But for once, let's forget about WEB development and write a simple software: a complete CRON.
If you are too lazy to read, you can directly check the code on Github
Features
We want to build something simple yet production-ready. Therefore we need to be able to:
- define a list of tasks to run via a configuration file
- have a task runner to execute the tasks
- add the tasks to our Supervision Tree (so we can monitor them)
- log the result of each task
- important events should trigger emails
- ship the CRON as a binary
- have proper unit tests
In this article, we will focus on the 2 first items
Let's develop!
Setup the Elixir project
First things first. Let's create a new project:
$ mix new omycron --sup
This command creates an empty project. The --sup
option also generates an application
that is going to be the main hosts for our other processes - the first node of our supervision tree.
defmodule Omycron.Application do
@moduledoc """
Main supervisor for Omycron
"""
use Application
@impl true
def start(_type, _args) do
children = []
opts = [strategy: :one_for_one, name: Omycron.Supervisor]
Supervisor.start_link(children, opts)
end
end
An Application is just a Supervisor that is automatically started by IEX by running iex -S mix
. We will soon add more children.
The tasks configuration file
Now we need to define the format of our config file. I like the simplicity of TOML so this is what we are going to use.
We will define our list of tasks using a TOML array of table
:
[[task]]
name = "Im alive"
interval = {second = 3}
command = "echo"
params = ["i'am alive"]
[[task]]
name = "list folder"
interval = {second = 10, minute = 1}
command = "ls"
params = ['-a', '-l']
[[task]]
name = "a new day"
interval = {hour = 24}
command = "date"
For now, each task must have a name, a shell command, and an interval. Optionally, we can also send an array of parameters.
Now we created the tasks config file, we need a way to dynamically retrieve its pathname in our Elixir application. While we could use environment variables, Elixir provides a Config module that is much easier to maintain.
The config for an Elixir project is usually in config/config.exs
. So let's create this file and add the following:
import Config
config :omycron,
tasks_pathname: "./omycron.toml"
You can now retrieve the pathname of the config file using this command:
Application.fetch_env!(:omycron, :tasks_pathname)
Setup the tasks storage
We want to parse the TOML once and then store the list of tasks. In Elixir, the right way to store such data is with an Agent. An agent is just a process that is able to hold a state and provide some helpers to set and retrieve its state - yup, you got it right, the right way to store data in Elixir is through a sub-process!
In a folder tasks
, create a new file agent.ex
and following the documentation, initialize an empty Agent:
defmodule Omycron.Tasks.Agent do
@moduledoc """
Store the tasks
"""
use Agent
def start_link([config_pathname]) do
IO.puts("IM STARTING !")
Agent.start_link(fn -> %{} end, name: __MODULE__)
end
end
As you can see, the first parameter of start_link
is a function that should return the data to store. For now, it stores an empty map. All sub-processes started properly can also have a name
(2nd parameter). It allows us to retrieve the process using its name instead of its PID.
If you tried to run iex -S mix
, you might have noticed that our agent doesn't start. Since this is a sub-process, it needs to be added to our supervision tree. This is when our initial Application comes in handy! Edit omycron/application.ex
:
# ...
def start(_type, _args) do
children = [
Omycron.Tasks.Agent.child_spec()
]
# ...
end
Now we added our newly created Agent as a sub-process of our main application. It will be automatically started by IEX. And since our Application supervisor is set with the strategy one_for_one
, if anything happens to our Agent and it crashes, it will automatically be restarted!
Time to parse!
Now everything is ready to store our tasks, we need to write our parser. We will use the librarytoml_elixir
. It's TOML 0.5 compliant. Not perfect but good enough.
Add toml_elixir
as a dependency
defp deps do
[
{:toml_elixir, "~> 2.0.0"}
]
end
Once you have installed the dependency with mix deps.get
, create a new file tasks/parser.ex
that looks like this:
defmodule Omycron.Tasks.Parser do
@moduledoc """
Parse the TOML config file
"""
defmodule __MODULE__.Task do
@moduledoc """
Define a struct that contains all the properties for a Task
"""
defstruct name: nil, command: nil, params: [], interval: nil
end
@doc """
Start the parsing
"""
@spec parse(String.t()) :: {:ok, list(any)} | {:error, String.t()}
def parse(pathname) do
# Use toml_elixir to read the file
TomlElixir.parse_file(pathname)
# parse_content use the value returned by toml_elixir
|> parse_content()
# `tap` execute a function and returns the value passed as parameters
# this is used temporary to print the result and still return the tasks
|> tap(&IO.puts("Parsed tasks > #{inspect(&1)}"))
end
## In case `toml_elixir` returns an error, we display a proper error message
## and exit the program
defp parse_content({:error, error_type}) do
case error_type do
:enoent -> write_error("Config file is missing")
_ -> write_error("Failed to parse config file")
end
write_error("Program exits.")
exit(:shutdown)
end
## If `toml_elixir` was successful, we parse the content
## we use pattern matching to direcly access the array of tasks
defp parse_content({:ok, %{"task" => tasks}}) do
parse_tasks(tasks, [])
end
## parse_tasks loop trough all tasks (thanks to pattern matching)
## and process them one by one.
## Once its done, it returns `parsed_tasks`
defp parse_tasks([], parsed_tasks), do: parsed_tasks
defp parse_tasks([task | tail], parsed_tasks),
do: parse_tasks(tail, [parse_task(task) | parsed_tasks])
## parse_task takes the raw data returned from `toml_elixir`
## and cast it into a Task struct defined earlier
## we use pattern matching to make sure the format is correct
defp parse_task(%{"name" => name, "command" => command, "params" => params, "interval" => interval}),
do: %__MODULE__.Task{
name: name,
command: command,
params: params,
interval: interval_to_ms(interval)
}
## In case the file has a invalid entry, we print an error message
## and exit the program
defp parse_task(invalid_task) do
write_error("Invalid command > #{inspect(invalid_task)}")
exit(:shutdown)
end
## we gradually merge hour to minute then minute to second and finally second to ms
defp interval_to_ms(%{"second" => second, "minute" => minute, "hour" => hour}),
do: interval_to_ms(%{"second" => second, "minute" => minute + hour * 60})
defp interval_to_ms(%{"second" => second, "minute" => minute}),
do: interval_to_ms(%{"second" => second + minute * 60})
defp interval_to_ms(%{"second" => second}),
do: second * 1000
# write on stderr
defp write_error(str), do: IO.write(:standard_error, str)
end
Ok, let's try our parser. Run iex -S mix
to start your program in an Interactive Elixir session and run Omycron.Tasks.Parser.parse
:
iex(1)> Omycron.Tasks.Parser.parse("./omc.toml")
Parsed tasks > [%OMC.Tasks.Parser.Task{command: "echo 'hello'", interval: %{"second" => 10}, name: "Hello world"}, %OMC.Tasks.Parser.Task{command: "echo 'im alive'", interval: %{"second" => 3}, name: "Im alive"}]
[
%Omycron.Tasks.Parser.Task{
command: "echo 'hello'",
interval: 10000,
name: "Hello world"
},
%Omycron.Tasks.Parser.Task{
command: "echo 'im alive'",
interval: 3000,
name: "Im alive"
}
]
iex(2)>
Alright, it's working well. Finally, we just have to merge this piece of code with the Agent we previously defined:
in tasks/agent.ex
, make the following changes to store the result of the parser into our Agent.
def start_link([config_pathname]) do
Agent.start_link(fn -> Omycron.Tasks.Parser.parse(config_pathname) end, name: __MODULE__)
end
and in application.ex
, retrieve the config pathname from using the Config module of Elixir:
children = [
Omycron.Tasks.Agent.child_spec([Application.fetch_env!(:omycron, :tasks_pathname)])
]
At this point, when you start the program, the tasks are loaded and stored in our agent. You can create a new file tasks.ex
where we will have a small helper to retrieve the list of tasks:
defmodule Omycron.Tasks do
@moduledoc """
Helpers for tasks
"""
@doc """
Retrive the list of tasks
"""
def get do
Agent.get(__MODULE__.Agent, & &1)
end
end
Our task runner
Let's wrap this chapter by writing our task runner. The task runner is a GenServer (understand a sub-process) that will send an event to itself every time a task should be executed.
Now everything is clear, create a runner.ex
file that looks like the following:
defmodule Omycron.Runner do
@moduledoc """
We use a GenServer (a sub-process) that will use the erlang `timer` module to send itself an event that will start the execution of a task
"""
use GenServer
@doc """
Follow the official documentation of GenServer
"""
def start_link(opts), do: GenServer.start_link(__MODULE__, opts, name: __MODULE__)
@impl true
def init(_opts) do
# small trick to wait for the tasks to load everything
:timer.send_after(1000, :start)
{:ok, nil}
end
@doc """
Load the list of tasks and start each of them one by one
"""
@impl true
def handle_info(:start, _params) do
with tasks <- OMC.Tasks.get(),
:ok <- Enum.each(tasks, &setup_task(&1)) do
{:noreply, tasks}
else
_err -> :error
end
end
@doc """
Execute a Task and print the result
"""
@impl true
def handle_info(
{:execute, %OMC.Tasks.Parser.Task{name: name, command: command, params: params}},
tasks
) do
{res, _} = System.cmd(command, params)
IO.puts("Task #{name} executed successfully:\n#{inspect(res)}\n")
{:noreply, tasks}
end
## use the `send_interval` function to trigger an event to itself everytime the task should run
defp setup_task(%OMC.Tasks.Parser.Task{interval_in_ms: interval_in_ms} = task) do
:timer.send_interval(interval_in_ms, {:execute, task})
end
end
Now everything is ready, let's add this GenServer to our application's children. Edit application.ex
:
@impl true
def start(_type, _args) do
children = [
# Loads and store the tasks
Omycron.Tasks.Agent.child_spec([Application.fetch_env!(:omycron, :tasks_pathname)]),
Omycron.Runner.child_spec([]) # <- add this line
]
opts = [strategy: :one_for_one, name: Omycron.Supervisor]
Supervisor.start_link(children, opts)
end
Et voilĂ ! Our CRON is finally working. You can test by running iex -S mix
.
I hope you enjoyed this article and stay tuned for the following chapter!
If you have a problem and no one else can help. Maybe you can hire the Kalvad-Team.