Showing preview only (206K chars total). Download the full file or copy to clipboard to get everything.
Repository: functional-rewire/dune
Branch: main
Commit: 8a6f7ddfb6a3
Files: 57
Total size: 191.6 KB
Directory structure:
gitextract_bo31q11i/
├── .formatter.exs
├── .github/
│ └── workflows/
│ └── ci.yml
├── .gitignore
├── CHANGELOG.md
├── LICENSE.md
├── README.md
├── lib/
│ ├── dune/
│ │ ├── allowlist/
│ │ │ ├── default.ex
│ │ │ ├── docs.ex
│ │ │ └── spec.ex
│ │ ├── allowlist.ex
│ │ ├── atom_mapping.ex
│ │ ├── eval/
│ │ │ ├── env.ex
│ │ │ ├── fake_module.ex
│ │ │ ├── function_clause_error.ex
│ │ │ ├── macro_env.ex
│ │ │ └── process.ex
│ │ ├── eval.ex
│ │ ├── failure.ex
│ │ ├── helpers/
│ │ │ ├── diagnostics.ex
│ │ │ └── term_checker.ex
│ │ ├── opts.ex
│ │ ├── parser/
│ │ │ ├── atom_encoder.ex
│ │ │ ├── compile_env.ex
│ │ │ ├── debug.ex
│ │ │ ├── real_module.ex
│ │ │ ├── safe_ast.ex
│ │ │ ├── sanitizer.ex
│ │ │ ├── string_parser.ex
│ │ │ └── unsafe_ast.ex
│ │ ├── parser.ex
│ │ ├── session.ex
│ │ ├── shims/
│ │ │ ├── atom.ex
│ │ │ ├── enum.ex
│ │ │ ├── io.ex
│ │ │ ├── json.ex
│ │ │ ├── kernel.ex
│ │ │ ├── list.ex
│ │ │ └── string.ex
│ │ └── success.ex
│ └── dune.ex
├── mix.exs
└── test/
├── dune/
│ ├── allowlist/
│ │ └── default_test.exs
│ ├── allowlist_test.exs
│ ├── atom_mapping_test.exs
│ ├── opts_test.exs
│ ├── parser/
│ │ ├── atom_encoder_test.exs
│ │ └── string_parser_test.exs
│ ├── session_test.exs
│ ├── shims_test.exs
│ └── validation_test.exs
├── dune_modules_test.exs
├── dune_oom_safety_test.exs
├── dune_quoted_test.exs
├── dune_string_test.exs
├── dune_string_to_quoted_test.exs
├── dune_test.exs
└── test_helper.exs
================================================
FILE CONTENTS
================================================
================================================
FILE: .formatter.exs
================================================
# Used by "mix format"
locals_without_parens = [allow: 2]
[
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"],
locals_without_parens: locals_without_parens,
export: [locals_without_parens: locals_without_parens]
]
================================================
FILE: .github/workflows/ci.yml
================================================
name: CI
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
name: OTP ${{matrix.otp}} / Elixir ${{matrix.elixir}}
env:
MIX_ENV: test
strategy:
matrix:
include:
- elixir: "1.14.5"
otp: "26.0"
testArgs: "--exclude=lts_only"
- elixir: "1.15.4"
otp: "26.0"
testArgs: "--exclude=lts_only"
- elixir: "1.16.0"
otp: "26.0"
testArgs: "--exclude=lts_only"
- elixir: "1.17.3"
otp: "27.0"
testArgs: "--exclude=lts_only"
- elixir: "1.18.0"
otp: "27.0"
testArgs: "--exclude=lts_only"
- elixir: "1.18.4"
otp: "27.0"
testArgs: "--exclude=lts_only"
- elixir: "1.19.0"
otp: "28.1"
testArgs: ""
steps:
- uses: actions/checkout@v2
- uses: erlef/setup-beam@v1
with:
otp-version: ${{matrix.otp}}
elixir-version: ${{matrix.elixir}}
- name: Install Dependencies
run: mix deps.get
- name: Check compile warnings
run: mix compile --warnings-as-errors
- name: Check format
run: mix format --check-formatted
# TODO add dialyzer?
- name: Unit tests
run: mix test ${{ matrix.testArgs }}
================================================
FILE: .gitignore
================================================
# The directory Mix will write compiled artifacts to.
/_build/
# If you run "mix test --cover", coverage assets end up here.
/cover/
# The directory Mix downloads your dependencies sources to.
/deps/
# Where third-party dependencies like ExDoc output generated docs.
/doc/
# Ignore .fetch files in case you like to edit your project deps locally.
/.fetch
# If the VM crashes, it generates a dump, let's ignore it too.
erl_crash.dump
# Also ignore archive artifacts (built via "mix archive.build").
*.ez
# Ignore package tarball (built via "mix hex.build").
dune-*.tar
# Temporary files, for example, from tests.
/tmp/
================================================
FILE: CHANGELOG.md
================================================
# Changelog
## Dev
## v0.3.15 (2025-10-19)
- Support Elixir 1.19
## v0.3.14 (2025-07-29)
### Security fixes
- Use safe shims for chardata -> string conversions (`List.to_string/1`, ...) in
`Dune.Allowlist.Default`
- Restrict more unsafe modules and functions in `Dune.Allowlist.Default`:
- `:unicode`
- `:erts_debug.flat_size`
## v0.3.13 (2025-07-27)
- Fix older versions pre-1.18 that don't have `JSON`
## v0.3.12 (2025-07-27)
- `Dune.Allowlist.Default` exposes a safe shim for the `JSON` module
## v0.3.11 (2024-12-21)
- Enable support for Elixir 1.18
## v0.3.10 (2024-07-14)
- Enable support for Elixir 1.17
## v0.3.9 (2024-06-25)
- `Dune.Allowlist.Default` allows the `Version` module
## v0.3.8 (2024-05-26)
### Bug fixes
- Make sure the `Duration` atom is available
## v0.3.7 (2024-05-26)
### Bug fixes
- Fix incorrect type definitions, remove unused ones
### Enhancements
- `Dune.Allowlist.Default` allows the new `Duration` module and new kernel
functions from Elixir 1.17
- Add an `:inspect_sort_maps` option for deterministic outputs
- Capture and return parser warnings in `stdio`
## v0.3.6 (2023-12-23)
- Support Elixir 1.16
- `Dune.Allowlist.Default` allows `**/2`
## v0.3.5 (2023-11-10)
### Enhancements
- Prepare Elixir 1.16 support(handle line-column positions in diagnostics)
## v0.3.4 (2023-09-14)
### Bug fixes
- Fix `UndefinedFunctionError` when using external modules in a custom allowlist
## v0.3.3 (2023-08-13)
### Bug fixes
- Fix vulnerability allowing an attacker to crash the VM using bitstrings
## v0.3.2 (2023-08-12)
### Enhancements
- `dbg/1` uses pretty printing
### Bug fixes
- Fix error message on restricted `dbg/0`
## v0.3.1 (2023-08-12)
### Enhancements
- Add support for `dbg/1`
### Bug fixes
- Properly distinguish user code `throw/1` from internal ones
## v0.3.0 (2023-08-09)
### Breaking changes
- Drop support for Elixir 1.13
- Compile errors are now returned as a separate type `:compile_error`
### Enhancements
- Support Elixir 1.15
- Capture compile diagnostics (Elixir >= 1.15)
### Bug fixes
- Better handle `UndefinedFunctionError` for dynamic module names
## v0.2.6 (2022-10-17)
### Enhancements
- Support Elixir 1.14
## v0.2.5 (2022-08-25)
### Bug fixes
- Restrict the use of `:counters` in `Dune.Allowlist.Default`, since it can leak
memory
## v0.2.4 (2022-07-13)
### Bug fixes
- Validate module names in `defmodule`, reject `nil` or booleans
## v0.2.3 (2022-04-13)
### Bug fixes
- `Dune.string_to_quoted/2` quotes modules with `.` correctly
- OTP 25 regression: keep a clean stacktrace for exceptions
## v0.2.2 (2022-04-05)
### Enhancements
- Add `Dune.string_to_quoted/2` to make it possible to visualize AST
- Merged parsing and eval options in a single `Dune.Opts` for simplicity
- Add a `pretty` option to inspect result
- Better error message when `def/2` and `defp/2` called outside a module
### Breaking changes
- Removed Dune.Parser.Opts and Dune.Eval.Opts
## v0.2.1 (2022-03-19)
### Bug fixes
- Handle default arguments in functions
- Handle conflicting `def` and `defp` with same name/arity
## v0.2.0 (2022-01-02)
### Breaking changes
- Support Elixir 1.13, drop support for 1.12
- This fixes a [bug in atoms](https://github.com/elixir-lang/elixir/pull/11313)
was due to the Elixir parser
## v0.1.2 (2021-10-17)
### Enhancements
- Allow safe functions from the `:erlang` module
### Bug fixes
- Fix bug when calling custom function in nested AST
## v0.1.1 (2021-10-16)
### Bug fixes
- Prevent atom leaks due to `Code.string_to_quoted/2` not respecting
`static_atoms_encoder`
- Handle Elixir 1.12 bug on single atom ASTs
- Handle atoms prefixed with `Elixir.` properly
- Fix inspect for quoted atoms
## v0.1.0 (2021-09-19)
- Initial release
================================================
FILE: LICENSE.md
================================================
MIT License
Copyright (c) 2021 Sabiwara
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
================================================
FILE: README.md
================================================
# Dune
[](https://hex.pm/packages/dune)
[](https://hexdocs.pm/dune/)
[](https://github.com/functional-rewire/dune/actions?query=workflow%3ACI)
A sandbox for Elixir to safely evaluate untrusted code from user input.
[**Try it out on our online playground!**](https://playground.functional-rewire.com/)
`Dune` can be useful to develop playgrounds, online REPL, coding games, or
customizable business logic.
**Warning:** `Dune` cannot offer strong security guarantees (see the
[Security guarantees](#security-guarantees) section below). Besides, it is still
early stage: expect bugs and vulnerabilities.
## Features
- allowlist mechanism (customizable) to restrict execution to safe modules and
functions: no access to environment variables, file system, network...
- code executed in an isolated process
- execution within configurable limits: timeout, maximum reductions and memory
(inspired by [Luerl](https://github.com/rvirding/luerl))
- captured standard output
- atoms, without atom leaks: parsing and runtime do not
[leak atoms](https://hexdocs.pm/elixir/String.html#to_atom/1) (i.e. does not
keep
[filling the atom table](https://learnyousomeerlang.com/starting-out-for-real#atoms)
until the VM crashes)
- modules, without actual module creation: Dune does not let users define any
actual module (would leak memory and modify the state of the VM globally), but
`defmodule` simulates the basic behavior of a module, including private and
recursive functions
```elixir
iex> Dune.eval_string("IO.puts(\"Hello world!\")")
%Dune.Success{inspected: ":ok", stdio: "Hello world!\n", value: :ok}
iex> Dune.eval_string("File.cwd!()")
%Dune.Failure{message: "** (DuneRestrictedError) function File.cwd!/0 is restricted", type: :restricted}
iex> Dune.eval_string("List.duplicate(:spam, 100_000)")
%Dune.Failure{message: "Execution stopped - memory limit exceeded", stdio: "", type: :memory}
iex> Dune.eval_string("Enum.product(1..100_000)")
%Dune.Failure{message: "Execution stopped - reductions limit exceeded", stdio: "", type: :reductions}
```
The list of modules and functions authorized by default is defined by the
[`Dune.Allowlist.Default`](https://hexdocs.pm/dune/Dune.Allowlist.Default.html#module-allowed-modules-functions)
module, but this list can be extended and customized (at your own risk!) using
[`Dune.Allowlist`](https://hexdocs.pm/dune/Dune.Allowlist.html).
If you need to keep the state between evaluations, you might consider
[`Dune.Session`](https://hexdocs.pm/dune/Dune.Session.html):
```elixir
iex> Dune.Session.new()
...> |> Dune.Session.eval_string("x = 1")
...> |> Dune.Session.eval_string("x + 2")
#Dune.Session<last_result: %Dune.Success{inspected: "3", stdio: "", value: 3}, ...>
```
`Dune.string_to_quoted/2` returns the AST corresponding to the provided
`string`, without leaking atoms:
```elixir
iex> Dune.string_to_quoted("foo(:bar)").inspected
"{:foo, [line: 1], [:bar]}"
```
## Limitations
`Dune` supports a fair subset of the base language, but it cannot safely support
advanced features (at least at this stage) such as:
- custom structs / behaviours / protocols
- concurrency / processes / OTP
- metaprogramming
## Security guarantees
Because of the approch being used, Dune cannot offer strong security guarantees,
and should not be considered a sufficient security layer by itself.
A best-effort approach is made to prevent attackers from getting outside of the
original process and from calling any function/macro outside of the allowlist.
However, it is impossible to prove that all escape paths have been completely
blocked. Due to how the Erlang VM works, an attacker able to escape the sandbox
could get full access to the VM without restriction.
See the
[EEF guidelines about sandboxing](https://erlef.github.io/security-wg/secure_coding_and_deployment_hardening/sandboxing)
for more information.
Use at your own risk and avoid running it directly on a server with any
sensitive access, e.g. to a database.
## Installation
The package can be installed by adding `dune` to your list of dependencies in
`mix.exs`:
```elixir
def deps do
[
{:dune, "~> 0.3.15"}
]
end
```
Documentation can be found at
[https://hexdocs.pm/dune](https://hexdocs.pm/dune).
## FAQ
### Why can I still create atoms?
Atoms are converted by the parser and mapped to always use a given set of atoms.
So when you type `foo = bar(:baz)`, the atoms `:foo`, `:bar` and `:baz` won't
actually be created, but other atoms like `:atom1`, `:atom2` and `:atom3` are
going to be used instead.
So even if many user run various codes using many different variable and
function names, they will all pull from the same pool of atoms.
### Why can I still create modules?
Modules are being defined globally within the VM, making it both an isolation
concern and a memory concern. But modules are an important part of the Elixir
language, and are especially important for learning platforms.
Dune implements an alternative `defmodule`, relying on maps of anonymous
functions at runtime and should reproduce the basic behavior of actual modules,
with support for recursive and private functions.
### Why is the behavior different than Elixir when doing X?
As explained above, some parts of the language are actually being completely
reimplemented because the original version could not be safely sandboxed: atoms,
modules... While these alternative implementation aim to be as close as possible
to the original ones, they might differ in some cases, because the code being
executed is actually different.
### Why can't I do X?
Some parts of the language are being restricted because they present a direct
security risk, while some other would need to be reimplemented in an alternative
way and therefore need a consequent amount of work that hasn't been done yet.
================================================
FILE: lib/dune/allowlist/default.ex
================================================
defmodule Dune.Allowlist.Default do
@moduledoc """
The default `Dune.Allowlist` module to be used to allow or restrict
functions and macros that can be safely executed.
## Examples
iex> Dune.Allowlist.Default.fun_status(Kernel, :+, 2)
:allowed
iex> Dune.Allowlist.Default.fun_status(String, :to_atom, 1)
:restricted
iex> Dune.Allowlist.Default.fun_status(Atom, :to_string, 1)
{:shimmed, Dune.Shims.Atom, :to_string}
iex> Dune.Allowlist.Default.fun_status(Kernel, :foo, 1)
:undefined_function
iex> Dune.Allowlist.Default.fun_status(Bar, :foo, 1)
:undefined_module
iex> Dune.Allowlist.Default.fun_status(Kernel.SpecialForms, :quote, 2)
:restricted
## Allowed modules / functions
__DUNE_ALLOWLIST_FUNCTIONS__
"""
use Dune.Allowlist
alias Dune.Shims
@special_forms_allowed ~w[
{}
%{}
<<>>
=
^
case
cond
fn
for
with
::
__aliases__
]a
@kernel_operators ~w[
|>
+
++
-
--
*
**
/
<>
==
===
!=
!==
=~
>
>=
<
<=
and
or
&&
||
!
..
..//
]a
@kernel_guards ~w[
is_integer
is_binary
is_bitstring
is_atom
is_boolean
is_integer
is_float
is_number
is_list
is_map
is_map_key
is_nil
is_reference
is_tuple
is_exception
is_struct
is_function
]a
@kernel_macros ~w[
if
unless
in
match?
then
tap
raise
]a
@kernel_sigils ~w[
sigil_C
sigil_D
sigil_N
sigil_R
sigil_S
sigil_T
sigil_U
sigil_c
sigil_r
sigil_s
sigil_w
]a
@kernel_functions ~w[
abs
binary_part
bit_size
byte_size
ceil
div
elem
floor
get_and_update_in
get_in
hd
length
make_ref
map_size
max
min
not
pop_in
put_elem
put_in
rem
round
self
tl
trunc
tuple_size
update_in
]a
# TODO Remove when dropping support for Elixir 1.16
extra_kernel_functions =
if System.version() |> Version.compare("1.17.0-rc.0") != :lt,
do: [:to_timeout, :is_non_struct_map],
else: []
@kernel_allowed extra_kernel_functions ++
@kernel_operators ++
@kernel_guards ++ @kernel_macros ++ @kernel_sigils ++ @kernel_functions
@kernel_shims [
apply: {Shims.Kernel, :safe_apply},
inspect: {Shims.Kernel, :safe_inspect},
to_string: {Shims.Kernel, :safe_to_string},
to_charlist: {Shims.Kernel, :safe_to_charlist},
sigil_w: {Shims.Kernel, :safe_sigil_w},
sigil_W: {Shims.Kernel, :safe_sigil_W},
throw: {Shims.Kernel, :safe_throw},
dbg: {Shims.Kernel, :safe_dbg}
]
@erlang_allowed [
:*,
:+,
:++,
:-,
:--,
:/,
:"/=",
:<,
:"=/=",
:"=:=",
:"=<",
:==,
:>,
:>=,
:abs,
:adler32,
:adler32_combine,
:and,
:append_element,
:band,
:binary_part,
:binary_to_float,
:binary_to_integer,
:binary_to_list,
:bit_size,
:bitstring_to_list,
:bnot,
:bor,
:bsl,
:bsr,
:bxor,
:byte_size,
:ceil,
:convert_time_unit,
:crc32,
:crc32_combine,
:date,
:delete_element,
:div,
:element,
:float,
:float_to_binary,
:float_to_list,
:floor,
:hd,
:insert_element,
:integer_to_binary,
:integer_to_list,
:iolist_size,
:iolist_to_binary,
:iolist_to_iovec,
:is_atom,
:is_binary,
:is_bitstring,
:is_boolean,
:is_float,
:is_function,
:is_integer,
:is_list,
:is_map,
:is_map_key,
:is_number,
:is_pid,
:is_port,
:is_record,
:is_reference,
:is_tuple,
:length,
:list_to_binary,
:list_to_bitstring,
:list_to_float,
:list_to_integer,
:localtime,
:localtime_to_universaltime,
:make_ref,
:make_tuple,
:map_get,
:map_size,
:max,
:md5,
:md5_final,
:md5_init,
:md5_update,
:min,
:monotonic_time,
:not,
:or,
:phash2,
:ref_to_list,
:rem,
:round,
:setelement,
:size,
:split_binary,
:system_time,
:time,
:time_offset,
:timestamp,
:tl,
:trunc,
:tuple_size,
:tuple_to_list,
:unique_integer,
:universaltime,
:universaltime_to_localtime,
:xor
]
@erlang_shims [
apply: {Shims.Kernel, :safe_apply}
]
@io_shims [
puts: {Shims.IO, :puts},
inspect: {Shims.IO, :inspect},
chardata_to_string: {Shims.List, :to_string}
]
allow Kernel.SpecialForms, only: @special_forms_allowed
allow Kernel, only: @kernel_allowed, shims: @kernel_shims
allow Access, :all
allow String, except: ~w[to_atom to_existing_atom]a
allow Regex, :all
allow Map, :all
allow MapSet, :all
allow Keyword, :all
allow Tuple, :all
allow List, shims: [to_string: {Shims.List, :to_string}], except: ~w[to_atom to_existing_atom]a
allow Enum, shims: [join: {Shims.Enum, :join}, map_join: {Shims.Enum, :map_join}]
# TODO double check
allow Stream, :all
allow Range, :all
allow Integer, :all
allow Float, :all
allow Atom,
except: ~w[to_char_list]a,
shims: [to_string: {Shims.Atom, :to_string}, to_charlist: {Shims.Atom, :to_charlist}]
if Code.ensure_loaded?(JSON) do
allow JSON,
only: ~w[decode decode!]a,
shims: Enum.map(~w[protocol_encode encode! encode_to_iodata!]a, &{&1, {Shims.JSON, &1}})
end
allow Date, :all
allow DateTime, :all
allow NaiveDateTime, :all
# TODO Remove when dropping support for Elixir 1.16
if System.version() |> Version.compare("1.17.0-rc.0") != :lt do
allow Duration, :all
end
allow Calendar, except: ~w[put_time_zone_database]a
allow Calendar.ISO, :all
allow Time, :all
allow Base, :all
allow URI, :all
allow Version, :all
allow Bitwise, :all
allow Function, only: ~w[identity]a
allow IO, only: ~w[iodata_length iodata_to_binary]a, shims: @io_shims
allow Process, only: [:sleep]
allow :erlang, only: @erlang_allowed, shims: @erlang_shims
allow :math, :all
allow :binary, :all
allow :lists, :all
allow :array, :all
allow :maps, :all
allow :gb_sets, :all
allow :gb_trees, :all
allow :ordsets, :all
allow :orddict, :all
allow :proplists, :all
allow :queue, :all
allow :string, :all
allow :rand, :all
# note: :unicode is not safe and should be shimmed due to "structural sharing bombs"
# note: flat_size is unsafe due to "structural sharing bombs"
allow :erts_debug, only: ~w[same size size_shared]a
allow :zlib, only: ~w[zip unzip gzip gunzip compress uncompress]a
end
================================================
FILE: lib/dune/allowlist/docs.ex
================================================
defmodule Dune.Allowlist.Docs do
@moduledoc false
def document_allowlist(spec) do
spec
|> Dune.Allowlist.Spec.list_ordered_modules()
|> Enum.map_join("\n", &do_doc_funs/1)
end
def public_functions(module) when is_atom(module) do
case Code.fetch_docs(module) do
{:docs_v1, _, _, _, _, _, list} ->
for {{:function, function_name, _}, _, _, %{}, %{}} <- list, into: MapSet.new() do
function_name
end
_ ->
[]
end
end
defp do_doc_funs({module, grouped_funs}) do
public_funs = public_functions(module)
head = ["- `", inspect(module), "`"]
tail =
Enum.map(grouped_funs, fn {status, funs} ->
[
"**",
format_status(status),
"**: " | Enum.map_intersperse(funs, ", ", &format_fun(module, &1, status, public_funs))
]
end)
Enum.intersperse([head | tail], "\n - ") |> to_string()
end
defp format_fun(module, {fun, arity}, status, public_funs) do
if fun in public_funs or module in [Kernel, Kernel.SpecialForms] do
[
?[,
maybe_strike(status),
?`,
Atom.to_string(fun),
?`,
maybe_strike(status),
"](`",
inspect(module),
?.,
to_string(fun),
?/,
to_string(arity),
"`)"
]
else
[
maybe_strike(status),
?`,
Atom.to_string(fun),
?`,
maybe_strike(status)
]
end
end
defp maybe_strike(:restricted), do: "~~"
defp maybe_strike(_status), do: []
defp format_status(:allowed), do: "Allowed"
defp format_status(:shimmed), do: "Alernative implementation"
defp format_status(:restricted), do: "Restricted"
end
Dune.Allowlist.Docs.public_functions(:rand)
================================================
FILE: lib/dune/allowlist/spec.ex
================================================
defmodule Dune.Allowlist.Spec do
@moduledoc false
alias Dune.Allowlist
alias Dune.Parser.RealModule
@type t :: %__MODULE__{
modules: %{optional(module) => [{atom, non_neg_integer, Allowlist.status()}]}
}
@enforce_keys [:modules]
defstruct @enforce_keys
def new do
%__MODULE__{modules: %{}}
end
@spec list_fun_statuses(t) :: list({module, atom, Allowlist.status()})
def list_fun_statuses(%__MODULE__{modules: modules}) do
for {module, funs} <- modules, {fun_name, _arity, status} <- funs do
{module, fun_name, status}
end
|> Enum.sort()
|> Enum.dedup()
end
@spec list_ordered_modules(t) :: list({module, {atom, Allowlist.status()}})
def list_ordered_modules(%__MODULE__{modules: modules}) do
modules
|> Enum.map(fn {module, funs} ->
{module, group_funs_by_status(funs)}
end)
|> Enum.sort()
end
defp group_funs_by_status(funs) do
Enum.group_by(
funs,
fn {_fun, _arity, status} -> extract_status_atom(status) end,
fn {fun, arity, _status} -> {fun, arity} end
)
|> Enum.map(fn {status, funs} -> {status, Enum.sort(funs) |> Enum.dedup_by(&elem(&1, 0))} end)
|> Enum.sort_by(fn {status, _} -> status_sort(status) end)
end
defp extract_status_atom(:restricted), do: :restricted
defp extract_status_atom(:allowed), do: :allowed
defp extract_status_atom({:shimmed, _, _}), do: :shimmed
defp status_sort(:allowed), do: 1
defp status_sort(:shimmed), do: 2
defp status_sort(:restricted), do: 3
@spec add_new_module(t, module, :all) :: t
def add_new_module(%__MODULE__{modules: modules}, module, _status)
when :erlang.map_get(module, modules) != nil do
# TODO proper error type
raise "ModuleConflict: module #{inspect(module)} already defined"
end
def add_new_module(spec = %__MODULE__{modules: modules}, module, status) when is_atom(module) do
Code.ensure_compiled!(module)
functions =
RealModule.list_functions(module)
|> classify_functions(status)
%{spec | modules: Map.put(modules, module, functions)}
end
defp classify_functions(functions, :all) do
Enum.map(functions, fn {fun_name, arity} -> {fun_name, arity, :allowed} end)
end
defp classify_functions(functions, only: only) when is_list(only) do
do_classify(functions, only, :allowed, :restricted)
end
defp classify_functions(functions, except: except) when is_list(except) do
do_classify(functions, except, :restricted, :allowed)
end
defp classify_functions(functions, opts) do
case Keyword.pop(opts, :shims) do
{shims, remaining_opts} when is_list(shims) ->
new_opts =
case remaining_opts do
[] -> :all
other -> other
end
classify_functions(functions, new_opts) |> shim_functions(shims)
{nil, _} ->
raise "Invalid opts #{inspect(opts)}"
end
end
defp do_classify(functions, set_list, member_atom, non_member_atom) do
set = to_atom_set(set_list)
functions
|> Enum.map_reduce(set, fn {fun_name, arity}, acc ->
case set do
%{^fun_name => _} -> {{fun_name, arity, member_atom}, Map.delete(acc, fun_name)}
_ -> {{fun_name, arity, non_member_atom}, acc}
end
end)
|> unwrap_classify()
end
defp to_atom_set(list) do
Enum.each(list, fn atom when is_atom(atom) -> :ok end)
:maps.from_keys(list, nil)
end
defp unwrap_classify({result, remaining}) when remaining == %{}, do: result
defp unwrap_classify({_, remaining}) do
[{key, _}] = Enum.take(remaining, 1)
raise "Unknown function #{key}"
end
defp shim_functions(functions, shims) do
# TODO validate shims
Enum.map(functions, fn fun = {fun_name, arity, _status} ->
case Keyword.get(shims, fun_name) do
nil ->
fun
{shim_module, shim_fun_name} ->
validate_shim!(shim_module, shim_fun_name, arity + 1)
{fun_name, arity, {:shimmed, shim_module, shim_fun_name}}
end
end)
end
defp validate_shim!(module, fun_name, arity) do
Code.ensure_compiled!(module)
unless function_exported?(module, fun_name, arity) or macro_exported?(module, fun_name, arity) do
raise "Invalid shim: function #{inspect(module)}.#{fun_name}/#{arity} doesn't exist!"
end
end
end
================================================
FILE: lib/dune/allowlist.ex
================================================
defmodule Dune.Allowlist do
@moduledoc """
Behaviour to customize the modules and functions that are allowed or restricted.
## Warning: security considerations
The default implementation is `Dune.Allowlist.Default`, and should only allow safe
functions: no atom leaks, no execution of arbitrary code, no access to the filesystem / network...
Defining or extending a custom `Dune.Allowlist` module can introduce security risks or bugs.
Please also note that using custom allowlists is still **experimental** and the API for it
might change faster than the rest of the library.
## Defining a new allowlist
In order to define a custom allowlist from scratch, `use Dune.Allowlist` can be used:
defmodule CustomAllowlist do
use Dune.Allowlist
allow Kernel, only: [:+, :*, :-, :/, :div, :rem]
end
Dune.eval_string("4 + 9", allowlist: CustomAllowlist)
## Extending an existing allowlist
Defining an allowlist from scratch can be both daunting and risky.
It is possible to extend an exisiting allowlist instead using the `extend` option:
defmodule ExtendedAllowlist do
use Dune.Allowlist, extend: Dune.Allowlist.Default
allow SomeModule, only: [:authorized]
end
Dune.eval_string("SomeModule.authorized(123)", allowlist: ExtendedAllowlist)
Note: currently, it is not possible to add or restrict functions from modules
that have already been specified.
## Documentation generation
The list of modules and functions with their status can be generated in the `@moduledoc`.
An example can be found in the `Dune.Allowlist.Default` documentation.
If the `__DUNE_ALLOWLIST_FUNCTIONS__` string is found in the `@moduledoc` string,
it will be replaced.
defmodule CustomAllowlist do
@moduledoc \"\"\"
Only allows simple arithmetic
## Allowlist functions
__DUNE_ALLOWLIST_FUNCTIONS__
\"\"\"
use Dune.Allowlist
allow Kernel, only: [:+, :*, :-, :/, :div, :rem]
end
"""
@type status :: :allowed | :restricted | {:shimmed, module, atom}
@doc """
Returns the trust status of a function or macro, specified as a `module`, `fun_name` and `arity` (`mfa`):
- `:allowed` if can be safely use
- `:restricted` if its usage should be forbidden
- a `{:shimmed, module, function_name}` if the function call should be replaced with an alternative implementation
"""
@callback fun_status(module, atom, non_neg_integer) :: Dune.Allowlist.status()
@doc """
Validates the fact that a module implements the `Dune.Allowlist` behaviour.
Raises if not the case.
## Examples
iex> Dune.Allowlist.ensure_implements_behaviour!(DoesNotExists)
** (ArgumentError) could not load module DoesNotExists due to reason :nofile
iex> Dune.Allowlist.ensure_implements_behaviour!(List)
** (ArgumentError) List does not implement the Dune.Allowlist behaviour
"""
@spec ensure_implements_behaviour!(module) :: module
def ensure_implements_behaviour!(module) when is_atom(module) do
Code.ensure_compiled!(module)
implemented? =
module.module_info(:attributes)
|> Keyword.get(:behaviour, [])
|> Enum.member?(Dune.Allowlist)
unless implemented? do
raise ArgumentError,
message: "#{inspect(module)} does not implement the Dune.Allowlist behaviour"
end
module
end
defmacro __using__(opts) do
extend = extract_extend_opt(opts, __CALLER__)
quote do
import Dune.Allowlist, only: [allow: 2]
@behaviour Dune.Allowlist
Module.register_attribute(__MODULE__, :allowlist, accumulate: true)
Module.put_attribute(__MODULE__, :extend_allowlist, unquote(extend))
@before_compile Dune.Allowlist
end
end
@doc """
Adds a new module to the allowlist and specifices which functions to use.
The module must not be already specified in the allowlist.
Must be called after `use Dune.Allowlist`.
## Examples
# allow all functions in a module
allow Time, :all
# only allow specific functions
allow Function, only: [:identity]
# exclude specific functions
allow Calendar, except: [:put_time_zone_database]
Note: `only` and `except` will cover all arities if several functions
share a name.
"""
defmacro allow(module, status) do
quote do
Module.put_attribute(__MODULE__, :allowlist, {unquote(module), unquote(status)})
end
end
defmacro __before_compile__(env) do
Dune.Allowlist.__postprocess__(env.module)
end
defp extract_extend_opt(opts, caller) do
case Keyword.fetch(opts, :extend) do
{:ok, module_ast} ->
Macro.expand(module_ast, caller) |> ensure_implements_behaviour!()
_ ->
nil
end
end
@doc false
def __postprocess__(module) do
extend = Module.get_attribute(module, :extend_allowlist)
spec = generate_spec(module, extend)
update_module_doc(module, spec)
quote do
unquote(def_spec(spec))
unquote(def_fun_status(spec))
@on_load :ensure_alias_atoms
# Aliases like Foo.Bar are represented on the AST level as {:alias, _, [:Foo, :Bar]}
# We need to force the creation of these atoms, which might otherwise not be
# available when we parse due to atom encoding logic.
def ensure_alias_atoms do
# do nothing and returns :ok
Enum.each(unquote(alias_atoms(spec)), & &1)
end
end
end
defp generate_spec(module, extend) do
base_spec =
case extend do
nil -> Dune.Allowlist.Spec.new()
allowlist when is_atom(allowlist) -> allowlist.spec()
end
Module.get_attribute(module, :allowlist)
|> Enum.reduce(base_spec, fn {module, status}, acc ->
Dune.Allowlist.Spec.add_new_module(acc, module, status)
end)
end
defp def_spec(spec) do
quote do
@doc false
@spec spec :: Dune.Allowlist.Spec.t()
def spec do
unquote(Macro.escape(spec))
end
end
end
defp def_fun_status(spec) do
defps =
for {m, f, status} = _ <- Dune.Allowlist.Spec.list_fun_statuses(spec) do
quote do
defp do_fun_status(unquote(m), unquote(f)),
do: unquote(Macro.escape(status))
end
end
quote do
@impl Dune.Allowlist
@doc "Implements `c:Dune.Allowlist.fun_status/3`"
def fun_status(module, fun_name, arity)
when is_atom(module) and is_atom(fun_name) and is_integer(arity) and arity >= 0 do
with :defined <- Dune.Parser.RealModule.fun_status(module, fun_name, arity) do
do_fun_status(module, fun_name)
end
end
unquote(defps)
defp do_fun_status(_module, _fun_name), do: :restricted
end
end
defp update_module_doc(module, spec) do
case Module.get_attribute(module, :moduledoc) do
{line, doc} when is_binary(doc) ->
doc =
String.replace(doc, "__DUNE_ALLOWLIST_FUNCTIONS__", fn _ ->
Dune.Allowlist.Docs.document_allowlist(spec)
end)
Module.put_attribute(module, :moduledoc, {line, doc})
_other ->
:ok
end
end
defp alias_atoms(spec) do
spec.modules
|> Enum.flat_map(fn {mod, _} ->
try do
Module.split(mod)
rescue
ArgumentError ->
# "expected an Elixir module" error -> erlang module
[]
end
end)
|> Enum.map(&String.to_atom/1)
|> Enum.uniq()
end
end
================================================
FILE: lib/dune/atom_mapping.ex
================================================
defmodule Dune.AtomMapping do
@moduledoc false
alias Dune.{Success, Failure}
@type substitute_atom :: atom
@type original_string :: String.t()
@type sub_mapping :: %{optional(substitute_atom) => original_string}
@type extra_info :: %{optional(substitute_atom) => :wrapped}
@typedoc """
Should be considered opaque
"""
@type t :: %__MODULE__{atoms: sub_mapping, modules: sub_mapping, extra_info: extra_info}
@enforce_keys [:atoms, :modules, :extra_info]
defstruct @enforce_keys
@spec new :: t()
def new do
%__MODULE__{atoms: %{}, modules: %{}, extra_info: %{}}
end
@spec from_atoms([{substitute_atom, original_string}], [{substitute_atom, :wrapped}]) :: t
def from_atoms(list, extra_info) when is_list(list) do
atoms = build_mapping(list)
extra_info = Map.new(extra_info)
%__MODULE__{atoms: atoms, modules: %{}, extra_info: extra_info}
end
@spec add_modules(t, [{substitute_atom, original_string}]) :: t
def add_modules(mapping = %__MODULE__{}, list) do
%{mapping | modules: build_mapping(list)}
end
defp build_mapping(list) do
Map.new(list, fn {substitute_atom, original_string}
when is_atom(substitute_atom) and is_binary(original_string) ->
{substitute_atom, original_string}
end)
end
@spec to_string(t, atom) :: String.t()
def to_string(mapping = %__MODULE__{}, atom) when is_atom(atom) do
case lookup_original_string(mapping, atom) do
{:atom, string} -> string
{:wrapped_atom, string} -> string
{:module, string} -> "Elixir.#{string}"
:error -> Atom.to_string(atom)
end
end
@spec inspect(t, atom) :: String.t()
def inspect(mapping = %__MODULE__{}, atom) when is_atom(atom) do
case lookup_original_string(mapping, atom) do
{:atom, string} -> ":#{string}"
{:wrapped_atom, string} -> ~s(:"#{string}")
{:module, string} -> string
:error -> inspect(atom)
end
end
@spec lookup_original_string(t, atom) :: {:atom | :wrapped_atom | :module, String.t()} | :error
def lookup_original_string(mapping = %__MODULE__{}, atom) when is_atom(atom) do
case mapping.modules do
%{^atom => string} ->
{:module, string}
_ ->
case mapping.atoms do
%{^atom => string} ->
case mapping.extra_info do
%{^atom => :wrapped} -> {:wrapped_atom, string}
_ -> {:atom, string}
end
_ ->
:error
end
end
end
@spec replace_in_string(t, String.t()) :: String.t()
def replace_in_string(mapping, string) when is_binary(string) do
if string =~ "Dune" do
do_replace_in_string(mapping, string)
else
string
end
end
@dune_atom_regex ~r/(Dune_Atom_\d+__|a?__Dune_atom_\d+__|Dune_Module_\d+__)/
defp do_replace_in_string(mapping = %__MODULE__{}, string) do
string_replace_map =
%{}
|> build_replace_map(mapping.modules, nil, &inspect/1)
|> build_replace_map(mapping.atoms, mapping.extra_info, &Atom.to_string/1)
String.replace(string, @dune_atom_regex, &Map.get(string_replace_map, &1, &1))
end
defp build_replace_map(map, sub_mapping, extra_info, to_string_fun) do
for {subsitute_atom, original_string} <- sub_mapping, into: map do
replace_by =
case extra_info do
%{^subsitute_atom => :wrapped} -> ~s("#{original_string}")
_ -> original_string
end
{to_string_fun.(subsitute_atom), replace_by}
end
end
@spec replace_in_result(t, Success.t() | Failure.t()) ::
Success.t() | Failure.t()
def replace_in_result(mapping, result)
def replace_in_result(mapping, %Success{} = success) do
%Success{
success
| inspected: replace_in_string(mapping, success.inspected),
stdio: replace_in_string(mapping, success.stdio)
}
end
def replace_in_result(mapping, %Failure{} = error) do
%Failure{
error
| message: replace_in_string(mapping, error.message),
stdio: replace_in_string(mapping, error.stdio)
}
end
@spec to_existing_atom(t, String.t()) :: atom
def to_existing_atom(mapping = %__MODULE__{}, string) when is_binary(string) do
case fetch_existing_atom(mapping, string) do
nil -> String.to_existing_atom(string)
atom -> atom
end
end
defp fetch_existing_atom(mapping, "Elixir." <> module_name) do
Enum.find_value(mapping.modules, fn {subsitute_atom, original_string} ->
if original_string == module_name do
subsitute_atom
end
end)
end
defp fetch_existing_atom(mapping, atom_name) do
Enum.find_value(mapping.atoms, fn {subsitute_atom, original_string} ->
if original_string == atom_name do
subsitute_atom
end
end)
end
end
================================================
FILE: lib/dune/eval/env.ex
================================================
defmodule Dune.Eval.Env do
@moduledoc false
alias Dune.AtomMapping
alias Dune.Eval.FakeModule
@type t :: %__MODULE__{
atom_mapping: AtomMapping.t(),
allowlist: module,
fake_modules: %{optional(atom) => FakeModule.t()}
}
@enforce_keys [:atom_mapping, :allowlist, :fake_modules]
defstruct @enforce_keys
def new(atom_mapping = %AtomMapping{}, allowlist) when is_atom(allowlist) do
%__MODULE__{atom_mapping: atom_mapping, allowlist: allowlist, fake_modules: %{}}
end
def add_module(env = %__MODULE__{fake_modules: modules}, module_name, module = %FakeModule{})
when is_atom(module_name) do
# TODO check a bunch of things here:
# - warn if module redefined
# - fail if overriding existing module
# - fail if overriding Kernel/Special forms
%{env | fake_modules: Map.put(modules, module_name, module)}
end
def apply_fake(env = %__MODULE__{}, module, fun_name, args)
when is_atom(module) and is_atom(fun_name) and is_list(args) do
arity = length(args)
case fetch_fake_function(env, module, fun_name, arity) do
{:def, fun} -> fun.(env, args)
other -> throw({other, module, fun_name, arity})
end
end
defp fetch_fake_function(%{fake_modules: modules}, module, fun_name, arity) do
case modules do
%{^module => fake_module} ->
case FakeModule.get_function(fake_module, fun_name, arity) do
nil -> :undefined_function
{:def, fun} -> {:def, fun}
end
_ ->
:undefined_module
end
end
end
================================================
FILE: lib/dune/eval/fake_module.ex
================================================
defmodule Dune.Eval.FakeModule do
@moduledoc false
@type t :: %__MODULE__{
public_funs: %{optional(atom) => %{required(non_neg_integer) => function}}
}
@enforce_keys [:public_funs]
defstruct @enforce_keys
def get_function(%__MODULE__{public_funs: funs}, fun_name, arity)
when is_atom(fun_name) and is_integer(arity) do
case funs do
%{^fun_name => %{^arity => fun}} -> {:def, fun}
_ -> nil
end
end
end
================================================
FILE: lib/dune/eval/function_clause_error.ex
================================================
defmodule Dune.Eval.FunctionClauseError do
@moduledoc false
defexception [:module, :function, :args]
def message(err = %__MODULE__{function: function, args: args}) do
module = inspect(err.module)
arity = length(args)
args = inspect(args) |> String.slice(1..-2//1)
"no function clause matching in #{module}.#{function}/#{arity}: #{module}.#{function}(#{args})"
end
end
================================================
FILE: lib/dune/eval/macro_env.ex
================================================
defmodule Dune.Eval.MacroEnv do
@moduledoc false
# Recommended way to generate a Macro.Env struct
# https://hexdocs.pm/elixir/main/Macro.Env.html
def make_env do
import Dune.Shims.Kernel, only: [safe_sigil_w: 3, safe_sigil_W: 3], warn: false
%Macro.Env{__ENV__ | file: "nofile", module: nil, function: nil, line: 1}
end
end
================================================
FILE: lib/dune/eval/process.ex
================================================
defmodule Dune.Eval.Process do
@moduledoc false
alias Dune.Failure
alias Dune.Helpers.Diagnostics
def run(fun, opts = %Dune.Opts{}) when is_function(fun, 0) do
with_string_io(fn string_io ->
do_run(fun, opts, string_io)
end)
end
defp do_run(fun, opts, string_io) do
task =
Task.async(fn ->
# spawn within a task to avoid trapping exits in the caller
spawn_trapped_process(
fun,
opts.max_heap_size,
opts.max_reductions,
string_io
)
end)
result =
case Task.yield(task, opts.timeout) || Task.shutdown(task) do
{:ok, result} ->
result
nil ->
%Failure{type: :timeout, message: "Execution timeout - #{opts.timeout}ms"}
end
case result do
%Failure{type: :compile_error} -> result
_ -> %{result | stdio: result.stdio <> StringIO.flush(string_io)}
end
end
defp with_string_io(fun) do
{:ok, string_io} = StringIO.open("")
try do
fun.(string_io)
after
StringIO.close(string_io)
end
end
# returns a Dune.Failure struct or exits
defp spawn_trapped_process(fun, max_heap_size, max_reductions, string_io) do
report_to = self()
Process.flag(:trap_exit, true)
# unlike plain spawn / Process.spawn, proc_lib doesn't trigger the logger:
# | Unlike in "plain Erlang", proc_lib processes will not generate error reports,
# | which are written to the terminal by the emulator. All exceptions are converted
# | to exits which are ignored by the default logger handler.
opts = [
:link,
priority: :low,
max_heap_size: %{size: max_heap_size, kill: true, error_logger: false}
]
pid =
:proc_lib.spawn_opt(
fn ->
Process.group_leader(self(), string_io)
fun
|> catch_diagnostics()
|> then(&send(report_to, &1))
end,
opts
)
spawn(fn -> check_max_reductions(pid, report_to, max_reductions) end)
receive do
{:ok, result, diagnostics} ->
Diagnostics.prepend_diagnostics(result, diagnostics)
{:compile_error, error, diagnostics, stacktrace} ->
format_compile_error(error, diagnostics, stacktrace)
{:EXIT, ^pid, reason} ->
case reason do
:normal ->
exit(:normal)
:killed ->
%Failure{type: :memory, message: "Execution stopped - memory limit exceeded"}
{error, stacktrace} ->
format_error(error, stacktrace)
end
{:EXIT, _other_pid, reason} ->
# avoid the process to become immune to parent death
exit(reason)
{:reductions_exceeded, _reductions} ->
%Failure{type: :reductions, message: "Execution stopped - reductions limit exceeded"}
end
end
defp catch_diagnostics(fun) do
{result, diagnostics} =
Diagnostics.with_diagnostics_polyfill(fn ->
try do
{:ok, fun.()}
rescue
err in CompileError ->
{err, __STACKTRACE__}
end
end)
case result do
{:ok, value} ->
{:ok, value, diagnostics}
{%CompileError{} = err, stacktrace} ->
{:compile_error, err, diagnostics, stacktrace}
end
end
defp check_max_reductions(pid, report_to, max_reductions) when is_integer(max_reductions) do
# approach inspired from luerl
# https://github.com/rvirding/luerl/blob/develop/src/luerl_sandbox.erl
case Process.info(pid, :reductions) do
nil ->
:ok
{:reductions, reductions} when reductions > max_reductions ->
# if send immediately, might arrive before an EXIT signal
Process.send_after(report_to, {:reductions_exceeded, reductions}, 1)
{:reductions, _reductions} ->
check_max_reductions(pid, report_to, max_reductions)
end
end
defp format_error(error, stacktrace)
defp format_error({:nocatch, value}, _stacktrace) do
case value do
{:undefined_module, module, fun, arity} -> Failure.undefined_module(module, fun, arity)
{:undefined_function, module, fun, arity} -> Failure.undefined_function(module, fun, arity)
{:safe_throw, thrown} -> %Failure{type: :throw, message: "** (throw) " <> inspect(thrown)}
end
end
defp format_error(error, stacktrace) do
[head | _] = stacktrace
parts =
case head do
{:erl_eval, :do_apply, _, _} -> 3
{:elixir_eval, :__FILE__, _, _} -> 3
_ -> 2
end
message =
{error, [head]}
|> Exception.format_exit()
|> String.split("\n ", parts: parts)
|> Enum.at(1)
# TODO properly pass stacktrace
%Failure{type: :exception, message: message}
end
defp format_compile_error(error, diagnostics, stacktrace) do
message =
{error, stacktrace}
|> Exception.format_exit()
|> String.split("\n ")
|> Enum.at(1)
%Failure{
type: :compile_error,
message: message,
stdio: Diagnostics.format_diagnostics(diagnostics)
}
end
end
================================================
FILE: lib/dune/eval.ex
================================================
defmodule Dune.Eval do
@moduledoc false
alias Dune.{AtomMapping, Success, Failure, Opts}
alias Dune.Eval.Env
alias Dune.Eval.MacroEnv
alias Dune.Parser.SafeAst
alias Dune.Shims
@typep previous_session :: %{:bindings => keyword, :env => Env.t(), optional(any) => any}
@spec run(SafeAst.t() | Failure.t(), Opts.t(), previous_session | nil) ::
Success.t() | Failure.t()
def run(parsed, opts, previous_session \\ nil)
def run(
%SafeAst{
ast: ast,
atom_mapping: atom_mapping,
compile_env: %{allowlist: allowlist},
stdio: parser_stdio
},
opts = %Opts{},
previous_session
) do
case previous_session do
nil ->
env = Env.new(atom_mapping, allowlist)
do_run(ast, atom_mapping, opts, env, nil)
%{bindings: bindings, env: env} ->
env = %{env | atom_mapping: atom_mapping, allowlist: allowlist}
do_run(ast, atom_mapping, opts, env, bindings)
end
|> prepend_parser_stdio(parser_stdio)
end
def run(%Failure{} = failure, _opts, _bindings), do: failure
defp do_run(ast, atom_mapping, opts, env, bindings) do
result =
Dune.Eval.Process.run(
fn ->
safe_eval(ast, env, bindings, opts.pretty, opts.inspect_sort_maps)
end,
opts
)
AtomMapping.replace_in_result(atom_mapping, result)
end
defp prepend_parser_stdio(result, ""), do: result
defp prepend_parser_stdio(result, parser_stdio) do
Map.update!(result, :stdio, &(parser_stdio <> &1))
end
defp safe_eval(safe_ast, env, bindings, pretty, sort_maps) do
try do
inspect_opts = [pretty: pretty, custom_options: [sort_maps: sort_maps]]
do_safe_eval(safe_ast, env, bindings, inspect_opts)
catch
failure = %Failure{} ->
failure
end
end
defp do_safe_eval(safe_ast, env, nil, inspect_opts) do
binding = [env__Dune__: env]
{value, new_env, _new_bindings} = eval_quoted(safe_ast, binding)
%Success{
value: value,
# another important thing about inspect is that it force-evaluates
# potentially huge shared structs => OOM before sending
inspected: Shims.Kernel.safe_inspect(new_env, value, inspect_opts),
stdio: ""
}
end
defp do_safe_eval(safe_ast, env, bindings, inspect_opts) when is_list(bindings) do
binding = [env__Dune__: env] ++ bindings
{value, new_env, new_bindings} = eval_quoted(safe_ast, binding)
%Success{
value: {value, new_env, new_bindings},
inspected: Shims.Kernel.safe_inspect(new_env, value, inspect_opts),
stdio: ""
}
end
defp eval_quoted(safe_ast, binding) do
{value, bindings, _env} = Code.eval_quoted_with_env(safe_ast, binding, MacroEnv.make_env())
{new_env, new_bindings} = Keyword.pop!(bindings, :env__Dune__)
{value, new_env, new_bindings}
end
end
================================================
FILE: lib/dune/failure.ex
================================================
defmodule Dune.Failure do
@moduledoc """
A struct returned when `Dune` parsing or evaluation fails.
Fields:
- `message` (string): the error message to display to the user
- `type` (atom): the nature of the error
- `stdio` (string): captured standard output
"""
@type error_type ::
:restricted
| :module_restricted
| :module_conflict
| :timeout
| :exception
| :compile_error
| :parsing
| :memory
| :reductions
@type t :: %__MODULE__{type: error_type, message: String.t(), stdio: binary}
@enforce_keys [:type, :message]
defstruct @enforce_keys ++ [stdio: ""]
@doc false
def restricted_function(module, fun, arity) do
formatted_fun = format_function(module, fun, arity)
message = "** (DuneRestrictedError) function #{formatted_fun} is restricted"
%__MODULE__{type: :restricted, message: message}
end
@doc false
def undefined_module(module, function, arity) do
base_message = base_undefined_message(module, function, arity)
message =
IO.iodata_to_binary([base_message, "(module ", inspect(module), " is not available)"])
%__MODULE__{type: :exception, message: message}
end
@doc false
def undefined_function(module, function, arity) do
base_message = base_undefined_message(module, function, arity)
message = IO.iodata_to_binary([base_message, "or private"])
%__MODULE__{type: :exception, message: message}
end
defp base_undefined_message(module, function, arity) do
formatted_fun = format_function(module, function, arity)
["** (UndefinedFunctionError) function ", formatted_fun, " is undefined "]
end
defp format_function(kernel, fun, arity) when kernel in [nil, Kernel, Kernel.SpecialForms] do
"#{fun}/#{arity}"
end
defp format_function(module, fun, arity) do
"#{inspect(module)}.#{fun}/#{arity}"
end
end
================================================
FILE: lib/dune/helpers/diagnostics.ex
================================================
defmodule Dune.Helpers.Diagnostics do
@moduledoc false
# used for formatting errors and warnings consistently
@type result_with_stdio :: %{stdio: binary()}
@spec prepend_diagnostics(
result_with_stdio(),
[Code.diagnostic(:warning | :error)]
) :: result_with_stdio()
def prepend_diagnostics(result, []), do: result
def prepend_diagnostics(result, diagnostics) do
%{result | stdio: format_diagnostics(diagnostics) <> "\n\n"}
end
@spec format_diagnostics([Code.diagnostic(:warning | :error)]) :: String.t()
def format_diagnostics(diagnostics) do
Enum.map_join(
diagnostics,
"\n",
&"#{&1.severity}: #{&1.message}\n #{&1.file}:#{format_pos(&1.position)}"
)
end
defp format_pos(integer) when is_integer(integer), do: Integer.to_string(integer)
defp format_pos({line, col}), do: [Integer.to_string(line), ?:, Integer.to_string(col)]
@doc """
A polyfill for `Code.with_diagnostics/1` for older versions of Elixir, which returns
an empty list of diagnostics if not available.
"""
# TODO remove then dropping support for 1.14
if System.version() |> Version.compare("1.15.0") != :lt do
defdelegate with_diagnostics_polyfill(fun), to: Code, as: :with_diagnostics
else
def with_diagnostics_polyfill(fun) do
{fun.(), []}
end
end
end
================================================
FILE: lib/dune/helpers/term_checker.ex
================================================
defmodule Dune.Helpers.TermChecker do
@moduledoc false
defguardp is_simple_term(term)
when is_atom(term) or is_bitstring(term) or is_number(term) or is_reference(term) or
is_function(term) or is_pid(term) or is_port(term) or term == []
@doc """
Walks the term recursively to make sure it is not a humongous tree built using structural sharing
"""
def check(term), do: do_check(term)
defp do_check(term) when is_simple_term(term), do: :ok
defp do_check([left | right]) when is_simple_term(left) do
do_check(right)
end
defp do_check([left | right]) do
do_check(left)
do_check(right)
end
defp do_check(map) when is_map(map) do
Map.to_list(map) |> do_check()
end
defp do_check(tuple) when is_tuple(tuple) do
Tuple.to_list(tuple) |> do_check()
end
end
================================================
FILE: lib/dune/opts.ex
================================================
defmodule Dune.Opts do
@moduledoc """
Defines and validates the options for `Dune`.
The available options are explained below:
### Parsing restriction options
- `atom_pool_size`:
Defines the maximum total number of atoms that can be created.
Must be an integer `>= 0`. Defaults to `5000`.
See the [section below](#module-extra-note-about-atom_pool_size) for more information.
- `max_length`:
Defines the maximum length of code strings that can be parsed.
Defaults to `5000`.
### Execution restriction options
- `allowlist`:
Defines which module and functions are considered safe or restricted.
Should be a module implementing the `Dune.Allowlist` behaviour.
Defaults to `Dune.Allowlist.Default`.
- `max_heap_size`:
Limits the memory usage of the evaluation process using the
[`max_heap_size` flag](https://erlang.org/doc/man/erlang.html#process_flag_max_heap_size).
Should be an integer `> 0`. Defaults to `30_000`.
- `max_reductions`:
Limits the number of CPU cycles of the evaluation process.
The erlang pre-emptive scheduler is using reductions to measure work being done by processes,
which is useful to prevent users to run CPU intensive code such as infinite loops.
Should be an integer `> 0`. Defaults to `30_000`.
- `timeout`:
Limits the time the evaluation process is authorized to run (in milliseconds).
Should be an integer `> 0`. Defaults to `50`.
The evaluation process will still need to parse and execute the sanitized AST, so using
too low limits here would leave only a small margin to actually run user code.
### Other options
- `pretty`:
Use pretty printing when inspecting the result.
Should be a boolean. Defaults to `false`.
- `inspect_sort_maps`:
Sort maps when inspecting the result, useful to keep the output deterministic.
Should be a boolean. Defaults to `false`. Only works since Elixir >= 1.14.4.
### Extra note about `atom_pool_size`
Atoms are reused from one evaluation to the other so the total is not
expected to grow. Atoms will not be leaked.
Also, the atom pool is actually split into several pools: regular atoms, module names,
unused variable names, ...
So defining a value of `100` does not mean that `100` atoms will be available, but
rather `25` of each type.
Atoms being very lightweight, there is no need to use a low value, as long
as there is an upper bound preventing atom leaks.
"""
alias Dune.Allowlist
@type t :: %__MODULE__{
atom_pool_size: non_neg_integer,
max_length: pos_integer,
allowlist: module,
max_heap_size: pos_integer,
max_reductions: pos_integer,
timeout: pos_integer,
pretty: boolean,
inspect_sort_maps: boolean
}
defstruct atom_pool_size: 5000,
max_length: 5000,
allowlist: Dune.Allowlist.Default,
max_heap_size: 50_000,
max_reductions: 30_000,
timeout: 50,
pretty: false,
inspect_sort_maps: false
@doc """
Validates untrusted options from a keyword or a map and returns a `Dune.Opts` struct.
## Examples
iex> Dune.Opts.validate!([])
%Dune.Opts{
allowlist: Dune.Allowlist.Default,
atom_pool_size: 5000,
max_heap_size: 50000,
max_length: 5000,
max_reductions: 30000,
pretty: false,
timeout: 50
}
iex> Dune.Opts.validate!(atom_pool_size: 10)
%Dune.Opts{atom_pool_size: 10, allowlist: Dune.Allowlist.Default}
iex> Dune.Opts.validate!(atom_pool_size: -10)
** (ArgumentError) atom_pool_size should be an integer >= 0
iex> Dune.Opts.validate!(max_length: 0)
** (ArgumentError) atom_pool_size should be an integer > 0
iex> Dune.Opts.validate!(allowlist: DoesNotExists)
** (ArgumentError) could not load module DoesNotExists due to reason :nofile
iex> Dune.Opts.validate!(allowlist: List)
** (ArgumentError) List does not implement the Dune.Allowlist behaviour
iex> Dune.Opts.validate!(max_reductions: 10_000, max_heap_size: 10_000, timeout: 20)
%Dune.Opts{max_heap_size: 10_000, max_reductions: 10_000, timeout: 20}
iex> Dune.Opts.validate!(max_heap_size: 0)
** (ArgumentError) max_heap_size should be an integer > 0
iex> Dune.Opts.validate!(max_reductions: 0)
** (ArgumentError) max_reductions should be an integer > 0
iex> Dune.Opts.validate!(timeout: "55")
** (ArgumentError) timeout should be an integer > 0
iex> Dune.Opts.validate!(pretty: :maybe)
** (ArgumentError) pretty should be a boolean
"""
@spec validate!(Keyword.t() | map) :: t
def validate!(opts) do
struct(__MODULE__, opts) |> do_validate()
end
defp do_validate(%{atom_pool_size: atom_pool_size})
when not (is_integer(atom_pool_size) and atom_pool_size >= 0) do
raise ArgumentError, message: "atom_pool_size should be an integer >= 0"
end
defp do_validate(%{max_length: max_length})
when not (is_integer(max_length) and max_length > 0) do
raise ArgumentError, message: "atom_pool_size should be an integer > 0"
end
defp do_validate(%{allowlist: allowlist}) when not is_atom(allowlist) do
raise ArgumentError, message: "allowlist should be a module"
end
defp do_validate(%{max_reductions: max_reductions})
when not (is_integer(max_reductions) and max_reductions > 0) do
raise ArgumentError, message: "max_reductions should be an integer > 0"
end
defp do_validate(%{max_heap_size: max_heap_size})
when not (is_integer(max_heap_size) and max_heap_size > 0) do
raise ArgumentError, message: "max_heap_size should be an integer > 0"
end
defp do_validate(%{timeout: timeout}) when not (is_integer(timeout) and timeout > 0) do
raise ArgumentError, message: "timeout should be an integer > 0"
end
defp do_validate(%{pretty: pretty}) when not is_boolean(pretty) do
raise ArgumentError, message: "pretty should be a boolean"
end
defp do_validate(%{inspect_sort_maps: sort}) when not is_boolean(sort) do
raise ArgumentError, message: "inspect_sort_maps should be a boolean"
end
defp do_validate(opts = %{allowlist: allowlist}) do
Allowlist.ensure_implements_behaviour!(allowlist)
opts
end
end
================================================
FILE: lib/dune/parser/atom_encoder.ex
================================================
defmodule Dune.Parser.AtomEncoder do
@moduledoc false
alias Dune.AtomMapping
@type atom_category :: :alias | :private_var | :public_var | :other
@atom_categories 4
# TODO Remove when dropping support for Elixir 1.16
extra_modules =
if System.version() |> Version.compare("1.17.0-rc.0") != :lt, do: [Duration], else: []
@elixir_modules [
Kernel,
Kernel.SpecialForms,
Atom,
Base,
Bitwise,
Date,
DateTime,
Duration,
Exception,
Float,
Function,
Integer,
Module,
NaiveDateTime,
Record,
Regex,
String,
Time,
Tuple,
URI,
Version,
Version.Requirement,
Access,
Date.Range,
Enum,
Keyword,
List,
Map,
MapSet,
Range,
Stream,
File,
File.Stat,
File.Stream,
IO,
IO.ANSI,
IO.Stream,
OptionParser,
Path,
Port,
StringIO,
System,
Calendar,
Calendar.ISO,
Calendar.TimeZoneDatabase,
Calendar.UTCOnlyTimeZoneDatabase,
Agent,
Application,
Config,
Config.Provider,
Config.Reader,
DynamicSupervisor,
GenServer,
Node,
Process,
Registry,
Supervisor,
Task,
Task.Supervisor,
Collectable,
Enumerable,
Inspect,
Inspect.Algebra,
Inspect.Opts,
List.Chars,
Protocol,
String.Chars,
Code,
Kernel.ParallelCompiler,
Macro,
Macro.Env,
Behaviour,
Dict,
GenEvent,
HashDict,
HashSet,
Set,
Supervisor.Spec,
ArgumentError,
ArithmeticError,
BadArityError,
BadBooleanError,
BadFunctionError,
BadMapError,
BadStructError,
CaseClauseError,
Code.LoadError,
CompileError,
CondClauseError,
Enum.EmptyError,
Enum.OutOfBoundsError,
ErlangError,
File.CopyError,
File.Error,
File.LinkError,
File.RenameError,
FunctionClauseError,
IO.StreamError,
Inspect.Error,
KeyError,
MatchError,
Module.Types.Error,
OptionParser.ParseError,
Protocol.UndefinedError,
Regex.CompileError,
RuntimeError,
SyntaxError,
SystemLimitError,
TokenMissingError,
TryClauseError,
DuneRestrictedError,
UnicodeConversionError,
Version.InvalidRequirementError,
Version.InvalidVersionError,
WithClauseError
] ++ extra_modules
@module_reprs @elixir_modules
|> Enum.flat_map(&Module.split/1)
|> Map.new(&{&1, String.to_existing_atom(&1)})
@spec load_atom_mapping(AtomMapping.t() | nil) :: :ok
def load_atom_mapping(nil), do: :ok
def load_atom_mapping(%AtomMapping{atoms: atoms}) do
count = Enum.count(atoms)
Process.put(:__Dune_atom_count__, count)
Enum.each(atoms, fn {atom, binary} ->
Process.put({:__Dune_atom__, binary}, atom)
end)
end
@spec static_atoms_encoder(String.t(), non_neg_integer()) :: {:ok, atom} | {:error, String.t()}
def static_atoms_encoder(binary, pool_size)
when is_binary(binary) and is_integer(pool_size) do
case @module_reprs do
%{^binary => atom} ->
{:ok, atom}
_ ->
if binary =~ "Dune" do
{:error, "Atoms containing `Dune` are restricted for safety"}
else
atom_category = categorize_atom_binary(binary)
do_static_atoms_encoder(binary, atom_category, pool_size)
end
end
end
@spec categorize_atom_binary(binary) :: atom_category
def categorize_atom_binary(atom_binary) do
charlist = String.to_charlist(atom_binary)
case {Code.Fragment.cursor_context(charlist), atom_binary} do
{{:alias, ^charlist}, _} -> :alias
{{:local_or_var, ^charlist}, "_" <> _} -> :private_var
{{:local_or_var, ^charlist}, _} -> :public_var
_ -> :other
end
end
defp do_static_atoms_encoder("Elixir." <> rest, :alias, pool_size) do
rest
|> String.split(".")
|> encode_many_atoms(pool_size, [])
end
defp do_static_atoms_encoder(binary, atom_category, pool_size) do
process_key = {:__Dune_atom__, binary}
case Process.get(process_key, nil) do
nil -> do_static_atoms_encoder(binary, atom_category, process_key, pool_size)
atom when is_atom(atom) -> {:ok, atom}
end
end
defp do_static_atoms_encoder(binary, atom_category, process_key, pool_size) do
{:ok, String.to_existing_atom(binary)}
rescue
ArgumentError ->
case new_atom(atom_category, pool_size) do
{:ok, atom} ->
Process.put(process_key, atom)
if atom_category == :other do
Process.put({:__Dune_atom_extra_info__, atom}, :wrapped)
end
{:ok, atom}
{:error, error} ->
{:error, error}
end
end
defp encode_many_atoms([], _pool_size, acc) do
{:ok, {:__aliases__, [], [Elixir | Enum.reverse(acc)]}}
end
defp encode_many_atoms([head | tail], pool_size, acc) do
case do_static_atoms_encoder(head, :alias, pool_size) do
{:ok, atom} -> encode_many_atoms(tail, pool_size, [atom | acc])
{:error, error} -> {:error, error}
end
end
@spec plain_atom_mapping :: AtomMapping.t()
def plain_atom_mapping() do
atoms =
for {{:__Dune_atom__, binary}, atom} <- Process.get() do
{atom, binary}
end
extra_info =
for {{:__Dune_atom_extra_info__, atom}, info} <- Process.get() do
{atom, info}
end
AtomMapping.from_atoms(atoms, extra_info)
end
defp new_atom(atom_category, pool_size) do
count = Process.get(:__Dune_atom_count__, 0) + 1
if count * @atom_categories > pool_size do
{:error, "atom_pool_size exceeded, failed to parse atom"}
else
Process.put(:__Dune_atom_count__, count)
atom = do_new_atom(atom_category, count)
{:ok, atom}
end
end
defp do_new_atom(:alias, count) do
:"Dune_Atom_#{count}__"
end
defp do_new_atom(:public_var, count) do
:"a__Dune_atom_#{count}__"
end
defp do_new_atom(category, count) when category in [:private_var, :other] do
:"__Dune_atom_#{count}__"
end
@spec encode_modules(Macro.t(), AtomMapping.t(), AtomMapping.t() | nil) ::
{Macro.t(), AtomMapping.t()}
def encode_modules(ast, plain_atom_mapping, existing_mapping) do
initial_acc = get_module_acc(existing_mapping)
{new_ast, acc} =
Macro.postwalk(ast, initial_acc, fn
{:__aliases__, ctx, atoms}, acc ->
{modules, new_acc} = remove_elixir_prefix(atoms) |> map_modules_ast(acc)
{{:__aliases__, ctx, modules}, new_acc}
other, acc ->
{other, acc}
end)
atom_mapping = build_module_mapping(acc, plain_atom_mapping)
{new_ast, atom_mapping}
end
defp get_module_acc(nil), do: %{}
defp get_module_acc(%AtomMapping{atoms: atoms, modules: modules}) do
reverse_atoms = Map.new(atoms, fn {atom, string} -> {string, atom} end)
Map.new(modules, fn {atom, string} ->
atoms = String.split(string, ".") |> Enum.map(&Map.fetch!(reverse_atoms, &1))
{atoms, atom}
end)
end
defp remove_elixir_prefix(atoms = [Elixir, Elixir | _]), do: atoms
defp remove_elixir_prefix([Elixir | atoms]) when atoms != [], do: atoms
defp remove_elixir_prefix(atoms), do: atoms
defp map_modules_ast(atoms, acc) do
case acc do
%{^atoms => module_name} ->
{[module_name], acc}
_ ->
try do
atoms |> Enum.join(".") |> then(&"Elixir.#{&1}") |> String.to_existing_atom()
rescue
ArgumentError ->
module_name = :"Dune_Module_#{map_size(acc) + 1}__"
{[module_name], Map.put(acc, atoms, module_name)}
else
_ ->
{atoms, acc}
end
end
end
defp build_module_mapping(acc, plain_atom_mapping) do
modules =
Enum.map(acc, fn {atoms, module_name} ->
string = Enum.map_join(atoms, ".", &AtomMapping.to_string(plain_atom_mapping, &1))
module = Module.concat([module_name])
{module, string}
end)
AtomMapping.add_modules(plain_atom_mapping, modules)
end
end
================================================
FILE: lib/dune/parser/compile_env.ex
================================================
defmodule Dune.Parser.CompileEnv do
@moduledoc false
@type name_arity :: {atom, non_neg_integer}
@type maybe_fake_module :: {:real | :fake, module}
@type t :: %__MODULE__{
module: module | nil,
allowlist: module,
fake_modules: %{optional(module) => %{optional(name_arity) => :def | :defp}}
# aliases
# struct info
# requires
}
@enforce_keys [:module, :allowlist, :fake_modules]
defstruct @enforce_keys
def new(allowlist) do
%__MODULE__{
allowlist: allowlist,
module: nil,
fake_modules: %{}
}
end
def define_fake_module(env = %__MODULE__{fake_modules: fake_modules}, module, name_arities)
when is_atom(module) and is_map(name_arities) do
if module_already_exists?(module, fake_modules) do
throw({:module_conflict, module})
end
new_modules = Map.put(fake_modules, module, name_arities)
%{env | fake_modules: new_modules}
end
defp module_already_exists?(module, fake_modules) do
case fake_modules do
%{^module => _conflict} -> true
_ -> Code.ensure_loaded?(module)
end
end
def resolve_mfa(%__MODULE__{}, module, fun_name, arity)
when module in [Kernel, nil] and fun_name in [:def, :defp] and arity in [1, 2] do
:outside_module
end
def resolve_mfa(env = %__MODULE__{}, module, fun_name, arity)
when is_atom(module) and is_atom(fun_name) and is_integer(arity) do
actual_module = resolve_module(module, fun_name, arity)
case env.allowlist.fun_status(actual_module, fun_name, arity) do
:undefined_module ->
resolve_fake_module(env, module, fun_name, arity)
:undefined_function ->
case module do
nil -> resolve_fake_module(env, nil, fun_name, arity)
_ -> :undefined_function
end
:restricted ->
{:restricted, actual_module}
other ->
other
end
end
defp resolve_module(nil, fun_name, arity) do
if Macro.special_form?(fun_name, arity) do
Kernel.SpecialForms
else
Kernel
end
end
defp resolve_module(module, _fun_name, _arity), do: module
defp resolve_fake_module(%{module: nil}, nil, _fun_name, _arity), do: :undefined_function
defp resolve_fake_module(env = %{module: module}, nil, fun_name, arity) do
resolve_fake_module(env, module, fun_name, arity)
end
defp resolve_fake_module(env, module, fun_name, arity) do
# TODO check current module to know if defp OK
fun_with_arity = {fun_name, arity}
case env.fake_modules do
%{^module => %{^fun_with_arity => def_or_defp}} -> check_private(env, module, def_or_defp)
_ -> :undefined_module
end
end
defp check_private(_env, module, :def), do: {:fake, module}
defp check_private(%{module: module}, module, :defp), do: {:fake, module}
defp check_private(_env, _module, _def), do: :undefined_function
end
================================================
FILE: lib/dune/parser/debug.ex
================================================
defmodule Dune.Parser.Debug do
@moduledoc false
def io_debug(ast) do
debug(ast) |> IO.puts()
ast
end
def debug(%{ast: ast}) when is_tuple(ast) do
ast_to_string(ast)
end
def debug(ast) when is_tuple(ast) do
ast_to_string(ast)
end
defp ast_to_string({:__block__, _, list}) do
Enum.map_join(list, "\n", &Macro.to_string/1)
end
defp ast_to_string(ast) do
Macro.to_string(ast)
end
end
================================================
FILE: lib/dune/parser/real_module.ex
================================================
defmodule Dune.Parser.RealModule do
@moduledoc false
@spec elixir_module?(module) :: boolean
def elixir_module?(module) do
module
|> Atom.to_string()
|> String.starts_with?("Elixir.")
end
def list_functions(module)
def list_functions(Kernel.SpecialForms) do
[{:%{}, 2}] ++ Kernel.SpecialForms.__info__(:macros)
end
@spec list_functions(module) :: [{atom, non_neg_integer}]
def list_functions(module) when is_atom(module) do
if elixir_module?(module) do
module.__info__(:functions) ++ module.__info__(:macros)
else
for {f, _a} = fa <- module.module_info(:exports), f != :module_info, do: fa
end
end
def fun_exists?(module, fun_name, arity) do
# TODO replace with fun_status
fun_status(module, fun_name, arity) == :defined
end
def fun_status(module, fun_name, arity)
def fun_status(Kernel.SpecialForms, fun_name, arity) do
if Macro.special_form?(fun_name, arity) do
:defined
else
:undefined_function
end
end
def fun_status(module, fun_name, arity) do
cond do
not Code.ensure_loaded?(module) -> :undefined_module
function_exported?(module, fun_name, arity) -> :defined
macro_exported?(module, fun_name, arity) -> :defined
true -> :undefined_function
end
end
end
================================================
FILE: lib/dune/parser/safe_ast.ex
================================================
defmodule Dune.Parser.SafeAst do
@moduledoc false
@type t :: %__MODULE__{
ast: Macro.t(),
atom_mapping: Dune.AtomMapping.t(),
compile_env: Dune.Parser.CompileEnv.t(),
stdio: binary()
}
@enforce_keys [:ast, :atom_mapping, :compile_env]
defstruct @enforce_keys ++ [stdio: <<>>]
end
================================================
FILE: lib/dune/parser/sanitizer.ex
================================================
defmodule Dune.Parser.Sanitizer do
@moduledoc false
alias Dune.{Failure, AtomMapping, Opts}
alias Dune.Parser.{CompileEnv, RealModule, UnsafeAst, SafeAst}
@env_variable_name :env__Dune__
@spec sanitize(UnsafeAst.t() | Failure.t(), Opts.t()) :: SafeAst.t() | Failure.t()
def sanitize(unsafe = %UnsafeAst{}, compile_env = %CompileEnv{}) do
case try_sanitize(unsafe.ast, compile_env) do
{:ok, safe_ast, new_env} ->
%SafeAst{
ast: safe_ast,
atom_mapping: unsafe.atom_mapping,
compile_env: new_env,
stdio: unsafe.stdio
}
{:restricted, module, fun, arity} ->
failure = Failure.restricted_function(module, fun, arity)
AtomMapping.replace_in_result(unsafe.atom_mapping, failure)
{:undefined_module, module, func_name, arity} ->
failure = Failure.undefined_module(module, func_name, arity)
AtomMapping.replace_in_result(unsafe.atom_mapping, failure)
{:undefined_function, module, func_name, arity} ->
failure = Failure.undefined_function(module, func_name, arity)
AtomMapping.replace_in_result(unsafe.atom_mapping, failure)
{:outside_module, def_or_defp} ->
message = "** (ArgumentError) cannot invoke #{def_or_defp}/2 inside function/macro"
new_failure(:exception, message, unsafe.atom_mapping)
{:module_invalid, module_ast, error_ctx} ->
line = Keyword.get(error_ctx, :line)
name = Macro.to_string(module_ast)
message = "** (Dune.Eval.CompileError) nofile:#{line}: invalid module name: #{name}"
new_failure(:exception, message, unsafe.atom_mapping)
{:module_restricted, ast} ->
message =
"** (DuneRestrictedError) the following syntax is restricted inside defmodule:\n #{Macro.to_string(ast)}"
new_failure(:module_restricted, message, unsafe.atom_mapping)
{:module_conflict, module} ->
message =
"** (DuneRestrictedError) Following module cannot be defined/redefined: #{inspect(module)}"
new_failure(:module_conflict, message, unsafe.atom_mapping)
{:definition_conflict, name_arity, previous_def, previous_ctx, conflict_def, conflict_ctx} ->
conflict_line = Keyword.get(conflict_ctx, :line)
previous_line = Keyword.get(previous_ctx, :line)
{name, arity} = name_arity
message =
"** (Dune.Eval.CompileError) nofile:#{conflict_line}: " <>
"#{conflict_def} #{name}/#{arity} already defined as #{previous_def} in nofile:#{previous_line}"
new_failure(:exception, message, unsafe.atom_mapping)
{:parsing_error, ast} ->
message = "dune parsing error: failed to safe parse\n #{Macro.to_string(ast)}"
new_failure(:parsing, message, unsafe.atom_mapping)
{:bin_modifier_restricted, ast} ->
message =
"** (DuneRestrictedError) bitstring modifier is restricted:\n #{Macro.to_string(ast)}"
new_failure(:restricted, message, unsafe.atom_mapping)
{:bin_modifier_size, max_size} ->
message = "** (DuneRestrictedError) size modifiers above #{max_size} are restricted"
new_failure(:restricted, message, unsafe.atom_mapping)
{:exception, error} ->
message = Exception.format(:error, error)
new_failure(:exception, message, unsafe.atom_mapping)
end
end
def sanitize(%Failure{} = failure, _opts), do: failure
defp new_failure(type, message, atom_mapping) when is_atom(type) and is_binary(message) do
failure = %Failure{type: type, message: message, stdio: ""}
AtomMapping.replace_in_result(atom_mapping, failure)
end
# XXX this is a bit hacky and brute-force approach!
# ideally the AST transformation is robust enough so we don't need it
defp try_sanitize(ast, env) do
do_sanitize_main(ast, env)
rescue
error ->
error
|> then(&Exception.blame(:error, &1, __STACKTRACE__))
|> elem(0)
|> then(&Exception.format(:error, &1))
|> IO.warn()
{:parsing_error, ast}
catch
thrown -> thrown
end
defp do_sanitize_main({:__block__, ctx, list}, env) do
{list_ast, env} = do_sanitize_main_list(list, env)
block_ast = {:__block__, ctx, list_ast}
{:ok, block_ast, env}
end
defp do_sanitize_main(single, env) do
case do_sanitize_main_list([single], env) do
{[safe_single], env} ->
{:ok, safe_single, env}
{list_ast, env} when is_list(list_ast) ->
block_ast = {:__block__, [], list_ast}
{:ok, block_ast, env}
end
end
defp do_sanitize_main_list(list, env) when is_list(list) do
{defmodules, instructions} = Enum.split_with(list, &defmodule_block?/1)
raw_fun_definitions = Enum.map(defmodules, &parse_module_definition/1)
env =
Enum.reduce(raw_fun_definitions, env, fn {module, fun_defs}, acc ->
fun_name_arities =
Map.new(fun_defs, fn {name_arity, [raw_definition | _]} ->
{name_arity, elem(raw_definition, 0)}
end)
CompileEnv.define_fake_module(acc, module, fun_name_arities)
end)
module_definitions = Enum.map(raw_fun_definitions, &sanitize_module_definition(&1, env))
sanitized_instructions =
case {raw_fun_definitions, do_sanitize(instructions, env)} do
{[_ | _], []} ->
{last_module, _} = List.last(raw_fun_definitions)
[quote(do: {:module, unquote(last_module), nil, nil})]
{_, sanitized_instructions} ->
sanitized_instructions
end
{module_definitions ++ sanitized_instructions, env}
end
defp defmodule_block?({:defmodule, _, _}), do: true
defp defmodule_block?(_), do: false
defp parse_module_definition({:defmodule, ctx, [module_name, [do: do_ast]]}) do
module_name
|> validate_module_name(ctx)
|> do_parse_module_definition(do_ast)
end
defp parse_module_definition(ast = {:defmodule, _, _}) do
throw({:parsing_error, ast})
end
defp validate_module_name(module, _ctx)
when is_atom(module) and module not in [nil, false, true] do
module
end
defp validate_module_name(module_def = {:__aliases__, _, [_module_atom]}, _ctx) do
Macro.expand_once(module_def, __ENV__)
end
defp validate_module_name(module_ast, ctx) do
throw({:module_invalid, module_ast, ctx})
end
defp do_parse_module_definition(module_name, do_ast) do
fun_definitions =
block_to_list(do_ast)
|> Enum.map(&parse_fun_definition/1)
|> Enum.filter(& &1)
|> Enum.flat_map(&expand_defaults/1)
|> Enum.group_by(&elem(&1, 0), &elem(&1, 1))
|> tap(&check_definition_conflicts/1)
{module_name, fun_definitions}
end
defp block_to_list({:__block__, _, list}) when is_list(list), do: list
defp block_to_list(single) when is_tuple(single), do: [single]
defp parse_fun_definition({def_or_defp, ctx, [signature, [do: body]]})
when def_or_defp in [:def, :defp] do
{header, guards} = parse_fun_signature(signature)
{name, args} = Macro.decompose_call(header)
{args, defaults} = extract_default_args(args, 0, [], [])
name_arity = {name, length(args)}
definition = {def_or_defp, ctx, args, body, guards}
{name_arity, definition, defaults}
end
defp parse_fun_definition({:@, _, [{doc, _, [value]}]})
when doc in ~w[moduledoc doc]a and (value == false or is_binary(value)) do
nil
end
defp parse_fun_definition({:@, _, [{typespec, _, [{:"::", _, _}]}]})
when typespec in ~w[spec type typep opaque]a do
nil
end
defp parse_fun_definition(unsupported_ast) do
throw({:module_restricted, unsupported_ast})
end
# TODO else raise unsupported!
defp extract_default_args([], _index, arg_acc, defaults) do
{Enum.reverse(arg_acc), defaults}
end
defp extract_default_args([{:\\, _, [arg, default]} | args], index, arg_acc, defaults) do
extract_default_args(args, index + 1, [arg | arg_acc], [{index, default} | defaults])
end
defp extract_default_args([arg | args], index, arg_acc, defaults) do
extract_default_args(args, index + 1, [arg | arg_acc], defaults)
end
defp expand_defaults({name_arity, definition, _defaults = []}) do
[{name_arity, definition}]
end
defp expand_defaults({name_arity = {name, arity}, definition, defaults}) do
def_or_defp = elem(definition, 0)
do_expand_defaults(name, arity, def_or_defp, defaults, [{name_arity, definition}])
end
defp do_expand_defaults(_name, _arity, _def_or_defp, [], acc) do
acc
end
defp do_expand_defaults(
name,
arity,
def_or_defp,
[{default_index, default_value} | defaults],
acc
) do
args = Macro.generate_arguments(arity, nil)
arity = arity - 1
args_without_default = List.delete_at(args, default_index)
args_in_expr =
Enum.with_index(args, fn
_arg, ^default_index -> default_value
arg, _index -> arg
end)
definition = {def_or_defp, [], args_without_default, {name, [], args_in_expr}, nil}
acc = [{{name, arity}, definition} | acc]
do_expand_defaults(name, arity, def_or_defp, defaults, acc)
end
defp check_definition_conflicts(grouped_definitions) do
Enum.each(grouped_definitions, fn {name_arity, [head | tail]} ->
check_definition_conflict(name_arity, head, tail)
end)
end
defp check_definition_conflict(_name_arity, _, []), do: :ok
defp check_definition_conflict(
name_arity,
head = {def_or_defp, _, _, _, _},
[
{def_or_defp, _, _, _, _} | rest
]
) do
check_definition_conflict(name_arity, head, rest)
end
defp check_definition_conflict(name_arity, {previous_def, previous_ctx, _, _, _}, [
{conflict_def, conflict_ctx, _, _, _} | _
]) do
throw(
{:definition_conflict, name_arity, previous_def, previous_ctx, conflict_def, conflict_ctx}
)
end
defp parse_fun_signature({:when, _, [header, guards]}) do
{header, guards}
end
defp parse_fun_signature(header) do
{header, nil}
end
defp sanitize_module_definition({module, fun_defs}, env) do
env = %{env | module: module}
public_funs_ast =
fun_defs
|> Enum.map(&sanitize_fun(&1, env))
|> Enum.group_by(&elem(&1, 0), fn {_fun, arity, ast} -> {arity, ast} end)
|> Enum.map(fn {fun_name, list} ->
{fun_name, to_map_ast(list)}
end)
|> to_map_ast()
quote do
unquote(env_variable()) =
Dune.Eval.Env.add_module(
unquote(env_variable()),
unquote(module),
%Dune.Eval.FakeModule{public_funs: unquote(public_funs_ast)}
)
end
end
defp to_map_ast(list_ast) when is_list(list_ast), do: {:%{}, [], list_ast}
defp sanitize_fun({{fun_name, arity}, definitions}, env) do
args = Macro.var(:args, nil)
bottom_clause =
quote do
args ->
raise %Dune.Eval.FunctionClauseError{
module: unquote(env.module),
function: unquote(fun_name),
args: args
}
end
clauses = Enum.map(definitions, &sanitize_fun_clause(&1, env)) ++ bottom_clause
env_var = env_variable_if_used(clauses)
anonymous_ast =
{:fn, [],
[
{:->, [],
[
[env_var, args],
{:case, [],
[
args,
[do: clauses]
]}
]}
]}
{fun_name, arity, anonymous_ast}
end
defp sanitize_fun_clause({_def_or_defp, ctx, args, body, guards}, env) do
safe_args = do_sanitize(args, env)
safe_body = do_sanitize(body, env)
safe_guards = do_sanitize(guards, env)
definition_to_clause(ctx, safe_args, safe_body, safe_guards)
end
defp definition_to_clause(ctx, args, body, _guards = nil) do
{:->, ctx, [[args], body]}
end
defp definition_to_clause(ctx, args, body, guards) when is_tuple(guards) do
args_and_guards = [{:when, [], [args, guards]}]
{:->, ctx, [args_and_guards, body]}
end
defp env_variable_if_used(asts) do
uses_variable?(asts, @env_variable_name)
case uses_variable?(asts, @env_variable_name) do
true -> env_variable()
false -> underscore_env_variable()
end
end
defp uses_variable?([], _variable_name), do: false
defp uses_variable?([head | tail], variable_name) do
case uses_variable?(head, variable_name) do
true -> true
false -> uses_variable?(tail, variable_name)
end
end
defp uses_variable?({variable_name, _, nil}, variable_name), do: true
defp uses_variable?({_, _, list}, variable_name) when is_list(list) do
uses_variable?(list, variable_name)
end
defp uses_variable?({x, y}, variable_name) do
uses_variable?(x, variable_name) or uses_variable?(y, variable_name)
end
defp uses_variable?(_ast, _variable_name), do: false
defp do_sanitize(ast, env)
defp do_sanitize(raw_value, _env)
when is_atom(raw_value) or is_number(raw_value) or is_binary(raw_value) do
raw_value
end
defp do_sanitize(list, env)
when is_list(list) do
sanitize_args(list, env)
end
defp do_sanitize({arg1, arg2}, env) do
[safe_arg1, safe_arg2] = sanitize_args([arg1, arg2], env)
{safe_arg1, safe_arg2}
end
defp do_sanitize({atom, _, _} = raw, env) when atom in [:__block__, :when, :<-, :->, :|] do
sanitize_args_in_node(raw, env)
end
defp do_sanitize({name, _, atom} = variable, _env)
when is_atom(name) and atom in [nil, Elixir] do
unless authorized_var_name?(name) do
throw({:restricted, Kernel, name, 0})
end
variable
end
defp do_sanitize({:&, _, args}, env) do
[ast] = args
sanitize_capture(ast, env)
end
defp do_sanitize({:<<>>, meta, args}, env) do
sanitized_args =
Enum.map(args, fn
{:"::", meta, [expr, modifier]} ->
{:"::", meta, [do_sanitize(expr, env), check_bin_modifier(modifier)]}
arg ->
do_sanitize(arg, env)
end)
{:<<>>, meta, sanitized_args}
end
defp do_sanitize({{:., _, [left, right]}, ctx, args} = raw, env)
when is_atom(right) and is_list(args) do
case left do
atom when is_atom(atom) ->
do_sanitize_function(raw, env)
{:__aliases__, _, list} when is_list(list) ->
do_sanitize_function(raw, env)
_ ->
do_sanitize_dot(left, right, args, ctx, env)
end
end
defp do_sanitize({{:., dot_ctx, [{fn_or_ampersand, _, _} = anonymous]}, ctx, args}, env)
when fn_or_ampersand in [:fn, :&] do
safe_anonymous = do_sanitize(anonymous, env)
safe_args = sanitize_args(args, env)
{{:., dot_ctx, [safe_anonymous]}, ctx, safe_args}
end
defp do_sanitize({:dbg, meta, [expr]}, env) do
quote do
value = unquote(do_sanitize(expr, env))
IO.puts([
"[nofile:",
to_string(unquote(meta[:line])),
": (file)]\n",
unquote(Macro.to_string(expr)),
" #=> ",
inspect(value, pretty: true),
?\n
])
value
end
end
defp do_sanitize({:|>, _, [expr, {:dbg, meta, []}]}, env) do
header =
quote do
["[nofile:", to_string(unquote(meta[:line])), ": (file)]\n"]
end
quote do
value = unquote(dbg_pipeline(expr, env, header))
IO.write("\n")
value
end
end
defp do_sanitize({:|>, _, _} = ast, env) do
case try_expand_once(ast) do
{atom, _, _} = expanded when atom != :|> ->
do_sanitize(expanded, env)
end
end
defp do_sanitize({_, _, args} = raw, env) when is_list(args) do
do_sanitize_function(raw, env)
end
defp try_expand_once(ast) do
Macro.expand_once(ast, __ENV__)
rescue
error ->
throw({:exception, error})
end
defp do_sanitize_dot(left, key, args, ctx, env) do
safe_left = do_sanitize(left, env)
safe_args = sanitize_args(args, env)
if args == [] and {:no_parens, true} in ctx do
quote do
Dune.Shims.Kernel.safe_dot(
unquote(env_variable()),
unquote(safe_left),
unquote(key)
)
end
else
quote do
Dune.Shims.Kernel.safe_apply(
unquote(env_variable()),
unquote(safe_left),
unquote(key),
unquote(safe_args)
)
end
end
end
defp do_sanitize_function({func, ctx, atom}, env) when atom in [nil, Elixir] do
do_sanitize_function({func, ctx, []}, env)
end
defp do_sanitize_function({{:., _, [{_variable_function, _, atom}]}, _, args} = raw, env)
when atom in [nil, Elixir] and is_list(args) do
sanitize_args_in_node(raw, env)
end
defp do_sanitize_function({func, _, args} = raw, env)
when is_list(args) do
{module, func_name} = extract_module_and_fun(func)
arity = length(args)
case CompileEnv.resolve_mfa(env, module, func_name, arity) do
{:restricted, resolved_module} ->
throw({:restricted, resolved_module, func_name, arity})
{:shimmed, shim_module, shim_func} ->
safe_args = sanitize_args(args, env)
quote do
unquote(shim_module).unquote(shim_func)(
unquote(env_variable()),
unquote_splicing(safe_args)
)
end
{:fake, fake_module} ->
safe_args = sanitize_args(args, env)
quote do
Dune.Eval.Env.apply_fake(
unquote(env_variable()),
unquote(fake_module),
unquote(func_name),
unquote(safe_args)
)
end
:allowed ->
sanitize_args_in_node(raw, env)
error ->
handle_mfa_error(error, module, func_name, arity)
end
end
defp extract_module_and_fun({:., _, [{:__aliases__, _, modules}, func_name]}) do
{modules |> Module.concat(), func_name}
end
defp extract_module_and_fun({:., _, [erlang_module, func_name]}) when is_atom(erlang_module) do
{erlang_module, func_name}
end
defp extract_module_and_fun(func_name) when is_atom(func_name) do
{nil, func_name}
end
defp sanitize_capture({:/, _, [{func, _, _}, arity]} = raw, env) when is_integer(arity) do
{module, func_name} = extract_module_and_fun(func)
case CompileEnv.resolve_mfa(env, module, func_name, arity) do
{:restricted, resolved_module} ->
throw({:restricted, resolved_module, func_name, arity})
{:fake, fake_module} ->
args = Macro.generate_unique_arguments(arity, nil)
quote do
fn unquote_splicing(args) ->
Dune.Eval.Env.apply_fake(
unquote(env_variable()),
unquote(fake_module),
unquote(func_name),
unquote(args)
)
end
end
{:shimmed, shim_module, shim_func} ->
args = Macro.generate_unique_arguments(arity, nil)
quote do
# FIXME pass env here!
fn unquote_splicing(args) ->
unquote(shim_module).unquote(shim_func)(
unquote(env_variable()),
unquote_splicing(args)
)
end
end
:allowed ->
{:&, [], [raw]}
error ->
handle_mfa_error(error, module, func_name, arity)
end
end
defp handle_mfa_error(:undefined_module, module, func_name, arity) do
throw({:undefined_module, module, func_name, arity})
end
defp handle_mfa_error(:undefined_function, module, func_name, arity) do
throw({:undefined_function, module, func_name, arity})
end
defp handle_mfa_error(:outside_module, _module, func_name, _arity)
when func_name in [:def, :defp] do
throw({:outside_module, func_name})
end
defp sanitize_capture(capture_arg, env) do
safe_capture_arg = do_sanitize(capture_arg, env)
{:&, [], [safe_capture_arg]}
end
defp sanitize_args(args, env) when is_list(args) do
Enum.map(args, &do_sanitize(&1, env))
end
defp sanitize_args_in_node({_, _, args} = raw, env) when is_list(args) do
safe_args = sanitize_args(args, env)
put_elem(raw, 2, safe_args)
end
defp dbg_pipeline({:|>, _, [left, {fun, meta, args} = right]}, env, header)
when is_list(args) and fun != :dbg do
ast_with_placeholder = do_sanitize({fun, meta, [:__DUNE_RESERVED__ | args]}, env)
ast =
Macro.prewalk(ast_with_placeholder, fn
:__DUNE_RESERVED__ -> dbg_pipeline(left, env, header)
other -> other
end)
quote do
value = unquote(ast)
IO.puts([
"|> ",
unquote(Macro.to_string(right)),
" #=> ",
inspect(value, pretty: true)
])
value
end
end
defp dbg_pipeline(expr, env, header) do
quote do
value = unquote(do_sanitize(expr, env))
IO.puts([
unquote_splicing(header),
unquote(Macro.to_string(expr)),
" #=> ",
inspect(value, pretty: true)
])
value
end
end
@max_segment_size 256
@binary_modifiers [:binary, :bytes]
@allowed_modifiers [:integer, :float, :bits, :bitstring, :utf8, :utf16, :utf32] ++
[:signed, :unsigned, :little, :big, :native]
defp check_bin_modifier(modifier) do
{size, unit} = check_bin_modifier_size(modifier, 8, nil)
unit = unit || 1
if size * unit > @max_segment_size do
throw({:bin_modifier_size, @max_segment_size})
end
modifier
end
defp check_bin_modifier_size({:-, _, [left, right]}, size, unit) do
{size, unit} = check_bin_modifier_size(left, size, unit)
check_bin_modifier_size(right, size, unit)
end
defp check_bin_modifier_size(modifier, size, unit) do
case modifier do
new_size when is_integer(new_size) ->
{new_size, unit}
{:size, _, [new_size]} when is_integer(new_size) ->
{new_size, unit}
{:unit, _, [new_unit]} when is_integer(new_unit) ->
{size, new_unit}
{:*, _, [new_size, new_unit]} when is_integer(new_size) and is_integer(new_unit) ->
{new_size, new_unit}
{:size, _, [{:^, _, [{var, _, ctx}]}]} when is_atom(var) and is_atom(ctx) ->
{size, unit}
{atom, _, ctx} when atom in @binary_modifiers and is_atom(ctx) ->
{size, unit || 8}
{atom, _, ctx} when atom in @allowed_modifiers and is_atom(ctx) ->
{size, unit}
other ->
throw({:bin_modifier_restricted, other})
end
end
defp env_variable do
Macro.var(@env_variable_name, nil)
end
defp underscore_env_variable do
Macro.var(:_env__Dune__, nil)
end
defp authorized_var_name?(name) do
# e.g. recompile could be interpreted as recompile/0
not (RealModule.fun_exists?(Kernel, name, 0) or Macro.special_form?(name, 0))
end
end
================================================
FILE: lib/dune/parser/string_parser.ex
================================================
defmodule Dune.Parser.StringParser do
@moduledoc false
alias Dune.{AtomMapping, Failure, Opts}
alias Dune.Helpers.Diagnostics
alias Dune.Parser.{AtomEncoder, UnsafeAst}
@typep previous_session :: %{atom_mapping: AtomMapping.t()}
# TODO options: parse timeout & max atoms
@spec parse_string(String.t(), Opts.t(), previous_session | nil, boolean) ::
UnsafeAst.t() | Failure.t()
def parse_string(string, opts, previous_session, encode_modules? \\ true)
when is_binary(string) and is_boolean(encode_modules?) do
# import: do in a different process because the AtomEncoder pollutes the Process dict
fn -> do_parse_string(string, opts, previous_session, encode_modules?) end
|> Task.async()
|> Task.await()
end
defp do_parse_string(
string,
%Opts{atom_pool_size: pool_size},
previous_session,
encode_modules?
) do
maybe_load_atom_mapping(previous_session)
encoder = fn binary, _ctx -> AtomEncoder.static_atoms_encoder(binary, pool_size) end
{result, diagnostics} =
Diagnostics.with_diagnostics_polyfill(fn ->
Code.string_to_quoted(string, static_atoms_encoder: encoder, existing_atoms_only: true)
end)
case result do
{:ok, ast} ->
maybe_encode_modules(ast, previous_session, encode_modules?)
|> Diagnostics.prepend_diagnostics(diagnostics)
{:error, {_ctx, error, token}} ->
handle_failure(error, token)
end
end
defp maybe_load_atom_mapping(nil), do: :ok
defp maybe_load_atom_mapping(%{atom_mapping: atom_mapping}) do
AtomEncoder.load_atom_mapping(atom_mapping)
end
defp maybe_encode_modules(ast, previous_session, encode_modules?) do
plain_atom_mapping = AtomEncoder.plain_atom_mapping()
{new_ast, atom_mapping} =
if encode_modules? do
existing_mapping = previous_session[:atom_mapping]
AtomEncoder.encode_modules(ast, plain_atom_mapping, existing_mapping)
else
{ast, plain_atom_mapping}
end
%UnsafeAst{ast: new_ast, atom_mapping: atom_mapping}
end
defp handle_failure("Atoms containing" <> _ = error, token) do
%Failure{message: error <> token, type: :restricted}
end
defp handle_failure(error, token) do
failure = do_handle_failure(error, token)
AtomEncoder.plain_atom_mapping()
|> AtomMapping.replace_in_result(failure)
end
defp do_handle_failure({error, explanation}, token)
when is_binary(error) and is_binary(explanation) do
message = IO.iodata_to_binary([error, token, explanation])
%Failure{message: message, type: :parsing}
end
defp do_handle_failure(error, token) when is_binary(error) do
%Failure{message: error <> token, type: :parsing}
end
end
================================================
FILE: lib/dune/parser/unsafe_ast.ex
================================================
defmodule Dune.Parser.UnsafeAst do
@moduledoc false
@type t :: %__MODULE__{
ast: Macro.t(),
atom_mapping: Dune.AtomMapping.t(),
stdio: binary()
}
@enforce_keys [:ast, :atom_mapping]
defstruct @enforce_keys ++ [stdio: <<>>]
end
================================================
FILE: lib/dune/parser.ex
================================================
defmodule Dune.Parser do
@moduledoc false
alias Dune.{AtomMapping, Success, Failure, Opts}
alias Dune.Parser.{CompileEnv, StringParser, Sanitizer, SafeAst, UnsafeAst}
@typep previous_session :: %{
atom_mapping: AtomMapping.t(),
compile_env: Dune.Parser.CompileEnv.t()
}
@spec parse_string(String.t(), Opts.t(), previous_session | nil) :: SafeAst.t() | Failure.t()
def parse_string(string, opts = %Opts{}, previous_session \\ nil) when is_binary(string) do
compile_env = get_compile_env(opts, previous_session)
string
|> do_parse_string(opts, previous_session)
|> Sanitizer.sanitize(compile_env)
end
defp do_parse_string(string, opts = %{max_length: max_length}, previous_session) do
case String.length(string) do
length when length > max_length ->
%Failure{type: :parsing, message: "max code length exceeded: #{length} > #{max_length}"}
_ ->
StringParser.parse_string(string, opts, previous_session)
end
end
@spec parse_quoted(Macro.t(), Opts.t(), previous_session | nil) :: SafeAst.t()
def parse_quoted(quoted, opts = %Opts{}, previous_session \\ nil) do
compile_env = get_compile_env(opts, previous_session)
quoted
|> unsafe_quoted()
|> Sanitizer.sanitize(compile_env)
end
def unsafe_quoted(ast) do
%UnsafeAst{ast: ast, atom_mapping: AtomMapping.new()}
end
defp get_compile_env(opts, nil) do
CompileEnv.new(opts.allowlist)
end
defp get_compile_env(opts, %{compile_env: compile_env}) do
%{compile_env | allowlist: opts.allowlist}
end
@spec string_to_quoted(String.t(), Opts.t()) :: Success.t() | Failure.t()
def string_to_quoted(string, opts) do
with unsafe = %UnsafeAst{} <- StringParser.parse_string(string, opts, nil, false) do
inspected = inspect(unsafe.ast, pretty: opts.pretty)
inspected = AtomMapping.replace_in_string(unsafe.atom_mapping, inspected)
%Success{
value: unsafe.ast,
inspected: inspected,
stdio: unsafe.stdio
}
end
end
end
================================================
FILE: lib/dune/session.ex
================================================
defmodule Dune.Session do
@moduledoc """
Sessions provide a way to evaluate code and keep state (bindings, modules...) between evaluations.
- Use `Dune.eval_string/2` to execute code as a one-off
- Use `Dune.Session.eval_string/3` to execute consecutive code blocks
`Dune.Session` could be used to implement something like a safe IEx shell, or to compile a module
once and call it several times without the overhead of parsing.
`Dune.Session` is also a struct that is used to store the state of an evaluation.
Only the following fields are public:
- `last_result`: contains the result of the last evaluation, or `nil` for empty sessions
Other fields are private and shouldn't be accessed directly.
"""
alias Dune.{Allowlist, Eval, Parser, Success, Failure, Opts}
@opaque private_env :: Eval.Env.t()
@opaque private_compile_env :: Parser.CompileEnv.t()
@typedoc """
The type of a `Dune.Session`.
"""
@type t :: %__MODULE__{
last_result: nil | Success.t() | Failure.t(),
env: private_env,
compile_env: private_compile_env,
bindings: keyword
}
@enforce_keys [:env, :compile_env, :bindings, :last_result]
defstruct @enforce_keys
@default_env Eval.Env.new(Dune.AtomMapping.new(), Allowlist.Default)
@default_compile_env Parser.CompileEnv.new(Allowlist.Default)
@doc """
Returns a new empty session.
## Examples
iex> Dune.Session.new()
#Dune.Session<last_result: nil, ...>
"""
@spec new :: t
def new do
%__MODULE__{
env: @default_env,
compile_env: @default_compile_env,
bindings: [],
last_result: nil
}
end
@doc """
Evaluates the provided `string` in the context of the `session` and returns a new session.
The result will be available in the `last_result` key.
In case of a success, the variable bindings or created modules will be saved in the session.
In case of a failure, the rest of the session state won't be updated, so it is possible to
keep executing instructions after a failure
## Examples
iex> Dune.Session.new()
...> |> Dune.Session.eval_string("x = 1")
...> |> Dune.Session.eval_string("x + 2")
#Dune.Session<last_result: %Dune.Success{value: 3, inspected: "3", stdio: ""}, ...>
iex> Dune.Session.new()
...> |> Dune.Session.eval_string("x = 1")
...> |> Dune.Session.eval_string("x = x / 0") # will fail, but the previous state is kept
...> |> Dune.Session.eval_string("x + 2")
#Dune.Session<last_result: %Dune.Success{value: 3, inspected: "3", stdio: ""}, ...>
"""
@spec eval_string(t, String.t(), keyword) :: t
def eval_string(session = %__MODULE__{}, string, opts \\ []) do
opts = Opts.validate!(opts)
parse_state = %{atom_mapping: session.env.atom_mapping, compile_env: session.compile_env}
parsed = Parser.parse_string(string, opts, parse_state)
parsed
|> Eval.run(opts, session)
|> add_result_to_session(session, parsed)
end
defp add_result_to_session(result = %Success{value: {value, env, bindings}}, session, %{
compile_env: compile_env
}) do
result = %{result | value: value}
%{session | env: env, compile_env: compile_env, last_result: result, bindings: bindings}
end
defp add_result_to_session(result = %Failure{}, session, _) do
%{session | last_result: result}
end
defimpl Inspect do
import Inspect.Algebra
def inspect(session, opts) do
container_doc(
"#Dune.Session<",
[last_result: session.last_result],
", ...>",
opts,
&do_inspect/2,
break: :strict
)
end
defp do_inspect({key, value}, opts) do
key = inspect_as_key(key) |> color(:atom, opts)
concat(key, concat(" ", to_doc(value, opts)))
end
if Code.ensure_loaded?(Macro) and function_exported?(Macro, :inspect_atom, 2) do
defp inspect_as_key(key), do: Macro.inspect_atom(:key, key)
else
defp inspect_as_key(key), do: Code.Identifier.inspect_as_key(key)
end
end
end
================================================
FILE: lib/dune/shims/atom.ex
================================================
defmodule Dune.Shims.Atom do
@moduledoc false
alias Dune.AtomMapping
def to_string(env, atom) when is_atom(atom) do
AtomMapping.to_string(env.atom_mapping, atom)
end
def to_charlist(env, atom) when is_atom(atom) do
__MODULE__.to_string(env, atom) |> String.to_charlist()
end
end
================================================
FILE: lib/dune/shims/enum.ex
================================================
defmodule Dune.Shims.Enum do
@moduledoc false
def join(env, enumerable, joiner \\ "") when is_binary(joiner) do
enumerable
|> Enum.map_intersperse(joiner, &Dune.Shims.Kernel.safe_to_string(env, &1))
|> IO.iodata_to_binary()
end
def map_join(env, enumerable, joiner \\ "", mapper)
when is_binary(joiner) and is_function(mapper, 1) do
enumerable
|> Enum.map_intersperse(joiner, &Dune.Shims.Kernel.safe_to_string(env, mapper.(&1)))
|> IO.iodata_to_binary()
end
end
================================================
FILE: lib/dune/shims/io.ex
================================================
defmodule Dune.Shims.IO do
@moduledoc false
alias Dune.{Failure, Shims}
def puts(env, device \\ :stdio, item)
def puts(env, :stdio, item) do
env
|> Shims.Kernel.safe_to_string(item)
|> then(&IO.puts(:stdio, &1))
end
def puts(_env, _device, _item) do
error = Failure.restricted_function(IO, :puts, 2)
throw(error)
end
def inspect(env, item, opts \\ []) do
inspect(env, :stdio, item, opts)
end
def inspect(env, :stdio, item, opts) when is_list(opts) do
inspected = Shims.Kernel.safe_inspect(env, item, opts)
chardata =
if label_opt = opts[:label] do
[Shims.Kernel.safe_to_string(env, label_opt), ": ", inspected]
else
inspected
end
IO.puts(:stdio, chardata)
item
end
def inspect(_env, _device, _item, opts) when is_list(opts) do
error = Failure.restricted_function(IO, :inspect, 3)
throw(error)
end
end
================================================
FILE: lib/dune/shims/json.ex
================================================
if Code.ensure_loaded?(JSON) do
defmodule Dune.Shims.JSON do
@moduledoc false
alias Dune.AtomMapping
alias Dune.Shims
def protocol_encode(env, value, encoder) when is_non_struct_map(value) do
case :maps.next(:maps.iterator(value)) do
:none ->
"{}"
{key, value, iterator} ->
[
?{,
key(env, key, encoder),
?:,
encoder.(value, encoder) | next(env, iterator, encoder)
]
end
end
def protocol_encode(env, value, encoder)
when is_atom(value) and value not in [nil, true, false] do
encoder.(AtomMapping.to_string(env.atom_mapping, value), encoder)
end
def protocol_encode(_env, value, encoder) do
JSON.protocol_encode(value, encoder)
end
defp next(env, iterator, encoder) do
case :maps.next(iterator) do
:none ->
"}"
{key, value, iterator} ->
[
?,,
key(env, key, encoder),
?:,
encoder.(value, encoder) | next(env, iterator, encoder)
]
end
end
defp key(_env, key, encoder) when is_binary(key), do: encoder.(key, encoder)
defp key(env, key, encoder), do: encoder.(Shims.Kernel.safe_to_string(env, key), encoder)
def encode!(env, term) do
encode!(env, term, &protocol_encode(env, &1, &2))
end
def encode!(_env, term, encoder) do
IO.iodata_to_binary(encoder.(term, encoder))
end
def encode_to_iodata!(env, term) do
encode_to_iodata!(env, term, &protocol_encode(env, &1, &2))
end
def encode_to_iodata!(_env, term, encoder) do
encoder.(term, encoder)
end
end
end
================================================
FILE: lib/dune/shims/kernel.ex
================================================
defmodule Dune.Shims.Kernel do
@moduledoc false
alias Dune.{AtomMapping, Failure, Shims}
alias Dune.Helpers.TermChecker
defmacro safe_sigil_w(_env, _, ~c"a") do
error = Failure.restricted_function(Kernel, :sigil_w, 2)
throw(error)
end
defmacro safe_sigil_w(_env, term, modifiers) do
quote do
sigil_w(unquote(term), unquote(modifiers))
end
end
defmacro safe_sigil_W(_env, _, ~c"a") do
error = Failure.restricted_function(Kernel, :sigil_W, 2)
throw(error)
end
defmacro safe_sigil_W(_env, term, modifiers) do
quote do
sigil_W(unquote(term), unquote(modifiers))
end
end
def safe_throw(_env, value) do
throw({:safe_throw, value})
end
defmacro safe_dbg(_env) do
error = Failure.restricted_function(Kernel, :dbg, 0)
throw(error)
end
defmacro safe_dbg(_env, _term) do
# should never be called because the sanitizer handles it
raise "unexpected call safe_dbg/2"
end
defmacro safe_dbg(_env, _term, _opts) do
error = Failure.restricted_function(Kernel, :dbg, 2)
throw(error)
end
def safe_dot(_env, %{} = map, key) do
# TODO test key error
Map.fetch!(map, key)
end
def safe_dot(env, module, fun) when is_atom(module) do
safe_apply(env, module, fun, [])
end
def safe_apply(_env, fun, args) when is_function(fun, 1) do
# TODO check if there is a risk / why it is here
apply(fun, args)
end
def safe_apply(env, module, fun, args) when is_atom(module) do
arity = length(args)
case env.allowlist.fun_status(module, fun, arity) do
:restricted ->
error = Failure.restricted_function(module, fun, arity)
throw(error)
{:shimmed, shim_module, shim_fun} ->
apply(shim_module, shim_fun, [env | args])
:allowed ->
apply(module, fun, args)
:undefined_module ->
Dune.Eval.Env.apply_fake(env, module, fun, args)
:undefined_function ->
throw({:undefined_function, module, fun, arity})
other when other in [:undefined_module, :undefined_function] ->
Dune.Eval.Env.apply_fake(env, module, fun, args)
end
end
def safe_inspect(env, term, opts \\ [])
def safe_inspect(_env, term, opts)
when is_number(term) or is_binary(term) or is_boolean(term) do
inspect(term, opts)
end
def safe_inspect(env, atom, _opts) when is_atom(atom) do
AtomMapping.inspect(env.atom_mapping, atom)
end
def safe_inspect(env, term, opts) do
TermChecker.check(term)
inspected = inspect(term, opts)
AtomMapping.replace_in_string(env.atom_mapping, inspected)
end
def safe_to_string(env, atom) when is_atom(atom) do
Shims.Atom.to_string(env, atom)
end
def safe_to_string(env, atom) when is_list(atom) do
Shims.List.to_string(env, atom)
end
def safe_to_string(_env, other), do: to_string(other)
def safe_to_charlist(env, atom) when is_atom(atom) do
Shims.Atom.to_charlist(env, atom)
end
def safe_to_charlist(_env, other), do: to_charlist(other)
end
================================================
FILE: lib/dune/shims/list.ex
================================================
defmodule Dune.Shims.List do
@moduledoc false
alias Dune.Shims
def to_string(_env, list) when is_list(list) do
do_to_string(list)
end
defp do_to_string(list) when is_list(list) do
if Enum.any?(list, &is_list/1) do
# eagerly convert lists to binary to prevent OOM on
# structural sharing bombs
Enum.map(list, &do_to_string/1)
else
list
end
|> List.to_string()
end
defp do_to_string(elem), do: elem
# note: this is probably not safe so not actually used
def to_existing_atom(env, list) when is_list(list) do
string = to_string(list)
# make sure it was actually a flat charlist and not an IO-list
case to_charlist(string) do
^list -> Shims.String.to_existing_atom(env, string)
_ -> List.to_existing_atom(list)
end
end
def to_existing_atom(_env, list) do
List.to_existing_atom(list)
end
end
================================================
FILE: lib/dune/shims/string.ex
================================================
defmodule Dune.Shims.String do
@moduledoc false
alias Dune.AtomMapping
# note: this is probably not safe so not actually used
def to_existing_atom(env, string) when is_binary(string) do
AtomMapping.to_existing_atom(env, string)
end
def to_existing_atom(_env, string) do
String.to_existing_atom(string)
end
end
================================================
FILE: lib/dune/success.ex
================================================
defmodule Dune.Success do
@moduledoc """
A struct returned when `Dune` evaluation succeeds.
Fields:
- `value` (term): the value which was actually returned at runtime.
Should not be displayed to the user, might be different from what the user expects.
- `inspected` (string): safely inspected `value` to be displayed to the user
- `stdio` (string): captured standard output
`value` contains the actual value used at runtime, so atoms will be different from the ones
displayed to the user (see `Dune.eval_string/2`).
"""
@type t :: %__MODULE__{
value: term,
inspected: String.t(),
stdio: binary
}
@enforce_keys [:value, :inspected, :stdio]
defstruct @enforce_keys
end
================================================
FILE: lib/dune.ex
================================================
defmodule Dune do
@moduledoc """
A sandbox for Elixir to safely evaluate untrusted code from user input.
## Features
- only authorized modules and functions can be executed (see
`Dune.Allowlist.Default`)
- no access to environment variables, file system, network...
- code executed in an isolated process
- execution within configurable limits: timeout, maximum reductions and memory
(inspired by [Luerl](https://github.com/rvirding/luerl))
- captured standard output
- atoms, without atom leaks: parsing and runtime do not
[leak atoms](https://hexdocs.pm/elixir/String.html#to_atom/1) (i.e. does not
keep
[filling the atom table](https://learnyousomeerlang.com/starting-out-for-real#atoms)
until the VM crashes)
- modules, without actual module creation: Dune does not let users define any
actual module (would leak memory and modify the state of the VM globally), but
`defmodule` simulates the basic behavior of a module, including private and
recursive functions
The list of modules and functions authorized by default is defined by the
`Dune.Allowlist.Default` module, but this list can be extended and customized
(at your own risk!) using `Dune.Allowlist`.
If you need to keep the state between evaluations, you might consider
`Dune.Session`.
"""
alias Dune.{Success, Failure, Parser, Eval, Opts}
@doc ~S"""
Evaluates the `string` in the sandbox.
Available options are detailed in `Dune.Opts`.
Returns a `Dune.Success` struct if the execution went successfully,
a `Dune.Failure` else.
## Examples
iex> Dune.eval_string("IO.puts(\"Hello world!\")")
%Dune.Success{inspected: ":ok", stdio: "Hello world!\n", value: :ok}
iex> Dune.eval_string("File.cwd!()")
%Dune.Failure{message: "** (DuneRestrictedError) function File.cwd!/0 is restricted", type: :restricted}
iex> Dune.eval_string("List.duplicate(:spam, 100_000)")
%Dune.Failure{message: "Execution stopped - memory limit exceeded", stdio: "", type: :memory}
iex> Dune.eval_string("Foo.bar()")
%Dune.Failure{message: "** (UndefinedFunctionError) function Foo.bar/0 is undefined (module Foo is not available)", type: :exception}
iex> Dune.eval_string("][")
%Dune.Failure{message: "unexpected token: ]", type: :parsing}
Atoms used during parsing and execution might be transformed to prevent atom leaks:
iex> Dune.eval_string("some_variable = IO.inspect(:some_atom)")
%Dune.Success{inspected: ":some_atom", stdio: ":some_atom\n", value: :a__Dune_atom_2__}
The `value` field shows the actual runtime value, but `inspected` and `stdio` are safe to display to the user.
"""
@spec eval_string(String.t(), Keyword.t()) :: Success.t() | Failure.t()
def eval_string(string, opts \\ []) when is_binary(string) do
opts = Opts.validate!(opts)
string
|> Parser.parse_string(opts)
|> Eval.run(opts)
end
@doc ~S"""
Evaluates the quoted `ast` in the sandbox.
Available options are detailed in `Dune.Opts` (parsing restrictions have no effect)..
Returns a `Dune.Success` struct if the execution went successfully,
a `Dune.Failure` else.
## Examples
iex> Dune.eval_quoted(quote do: [1, 2] ++ [3, 4])
%Dune.Success{inspected: "[1, 2, 3, 4]", stdio: "", value: [1, 2, 3, 4]}
iex> Dune.eval_quoted(quote do: System.get_env())
%Dune.Failure{message: "** (DuneRestrictedError) function System.get_env/0 is restricted", type: :restricted}
iex> Dune.eval_quoted(quote do: Process.sleep(500))
%Dune.Failure{message: "Execution timeout - 50ms", type: :timeout}
"""
@spec eval_quoted(Macro.t(), Keyword.t()) :: Success.t() | Failure.t()
def eval_quoted(ast, opts \\ []) do
opts = Opts.validate!(opts)
ast
|> Parser.parse_quoted(opts)
|> Eval.run(opts)
end
@doc ~S"""
Returns the AST corresponding to the provided `string`, without leaking atoms.
Available options are detailed in `Dune.Opts` (runtime restrictions have no effect).
Returns a `Dune.Success` struct if the execution went successfully,
a `Dune.Failure` else.
## Examples
iex> Dune.string_to_quoted("1 + 2")
%Dune.Success{inspected: "{:+, [line: 1], [1, 2]}", stdio: "", value: {:+, [line: 1], [1, 2]}}
iex> Dune.string_to_quoted("[invalid")
%Dune.Failure{stdio: "", message: "missing terminator: ]", type: :parsing}
The `pretty` option can make the AST more readable by adding newlines to `inspected`:
iex> Dune.string_to_quoted("IO.puts(:hello)", pretty: true).inspected
"{{:., [line: 1], [{:__aliases__, [line: 1], [:IO]}, :puts]}, [line: 1],\n [:hello]}"
iex> Dune.string_to_quoted("IO.puts(:hello)").inspected
"{{:., [line: 1], [{:__aliases__, [line: 1], [:IO]}, :puts]}, [line: 1], [:hello]}"
Since the code isn't executed, there is no allowlist restriction:
iex> Dune.string_to_quoted("System.halt()")
%Dune.Success{
inspected: "{{:., [line: 1], [{:__aliases__, [line: 1], [:System]}, :halt]}, [line: 1], []}",
stdio: "",
value: {{:., [line: 1], [{:__aliases__, [line: 1], [:System]}, :halt]}, [line: 1], []}
}
Atoms might be transformed during parsing to prevent atom leaks:
iex> Dune.string_to_quoted("some_variable = :some_atom")
%Dune.Success{
inspected: "{:=, [line: 1], [{:some_variable, [line: 1], nil}, :some_atom]}",
stdio: "",
value: {:=, [line: 1], [{:a__Dune_atom_1__, [line: 1], nil}, :a__Dune_atom_2__]}
}
The `value` field shows the actual runtime value, but `inspected` is safe to display to the user.
"""
@spec string_to_quoted(String.t(), Keyword.t()) :: Success.t() | Failure.t()
def string_to_quoted(string, opts \\ []) when is_binary(string) do
opts = Opts.validate!(opts)
Parser.string_to_quoted(string, opts)
end
end
================================================
FILE: mix.exs
================================================
defmodule Dune.MixProject do
use Mix.Project
@version "0.3.15"
@github_url "https://github.com/functional-rewire/dune"
def project do
[
app: :dune,
version: @version,
elixir: ">= 1.14.0 and < 1.20.0",
start_permanent: Mix.env() == :prod,
deps: deps(),
dialyzer: [flags: [:missing_return, :extra_return]],
# Hex
description: "A sandbox for Elixir to safely evaluate untrusted code from user input",
package: package(),
aliases: aliases(),
docs: docs()
]
end
# Run "mix help compile.app" to learn about applications.
def application do
[
extra_applications: [:logger]
]
end
# Run "mix help deps" to learn about dependencies.
defp deps do
[
# CI
{:dialyxir, "~> 1.0", only: :test, runtime: false},
# DOCS
{:ex_doc, "~> 0.24", only: :docs, runtime: false}
]
end
defp package do
[
maintainers: ["functional-rewire", "sabiwara"],
licenses: ["MIT"],
links: %{"GitHub" => @github_url},
files: ~w(lib mix.exs .formatter.exs README.md LICENSE.md CHANGELOG.md)
]
end
defp aliases do
[docs: ["compile --force", "docs"]]
end
def cli do
[preferred_envs: [docs: :docs, "hex.publish": :docs, dialyzer: :test]]
end
defp docs do
[
main: "Dune",
source_ref: "v#{@version}",
source_url: @github_url,
homepage_url: @github_url,
extras: ["README.md", "CHANGELOG.md", "LICENSE.md"]
]
end
end
================================================
FILE: test/dune/allowlist/default_test.exs
================================================
defmodule Dune.Allowlist.DefaultTest do
use ExUnit.Case, async: true
doctest Dune.Allowlist.Default
alias Dune.Allowlist.Default
describe "fun_status/3" do
test "should not allow module_info/N" do
assert :restricted = Default.fun_status(Float, :module_info, 0)
assert :restricted = Default.fun_status(Float, :module_info, 1)
assert :restricted = Default.fun_status(:math, :module_info, 0)
assert :restricted = Default.fun_status(:math, :module_info, 1)
end
end
end
================================================
FILE: test/dune/allowlist_test.exs
================================================
defmodule Dune.AllowlistTest do
use ExUnit.Case, async: true
doctest Dune.Allowlist
describe "use/2" do
test "creates a new sandbox " do
defmodule CustomAllowlist do
use Dune.Allowlist
allow Kernel, only: [:+, :*, :-, :/, :div, :rem]
allow Integer, only: [:pow]
end
assert :allowed = CustomAllowlist.fun_status(Kernel, :+, 2)
assert :restricted = CustomAllowlist.fun_status(Kernel, :<>, 2)
assert :undefined_function = CustomAllowlist.fun_status(Kernel, :foo, 1)
assert :allowed = CustomAllowlist.fun_status(Integer, :pow, 2)
assert :restricted = CustomAllowlist.fun_status(Integer, :to_string, 1)
assert :undefined_function = CustomAllowlist.fun_status(Integer, :foo, 1)
assert :restricted = CustomAllowlist.fun_status(String, :upcase, 1)
assert :undefined_module = CustomAllowlist.fun_status(Foo, :foo, 1)
end
test "extends an existing sandbox " do
defmodule CustomModule do
def authorized(i), do: i + 1
def forbidden(i), do: i - 1
end
defmodule ExtendedAllowlist do
use Dune.Allowlist, extend: Dune.Allowlist.Default
allow CustomModule, only: [:authorized]
end
assert :allowed = ExtendedAllowlist.fun_status(String, :upcase, 1)
assert :restricted = ExtendedAllowlist.fun_status(String, :to_atom, 1)
assert :allowed = ExtendedAllowlist.fun_status(CustomModule, :authorized, 1)
assert :restricted = ExtendedAllowlist.fun_status(CustomModule, :forbidden, 1)
assert :undefined_module = ExtendedAllowlist.fun_status(Foo, :foo, 1)
end
end
end
================================================
FILE: test/dune/atom_mapping_test.exs
================================================
defmodule Dune.AtomMappingTest do
use ExUnit.Case, async: true
doctest Dune.AtomMapping
end
================================================
FILE: test/dune/opts_test.exs
================================================
defmodule Dune.OptsTest do
use ExUnit.Case, async: true
doctest Dune.Opts
end
================================================
FILE: test/dune/parser/atom_encoder_test.exs
================================================
defmodule Dune.Parser.AtomEncoderTest do
use ExUnit.Case, async: true
import Dune.Parser.AtomEncoder
describe "categorize_atom_binary/1" do
test "categorizes aliases cases" do
assert :alias = categorize_atom_binary("Elixir")
assert :alias = categorize_atom_binary("String")
assert :alias = categorize_atom_binary("Foo.Bar")
end
test "categorizes valid public variable names" do
assert :public_var = categorize_atom_binary("abc")
assert :public_var = categorize_atom_binary("erlang")
assert :public_var = categorize_atom_binary("あ")
end
test "categorizes valid private variable names" do
assert :private_var = categorize_atom_binary("_")
assert :private_var = categorize_atom_binary("_abc")
assert :private_var = categorize_atom_binary("_あ")
end
test "categorizes other cases" do
assert :other = categorize_atom_binary("")
assert :other = categorize_atom_binary(" ")
assert :other = categorize_atom_binary("a b")
assert :other = categorize_atom_binary("A B")
assert :other = categorize_atom_binary("Elixir. A")
assert :other = categorize_atom_binary("Foo.Bar ")
assert :other = categorize_atom_binary(" Foo.Bar")
end
end
end
================================================
FILE: test/dune/parser/string_parser_test.exs
================================================
defmodule Dune.Parser.StringParserTest do
use ExUnit.Case, async: true
alias Dune.{AtomMapping, Opts}
alias Dune.Parser.{StringParser, UnsafeAst}
describe "parse_string/2" do
test "existing atoms" do
assert %UnsafeAst{ast: nil, atom_mapping: AtomMapping.new()} ==
StringParser.parse_string("nil", %Opts{}, nil)
assert %UnsafeAst{ast: true, atom_mapping: AtomMapping.new()} ==
StringParser.parse_string("true", %Opts{}, nil)
assert %UnsafeAst{ast: :atom, atom_mapping: AtomMapping.new()} ==
StringParser.parse_string(":atom", %Opts{}, nil)
assert %UnsafeAst{ast: :Atom, atom_mapping: AtomMapping.new()} ==
StringParser.parse_string(":Atom", %Opts{}, nil)
end
test "existing modules" do
assert %UnsafeAst{
ast: {:__aliases__, [line: 1], [:Module]},
atom_mapping: AtomMapping.new()
} == StringParser.parse_string("Module", %Opts{}, nil)
assert %UnsafeAst{
ast: {:__aliases__, [line: 1], [:Date, :Range]},
atom_mapping: AtomMapping.new()
} == StringParser.parse_string("Date.Range", %Opts{}, nil)
end
test "non-existing atoms" do
assert %UnsafeAst{
ast: :a__Dune_atom_1__,
atom_mapping: %AtomMapping{
atoms: %{a__Dune_atom_1__: "my_atom"},
modules: %{},
extra_info: %{}
}
} == StringParser.parse_string(":my_atom", %Opts{}, nil)
assert %UnsafeAst{
ast: :__Dune_atom_1__,
atom_mapping: %AtomMapping{
atoms: %{__Dune_atom_1__: "_my_atom"},
modules: %{},
extra_info: %{}
}
} == StringParser.parse_string(":_my_atom", %Opts{}, nil)
assert %UnsafeAst{
ast: :Dune_Atom_1__,
atom_mapping: %AtomMapping{
atoms: %{Dune_Atom_1__: "MyAtom"},
modules: %{},
extra_info: %{}
}
} == StringParser.parse_string(":MyAtom", %Opts{}, nil)
end
test "non-existing modules" do
assert %UnsafeAst{
ast: {:__aliases__, [line: 1], [:Dune_Module_1__]},
atom_mapping: %AtomMapping{
atoms: %{Dune_Atom_1__: "MyModule"},
modules: %{Dune_Module_1__ => "MyModule"},
extra_info: %{}
}
} == StringParser.parse_string("MyModule", %Opts{}, nil)
assert %UnsafeAst{
ast: {:__aliases__, [line: 1], [:Dune_Module_1__]},
atom_mapping: %AtomMapping{
atoms: %{Dune_Atom_1__: "My", Dune_Atom_2__: "AwesomeModule"},
modules: %{Dune_Module_1__ => "My.AwesomeModule"},
extra_info: %{}
}
} == StringParser.parse_string("My.AwesomeModule", %Opts{}, nil)
assert %UnsafeAst{
ast: {:__aliases__, [line: 1], [:Dune_Module_1__]},
atom_mapping: %AtomMapping{
atoms: %{Dune_Atom_1__: "My"},
modules: %{Dune_Module_1__ => "My.Module"},
extra_info: %{}
}
} == StringParser.parse_string("My.Module", %Opts{}, nil)
end
test ~S[non-existing "wrapped" atoms (with whitespace)] do
assert %UnsafeAst{
ast: :__Dune_atom_1__,
atom_mapping: %AtomMapping{
atoms: %{__Dune_atom_1__: " "},
modules: %{},
extra_info: %{__Dune_atom_1__: :wrapped}
}
} == StringParser.parse_string(~S(:" "), %Opts{}, nil)
assert %UnsafeAst{
ast: :__Dune_atom_1__,
atom_mapping: %AtomMapping{
atoms: %{__Dune_atom_1__: "my atom"},
modules: %{},
extra_info: %{__Dune_atom_1__: :wrapped}
}
} == StringParser.parse_string(~S(:"my atom"), %Opts{}, nil)
end
end
end
================================================
FILE: test/dune/session_test.exs
================================================
defmodule Dune.SessionTest do
use ExUnit.Case
doctest Dune.Session, tags: [lts_only: true]
alias Dune.{Session, Success, Failure}
@module_code """
defmodule MySum do
def sum(xs), do: sum(xs, 0)
defp sum([], acc), do: acc
defp sum([x | xs], acc) do
sum(xs, acc + x)
end
end
"""
describe "eval_string/3" do
test "keeps variable bindings" do
session =
Session.new()
|> Session.eval_string("abcd = 5")
|> Session.eval_string("abcd")
assert %Session{
last_result: %Success{inspected: "5"},
bindings: [a__Dune_atom_1__: 5]
} = session
end
test "ignores failed steps" do
session =
Session.new()
|> Session.eval_string("abcd = 5")
|> Session.eval_string("abcd = 1 / 0")
|> Session.eval_string("abcd")
assert %Session{
last_result: %Success{inspected: "5"},
bindings: [a__Dune_atom_1__: 5]
} = session
end
test "keeps variable bindings on errors" do
session =
Session.new()
|> Session.eval_string("abcd = 5")
|> Session.eval_string("abcd / 0")
assert %Session{
last_result: %Failure{
message: "** (ArithmeticError) bad argument in arithmetic expression" <> _
},
bindings: [a__Dune_atom_1__: 5]
} = session
end
test "keeps the atom mapping" do
session =
Session.new()
|> Session.eval_string("abcd = :abcd")
|> Session.eval_string("efgh = :efgh")
|> Session.eval_string("[abcd, efgh]")
assert %Session{
last_result: %Success{inspected: "[:abcd, :efgh]"},
bindings: [
a__Dune_atom_2__: :a__Dune_atom_2__,
a__Dune_atom_1__: :a__Dune_atom_1__
]
} = session
end
test "keeps the module mapping" do
session =
Session.new()
|> Session.eval_string("acbd = [Aa]")
|> Session.eval_string("acbd = acbd ++ [Bb]")
|> Session.eval_string("acbd = acbd ++ [Aa.Bb]")
|> Session.eval_string("acbd ++ [Cc.Dd]")
assert %Session{
last_result: %Success{inspected: "[Aa, Bb, Aa.Bb, Cc.Dd]"},
bindings: [a__Dune_atom_1__: [Dune_Module_1__, Dune_Module_2__, Dune_Module_3__]]
} = session
end
test "handles modules" do
session =
Session.new()
|> Session.eval_string(@module_code)
|> Session.eval_string("MySum.sum([1, 2, 100])")
assert %Session{
last_result: %Success{inspected: "103"},
bindings: []
} = session
end
test "does not break due to Elixir single atom bug" do
session = Session.new() |> Session.eval_string(":foo")
assert %Session{
last_result: %Success{inspected: ":foo"},
bindings: []
} = session
end
end
end
================================================
FILE: test/dune/shims_test.exs
================================================
defmodule Dune.ShimsTest do
use ExUnit.Case, async: true
alias Dune.Success
alias Dune.Failure
defmacrop sigil_E(call, _expr) do
quote do
Dune.eval_string(unquote(call), timeout: 100, inspect_sort_maps: true)
end
end
describe "JSON" do
@describetag :lts_only
test "encode atoms" do
assert %Success{value: ~S("json101"), inspected: ~S("\"json101\"")} =
~E'JSON.encode!(:json101)'
assert %Success{value: ~S("json102"), inspected: ~S("\"json102\"")} =
~E'JSON.encode_to_iodata!(:json102) |> IO.iodata_to_binary()'
assert %Success{
value: ~S({"json201":["json202",123,"foo",null,true]}),
inspected: ~S("{\"json201\":[\"json202\",123,\"foo\",null,true]}")
} = ~E'JSON.encode!(%{json201: [:json202, 123, "foo", nil, true]})'
end
test "decode atoms" do
assert %Success{value: "json301", inspected: ~S("json301")} =
~E'JSON.decode!("\"json301\"")'
end
end
describe "iodata / chardata" do
test "List.to_string" do
assert %Success{value: <<1, 2, 3>>} = ~E'List.to_string([1, 2, 3])'
assert %Success{value: <<1, 2, 3>>} = ~E'List.to_string([1, [[2], 3]])'
assert %Success{value: "abc"} = ~E'List.to_string(["a", [["b"], "c"]])'
assert %Failure{message: "** (ArgumentError) cannot convert the given list" <> _} =
~E'List.to_string([1, :foo])'
end
test "IO.chardata_to_string" do
assert %Success{value: <<1, 2, 3>>} = ~E'IO.chardata_to_string([1, 2, 3])'
assert %Success{value: <<1, 2, 3>>} = ~E'IO.chardata_to_string([1, [[2], 3]])'
assert %Success{value: "abc"} = ~E'IO.chardata_to_string(["a", [["b"], "c"]])'
assert %Failure{message: "** (ArgumentError) cannot convert the given list" <> _} =
~E'IO.chardata_to_string([1, :foo])'
end
end
end
================================================
FILE: test/dune/validation_test.exs
================================================
defmodule Dune.ValidationTest do
end
================================================
FILE: test/dune_modules_test.exs
================================================
defmodule DuneModulesTest do
use ExUnit.Case, async: true
alias Dune.{Success, Failure}
defmacro sigil_E(call, _expr) do
# TODO fix memory needs
quote do
Dune.eval_string(unquote(call), max_reductions: 25_000, max_heap_size: 50_000, timeout: 100)
end
end
describe "Dune authorized" do
test "basic module" do
result = ~E'''
defmodule Hello do
def greet(value) do
IO.puts "Hello #{value}"
end
end
Hello.greet(:world!)
'''
assert %Success{value: :ok, stdio: "Hello world!\n"} = result
end
test "plain atom module" do
result = ~E'''
defmodule :hello do
def greet(value) do
IO.puts "Hello #{value}"
end
end
:hello.greet(:world!)
'''
assert %Success{value: :ok, stdio: "Hello world!\n"} = result
end
test "module without other code" do
result = ~E'''
defmodule Hello do
end
'''
assert %Success{
value: {:module, Dune_Module_1__, nil, nil},
inspected: "{:module, Hello, nil, nil}",
stdio: ""
} = result
end
test "default argument" do
result = ~E'''
defmodule My.Default do
def incr(x \\ 0), do: x + 1
end
[My.Default.incr(), My.Default.incr(100)]
'''
assert %Success{value: [1, 101]} = result
end
test "default arguments" do
result = ~E'''
defmodule My.Defaults do
def defaults(a \\ 1, b \\ 2, c) do
[a, b, c]
end
end
{My.Defaults.defaults(:c), My.Defaults.defaults(:a, :c)}
'''
assert %Success{value: {[1, 2, :c], [:a, 2, :c]}} = result
end
test "recursive functions with guards" do
result = ~E'''
defmodule My.List do
def my_sum([]), do: 0
def my_sum([h | t]) when is_number(h), do: h + my_sum(t)
end
My.List.my_sum([1, 100, 1000])
'''
assert %Success{value: 1101} = result
end
test "recursive functions in a nested block" do
result = ~E'''
defmodule My.List do
def my_sum([]), do: 0
def my_sum([h | t]) do
if is_number(h) do
h + my_sum(t)
else
:NaN
end
end
end
My.List.my_sum([1, 100, 1000])
'''
assert %Success{value: 1101} = result
end
test "public and private functions" do
assert %Success{value: "success!"} = ~E'''
defmodule My.Module do
def public, do: private()
defp private, do: "success!"
end
My.Module.public
'''
end
test "recursive private function with guards" do
result = ~E'''
defmodule My.List do
def my_sum(list) when is_list(list), do: my_sum(list, 0)
defp my_sum([], acc), do: acc
defp my_sum([h | t], acc) when is_number(h), do: my_sum(t, h + acc)
end
My.List.my_sum([1, 100, 1000])
'''
assert %Success{value: 1101} = result
end
test "captured fake module functions (external)" do
assert %Success{value: ["Joe (20)", "Jane (27)"]} = ~E'''
defmodule My.Captures do
def format(%{name: name, age: age}) do
"#{name} (#{age})"
end
end
Enum.map(
[%{name: "Joe", age: 20}, %{name: "Jane", age: 27}],
&My.Captures.format/1
)
'''
assert %Success{inspected: ":foo"} = ~E'''
defmodule My.Captures do
def const, do: :foo
end
f = &My.Captures.const/0
f.()
'''
end
test "captured fake module functions (internal)" do
assert %Success{value: ["Joe (20)", "Jane (27)"]} = ~E'''
defmodule My.Captures do
def format_many(list) do
Enum.map(list, &format_one/1)
end
def format_one(%{name: name, age: age}) do
"#{name} (#{age})"
end
end
My.Captures.format_many([%{name: "Joe", age: 20}, %{name: "Jane", age: 27}])
'''
end
test "apply fake module functions" do
assert %Success{value: "Joe (20)"} = ~E'''
defmodule My.Formatter do
def format(%{name: name, age: age}) do
"#{name} (#{age})"
end
end
apply(
My.Formatter,
:format,
[%{name: "Joe", age: 20}]
)
'''
end
test "accept docs and typespecs" do
assert %Success{value: "Joe (20)"} = ~E'''
defmodule My.Formatter do
@moduledoc "Format all the things!"
@typep name :: String.t()
@type user :: %{name: name, age: integer}
@doc "Formats a user"
@spec format(user) :: String.t()
def format(%{name: name, age: age}) do
"#{name} (#{age})"
end
end
My.Formatter.format(%{name: "Joe", age: 20})
'''
end
test "0-arity call without parenthesis" do
assert %Success{value: "success!"} = ~E'''
defmodule My.Module do
def public, do: private()
defp private, do: "success!"
end
My.Module.public
'''
end
# TODO: apply private function
end
describe "exceptions" do
test "function clause error" do
assert %Failure{
type: :exception,
message:
"** (Dune.Eval.FunctionClauseError) no function clause matching in My.Checker.check_age/1: My.Checker.check_age(:invalid)"
} = ~E'''
defmodule My.Checker do
def check_age(age) when is_integer(age) and age >= 18, do: :ok
end
My.Checker.check_age(:invalid)
'''
end
test "calling private function" do
assert %Failure{
type: :exception,
message:
"** (UndefinedFunctionError) function My.Module.private/0 is undefined or private"
} = ~E'''
defmodule My.Module do
def public, do: :public
defp private, do: :private
end
My.Module.private
'''
end
test "conflicting public and private functions" do
assert %Failure{
type: :exception,
message:
"** (Dune.Eval.CompileError) nofile:3: defp conflicting/0 already defined as def in nofile:2"
} = ~E'''
defmodule My.Module do
def conflicting, do: "public"
defp conflicting, do: "private"
end
'''
end
end
describe "Dune restricted" do
test "unsafe function body" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function System.get_env/0 is restricted"
} = ~E'''
defmodule Danger do
def danger() do
System.get_env()
end
end
Danger.danger()
'''
end
test "unsafe function default arg" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function System.get_env/0 is restricted"
} = ~E'''
defmodule Danger do
def danger(env \\ System.get_env()) do
env
end
end
Danger.danger()
'''
end
test "restrictions in the module top level" do
assert %Failure{
type: :module_restricted,
message:
"** (DuneRestrictedError) the following syntax is restricted inside defmodule:\n def no_block"
} = ~E'''
defmodule My.Module do
def no_block
end
'''
assert %Failure{
type: :module_restricted,
message:
"** (DuneRestrictedError) the following syntax is restricted inside defmodule:\n @foo 1 + 1"
} = ~E'''
defmodule My.Module do
@foo 1 + 1
end
'''
end
test "trying to redefine existing module" do
assert %Failure{
type: :module_conflict,
message:
"** (DuneRestrictedError) Following module cannot be defined/redefined: System"
} = ~E'''
defmodule System do
def foo, do: :bar
end
'''
assert %Failure{
type: :module_conflict,
message:
"** (DuneRestrictedError) Following module cannot be defined/redefined: String"
} = ~E'''
defmodule String do
def foo, do: :bar
end
'''
assert %Failure{
type: :module_conflict,
message:
"** (DuneRestrictedError) Following module cannot be defined/redefined: ExUnit"
} = ~E'''
defmodule ExUnit do
def foo, do: :bar
end
'''
assert %Failure{
type: :module_conflict,
message:
"** (DuneRestrictedError) Following module cannot be defined/redefined: Foo"
} = ~E'''
defmodule Foo do
def foo, do: :bar
end
defmodule Foo do
def foo, do: :bar
end
'''
end
test "defmodule used with different arities" do
assert %Failure{
type: :parsing,
message: "dune parsing error: failed to safe parse\n defmodule"
} = ~E'defmodule'
assert %Failure{
type: :parsing,
message: "dune parsing error: failed to safe parse\n defmodule do\n" <> _
} = ~E'''
defmodule do
def foo, do: :bar
end
'''
end
test "defmodule with invalid module names" do
assert %Failure{
type: :exception,
message: "** (Dune.Eval.CompileError) nofile:1: invalid module name: true"
} = ~E'''
defmodule true do
end
'''
assert %Failure{
type: :exception,
message: "** (Dune.Eval.CompileError) nofile:1: invalid module name: nil"
} = ~E'''
defmodule nil do
end
'''
assert %Failure{
type: :exception,
message: "** (Dune.Eval.CompileError) nofile:1: invalid module name: 1"
} = ~E'''
defmodule 1 do
end
'''
end
end
end
================================================
FILE: test/dune_oom_safety_test.exs
================================================
defmodule Dune.AssertionHelper do
alias Dune.Failure
defmacro test_execution_stops(test_name, do: expr) do
quote do
test unquote(test_name) do
# not sure if reductions or memory limit this first
assert %Failure{message: "Execution stopped - " <> _} =
Dune.eval_quoted(unquote(Macro.escape(expr)), timeout: 100)
end
end
end
end
defmodule Dune.OOMSafetyTest do
# Safety integration tests for "structural-sharing bombs" edge cases
# that would cause BIFs to hang and use enormous amounts of memory
use ExUnit.Case, async: true
import Dune.AssertionHelper
test_execution_stops "List.duplicate" do
List.duplicate(:foo, 200_000)
end
# TODO figure out why this fails since Elixir 1.18
@tag :skip
test_execution_stops "String.duplicate" do
String.duplicate("foo", 200_000)
end
describe "structural sharing bombs" do
test_execution_stops "returning value directly" do
Enum.reduce(1..100, ["foo", "bar"], fn _, acc -> [acc, acc] end)
end
test_execution_stops "inspect" do
Enum.reduce(1..100, ["foo", "bar"], fn _, acc -> [acc, acc] end)
|> inspect(limit: :infinity)
end
test_execution_stops "string interpolation" do
bomb = Enum.reduce(1..100, ["foo", "bar"], fn _, acc -> [acc, acc] end)
"#{bomb}!"
end
test_execution_stops "to_string" do
Enum.reduce(1..100, ["foo", "bar"], fn _, acc -> [acc, acc] end) |> to_string()
end
test_execution_stops "List.to_string" do
Enum.reduce(1..100, ["foo", "bar"], fn _, acc -> [acc, acc] end) |> List.to_string()
end
test_execution_stops "IO.iodata_to_binary" do
Enum.reduce(1..100, ["foo", "bar"], fn _, acc -> [acc, acc] end) |> IO.iodata_to_binary()
end
test_execution_stops "IO.iodata_length" do
Enum.reduce(1..100, ["foo", "bar"], fn _, acc -> [acc, acc] end) |> IO.iodata_length()
end
test_execution_stops "IO.chardata_to_string" do
Enum.reduce(1..100, ["foo", "bar"], fn _, acc -> [acc, acc] end) |> IO.chardata_to_string()
end
test_execution_stops "Enum.join" do
Enum.reduce(1..100, ["foo", "bar"], fn _, acc -> [acc, acc] end) |> Enum.join()
end
@tag :lts_only
test_execution_stops "JSON encode key" do
bomb = Enum.reduce(1..100, ["foo", "bar"], fn _, acc -> [acc, acc] end)
JSON.encode!(%{bomb => 123})
end
end
end
================================================
FILE: test/dune_quoted_test.exs
================================================
defmodule DuneQuotedTest do
use ExUnit.Case, async: true
import ExUnit.CaptureIO
alias Dune.{Success, Failure}
defmacrop dune(do: ast) do
escaped_ast = Macro.escape(ast)
quote do
unquote(escaped_ast) |> Dune.eval_quoted(timeout: 100)
end
end
describe "Dune authorized" do
test "simple operations" do
assert %Success{value: 5} = dune(do: 2 + 3)
assert %Success{value: 15} = dune(do: 5 * 3)
assert %Success{value: 0.5} = dune(do: 1 / 2)
assert %Success{value: [1, 2, 3, 4]} = dune(do: [1, 2] ++ [3, 4])
assert %Success{value: "abcd"} = dune(do: "ab" <> "cd")
end
test "basic Kernel functions" do
assert %Success{value: 10} = dune(do: max(5, 10))
assert %Success{value: "foo"} = dune(do: to_string(:foo))
assert %Success{value: true} = dune(do: is_atom(:foo))
end
test "Kernel guards" do
assert %Success{value: false} = dune(do: is_binary(55))
assert %Success{value: true} = dune(do: is_number(55))
end
test "basic String functions" do
assert %Success{value: "JoJo"} = dune(do: String.replace("jojo", "j", "J"))
end
test "basic Map functions" do
assert %Success{value: %{foo: 5}} = dune(do: Map.put(%{}, :foo, 5))
assert %Success{value: %{}} = dune(do: Map.new())
end
test "basic :math functions" do
assert %Success{value: 3.0} = dune(do: :math.log10(1000))
assert %Success{value: 3.141592653589793} = dune(do: :math.pi())
end
test "tuples" do
assert %Success{value: {}} = dune(do: {})
assert %Success{value: {:foo}} = dune(do: {:foo})
assert %Success{value: {"hello", ~c"world"}} = dune(do: {"hello", ~c"world"})
assert %Success{value: {1, 2, 3}} = dune(do: {1, 2, 3})
end
test "map operations" do
assert %Success{value: %{a: :foo, b: 6}} =
(dune do
map = %{a: 5, b: 6}
%{map | a: :foo}
end)
assert %Success{value: "Dio"} =
(dune do
user = %{first_name: "Dio", last_name: "Brando"}
user.first_name
end)
end
test "dynamic module names (authorized functions)" do
assert %Success{value: %{}} =
(dune do
module = Map
module.new()
end)
assert %Success{value: [%{}, %MapSet{}]} =
(dune do
Enum.map([Map, MapSet], fn module -> module.new end)
end)
assert %Success{value: [%{}, %MapSet{}]} =
(dune do
Enum.map([Map, MapSet], fn module -> module.new() end)
end)
end
test "captures" do
assert %Success{value: ["1", "2", "3"]} =
(dune do
1..3 |> Enum.map(&inspect/1)
end)
assert %Success{value: [[0], [1, 0], [2, 0], [3, 0]]} =
(dune do
0..30//10 |> Enum.map(&Integer.digits/1)
end)
assert %Success{value: 3.317550714905183e39} =
(dune do
1..100//10 |> Enum.map(&:math.exp/1) |> Enum.sum()
end)
end
test "sigils" do
assert %Success{value: %Regex{source: "(a|b)?c"}} = dune(do: ~r/(a|b)?c/)
assert %Success{value: ~U[2021-05-20 01:02:03Z]} = dune(do: ~U[2021-05-20 01:02:03Z])
assert %Success{value: [~c"foo", ~c"bar", ~c"baz"]} = dune(do: ~W[foo bar baz]c)
assert %Success{value: [~c"foo", ~c"bar", ~c"baz"]} =
dune(do: ~w[#{String.downcase("FOO")} bar baz]c)
assert %Dune.Failure{
message: "** (DuneRestrictedError) function sigil_W/2 is restricted",
type: :restricted
} = dune(do: ~W[foo bar baz]a)
end
test "block of code" do
assert %Success{value: "quick-brown-fox-jumps-over-lazy-dog"} =
(dune do
sentence = "the quick brown fox jumps over the lazy dog"
words = String.split(sentence)
filtered = Enum.reject(words, &(&1 == "the"))
Enum.join(filtered, "-")
end)
end
test "pipe operator" do
assert %Success{value: "quick-brown-fox-jumps-over-lazy-dog"} =
(dune do
"the quick brown fox jumps over the lazy dog"
|> String.split()
|> Enum.reject(&(&1 == "the"))
|> Enum.join("-")
end)
end
test "if block" do
assert %Success{value: {:foo, 6}} =
(dune do
x = 6
if x > 5 do
{:foo, x}
else
{:bar, x}
end
end)
end
test "case block" do
assert %Success{value: {:bar, 4}} =
(dune do
case 4 do
x when x > 5 -> {:foo, x}
y -> {:bar, y}
end
end)
end
test "for block" do
assert %Success{value: [:b, :c]} =
(dune do
for {x, y} <- [a: 1, b: 2, c: 3], y > 1, do: x
end)
end
end
describe "Dune restricted" do
test "System calls" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function System.get_env/0 is restricted"
} = dune(do: System.get_env())
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function System.get_env/1 is restricted"
} = dune(do: System.get_env("TEST"))
end
test "Code calls" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = dune(do: Code.eval_string("IO.puts(:hello)"))
end
test "String/List restricted methods" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function String.to_atom/1 is restricted"
} = dune(do: String.to_atom("foo"))
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function List.to_atom/1 is restricted"
} = dune(do: List.to_atom(~c"foo"))
assert %Failure{
type: :restricted,
message:
"** (DuneRestrictedError) function String.to_existing_atom/1 is restricted"
} = dune(do: String.to_existing_atom("foo"))
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function List.to_existing_atom/1 is restricted"
} = dune(do: List.to_existing_atom(~c"foo"))
end
test "atom interpolation" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function :erlang.binary_to_atom/2 is restricted"
} = dune(do: :"#{1 + 1} is two")
end
test "Kernel apply/3" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Process.get/0 is restricted"
} =
(dune do
apply(Process, :get, [])
end)
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function IO.puts/2 is restricted"
} =
(dune do
apply(IO, :puts, [:stderr, "Hello"])
end)
end
test ". operator with variable modules" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function :erlang.yield/0 is restricted"
} =
(dune do
module = :erlang
module.yield
end)
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function :erlang.processes/0 is restricted"
} =
(dune do
Enum.map([:erlang], fn module -> module.processes end)
end)
end
test "capture operator" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = dune(do: f = &Code.eval_string/1)
end
test "Kernel 0-arity functions" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function binding/0 is restricted"
} =
(dune do
binding
end)
end
test "erlang unsafe libs" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function :erlang.processes/0 is restricted"
} = dune(do: :erlang.processes())
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function :erlang.get/0 is restricted"
} = dune(do: :erlang.get())
end
test "nested restricted code" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} =
(dune do
f = fn -> Code.eval_string("IO.puts(:hello)") end
end)
end
end
describe "process restrictions" do
test "execution timeout" do
assert %Failure{type: :timeout, message: "Execution timeout - 100ms"} =
dune(do: Process.sleep(101))
end
test "too many reductions" do
assert %Failure{type: :reductions, message: "Execution stopped - reductions limit exceeded"} =
dune(do: Enum.any?(1..1_000_000, &(&1 < 0)))
end
test "uses to much memory" do
assert %Failure{type: :memory, message: "Execution stopped - memory limit exceeded"} =
dune(do: List.duplicate(:foo, 100_000))
end
end
describe "error handling" do
test "math error" do
assert %Failure{
type: :exception,
message: "** (ArithmeticError) bad argument in arithmetic expression\n" <> rest
} = dune(do: 42 / 0)
assert rest =~ ":erlang./(42, 0)"
end
test "throw" do
assert %Failure{type: :throw, message: "** (throw) :yo"} = dune(do: throw(:yo))
end
test "raise" do
assert %Failure{type: :exception, message: "** (ArgumentError) kaboom!"} =
(dune do
raise ArgumentError, "kaboom!"
end)
end
test "actual UndefinedFunctionError" do
assert %Failure{
type: :exception,
message: "** (UndefinedFunctionError) function Code.baz/0 is undefined or private"
} = dune(do: Code.baz())
assert %Failure{
type: :exception,
message:
"** (UndefinedFunctionError) function FooBar.baz/0 is undefined (module FooBar is not available)"
} = dune(do: FooBar.baz())
assert %Failure{
type: :exception,
message:
"** (UndefinedFunctionError) function :foo_bar.baz/0 is undefined (module :foo_bar is not available)"
} = dune(do: :foo_bar.baz())
end
@tag :lts_only
test "undefined variable" do
capture_io(:stderr, fn ->
# TODO better error messages
assert %Failure{
type: :compile_error,
message:
"** (CompileError) nofile: cannot compile file (errors have been logged)",
stdio: "error: undefined variable \"y\"\n" <> _
} = dune(do: y)
assert %Failure{
type: :compile_error,
message:
"** (CompileError) nofile: cannot compile file (errors have been logged)",
stdio: "error: undefined variable \"x\"\n" <> _
} = dune(do: if(x, do: x))
end)
end
end
end
================================================
FILE: test/dune_string_test.exs
================================================
defmodule DuneStringTest do
use ExUnit.Case, async: true
alias Dune.{Success, Failure}
defmacro sigil_E(call, _expr) do
quote do
Dune.eval_string(unquote(call), timeout: 100, inspect_sort_maps: true)
end
end
describe "Dune authorized" do
test "simple operations" do
assert %Success{value: 5, inspected: ~S'5'} = ~E'2 + 3'
assert %Success{value: 15, inspected: ~S'15'} = ~E'5 * 3'
assert %Success{value: 0.5, inspected: ~S'0.5'} = ~E'1 / 2'
assert %Success{value: [1, 2, 3, 4], inspected: ~S'[1, 2, 3, 4]'} = ~E'[1, 2] ++ [3, 4]'
assert %Success{value: "abcd", inspected: ~S'"abcd"'} = ~E'"ab" <> "cd"'
assert %Success{value: true, inspected: ~S'true'} = ~E'"abc!" =~ ~r/abc/'
end
test "basic Kernel functions" do
assert %Success{value: 10, inspected: ~S'10'} = ~E'max(5, 10)'
assert %Success{value: "foo", inspected: ~S'"foo"'} = ~E'to_string(:foo)'
assert %Success{value: true, inspected: ~S'true'} = ~E'is_atom(:foo)'
end
test "Kernel guards" do
assert %Success{value: false, inspected: ~S'false'} = ~E'is_binary(55)'
assert %Success{value: true, inspected: ~S'true'} = ~E'is_number(55)'
end
test "basic String functions" do
assert %Success{value: "JoJo", inspected: ~S'"JoJo"'} = ~E'String.replace("jojo", "j", "J")'
end
test "basic Map functions" do
assert %Success{value: %{foo: 5}, inspected: ~S'%{foo: 5}'} = ~E'Map.put(%{}, :foo, 5)'
assert %Success{value: %{}, inspected: ~S'%{}'} = ~E'Map.new()'
end
test "basic :math functions" do
assert %Success{value: 3.0, inspected: ~S'3.0'} = ~E':math.log10(1000)'
assert %Success{value: 3.141592653589793, inspected: ~S'3.141592653589793'} = ~E':math.pi()'
end
@tag :lts_only
test "tuples" do
assert %Success{value: {}, inspected: ~S'{}'} = ~E'{}'
assert %Success{value: {:foo}, inspected: ~S'{:foo}'} = ~E'{:foo}'
assert %Success{value: {"hello", ~c"world"}, inspected: ~S/{"hello", ~c"world"}/} =
~E/{"hello", 'world'}/
assert %Success{value: {1, 2, 3}, inspected: ~S'{1, 2, 3}'} = ~E'{1, 2, 3}'
end
test "map operations" do
assert %Success{value: %{a: :foo, b: 6}, inspected: ~S'%{a: :foo, b: 6}'} = ~E'
map = %{a: 5, b: 6}
%{map | a: :foo}
'
assert %Success{
value: "Dio",
inspected: ~S'"Dio"'
} = ~E'
user = %{first_name: "Dio", last_name: "Brando"}
user.first_name
'
end
test "dynamic module names (authorized functions)" do
assert %Success{value: %{}, inspected: ~S'%{}'} = ~E'module = Map; module.new()'
assert %Success{value: [%{}], inspected: ~S'[%{}]'} =
~E'Enum.map([Map], fn module -> module.new end)'
assert %Success{value: [%{}], inspected: ~S'[%{}]'} =
~E'Enum.map([Map], fn module -> module.new() end)'
assert %Success{value: 3, inspected: ~S'3'} = ~E'apply(List, :last, [[1, 2, 3]])'
end
test "captures" do
assert %Success{value: 33, inspected: ~S'33'} = ~E'f = &+/2; f.(11, 22)'
assert %Success{value: 35, inspected: ~S'35'} = ~E'f = & &1*&2; f.(5, 7)'
assert %Success{value: 1.5, inspected: ~S'1.5'} = ~E'f = & &1/&2; 3 |> f.(2)'
assert %Success{value: 20, inspected: ~S'20'} = ~E'apply(& &1 * 2, [10])'
assert %Success{value: ["1", "2", "3"], inspected: ~S'["1", "2", "3"]'} =
~E'1..3 |> Enum.map(&inspect/1)'
assert %Success{
value: [[0], [1, 0], [2, 0], [3, 0]],
inspected: ~S'[[0], [1, 0], [2, 0], [3, 0]]'
} = ~E'0..30//10 |> Enum.map(&Integer.digits/1)'
assert %Success{value: 3.317550714905183e39, inspected: ~S'3.317550714905183e39'} =
~E'1..100//10 |> Enum.map(&:math.exp/1) |> Enum.sum()'
end
test "anonymous functions" do
assert %Success{value: 0, inspected: ~S'0'} = ~E'f = fn -> _x = 0 end; f.()'
assert %Success{value: 0, inspected: ~S'0'} = ~E'(fn -> _x = 0 end).()'
assert %Success{value: 3, inspected: ~S'3'} = ~E'2 |> (fn x -> x + 1 end).()'
assert %Success{value: -4, inspected: ~S'-4'} = ~E'(&(&2 - &1)).(7, 3)'
assert %Success{value: -4, inspected: ~S'-4'} = ~E'(& &2 - &1).(7, 3)'
assert %Success{value: 3, inspected: ~S'3'} = ~E'2 |> (& &1 + 1).()'
assert %Success{value: 0.2, inspected: ~S'0.2'} = ~E'(& &2 / &1).(10, 2)'
end
@tag :lts_only
test "sigils" do
assert %Success{value: %Regex{source: "(a|b)?c"}, inspected: ~S'~r/(a|b)?c/'} =
~E'~r/(a|b)?c/'
assert %Success{value: ~U[2021-05-20 01:02:03Z], inspected: ~S'~U[2021-05-20 01:02:03Z]'} =
~E'~U[2021-05-20 01:02:03Z]'
assert %Success{
value: [~c"foo", ~c"bar", ~c"baz"],
inspected: ~S'[~c"foo", ~c"bar", ~c"baz"]'
} = ~E'~W[foo bar baz]c'
assert %Success{
value: [~c"foo", ~c"bar", ~c"baz"],
inspected: ~S'[~c"foo", ~c"bar", ~c"baz"]'
} = ~E'~w[#{String.downcase("FOO")} bar baz]c'
assert %Dune.Failure{
message: "** (DuneRestrictedError) function sigil_W/2 is restricted",
type: :restricted
} = ~E'~W[foo bar baz]a'
assert %Dune.Failure{
message: "** (DuneRestrictedError) function sigil_w/2 is restricted",
type: :restricted
} = ~E'~w[#{String.downcase("FOO")} bar baz]a'
end
@tag :lts_only
test "bitstring modifiers" do
assert %Success{
value: <<3>>,
inspected: "<<3>>"
} = ~E'<<3>>'
assert %Success{
value: <<3::4>>,
inspected: "<<3::size(4)>>"
} = ~E'<<3::4>>'
assert %Success{
value: "ߧ",
inspected: ~S'"ߧ"'
} = ~E'<<2023::utf8>>'
assert %Success{
value: 12520,
inspected: "12520"
} = ~E'<<c::utf8>> = "ヨ"; c'
assert %Success{
value: <<3::4>>,
inspected: "<<3::size(4)>>"
} = ~E'<<3::size(4)>>'
assert %Success{
value: 6_382_179,
inspected: "6382179"
} = ~E'<<c::size(24)>> = "abc"; c'
assert %Success{
value: <<2, 3>>,
inspected: "<<2, 3>>"
} = ~E'<<_::binary-size(2), rest::binary>> = <<0, 1, 2, 3>>; rest'
assert %Success{
value: <<0, 0, 0, 1>>,
inspected: "<<0, 0, 0, 1>>"
} = ~E'''
x = 1
<<x::8*4>>
'''
assert %Success{
value: <<0, 0, 0, 1>>,
inspected: "<<0, 0, 0, 1>>"
} = ~E'''
x = 1
<<x::size(8)-unit(4)>>
'''
assert %Success{
value: {"Frank", "Walrus"},
inspected: ~S'{"Frank", "Walrus"}'
} = ~E'''
name_size = 5
<<name::binary-size(^name_size), " the ", species::binary>> = <<"Frank the Walrus">>
{name, species}
{"Frank", "Walrus"}
'''
end
test "binary comprehensions" do
assert %Success{
value: [{213, 45, 132}, {64, 76, 32}],
inspected: "[{213, 45, 132}, {64, 76, 32}]"
} = ~E'''
pixels = <<213, 45, 132, 64, 76, 32>>
for <<r::8, g::8, b::8 <- pixels>>, do: {r, g, b}
'''
end
test "block of code" do
assert %Success{
value: "quick-brown-fox-jumps-over-lazy-dog",
inspected: ~S'"quick-brown-fox-jumps-over-lazy-dog"'
} = ~E'
sentence = "the quick brown fox jumps over the lazy dog"
words = String.split(sentence)
filtered = Enum.reject(words, &(&1 == "the"))
Enum.join(filtered, "-")
'
end
test "pipe operator" do
assert %Success{
value: "quick-brown-fox-jumps-over-lazy-dog",
inspected: ~S'"quick-brown-fox-jumps-over-lazy-dog"'
} = ~E'
"the quick brown fox jumps over the lazy dog"
|> String.split()
|> Enum.reject(&(&1 == "the"))
|> Enum.join("-")
'
assert %Success{value: ":foo", inspected: ~S'":foo"'} = ~E':foo |> inspect()'
end
test "atoms" do
assert %Success{value: "foo51", inspected: ~s'"foo51"'} = ~E'to_string(:foo51)'
assert %Success{value: "foo52", inspected: ~s'"foo52"'} = ~E'Atom.to_string(:foo52)'
assert %Success{value: "foo53", inspected: ~s'"foo53"'} = ~E'Enum.join([:foo53])'
assert %Success{value: "boo57", inspected: ~s'"boo57"'} =
~E':foo57 |> to_string() |> String.replace("f", "b")'
assert %Success{value: "Hello boo58", inspected: ~s'"Hello boo58"'} =
~E'"Hello #{:foo58}" |> String.replace("f", "b")'
assert %Success{value: :Dune_Atom_1__, inspected: ~s':Foo12'} = ~E':Foo12'
assert %Success{value: Dune_Module_1__, inspected: ~s'Foo13'} = ~E'Foo13'
assert %Success{value: "Foo14", inspected: ~s'"Foo14"'} = ~E'Atom.to_string(:Foo14)'
assert %Success{value: "Elixir.Foo15", inspected: ~s("Elixir.Foo15")} =
~E'Atom.to_string(Foo15)'
assert %Success{value: ":Foo16", inspected: ~s'":Foo16"'} = ~E'inspect(:Foo16)'
assert %Success{value: "Foo17", inspected: ~s'"Foo17"'} = ~E'inspect(Foo17)'
assert %Success{value: "Elixir.Foo.Bar33", inspected: ~s("Elixir.Foo.Bar33")} =
~E'Atom.to_string(Foo.Bar33)'
assert %Success{
value: [
Dune_Module_1__,
{:a__Dune_atom_2__, :Dune_Atom_1__},
[a__Dune_atom_2__: 15, Dune_Atom_1__: 6, __Dune_atom_3__: 33]
],
inspected: ~s([Foo91, {:foo91, :Foo91}, [foo91: 15, Foo91: 6, _foo92: 33]])
} = ~E'[Foo91, {:foo91, :Foo91}, [foo91: 15, Foo91: 6, _foo92: 33]]'
end
@tag :lts_only
test "atoms to charlist" do
assert %Success{value: ~c"Hello foo59", inspected: ~s'~c"Hello foo59"'} =
~E'~c"Hello #{:foo59}"'
assert %Success{value: ~c"Elixir.Foo15", inspected: ~s(~c"Elixir.Foo15")} =
~E'Atom.to_charlist(Foo15)'
end
test "atoms (prefixed by Elixir)" do
assert %Success{value: Elixir, inspected: ~s'Elixir'} = ~E'Elixir'
assert %Success{value: Dune_Module_1__, inspected: ~s'Foo13'} = ~E'Elixir.Foo13'
assert %Success{value: Dune_Module_1__, inspected: ~s'Foo13'} = ~E':"Elixir.Foo13"'
assert %Success{value: true, inspected: ~s'true'} = ~E'Elixir.Foo13 == Foo13'
assert %Success{value: true, inspected: ~s'true'} = ~E':"Elixir.Foo13" == Foo13'
assert %Success{value: true, inspected: ~s'true'} = ~E'Elixir.Foo13.Foo13 == Foo13.Foo13'
assert %Success{value: true, inspected: ~s'true'} = ~E':"Elixir.Foo13.Foo13" == Foo13.Foo13'
assert %Success{value: String, inspected: ~s'String'} = ~E':"Elixir.String"'
assert %Success{value: Dune_Module_1__, inspected: ~s'Elixir.Elixir'} = ~E'Elixir.Elixir'
assert %Success{value: Dune_Module_1__, inspected: ~s'Elixir.Elixir'} = ~E':"Elixir.Elixir"'
end
test "atoms (wrapped with quotes)" do
assert %Success{value: :__Dune_atom_1__, inspected: ~s':" "'} = ~E':" "'
assert %Success{value: :__Dune_atom_1__, inspected: ~s':"foo/bar"'} = ~E':"foo/bar"'
assert %Success{value: " ", inspected: ~s'" "'} = ~E'to_string :" "'
assert %Success{value: "foo/bar", inspected: ~s'"foo/bar"'} = ~E'to_string :"foo/bar"'
assert %Success{
value: [
__Dune_atom_1__: {:__Dune_atom_2__, :__Dune_atom_3__},
__Dune_atom_4__: %{__Dune_atom_5__: :__Dune_atom_6__, a__Dune_atom_7__: 6}
],
inspected: ~s([" ": {:"\t", :" A"}, "ab cd": %{"foo+91": :"15", abc: 6}])
} = ~E([" ": {:"\t", :" A"}, "ab cd": %{"foo+91": :'15', abc: 6}])
end
test "function and atom parameters" do
assert ":digits" = ~E':digits'.value |> inspect()
assert ":turkic" = ~E':turkic'.value |> inspect()
end
test "stdio capture" do
assert %Success{value: :ok, inspected: ~s(:ok), stdio: "yo!\n"} = ~E'IO.puts("yo!")'
assert %Success{value: :ok, inspected: ~s(:ok), stdio: "foo987\n"} = ~E'IO.puts(:foo987)'
assert %Success{value: :ok, inspected: ~s(:ok), stdio: "hello world!\n"} =
~E'io = IO; io.puts(["hello", ?\s, "world", ?!])'
assert %Success{value: :ok, inspected: ~s(:ok), stdio: "1\n2\n3\n"} =
~E'Enum.each(1..3, &IO.puts/1)'
assert %Success{value: :a__Dune_atom_1__, inspected: ~s(:foo912), stdio: ":foo912\n"} =
~E'IO.inspect(:foo912)'
assert %Success{
value: %{a__Dune_atom_1__: 581},
inspected: ~s(%{foo9101: 581}),
stdio: "bar777: %{foo9101: 581}\n"
} = ~E'%{foo9101: 581} |> IO.inspect(label: :bar777)'
assert %Success{
value: :ok,
inspected: ":ok",
stdio: ":foo9321\nfoo9321\n"
} = ~E'io = IO; io.puts(io.inspect(:foo9321))'
end
test "dbg" do
assert %Success{value: :a__Dune_atom_1__, inspected: ~s(:foo913), stdio: stdio} =
~E'dbg(:foo913)'
assert stdio == """
[nofile:1: (file)]
:foo913 #=> :foo913
"""
assert %Success{value: :a__Dune_atom_1__, inspected: ~s(:foo914), stdio: stdio} =
~E':foo914 |> dbg()'
assert stdio == """
[nofile:1: (file)]
:foo914 #=> :foo914
"""
assert %Success{value: ":foo915", inspected: ~s(":foo915"), stdio: stdio} =
~E':foo915 |> inspect() |> dbg()'
assert stdio == """
[nofile:1: (file)]
:foo915 #=> :foo915
|> inspect() #=> ":foo915"
"""
assert %Success{value: "Hello World", inspected: ~s("Hello World"), stdio: stdio} =
~E'"hello world" |> String.split() |> Enum.map_join(" ", &String.capitalize/1) |> dbg()'
assert stdio == """
[nofile:1: (file)]
"hello world" #=> "hello world"
|> String.split() #=> ["hello", "world"]
|> Enum.map_join(" ", &String.capitalize/1) #=> "Hello World"
"""
assert %Success{value: ":foo987398", inspected: ~s(":foo987398"), stdio: stdio} =
~E':foo987398 |> inspect() |> dbg() |> dbg()'
assert stdio == """
[nofile:1: (file)]
:foo987398 #=> :foo987398
|> inspect() #=> ":foo987398"
[nofile:1: (file)]
:foo987398 |> inspect() |> dbg() #=> ":foo987398"
"""
assert %Dune.Failure{
type: :restricted,
message: "** (DuneRestrictedError) function dbg/0 is restricted"
} = ~E'dbg()'
assert %Dune.Failure{
type: :restricted,
message: "** (DuneRestrictedError) function dbg/2 is restricted"
} = ~E'dbg(:abc, syntax_colors: [])'
assert %Dune.Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = ~E'Code.eval_string(":hello") |> dbg()'
assert %Dune.Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = ~E'":hello" |> Code.eval_string() |> dbg()'
end
test "pretty option" do
raw_string =
~S'{"This line is really long, maybe we should break", [%{bar: 1, baz: 2}, %{bar: 55}]}'
with_break =
~s'{"This line is really long, maybe we should break",\n [%{bar: 1, baz: 2}, %{bar: 55}]}'
assert %Success{inspected: ^raw_string} =
Dune.eval_string(raw_string, inspect_sort_maps: true)
assert %Success{inspected: ^with_break} =
Dune.eval_string(raw_string, pretty: true, inspect_sort_maps: true)
end
end
describe "Dune restricted" do
test "System calls" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function System.get_env/0 is restricted"
} = ~E'System.get_env()'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function System.get_env/1 is restricted"
} = ~E'System.get_env("TEST")'
end
test "Code calls" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = ~E'Code.eval_string("IO.puts(:hello)")'
end
test "String/List restricted methods" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function String.to_atom/1 is restricted"
} = ~E'String.to_atom("foo")'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function List.to_atom/1 is restricted"
} = ~E/List.to_atom('foo')/
end
test "atom interpolation" do
assert %Failure{
type: :restricted,
message:
"** (DuneRestrictedError) function :erlang.binary_to_existing_atom/2 is restricted"
} = ~E':"#{1 + 1} is two"'
end
test "Kernel apply/3" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Process.get/0 is restricted"
} = ~E'apply(Process, :get, [])'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function IO.puts/2 is restricted"
} = ~E'apply(IO, :puts, [:stderr, "Hello"])'
end
test ". operator with variable modules" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function :erlang.yield/0 is restricted"
} = ~E'
module = :erlang
module.yield
'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function :erlang.processes/0 is restricted"
} = ~E'
Enum.map([:erlang], fn module -> module.processes end)
'
end
test ". operator as key access" do
assert %Success{value: 100, inspected: ~S'100'} =
~E'users = [john: %{age: 100}]; users[:john].age'
end
test ". operator various failures" do
assert %Failure{
type: :exception,
message:
"** (UndefinedFunctionError) function :foo.bar/0 is undefined (module :foo is not available)"
} = ~E'module = :foo; module.bar()'
assert %Failure{
type: :exception,
message: "** (UndefinedFunctionError) function List.bar/0 is undefined or private"
} = ~E'module = List; module.bar()'
assert %Failure{
type: :exception,
message: "** (KeyError) key :job not found in" <> rest_message
} = ~E'users = [john: %{age: 100}]; users[:john].job'
assert rest_message =~ "%{age: 100}"
assert %Failure{
type: :exception,
message:
"** (UndefinedFunctionError) function Foo1234.bar567/0 is undefined (module Foo1234 is not available)"
} = ~E'Foo1234.bar567.baz890'
end
test "pipe operator" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = ~E'":foo" |> Code.eval_string()'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = ~E'code = Code; "1 + 1" |> code.eval_string()'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = ~E'code = Code; "1 + 1" |> code.eval_string'
end
test "capture operator" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = ~E'f = &Code.eval_string/1'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = ~E'f = &Code.eval_string(&1)'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = ~E'(&Code.eval_string/1).(":pawned!")'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = ~E'(&Code.eval_string(&1)).(":pawned!")'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = ~E'":pawned!" |> (&Code.eval_string/1).()'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = ~E'":pawned!" |> (&Code.eval_string(&1)).()'
end
test "Kernel 0-arity functions" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function binding/0 is restricted"
} = ~E'binding'
end
test "erlang unsafe libs" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function :erlang.processes/0 is restricted"
} = ~E':erlang.processes()'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function :erlang.get/0 is restricted"
} = ~E':erlang.get()'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function :erlang.process_info/1 is restricted"
} = ~E':erlang.process_info(self)'
end
test "nested restricted code" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function Code.eval_string/1 is restricted"
} = ~E'f = fn -> Code.eval_string("IO.puts(:hello)") end'
end
test "partially restricted shims" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function IO.puts/2 is restricted"
} = ~E'IO.puts(:stderr, "foo")'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function IO.puts/2 is restricted"
} = ~E':stderr |> IO.puts("foo")'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function IO.inspect/3 is restricted"
} = ~E'IO.inspect(:stderr, "foo", [])'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function IO.inspect/3 is restricted"
} = ~E'io = IO; io.inspect(:stderr, "foo", [])'
end
test "forbidden atoms" do
assert %Failure{
type: :restricted,
message: "Atoms containing `Dune` are restricted for safety: Dune"
} = ~E'Dune'
assert %Failure{
type: :restricted,
message: "Atoms containing `Dune` are restricted for safety: Dune"
} = ~E'Dune.Foo'
assert %Failure{
type: :restricted,
message: "Atoms containing `Dune` are restricted for safety: Dune"
} = ~E':Dune'
assert %Failure{
type: :restricted,
message: "Atoms containing `Dune` are restricted for safety: __Dune__"
} = ~E':__Dune__'
end
test "forbidden use" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function use/1 is restricted"
} = ~E'use GenServer'
end
test "forbidden import/requires" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function import/1 is restricted"
} = ~E'import Logger'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function import/2 is restricted"
} = ~E'import Logger, only: [info: 2]'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function require/1 is restricted"
} = ~E'require Logger'
end
test "forbidden alias" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function alias/1 is restricted"
} = ~E'alias Task.Supervised'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function alias/2 is restricted"
} = ~E'alias Process, as: P; P.get'
end
test "forbidden quote/unquote" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function quote/1 is restricted"
} = ~E'quote do: 1 + 1'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function quote/1 is restricted"
} = ~E'quote do: unquote(a)'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function unquote/1 is restricted"
} = ~E'unquote(10)'
end
test "forbidden receive" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function receive/1 is restricted"
} = ~E'''
receive do
{:ok, foo} -> foo
end
'''
end
test "forbidden __ENV__" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function __ENV__/0 is restricted"
} = ~E'__ENV__'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) function __ENV__/0 is restricted"
} = ~E'__ENV__.requires'
end
test "bitstring modifiers" do
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) size modifiers above 256 are restricted"
} = ~E'<<0::123456789123456789>>'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) size modifiers above 256 are restricted"
} = ~E'<<0::size(257)>>'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) size modifiers above 256 are restricted"
} = ~E'<<0::256*2>>'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) size modifiers above 256 are restricted"
} = ~E'<<0::integer-size(257)>>'
assert %Failure{
type: :restricted,
message: "** (DuneRestrictedError) size modifiers above 256 are restricted"
} = ~E'<<0::integer-size(256)-unit(2)>>'
assert %Failure{
type: :restricted,
message:
"** (DuneRestrictedError) bitstring modifier is restricted:\n size(x)"
} = ~E'''
x = 123456789123456789
<<0::size(x)>>
'''
assert %Failure{
message:
"** (DuneRestrictedError) bitstring modifier is restricted:\n unquote(1)"
} = ~E'<<1::unquote(1)>>'
assert %Failure{
message:
"** (DuneRestrictedError) bitstring modifier is restricted:\n unquote(1)"
} = ~E'<<1::integer-unquote(1)>>'
assert %Failure{
message: "** (DuneRestrictedError) function unquote/1 is restricted"
} = ~E'<<unquote(1)>>'
end
end
describe "process restrictions" do
test "execution timeout" do
assert %Failure{type: :timeout, message: "Execution timeout - 100ms"} =
~E'Process.sleep(101)'
end
test "too many reductions" do
assert %Failure{type: :reductions, message: "Execution stopped - reductions limit exceeded"} =
~E'Enum.any?(1..1_000_000, &(&1 < 0))'
end
test "uses to much memory" do
assert %Failure{type: :memory, message: "Execution stopped - memory limit exceeded"} =
~E'List.duplicate(:foo, 100_000)'
end
end
describe "error handling" do
test "math error" do
assert %Failure{
type: :exception,
message: "** (ArithmeticError) bad argument in arithmetic expression\n" <> rest
} = ~E'42 / 0'
assert rest =~ ":erlang./(42, 0)"
end
test "throw" do
assert %Failure{type: :throw, message: "** (throw) :yo"} = ~E'throw(:yo)'
assert %Failure{type: :throw, message: "** (throw) {:undefined_function, Kernel, :+, 2}"} =
~E'throw({:undefined_function, Kernel, :+, 2})'
end
test "raise" do
assert %Failure{type: :exception, message: "** (ArgumentError) kaboom!"} =
~E'raise ArgumentError, "kaboom!"'
end
test "actual UndefinedFunctionError" do
assert %Failure{
type: :exception,
message: "** (UndefinedFunctionError) function Code.baz/0 is undefined or private"
} = ~E'Code.baz()'
assert %Failure{
type: :exception,
message:
"** (UndefinedFunctionError) function FooBar.baz/0 is undefined (module FooBar is not available)"
} = ~E'FooBar.baz()'
assert %Failure{
type: :exception,
message:
"** (UndefinedFunctionError) function :foo_bar.baz/0 is undefined (module :foo_bar is not available)"
} = ~E':foo_bar.baz()'
end
@tag :lts_only
test "syntax error" do
assert %Failure{
type: :parsing,
message: "missing terminator: )"
} = ~E'foo('
assert %Failure{
type: :parsing,
message: "missing terminator: }"
} = ~E'{'
assert %Failure{
type: :parsing,
message: "unexpected reserved word: do. In case you wanted to write " <> _
} = ~E'if true, do'
# TODO improve message
assert %Failure{type: :parsing, message: "syntax error before: "} = ~E'%'
assert %Failure{type: :parsing, message: "syntax error before: foo120987"} =
~E'<<>>foo120987'
end
test "max length" do
assert %Failure{
type: :parsing,
message: "max code length exceeded: 26 > 10"
} = Dune.eval_string("exceeeds_max_length = true", max_length: 10)
end
test "atom pool" do
assert %Failure{
type: :parsing,
message: "atom_pool_size exceeded, failed to parse atom: bar1462"
} = Dune.eval_string("{foo5345, bar1462} = {9, 10}", atom_pool_size: 4)
end
test "invalid pipe" do
assert %Failure{
type: :exception,
message: "** (ArgumentError) cannot pipe 1 into 2, can only pipe into " <> _
} = ~E'1 |> 2'
assert %Failure{
type: :exception,
m
gitextract_bo31q11i/
├── .formatter.exs
├── .github/
│ └── workflows/
│ └── ci.yml
├── .gitignore
├── CHANGELOG.md
├── LICENSE.md
├── README.md
├── lib/
│ ├── dune/
│ │ ├── allowlist/
│ │ │ ├── default.ex
│ │ │ ├── docs.ex
│ │ │ └── spec.ex
│ │ ├── allowlist.ex
│ │ ├── atom_mapping.ex
│ │ ├── eval/
│ │ │ ├── env.ex
│ │ │ ├── fake_module.ex
│ │ │ ├── function_clause_error.ex
│ │ │ ├── macro_env.ex
│ │ │ └── process.ex
│ │ ├── eval.ex
│ │ ├── failure.ex
│ │ ├── helpers/
│ │ │ ├── diagnostics.ex
│ │ │ └── term_checker.ex
│ │ ├── opts.ex
│ │ ├── parser/
│ │ │ ├── atom_encoder.ex
│ │ │ ├── compile_env.ex
│ │ │ ├── debug.ex
│ │ │ ├── real_module.ex
│ │ │ ├── safe_ast.ex
│ │ │ ├── sanitizer.ex
│ │ │ ├── string_parser.ex
│ │ │ └── unsafe_ast.ex
│ │ ├── parser.ex
│ │ ├── session.ex
│ │ ├── shims/
│ │ │ ├── atom.ex
│ │ │ ├── enum.ex
│ │ │ ├── io.ex
│ │ │ ├── json.ex
│ │ │ ├── kernel.ex
│ │ │ ├── list.ex
│ │ │ └── string.ex
│ │ └── success.ex
│ └── dune.ex
├── mix.exs
└── test/
├── dune/
│ ├── allowlist/
│ │ └── default_test.exs
│ ├── allowlist_test.exs
│ ├── atom_mapping_test.exs
│ ├── opts_test.exs
│ ├── parser/
│ │ ├── atom_encoder_test.exs
│ │ └── string_parser_test.exs
│ ├── session_test.exs
│ ├── shims_test.exs
│ └── validation_test.exs
├── dune_modules_test.exs
├── dune_oom_safety_test.exs
├── dune_quoted_test.exs
├── dune_string_test.exs
├── dune_string_to_quoted_test.exs
├── dune_test.exs
└── test_helper.exs
SYMBOL INDEX (257 symbols across 49 files)
FILE: lib/dune.ex
class Dune (line 1) | defmodule Dune
method eval_quoted (line 98) | def eval_quoted(ast, opts \\ []) do
FILE: lib/dune/allowlist.ex
class Dune.Allowlist (line 1) | defmodule Dune.Allowlist
method extract_extend_opt (line 155) | defp extract_extend_opt(opts, caller) do
method __postprocess__ (line 166) | def __postprocess__(module) do
method generate_spec (line 187) | defp generate_spec(module, extend) do
method def_spec (line 200) | defp def_spec(spec) do
method def_fun_status (line 210) | defp def_fun_status(spec) do
method update_module_doc (line 235) | defp update_module_doc(module, spec) do
method alias_atoms (line 250) | defp alias_atoms(spec) do
FILE: lib/dune/allowlist/default.ex
class Dune.Allowlist.Default (line 1) | defmodule Dune.Allowlist.Default
FILE: lib/dune/allowlist/docs.ex
class Dune.Allowlist.Docs (line 1) | defmodule Dune.Allowlist.Docs
method document_allowlist (line 4) | def document_allowlist(spec) do
method do_doc_funs (line 22) | defp do_doc_funs({module, grouped_funs}) do
method format_fun (line 39) | defp format_fun(module, {fun, arity}, status, public_funs) do
method maybe_strike (line 67) | defp maybe_strike(:restricted), do: "~~"
method maybe_strike (line 68) | defp maybe_strike(_status), do: []
method format_status (line 70) | defp format_status(:allowed), do: "Allowed"
method format_status (line 71) | defp format_status(:shimmed), do: "Alernative implementation"
method format_status (line 72) | defp format_status(:restricted), do: "Restricted"
FILE: lib/dune/allowlist/spec.ex
class Dune.Allowlist.Spec (line 1) | defmodule Dune.Allowlist.Spec
method new (line 13) | def new do
method list_fun_statuses (line 18) | def list_fun_statuses(%__MODULE__{modules: modules}) do
method list_ordered_modules (line 27) | def list_ordered_modules(%__MODULE__{modules: modules}) do
method group_funs_by_status (line 35) | defp group_funs_by_status(funs) do
method extract_status_atom (line 45) | defp extract_status_atom(:restricted), do: :restricted
method extract_status_atom (line 46) | defp extract_status_atom(:allowed), do: :allowed
method extract_status_atom (line 47) | defp extract_status_atom({:shimmed, _, _}), do: :shimmed
method status_sort (line 49) | defp status_sort(:allowed), do: 1
method status_sort (line 50) | defp status_sort(:shimmed), do: 2
method status_sort (line 51) | defp status_sort(:restricted), do: 3
method classify_functions (line 70) | defp classify_functions(functions, :all) do
method classify_functions (line 82) | defp classify_functions(functions, opts) do
method do_classify (line 98) | defp do_classify(functions, set_list, member_atom, non_member_atom) do
method to_atom_set (line 111) | defp to_atom_set(list) do
method unwrap_classify (line 118) | defp unwrap_classify({_, remaining}) do
method shim_functions (line 123) | defp shim_functions(functions, shims) do
method validate_shim! (line 137) | defp validate_shim!(module, fun_name, arity) do
FILE: lib/dune/atom_mapping.ex
class Dune.AtomMapping (line 1) | defmodule Dune.AtomMapping
method new (line 19) | def new do
method add_modules (line 31) | def add_modules(mapping = %__MODULE__{}, list) do
method build_mapping (line 35) | defp build_mapping(list) do
method do_replace_in_string (line 92) | defp do_replace_in_string(mapping = %__MODULE__{}, string) do
method build_replace_map (line 101) | defp build_replace_map(map, sub_mapping, extra_info, to_string_fun) do
method replace_in_result (line 115) | def replace_in_result(mapping, result)
method replace_in_result (line 117) | def replace_in_result(mapping, %Success{} = success) do
method replace_in_result (line 125) | def replace_in_result(mapping, %Failure{} = error) do
method fetch_existing_atom (line 141) | defp fetch_existing_atom(mapping, "Elixir." <> module_name) do
method fetch_existing_atom (line 149) | defp fetch_existing_atom(mapping, atom_name) do
FILE: lib/dune/eval.ex
class Dune.Eval (line 1) | defmodule Dune.Eval
method run (line 14) | def run(parsed, opts, previous_session \\ nil)
method run (line 16) | def run(
method run (line 38) | def run(%Failure{} = failure, _opts, _bindings), do: failure
method do_run (line 40) | defp do_run(ast, atom_mapping, opts, env, bindings) do
method prepend_parser_stdio (line 52) | defp prepend_parser_stdio(result, ""), do: result
method prepend_parser_stdio (line 54) | defp prepend_parser_stdio(result, parser_stdio) do
method safe_eval (line 58) | defp safe_eval(safe_ast, env, bindings, pretty, sort_maps) do
method do_safe_eval (line 68) | defp do_safe_eval(safe_ast, env, nil, inspect_opts) do
method eval_quoted (line 92) | defp eval_quoted(safe_ast, binding) do
FILE: lib/dune/eval/env.ex
class Dune.Eval.Env (line 1) | defmodule Dune.Eval.Env
method fetch_fake_function (line 38) | defp fetch_fake_function(%{fake_modules: modules}, module, fun_name, a...
FILE: lib/dune/eval/fake_module.ex
class Dune.Eval.FakeModule (line 1) | defmodule Dune.Eval.FakeModule
FILE: lib/dune/eval/function_clause_error.ex
class Dune.Eval.FunctionClauseError (line 1) | defmodule Dune.Eval.FunctionClauseError
method message (line 6) | def message(err = %__MODULE__{function: function, args: args}) do
FILE: lib/dune/eval/macro_env.ex
class Dune.Eval.MacroEnv (line 1) | defmodule Dune.Eval.MacroEnv
method make_env (line 6) | def make_env do
FILE: lib/dune/eval/process.ex
class Dune.Eval.Process (line 1) | defmodule Dune.Eval.Process
method do_run (line 13) | defp do_run(fun, opts, string_io) do
method with_string_io (line 40) | defp with_string_io(fun) do
method spawn_trapped_process (line 51) | defp spawn_trapped_process(fun, max_heap_size, max_reductions, string_...
method catch_diagnostics (line 109) | defp catch_diagnostics(fun) do
method format_error (line 145) | defp format_error(error, stacktrace)
method format_error (line 147) | defp format_error({:nocatch, value}, _stacktrace) do
method format_error (line 155) | defp format_error(error, stacktrace) do
method format_compile_error (line 175) | defp format_compile_error(error, diagnostics, stacktrace) do
FILE: lib/dune/failure.ex
class Dune.Failure (line 1) | defmodule Dune.Failure
method restricted_function (line 28) | def restricted_function(module, fun, arity) do
method undefined_module (line 36) | def undefined_module(module, function, arity) do
method undefined_function (line 46) | def undefined_function(module, function, arity) do
method base_undefined_message (line 53) | defp base_undefined_message(module, function, arity) do
method format_function (line 62) | defp format_function(module, fun, arity) do
FILE: lib/dune/helpers/diagnostics.ex
class Dune.Helpers.Diagnostics (line 1) | defmodule Dune.Helpers.Diagnostics
method prepend_diagnostics (line 12) | def prepend_diagnostics(result, []), do: result
method prepend_diagnostics (line 14) | def prepend_diagnostics(result, diagnostics) do
method format_diagnostics (line 19) | def format_diagnostics(diagnostics) do
method format_pos (line 28) | defp format_pos({line, col}), do: [Integer.to_string(line), ?:, Intege...
FILE: lib/dune/helpers/term_checker.ex
class Dune.Helpers.TermChecker (line 1) | defmodule Dune.Helpers.TermChecker
method check (line 11) | def check(term), do: do_check(term)
method do_check (line 19) | defp do_check([left | right]) do
FILE: lib/dune/opts.ex
class Dune.Opts (line 1) | defmodule Dune.Opts
method validate! (line 134) | def validate!(opts) do
method do_validate (line 174) | defp do_validate(opts = %{allowlist: allowlist}) do
FILE: lib/dune/parser.ex
class Dune.Parser (line 1) | defmodule Dune.Parser
method do_parse_string (line 21) | defp do_parse_string(string, opts = %{max_length: max_length}, previou...
method parse_quoted (line 32) | def parse_quoted(quoted, opts = %Opts{}, previous_session \\ nil) do
method unsafe_quoted (line 40) | def unsafe_quoted(ast) do
method get_compile_env (line 44) | defp get_compile_env(opts, nil) do
method get_compile_env (line 48) | defp get_compile_env(opts, %{compile_env: compile_env}) do
method string_to_quoted (line 53) | def string_to_quoted(string, opts) do
FILE: lib/dune/parser/atom_encoder.ex
class Dune.Parser.AtomEncoder (line 1) | defmodule Dune.Parser.AtomEncoder
method load_atom_mapping (line 137) | def load_atom_mapping(nil), do: :ok
method load_atom_mapping (line 139) | def load_atom_mapping(%AtomMapping{atoms: atoms}) do
method categorize_atom_binary (line 166) | def categorize_atom_binary(atom_binary) do
method do_static_atoms_encoder (line 177) | defp do_static_atoms_encoder("Elixir." <> rest, :alias, pool_size) do
method do_static_atoms_encoder (line 183) | defp do_static_atoms_encoder(binary, atom_category, pool_size) do
method do_static_atoms_encoder (line 192) | defp do_static_atoms_encoder(binary, atom_category, process_key, pool_...
method encode_many_atoms (line 211) | defp encode_many_atoms([], _pool_size, acc) do
method encode_many_atoms (line 215) | defp encode_many_atoms([head | tail], pool_size, acc) do
method plain_atom_mapping (line 223) | def plain_atom_mapping() do
method new_atom (line 237) | defp new_atom(atom_category, pool_size) do
method do_new_atom (line 249) | defp do_new_atom(:alias, count) do
method do_new_atom (line 253) | defp do_new_atom(:public_var, count) do
method encode_modules (line 263) | def encode_modules(ast, plain_atom_mapping, existing_mapping) do
method get_module_acc (line 281) | defp get_module_acc(nil), do: %{}
method get_module_acc (line 283) | defp get_module_acc(%AtomMapping{atoms: atoms, modules: modules}) do
method remove_elixir_prefix (line 292) | defp remove_elixir_prefix(atoms = [Elixir, Elixir | _]), do: atoms
method remove_elixir_prefix (line 294) | defp remove_elixir_prefix(atoms), do: atoms
method map_modules_ast (line 296) | defp map_modules_ast(atoms, acc) do
method build_module_mapping (line 315) | defp build_module_mapping(acc, plain_atom_mapping) do
FILE: lib/dune/parser/compile_env.ex
class Dune.Parser.CompileEnv (line 1) | defmodule Dune.Parser.CompileEnv
method new (line 17) | def new(allowlist) do
method module_already_exists? (line 36) | defp module_already_exists?(module, fake_modules) do
method resolve_module (line 70) | defp resolve_module(nil, fun_name, arity) do
method resolve_module (line 78) | defp resolve_module(module, _fun_name, _arity), do: module
method resolve_fake_module (line 80) | defp resolve_fake_module(%{module: nil}, nil, _fun_name, _arity), do: ...
method resolve_fake_module (line 82) | defp resolve_fake_module(env = %{module: module}, nil, fun_name, arity...
method resolve_fake_module (line 86) | defp resolve_fake_module(env, module, fun_name, arity) do
method check_private (line 96) | defp check_private(_env, module, :def), do: {:fake, module}
method check_private (line 97) | defp check_private(%{module: module}, module, :defp), do: {:fake, module}
method check_private (line 98) | defp check_private(_env, _module, _def), do: :undefined_function
FILE: lib/dune/parser/debug.ex
class Dune.Parser.Debug (line 1) | defmodule Dune.Parser.Debug
method io_debug (line 4) | def io_debug(ast) do
method ast_to_string (line 17) | defp ast_to_string({:__block__, _, list}) do
method ast_to_string (line 21) | defp ast_to_string(ast) do
FILE: lib/dune/parser/real_module.ex
class Dune.Parser.RealModule (line 1) | defmodule Dune.Parser.RealModule
method elixir_module? (line 5) | def elixir_module?(module) do
method list_functions (line 11) | def list_functions(module)
method list_functions (line 13) | def list_functions(Kernel.SpecialForms) do
method fun_exists? (line 26) | def fun_exists?(module, fun_name, arity) do
method fun_status (line 31) | def fun_status(module, fun_name, arity)
method fun_status (line 33) | def fun_status(Kernel.SpecialForms, fun_name, arity) do
method fun_status (line 41) | def fun_status(module, fun_name, arity) do
FILE: lib/dune/parser/safe_ast.ex
class Dune.Parser.SafeAst (line 1) | defmodule Dune.Parser.SafeAst
FILE: lib/dune/parser/sanitizer.ex
class Dune.Parser.Sanitizer (line 1) | defmodule Dune.Parser.Sanitizer
method sanitize (line 10) | def sanitize(unsafe = %UnsafeAst{}, compile_env = %CompileEnv{}) do
method sanitize (line 86) | def sanitize(%Failure{} = failure, _opts), do: failure
method try_sanitize (line 95) | defp try_sanitize(ast, env) do
method do_sanitize_main (line 110) | defp do_sanitize_main({:__block__, ctx, list}, env) do
method do_sanitize_main (line 116) | defp do_sanitize_main(single, env) do
method defmodule_block? (line 157) | defp defmodule_block?({:defmodule, _, _}), do: true
method defmodule_block? (line 158) | defp defmodule_block?(_), do: false
method parse_module_definition (line 160) | defp parse_module_definition({:defmodule, ctx, [module_name, [do: do_a...
method parse_module_definition (line 166) | defp parse_module_definition(ast = {:defmodule, _, _}) do
method validate_module_name (line 175) | defp validate_module_name(module_def = {:__aliases__, _, [_module_atom...
method validate_module_name (line 179) | defp validate_module_name(module_ast, ctx) do
method do_parse_module_definition (line 183) | defp do_parse_module_definition(module_name, do_ast) do
method parse_fun_definition (line 218) | defp parse_fun_definition(unsupported_ast) do
method extract_default_args (line 224) | defp extract_default_args([], _index, arg_acc, defaults) do
method extract_default_args (line 228) | defp extract_default_args([{:\\, _, [arg, default]} | args], index, ar...
method extract_default_args (line 232) | defp extract_default_args([arg | args], index, arg_acc, defaults) do
method expand_defaults (line 236) | defp expand_defaults({name_arity, definition, _defaults = []}) do
method expand_defaults (line 240) | defp expand_defaults({name_arity = {name, arity}, definition, defaults...
method do_expand_defaults (line 245) | defp do_expand_defaults(_name, _arity, _def_or_defp, [], acc) do
method do_expand_defaults (line 249) | defp do_expand_defaults(
method check_definition_conflicts (line 274) | defp check_definition_conflicts(grouped_definitions) do
method check_definition_conflict (line 280) | defp check_definition_conflict(_name_arity, _, []), do: :ok
method check_definition_conflict (line 282) | defp check_definition_conflict(
method check_definition_conflict (line 292) | defp check_definition_conflict(name_arity, {previous_def, previous_ctx...
method parse_fun_signature (line 300) | defp parse_fun_signature({:when, _, [header, guards]}) do
method parse_fun_signature (line 304) | defp parse_fun_signature(header) do
method sanitize_module_definition (line 308) | defp sanitize_module_definition({module, fun_defs}, env) do
method sanitize_fun (line 332) | defp sanitize_fun({{fun_name, arity}, definitions}, env) do
method sanitize_fun_clause (line 365) | defp sanitize_fun_clause({_def_or_defp, ctx, args, body, guards}, env) do
method definition_to_clause (line 372) | defp definition_to_clause(ctx, args, body, _guards = nil) do
method env_variable_if_used (line 381) | defp env_variable_if_used(asts) do
method uses_variable? (line 390) | defp uses_variable?([], _variable_name), do: false
method uses_variable? (line 392) | defp uses_variable?([head | tail], variable_name) do
method uses_variable? (line 399) | defp uses_variable?({variable_name, _, nil}, variable_name), do: true
method uses_variable? (line 405) | defp uses_variable?({x, y}, variable_name) do
method uses_variable? (line 409) | defp uses_variable?(_ast, _variable_name), do: false
method do_sanitize (line 411) | defp do_sanitize(ast, env)
method do_sanitize (line 423) | defp do_sanitize({arg1, arg2}, env) do
method do_sanitize (line 441) | defp do_sanitize({:&, _, args}, env) do
method do_sanitize (line 447) | defp do_sanitize({:<<>>, meta, args}, env) do
method do_sanitize (line 481) | defp do_sanitize({:dbg, meta, [expr]}, env) do
method do_sanitize (line 499) | defp do_sanitize({:|>, _, [expr, {:dbg, meta, []}]}, env) do
method do_sanitize (line 512) | defp do_sanitize({:|>, _, _} = ast, env) do
method try_expand_once (line 523) | defp try_expand_once(ast) do
method do_sanitize_dot (line 530) | defp do_sanitize_dot(left, key, args, ctx, env) do
method extract_module_and_fun (line 603) | defp extract_module_and_fun({:., _, [{:__aliases__, _, modules}, func_...
method handle_mfa_error (line 657) | defp handle_mfa_error(:undefined_module, module, func_name, arity) do
method handle_mfa_error (line 661) | defp handle_mfa_error(:undefined_function, module, func_name, arity) do
method sanitize_capture (line 670) | defp sanitize_capture(capture_arg, env) do
method dbg_pipeline (line 708) | defp dbg_pipeline(expr, env, header) do
method check_bin_modifier (line 728) | defp check_bin_modifier(modifier) do
method check_bin_modifier_size (line 739) | defp check_bin_modifier_size({:-, _, [left, right]}, size, unit) do
method check_bin_modifier_size (line 744) | defp check_bin_modifier_size(modifier, size, unit) do
method env_variable (line 772) | defp env_variable do
method underscore_env_variable (line 776) | defp underscore_env_variable do
method authorized_var_name? (line 780) | defp authorized_var_name?(name) do
FILE: lib/dune/parser/string_parser.ex
class Dune.Parser.StringParser (line 1) | defmodule Dune.Parser.StringParser
method do_parse_string (line 21) | defp do_parse_string(
method maybe_load_atom_mapping (line 45) | defp maybe_load_atom_mapping(nil), do: :ok
method maybe_load_atom_mapping (line 47) | defp maybe_load_atom_mapping(%{atom_mapping: atom_mapping}) do
method maybe_encode_modules (line 51) | defp maybe_encode_modules(ast, previous_session, encode_modules?) do
method handle_failure (line 65) | defp handle_failure("Atoms containing" <> _ = error, token) do
method handle_failure (line 69) | defp handle_failure(error, token) do
FILE: lib/dune/parser/unsafe_ast.ex
class Dune.Parser.UnsafeAst (line 1) | defmodule Dune.Parser.UnsafeAst
FILE: lib/dune/session.ex
class Dune.Session (line 1) | defmodule Dune.Session
method new (line 50) | def new do
method eval_string (line 82) | def eval_string(session = %__MODULE__{}, string, opts \\ []) do
method add_result_to_session (line 93) | defp add_result_to_session(result = %Success{value: {value, env, bindi...
method add_result_to_session (line 100) | defp add_result_to_session(result = %Failure{}, session, _) do
FILE: lib/dune/shims/atom.ex
class Dune.Shims.Atom (line 1) | defmodule Dune.Shims.Atom
FILE: lib/dune/shims/enum.ex
class Dune.Shims.Enum (line 1) | defmodule Dune.Shims.Enum
FILE: lib/dune/shims/io.ex
class Dune.Shims.IO (line 1) | defmodule Dune.Shims.IO
method puts (line 6) | def puts(env, device \\ :stdio, item)
method puts (line 8) | def puts(env, :stdio, item) do
method puts (line 14) | def puts(_env, _device, _item) do
method inspect (line 19) | def inspect(env, item, opts \\ []) do
FILE: lib/dune/shims/kernel.ex
class Dune.Shims.Kernel (line 1) | defmodule Dune.Shims.Kernel
method safe_throw (line 29) | def safe_throw(_env, value) do
method safe_dot (line 48) | def safe_dot(_env, %{} = map, key) do
method safe_inspect (line 87) | def safe_inspect(env, term, opts \\ [])
method safe_inspect (line 98) | def safe_inspect(env, term, opts) do
method safe_to_string (line 112) | def safe_to_string(_env, other), do: to_string(other)
method safe_to_charlist (line 118) | def safe_to_charlist(_env, other), do: to_charlist(other)
FILE: lib/dune/shims/list.ex
class Dune.Shims.List (line 1) | defmodule Dune.Shims.List
method do_to_string (line 21) | defp do_to_string(elem), do: elem
method to_existing_atom (line 33) | def to_existing_atom(_env, list) do
FILE: lib/dune/shims/string.ex
class Dune.Shims.String (line 1) | defmodule Dune.Shims.String
method to_existing_atom (line 12) | def to_existing_atom(_env, string) do
FILE: lib/dune/success.ex
class Dune.Success (line 1) | defmodule Dune.Success
FILE: mix.exs
class Dune.MixProject (line 1) | defmodule Dune.MixProject
method project (line 7) | def project do
method application (line 25) | def application do
method deps (line 32) | defp deps do
method package (line 41) | defp package do
method aliases (line 50) | defp aliases do
method cli (line 54) | def cli do
method docs (line 58) | defp docs do
FILE: test/dune/allowlist/default_test.exs
class Dune.Allowlist.DefaultTest (line 1) | defmodule Dune.Allowlist.DefaultTest
FILE: test/dune/allowlist_test.exs
class Dune.AllowlistTest (line 1) | defmodule Dune.AllowlistTest
FILE: test/dune/atom_mapping_test.exs
class Dune.AtomMappingTest (line 1) | defmodule Dune.AtomMappingTest
FILE: test/dune/opts_test.exs
class Dune.OptsTest (line 1) | defmodule Dune.OptsTest
FILE: test/dune/parser/atom_encoder_test.exs
class Dune.Parser.AtomEncoderTest (line 1) | defmodule Dune.Parser.AtomEncoderTest
FILE: test/dune/parser/string_parser_test.exs
class Dune.Parser.StringParserTest (line 1) | defmodule Dune.Parser.StringParserTest
FILE: test/dune/session_test.exs
class Dune.SessionTest (line 1) | defmodule Dune.SessionTest
FILE: test/dune/shims_test.exs
class Dune.ShimsTest (line 1) | defmodule Dune.ShimsTest
FILE: test/dune/validation_test.exs
class Dune.ValidationTest (line 1) | defmodule Dune.ValidationTest
FILE: test/dune_modules_test.exs
class DuneModulesTest (line 1) | defmodule DuneModulesTest
FILE: test/dune_oom_safety_test.exs
class Dune.AssertionHelper (line 1) | defmodule Dune.AssertionHelper
class Dune.OOMSafetyTest (line 15) | defmodule Dune.OOMSafetyTest
FILE: test/dune_quoted_test.exs
class DuneQuotedTest (line 1) | defmodule DuneQuotedTest
FILE: test/dune_string_test.exs
class DuneStringTest (line 1) | defmodule DuneStringTest
FILE: test/dune_string_to_quoted_test.exs
class DuneStringToQuotedTest (line 1) | defmodule DuneStringToQuotedTest
FILE: test/dune_test.exs
class DuneTest (line 1) | defmodule DuneTest
Condensed preview — 57 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (208K chars).
[
{
"path": ".formatter.exs",
"chars": 239,
"preview": "# Used by \"mix format\"\nlocals_without_parens = [allow: 2]\n\n[\n inputs: [\"{mix,.formatter}.exs\", \"{config,lib,test}/**/*."
},
{
"path": ".github/workflows/ci.yml",
"chars": 1349,
"preview": "name: CI\non: [push, pull_request]\n\njobs:\n test:\n runs-on: ubuntu-latest\n name: OTP ${{matrix.otp}} / Elixir ${{ma"
},
{
"path": ".gitignore",
"chars": 626,
"preview": "# The directory Mix will write compiled artifacts to.\n/_build/\n\n# If you run \"mix test --cover\", coverage assets end up "
},
{
"path": "CHANGELOG.md",
"chars": 3796,
"preview": "# Changelog\n\n## Dev\n\n## v0.3.15 (2025-10-19)\n\n- Support Elixir 1.19\n\n## v0.3.14 (2025-07-29)\n\n### Security fixes\n\n- Use "
},
{
"path": "LICENSE.md",
"chars": 1065,
"preview": "MIT License\n\nCopyright (c) 2021 Sabiwara\n\nPermission is hereby granted, free of charge, to any person obtaining a copy o"
},
{
"path": "README.md",
"chars": 6006,
"preview": "# Dune\n\n[](https://hex.pm/packages/dune)\n[![docs](https://img.shi"
},
{
"path": "lib/dune/allowlist/default.ex",
"chars": 6704,
"preview": "defmodule Dune.Allowlist.Default do\n @moduledoc \"\"\"\n The default `Dune.Allowlist` module to be used to allow or restri"
},
{
"path": "lib/dune/allowlist/docs.ex",
"chars": 1781,
"preview": "defmodule Dune.Allowlist.Docs do\n @moduledoc false\n\n def document_allowlist(spec) do\n spec\n |> Dune.Allowlist.Sp"
},
{
"path": "lib/dune/allowlist/spec.ex",
"chars": 4339,
"preview": "defmodule Dune.Allowlist.Spec do\n @moduledoc false\n\n alias Dune.Allowlist\n alias Dune.Parser.RealModule\n\n @type t ::"
},
{
"path": "lib/dune/allowlist.ex",
"chars": 7500,
"preview": "defmodule Dune.Allowlist do\n @moduledoc \"\"\"\n Behaviour to customize the modules and functions that are allowed or rest"
},
{
"path": "lib/dune/atom_mapping.ex",
"chars": 4788,
"preview": "defmodule Dune.AtomMapping do\n @moduledoc false\n\n alias Dune.{Success, Failure}\n\n @type substitute_atom :: atom\n @ty"
},
{
"path": "lib/dune/eval/env.ex",
"chars": 1568,
"preview": "defmodule Dune.Eval.Env do\n @moduledoc false\n\n alias Dune.AtomMapping\n alias Dune.Eval.FakeModule\n\n @type t :: %__MO"
},
{
"path": "lib/dune/eval/fake_module.ex",
"chars": 459,
"preview": "defmodule Dune.Eval.FakeModule do\n @moduledoc false\n\n @type t :: %__MODULE__{\n public_funs: %{optional(atom) "
},
{
"path": "lib/dune/eval/function_clause_error.ex",
"chars": 395,
"preview": "defmodule Dune.Eval.FunctionClauseError do\n @moduledoc false\n\n defexception [:module, :function, :args]\n\n def message"
},
{
"path": "lib/dune/eval/macro_env.ex",
"chars": 344,
"preview": "defmodule Dune.Eval.MacroEnv do\n @moduledoc false\n\n # Recommended way to generate a Macro.Env struct\n # https://hexdo"
},
{
"path": "lib/dune/eval/process.ex",
"chars": 5076,
"preview": "defmodule Dune.Eval.Process do\n @moduledoc false\n\n alias Dune.Failure\n alias Dune.Helpers.Diagnostics\n\n def run(fun,"
},
{
"path": "lib/dune/eval.ex",
"chars": 2893,
"preview": "defmodule Dune.Eval do\n @moduledoc false\n\n alias Dune.{AtomMapping, Success, Failure, Opts}\n alias Dune.Eval.Env\n al"
},
{
"path": "lib/dune/failure.ex",
"chars": 1917,
"preview": "defmodule Dune.Failure do\n @moduledoc \"\"\"\n A struct returned when `Dune` parsing or evaluation fails.\n\n Fields:\n - `"
},
{
"path": "lib/dune/helpers/diagnostics.ex",
"chars": 1344,
"preview": "defmodule Dune.Helpers.Diagnostics do\n @moduledoc false\n\n # used for formatting errors and warnings consistently\n\n @t"
},
{
"path": "lib/dune/helpers/term_checker.ex",
"chars": 835,
"preview": "defmodule Dune.Helpers.TermChecker do\n @moduledoc false\n\n defguardp is_simple_term(term)\n when is_atom(term"
},
{
"path": "lib/dune/opts.ex",
"chars": 6377,
"preview": "defmodule Dune.Opts do\n @moduledoc \"\"\"\n Defines and validates the options for `Dune`.\n\n The available options are exp"
},
{
"path": "lib/dune/parser/atom_encoder.ex",
"chars": 9900,
"preview": "defmodule Dune.Parser.AtomEncoder do\n @moduledoc false\n\n alias Dune.AtomMapping\n\n @type atom_category :: :alias | :pr"
},
{
"path": "lib/dune/parser/compile_env.ex",
"chars": 2917,
"preview": "defmodule Dune.Parser.CompileEnv do\n @moduledoc false\n\n @type name_arity :: {atom, non_neg_integer}\n @type maybe_fake"
},
{
"path": "lib/dune/parser/debug.ex",
"chars": 431,
"preview": "defmodule Dune.Parser.Debug do\n @moduledoc false\n\n def io_debug(ast) do\n debug(ast) |> IO.puts()\n ast\n end\n\n d"
},
{
"path": "lib/dune/parser/real_module.ex",
"chars": 1305,
"preview": "defmodule Dune.Parser.RealModule do\n @moduledoc false\n\n @spec elixir_module?(module) :: boolean\n def elixir_module?(m"
},
{
"path": "lib/dune/parser/safe_ast.ex",
"chars": 337,
"preview": "defmodule Dune.Parser.SafeAst do\n @moduledoc false\n\n @type t :: %__MODULE__{\n ast: Macro.t(),\n atom_"
},
{
"path": "lib/dune/parser/sanitizer.ex",
"chars": 22740,
"preview": "defmodule Dune.Parser.Sanitizer do\n @moduledoc false\n\n alias Dune.{Failure, AtomMapping, Opts}\n alias Dune.Parser.{Co"
},
{
"path": "lib/dune/parser/string_parser.ex",
"chars": 2760,
"preview": "defmodule Dune.Parser.StringParser do\n @moduledoc false\n\n alias Dune.{AtomMapping, Failure, Opts}\n alias Dune.Helpers"
},
{
"path": "lib/dune/parser/unsafe_ast.ex",
"chars": 274,
"preview": "defmodule Dune.Parser.UnsafeAst do\n @moduledoc false\n\n @type t :: %__MODULE__{\n ast: Macro.t(),\n ato"
},
{
"path": "lib/dune/parser.ex",
"chars": 2066,
"preview": "defmodule Dune.Parser do\n @moduledoc false\n\n alias Dune.{AtomMapping, Success, Failure, Opts}\n alias Dune.Parser.{Com"
},
{
"path": "lib/dune/session.ex",
"chars": 4075,
"preview": "defmodule Dune.Session do\n @moduledoc \"\"\"\n Sessions provide a way to evaluate code and keep state (bindings, modules.."
},
{
"path": "lib/dune/shims/atom.ex",
"chars": 302,
"preview": "defmodule Dune.Shims.Atom do\n @moduledoc false\n\n alias Dune.AtomMapping\n\n def to_string(env, atom) when is_atom(atom)"
},
{
"path": "lib/dune/shims/enum.ex",
"chars": 504,
"preview": "defmodule Dune.Shims.Enum do\n @moduledoc false\n\n def join(env, enumerable, joiner \\\\ \"\") when is_binary(joiner) do\n "
},
{
"path": "lib/dune/shims/io.ex",
"chars": 919,
"preview": "defmodule Dune.Shims.IO do\n @moduledoc false\n\n alias Dune.{Failure, Shims}\n\n def puts(env, device \\\\ :stdio, item)\n\n "
},
{
"path": "lib/dune/shims/json.ex",
"chars": 1705,
"preview": "if Code.ensure_loaded?(JSON) do\n defmodule Dune.Shims.JSON do\n @moduledoc false\n\n alias Dune.AtomMapping\n alia"
},
{
"path": "lib/dune/shims/kernel.ex",
"chars": 3029,
"preview": "defmodule Dune.Shims.Kernel do\n @moduledoc false\n\n alias Dune.{AtomMapping, Failure, Shims}\n alias Dune.Helpers.TermC"
},
{
"path": "lib/dune/shims/list.ex",
"chars": 893,
"preview": "defmodule Dune.Shims.List do\n @moduledoc false\n\n alias Dune.Shims\n\n def to_string(_env, list) when is_list(list) do\n "
},
{
"path": "lib/dune/shims/string.ex",
"chars": 336,
"preview": "defmodule Dune.Shims.String do\n @moduledoc false\n\n alias Dune.AtomMapping\n\n # note: this is probably not safe so not "
},
{
"path": "lib/dune/success.ex",
"chars": 736,
"preview": "defmodule Dune.Success do\n @moduledoc \"\"\"\n A struct returned when `Dune` evaluation succeeds.\n\n Fields:\n - `value` ("
},
{
"path": "lib/dune.ex",
"chars": 5884,
"preview": "defmodule Dune do\n @moduledoc \"\"\"\n A sandbox for Elixir to safely evaluate untrusted code from user input.\n\n ## Featu"
},
{
"path": "mix.exs",
"chars": 1514,
"preview": "defmodule Dune.MixProject do\n use Mix.Project\n\n @version \"0.3.15\"\n @github_url \"https://github.com/functional-rewire/"
},
{
"path": "test/dune/allowlist/default_test.exs",
"chars": 508,
"preview": "defmodule Dune.Allowlist.DefaultTest do\n use ExUnit.Case, async: true\n doctest Dune.Allowlist.Default\n alias Dune.All"
},
{
"path": "test/dune/allowlist_test.exs",
"chars": 1647,
"preview": "defmodule Dune.AllowlistTest do\n use ExUnit.Case, async: true\n\n doctest Dune.Allowlist\n\n describe \"use/2\" do\n test"
},
{
"path": "test/dune/atom_mapping_test.exs",
"chars": 96,
"preview": "defmodule Dune.AtomMappingTest do\n use ExUnit.Case, async: true\n doctest Dune.AtomMapping\nend\n"
},
{
"path": "test/dune/opts_test.exs",
"chars": 82,
"preview": "defmodule Dune.OptsTest do\n use ExUnit.Case, async: true\n doctest Dune.Opts\nend\n"
},
{
"path": "test/dune/parser/atom_encoder_test.exs",
"chars": 1264,
"preview": "defmodule Dune.Parser.AtomEncoderTest do\n use ExUnit.Case, async: true\n import Dune.Parser.AtomEncoder\n\n describe \"ca"
},
{
"path": "test/dune/parser/string_parser_test.exs",
"chars": 4132,
"preview": "defmodule Dune.Parser.StringParserTest do\n use ExUnit.Case, async: true\n\n alias Dune.{AtomMapping, Opts}\n alias Dune."
},
{
"path": "test/dune/session_test.exs",
"chars": 3040,
"preview": "defmodule Dune.SessionTest do\n use ExUnit.Case\n\n doctest Dune.Session, tags: [lts_only: true]\n\n alias Dune.{Session, "
},
{
"path": "test/dune/shims_test.exs",
"chars": 1901,
"preview": "defmodule Dune.ShimsTest do\n use ExUnit.Case, async: true\n\n alias Dune.Success\n alias Dune.Failure\n\n defmacrop sigil"
},
{
"path": "test/dune/validation_test.exs",
"chars": 37,
"preview": "defmodule Dune.ValidationTest do\nend\n"
},
{
"path": "test/dune_modules_test.exs",
"chars": 11170,
"preview": "defmodule DuneModulesTest do\n use ExUnit.Case, async: true\n\n alias Dune.{Success, Failure}\n\n defmacro sigil_E(call, _"
},
{
"path": "test/dune_oom_safety_test.exs",
"chars": 2421,
"preview": "defmodule Dune.AssertionHelper do\n alias Dune.Failure\n\n defmacro test_execution_stops(test_name, do: expr) do\n quot"
},
{
"path": "test/dune_quoted_test.exs",
"chars": 12212,
"preview": "defmodule DuneQuotedTest do\n use ExUnit.Case, async: true\n\n import ExUnit.CaptureIO\n\n alias Dune.{Success, Failure}\n\n"
},
{
"path": "test/dune_string_test.exs",
"chars": 34698,
"preview": "defmodule DuneStringTest do\n use ExUnit.Case, async: true\n\n alias Dune.{Success, Failure}\n\n defmacro sigil_E(call, _e"
},
{
"path": "test/dune_string_to_quoted_test.exs",
"chars": 777,
"preview": "defmodule DuneStringToQuotedTest do\n use ExUnit.Case, async: true\n\n alias Dune.Success\n\n describe \"Dune.string_to_quo"
},
{
"path": "test/dune_test.exs",
"chars": 144,
"preview": "defmodule DuneTest do\n use ExUnit.Case, async: true\n\n # TODO remove then dropping support for 1.15\n doctest Dune, tag"
},
{
"path": "test/test_helper.exs",
"chars": 15,
"preview": "ExUnit.start()\n"
}
]
About this extraction
This page contains the full source code of the functional-rewire/dune GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 57 files (191.6 KB), approximately 54.4k tokens, and a symbol index with 257 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.