EEP 67: Internal exports #

Abstract #

This EEP introduces a new directive called internal_export which enables semantic separation of function exports. This EEP is mostly inspired by EEP 5.

Rationale #

Erlang application API is an ambigous term. In theory, it is a set of module APIs (exported functions). In practice, it is a set of modules’ APIs but without modules considered internal, undocumented exports, callback implementation, etc. While reading the docs, you can conclude what is application API, but while reading the source, it’s not so straightforward.

Not all exports are semantically the same. A function can be exported to be part of the application API, to implement a callback, to test some code, or to allow modules within the application to use it. Currently, from the user perspective, they are all the same. There is a convention not to use undocumented exports, but there is no way to enforce that convention in a codebase.

The goal of this EEP is to give a mechanism to separate application API exports from other exports.

A few terms are introduced for convenience:

• internally exported function - a function marked with internal_export attribute
• internal module - module containing only internally exported functions
• application dependencies - a set of applications found in applications, include_applications and optional_applications properties in .app file (and their dependencies, recursively)
• export scope - a set of modules that can legally access the exported function

Internally exported functions are still globally exported, but application users are discouraged from using them. The mechanism is similar to the deprecation mechanism, where you can call deprecated functions, although it’s not recommended. You also have a mechanism to check if there are static calls to that function so you don’t need to manually check if each function is deprecated.

Specification #

Syntax for internally exporting a function would be the following:

-internal_export([f/a, ...]).

where f is atom() and a is arity().

Internally exported functions can legally be called from within their applications and from their dependencies. The reason why the application’s dependencies can legally call internally exported functions are callbacks to other applications¹.

Example #

Application A contains module mod_A which exports public/1 and internally exports internal/1.

Application B contains module mod_B which exports x/1 and internally exports y/1.

Application B depends on application A.

mod_A:internal/1 can legally be called only from within A, while mod_B:y/1 can be legally called from both A and B. Static call from A to mod_B:y/1 will probably never occur, because A is not aware of B, but it could be that mod_B:y/1 is a callback implementation for mod_A. All the other applications can’t legally call mod_A:internal/1 and mod_B:y/1.

EEP 5 Modifications #

Although the idea is similar, there are a few key differences in design and implementation between this EEP and EEP 5:

Application-based approach #

There is an open question of whether to export functions to modules or applications. Exporting to modules is currently proposed in EEP 5.

Although the module-based approach is straightforward, it has 2 drawbacks:

1. Maintainance - you have to manually intervene after a module is added or renamed.

2. Exporting to behaviors - you would expect that

    -export_to(gen_server, [init/1, handle_cast/2, handle_call/3]).
   

would work.

It would be very convenient to export callbacks this way instead of

    %% gen_server API
    -export([init/1, handle_cast/2, handle_call/3]).

, but that notation would actually be wrong because calls to gen_server callbacks are done in gen module, not gen_server. Users shouldn’t even think about that implementation detail, let alone writing code dependent on it.

Implicit scope #

Currently, comment-based is a widespread approach to clarify module/application API, e.g.:

%% gen_server
-export([init/1, handle_cast/2, handle_call/3, code_change/3]).
%% system calls
-export(...).
%% test-only
-export(...).
%% API
-export(...).
%% internal exports
-export(...).

This of course works, but a comment-based semantical separation of exports limits any usage of code analysis tools. Can we do better?

In EEP 5, the responsibility of declaring export scope is on the programmer. This may result in convenient, more semantically valuable code, like:

-export_to([mod_x, mod_y], [f_1/1, f_2/0]).

It denotes what is the purpose of some particular export, i.e. which modules can call them. Notice that this is only convenient when limiting function usage to the application itself (conventionally called internal exports) - it was mentioned before that this approach is not appropriate when exporting functions to behavior modules.

This brings up a question: how much control do we really need/want? This EEP aims to limit ways users can misuse applications/modules. Missuage doesn’t come from within the application itself, or from its dependency applications, but from its users! I.e. besides private and global, there is a need for internal export scope.

There is no need for the user to specify the scope manually as it can be determined at compile-time: internally exported function can be called from the application where it is declared or from any of its dependency applications.

Implicit scoping is not as semantically valuable as explicit one, but it does the main task; it separates application/module public API from internal stuff.

Syntax #

The export_to attribute seems completely logical when used with explicit scoping, but with implicit scoping, it is unclear to what _to refer. Name internal_export is proposed instead of export_to. Notice that it only has one argument with the same syntax rules as for export.

Implementation strategy #

In EEP 5, calls would checked in the loader (for static calls) and at runtime (for dynamic calls), causing them to fail if an invalid call occurs. In contrast, this EEP suggests a code-analysis-based approach, with xref checking all static calls and disregarding all dynamic calls. Calls to all functions remain valid at runtime. This approach, of course, provides no runtime guarantees, but imposes no performance hit and requires significantly less work and maintenance.

Reference Implementation #

The current implementation is in PR 7407 in the OTP repository.

Besides this PR, there are forum threads about the need for internal exports and the implementation.

Backward compatibility #

Code that is already using -internal_export(FAs). attribute would be affected.

References #

important to denote those callbacks can legally be used from dependency applications. If someday dynamic checks become a thing mechanism could stay the same. That said, the current mechanism can be simplified and only check if internally exported is used only from within the same application.

Copyright #

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.