Tracing documentation for entities in big codebases is hard. For instance, finding documentation for a C++ function in the absence of a reference website is likely to require a lot of grep-ing and reading. Compared to C++, Common Lisp has enough features to make documentation retrieval easy enough. Even the largest of projects are made comprehensible.

There are many ways to sail a boat, here's our set of rules and conventions we developed while documenting Nyxt and the libraries around it.

General Ideas

Names

Good names explain additional context and use. Name things conventionally:

• Dynamic/special/global variables with *earmuffs*.
• Constants with +plus-signs+.
• Internal functions and variables with %percent-signs%.
• Predicates with -p suffix.
• etc.

Interlinked Docs

Whatever the nesting level and documentation convention, it's useful to link documented entities to each other. This way, a separate piece of documentation for a certain entity has more context and details to benefit from. Most of the following sections greatly benefit from using symbol cross-references.

Most of Nyxt-related code uses the backtick-quote syntax typical to Emacs and GNU code. It's not universally portable, but sufficient to cover most Lisp development environments and to enable jump-to-source there. Here's how one could cross-reference one of our library symbols:

"Based on `nkeymaps:keymap' key resolution."

More Examples

While Lisp code usually has clear and inspectable lambda lists, type information, and (given enough effort) documentation, it never hurts to list usage examples. Examples don't even have to be realistic— even the simplest ones show the patterns of use for a certain function/macro/class.

The more examples, the better.

Document Everything

It's better to over-document than under-document. So, every slot, variable, flet, and macrolet—all of these are better off with docs. Even the internal ones. Because getting back to the undocumented code always prompts for refactoring due to old entities being opaque. Better to explain implementation details to your future self than to make them guess.

Levels of Magnification

There are several levels—from toplevel mission statements to implementation details—for documentation. Most of these are covered by respective entity docstrings.

READMEs

README files are over-used in non-Lisp projects due to absence of good toplevel documentation alternatives. We have better ways to describe project APIs. Still, READMEs are good for a quick glance of what the library can and cannot do. Thus, our README structure is concise, yet sufficient for this cursory look:

Title and description

General information about the project. Goal and features.

Getting started

How to install and load.

Examples

Short code snippets for the most common use-cases.

How it works

Explanations of the underlying model and implementation details.

That's it. Everything else goes to more Lisp-native ways to document software.

Packages

Packages, being the bags for symbols, managing what is used, shadowed, imported, exported, and nicknamed, are the perfect place to reflect the toplevel API of the library—main functions, classes, and macros. Linking to the main entry points, which have more documentation (see Interlinked Docs). Package documentation is also a perfect place to flesh out possible usage scenarios and list code for them.

Classes

Not everyone uses CLOS. But we do, and heavily. As with every good object system, it's a useful and easy way to describe the actual entities of a domain.

• Implementing keymaps? Make a keymap entity to manage them.
• Working with debuggers? Make a condition-wrapper for the debugging context and debugger UI itself.
• Playing with hooks? hook is the noun you describe things with ¯\_(ツ)_/¯

Classes document the useful slots influencing their behavior and useful generics/methods that act on them. For mixins, supposed subclasses, or metaclasses—their behavior is only clear when explained. MOP is complex and every MOP extension is atypical by definition.

Structs

We don't use structs—they are less inspectable, flexible, and they retain much less metadata compared to classes.

Types

Types are good to validate function/variable/slot contract early and explain the set of accepted values. Define well-documented and well-named types, so that there's another source of information for the reader/user of the library.

Functions

Functions are the most fundamental entity in Lisp and beyond—everyone uses them, and every Lisp dialect is easy to use with composition and closures. That's why function docstrings should excel in their readability and usefulness.

Here's our structure for function docstrings:

"First line as a single sentence less than 80 chars long—for readability.
Return VALUE or
- VALUE-1
- VALUE-2
- etc.

ARGUMENT-1 represents ...
BOOLEAN-ARGUMENT mandates whether ...

See also `another-symbol'."

Notice that the docstring doesn't mention the exact shape of the argument list—arglist can be portably retrieved on every implementation. Docstring doesn't normally mention argument types either—function types are also retrievable and enforceable at runtime/compile-time.

Generics and methods

All the same function conventions apply to generics. Generics should come from a defgeneric form for increased inspectability and reliability of access. Arguments should be described, unless their specializers explain more than their docs.

Methods can be undocumented, because they are generally just implementations for a documented generic. Method documentation is where exceptional implementation details end up. Otherwise, docstrings are not really needed, especially given the difficulty of method documentation retrieval.

Macros

Beyond the fact that macros require a lot of discipline, their documentation should also be more involved. Macro arglists and types are harder to fetch and understand. Even macroexpand-1 often fails to explain the effect of the macro. Docstrings have to go a great length to explain the exact syntax of the macro, list the possible values and even their type/shape. Does it accept quoted symbols? Forms? Does it evaluate arguments? Which of them?

That's where examples are vital: even the best docstring cannot possibly explain the nested structure of a clever enough macro.

Variables

Variables are simple. Constant or not, variables all contain values that are atomic, and it's hard to break them. But there are non-obvious things about variables: what is the exact semantics for their type? What is their scope and extent?

That all should be documented: hash-table keys and values, list contents, magic values. Dynamic variables should be given extra care—how do these influence the behavior and of what?

Conclusions

Given a good discipline, Lisp lends itself to a very powerful documentation system. Its source code can be introspectable, explorable, and fun to understand. We hope we've given you some good pointers and ideas for how you might structure your code. Thanks for reading :-)