Rules let you control how Nogon indexes and surfaces documents. They live in {data_directory}/rules.json in single-user mode, or per-user in the database
when user handling is enabled. The Rules tab in the web
interface (and the TUI) is the easiest way to manage them.
Rule Types
Allow rules
Allow rules limit indexing to URLs matching at least one allow pattern. Patterns are combined with OR: a match against any entry is enough. Skip rules still take precedence, so you can allow a whole site while excluding particular paths.
An empty allow list permits any URL that does not match a skip rule. Removing the
last allow rule restores that behavior. Existing rules files without an allow field continue to work as before.
For example, allow these sites:
^https://example.com/
^https://docs.example.org/ Then add ^https://docs\.example\.org/private/ as a skip rule to exclude private
paths on the documentation site.
Allow rules apply to API submissions and browser imports as well as automatic
browser capture. Editing rules does not immediately remove saved documents.
Running nogon reindex removes documents excluded by the current allow or skip
rules, except documents carrying an explicit manual override. In multiple user
mode, reindexing uses each document owner’s rules; public documents use the
instance rules.
Skip rules
Skip rules prevent matching URLs from being added to the index. When a URL
matches any pattern in the skip list, Nogon silently discards the document
during indexing and during nogon reindex.
Manual indexing in the browser extension overrides allow and skip rules for that submission
and saves the choice with the document, so the
page survives nogon reindex. Automatic submissions still respect the rules.
From the command line, use --ignore-rules with nogon index or any nogon import subcommand to save the same explicit override with each submitted document. For an
already indexed URL, combine nogon index --ignore-rules with --force so that the
document is submitted again. The override does not bypass sensitive content checks,
robots rules, or crawl and file selection filters.
API clients can set metadata.ignore_skip_rules to the boolean true in JSON
documents submitted to /api/add. For /api/add_pdf, it belongs inside document.metadata; for /api/batch, each add operation has its own metadata.
The override is stored with the document and preserved in exported documents.
The override applies only to URL allow and skip rules. The metadata field keeps its
existing name for compatibility. Authentication, ownership, sensitive
content checks, and the exclusion of Nogon’s own URLs still apply. Accepted
updates preserve an existing override unless they explicitly submit metadata.ignore_skip_rules: false. Explicit deletion still removes the document.
Use cases: block ad networks, login pages, cookie-consent walls, or any site you never want in your search results.
^https://ads.example.com
^https?://(login|mail).example.com/
.*?utm_source= Priority rules
Priority rules push matching documents to the top of search results regardless of their relevance score. Nogon applies a large score boost to any document whose URL matches a priority pattern.
Use cases: always surface your personal wiki, your company’s internal documentation, or a trusted source before other results.
^https://wiki.example.com/
^https://docs.example.com/ Versioning rules
Versioning rules tell Nogon to track changes to a document every time it is re-indexed. When a URL matches a versioning pattern and the document content differs from the previously indexed version, Nogon stores diff match patch records for the HTML and text changes in the database.
Use cases: monitor pages for edits (privacy policies, documentation, news articles), build a personal changelog of sites you follow, or audit when a trusted resource last changed.
^https://example.com/privacy-policy$
^https://docs.example.com/ Viewing stored versions
Stored differences can reconstruct earlier document versions. Version history is available only after Nogon has recorded at least one change for that document.
Aliases
Aliases are query-time shortcuts: before Nogon executes a search, it replaces any alias keyword with its expanded form.
{
"gh": "domain:github.com",
"local": "type:file",
"work": "domain:(internal.example.com|jira.example.com)"
} With the work alias above, searching for work deployment is equivalent to domain:(internal.example.com|jira.example.com) deployment.
Pattern syntax
Allow, skip, priority, and versioning rules are matched against the full URL (including scheme, host, path, and query string). A few important details:
- Patterns follow Go regular expression syntax. Look-ahead and look-behind are not supported.
- Anchoring must include the scheme:
^https://foo.comor^https?://(login|mail)\.are valid;^foo.comis not. - The URL hash is stripped before matching
(
https://foo.com/#sectionbecomeshttps://foo.com/). - Query-string parameters are not reordered; only
utm_*parameters are stripped. - A trailing
$anchor will not match URLs that have a query string:/login$does not matchhttps://foo.com/login?auth=1.
Saving a skip rule does not delete existing documents by default. Optional deletion applies the rule to documents already in the index. Cancelling deletion leaves both the saved rule and the existing documents intact.
Storage
In single-user mode (user handling disabled), rules are saved to and read
from rules.json in the configured data directory. Changes made through the
web interface or API update this file directly.
In multi-user mode (user handling enabled), each user has a private copy of
their rules stored in the database. Changes affect only the authenticated user
and do not touch rules.json. See User Handling for details.