Self-hosting

Self-hosting support for Inngest is supported as of the 1.0 release.

• Why self-host Inngest?
• Inngest system architecture
• How to self-host Inngest

Why self-host Inngest?

While the easiest way to get started with Inngest is using our hosted platform, including our generous free tier, we understand that developers may want to self-host for a variety of reasons. If security or data privacy are concerns, review our security documentation for more information including details about end-to-end encryption.

Inngest system architecture

To best understand how to self-host Inngest, it's important to understand the system architecture and components.

The system is composed of the following services:

• Event API - Receives events from SDKs via HTTP requests. Authenticates client requests via Event Keys. The Event API publishes event payloads to an internal event stream.
• Event stream - Acts as buffer between the Event API and the Runner.
• Runner - Consumes incoming events and performs several actions:
  • Scheduling of new “function runs” (aka jobs) given the event type, creating initial run state in the State store database. Runs are added to queues given the function's flow control configuration.
  • Resume functions paused via waitForEvent with matching expressions.
  • Cancels running functions with matching cancelOn expressions
  • Writes ingested events to a database for historical record and future replay.
• Queue - A multi-tenant aware, multi-tier queue designed for fairness and various flow control methods (concurrency, throttling, prioritization, debouncing, rate limiting) and batching.
• Executor - Responsible for executing functions, from initial execution, step execution, writing incremental function run state to the State store, and retries after failures.
• State store (database) - Persists data for pending and ongoing function runs. Data includes initial triggering event(s), step output and step errors.
• Database - Persists system data and history including Apps, Functions, Events, Function run results.
• API - GraphQL and REST APIs for programmatic access and management of system resources.
• Dashboard UI - The UI to manage apps, functions and view function run history.

The source code for Inngest and all services is available on GitHub.

How to self-host Inngest

To begin self-hosting Inngest, you only need to install the Inngest CLI. The Inngest CLI is a single binary which includes all Inngest services and can be run in any environment. Alternatively, you download the binary directly from GitHub releases.

npm install -g inngest-cli

Now that you have the CLI installed, you can start the Inngest server using the inngest start command.

inngest start

This will start the Inngest server on the default port 8288 and use the default configuration, including SQLite for persistence.

Configuration

Configuring the server can be done via command line flags, environment variables, or a configuration file.

By default, the server will:

• Run on localhost:8288, including the Event API, API, and Dashboard UI.
• Use an in-memory Redis server for the queue and state store. (See Using external services for more information)
• Use SQLite for persistence. The default database is located at ./.inngest/main.db. Queue and state store snapshots are periodically saved to the SQLite database, including prior to shutdown.
• Disable app sync polling to check for new functions or updated configurations (see --poll-interval flag).

To securely configure your server, create your event and signing keys using whatever format that you choose and start the Inngest server using them. You can also pass them via environment variable (see below):

inngest start --event-key <YOUR_EVENT_KEY> --signing-key <YOUR_SIGNING_KEY>

Then you can use these same keys as environment variables when starting your application (INNGEST_EVENT_KEY and INNGEST_SIGNING_KEY). See below for an example Node.js startup command.

To see all the available options, run inngest start --help:

$ inngest start --help
[Beta] Run Inngest as a single-node service.

Usage:
  inngest start [flags]

Examples:
inngest start

Flags:
      --config string        Path to an Inngest configuration file
      --event-key string     Event key(s) that will be used by apps to send events to the server.
  -h, --help                 Output this help information
      --host string          Server hostname
      --poll-interval int    Enable app sync polling at a specific interval in seconds. (default disabled)
  -p, --port string          Server port (default "8288")
      --retry-interval int   Retry interval in seconds for linear backoff. Minimum: 1.
  -u, --sdk-url strings      SDK URLs to load functions from
      --signing-key string   Signing key used to sign and validate data between the server and apps.
      --tick int             Interval, in milliseconds, of which to check for new work. (default 150)

Persistence Flags:
      --redis-uri string    Redis server URI for external queue and run state. Defaults to self-contained, in-memory Redis server with periodic snapshot backups.
      --sqlite-dir string   Directory for where to write SQLite database.

Group Flags:
      --json               Output logs as JSON.  Set to true if stdout is not a TTY.
  -l, --log-level string   Set the log level.  One of: trace, debug, info, warn, error. (default "info")
  -v, --verbose            Enable verbose logging.

urls:
  - http://localhost:3000/api/inngest
poll-interval: 60
redis-uri: redis://localhost:6379
sqlite-dir: /app/data

Configuring Inngest SDKs to use self-hosted server

By default, the Inngest SDK will use URLs of the managed Inngest platform. To connect to a self-hosted server, set the INNGEST_DEV and INNGEST_BASE_URL environment variables. As mentioned above, you'll also need to set the INNGEST_EVENT_KEY and INNGEST_SIGNING_KEY environment variables for securely connecting your application to the Inngest server.

For example, to connect to a self-hosted server running on localhost:8288 for a Node.js app, set the following environment variables:

INNGEST_EVENT_KEY=<YOUR_EVENT_KEY> \
  INNGEST_SIGNING_KEY=<YOUR_SIGNING_KEY> \
  INNGEST_DEV=0 \
  INNGEST_BASE_URL=http://localhost:8288 \
  node ./server.js

Using external services

Inngest can be configured to use external services for the queue and state store, and soon, the database

External Redis server

With goal of simplifying the initial setup, the Inngest server will run an in-memory Redis server for the queue and state store. As this is running within the same process as the Inngest server, running your own Redis server can improve performance and reliability of the system. You may choose to run your own Redis server or use a cloud-based Redis service like AWS ElastiCache, Redis Cloud, etc.

To use an external Redis server, set the redis-uri flag to the Redis server URI.

External Postgres database (Coming soon)

In an pending release, Inngest will support using an external Postgres database for persistence. This is useful for those who want to use a more scalable and durable database than SQLite.

Roadmap & feature requests

Planned features for self-hosting include:

• Postgres external database support.
• Event key and signing key management via API and UI.
• Multi-node Inngest server support. Run standalone services (API, Executor, etc.) instead of all services in a single process.

To suggest additional features, please submit feedback on our public roadmap.

Check out the source code on GitHub to file issues or submit pull requests.