Squashed commit of the following:

commit 886429b
Author: Matt Aitken <[email protected]>
Date:   Tue Oct 8 18:36:46 2024 +0100

    Removed emails, @trigger.dev/database and @trigger.dev/otlp-importer from changesets config

commit f65157a
Author: Matt Aitken <[email protected]>
Date:   Tue Oct 8 18:31:27 2024 +0100

    Lockfile with run-engine removed

commit 3d67bb8
Author: Matt Aitken <[email protected]>
Date:   Tue Oct 8 18:24:31 2024 +0100

    Removed run-engine from the webapp package.json/tsconfig

commit d30e971
Author: Matt Aitken <[email protected]>
Date:   Tue Oct 8 18:06:04 2024 +0100

    Dockerfile fix because the database package has been moved

commit f2babbf
Author: Matt Aitken <[email protected]>
Date:   Tue Oct 8 09:41:22 2024 -0700

    Internal packages (testcontainers, redis-worker and zod-worker) (#1392)

    * Some notes on the new run engine

    * lockfile with setup for the run engine

    * Documenting where TaskRun is currently mutated, to try figure out the shape of the new system

    * Added notes about how triggering currently works

    * Details about when triggering happens

    * Lots of notes about waitpoints

    * Started scaffolding the RunEngine

    * Sketch of Prisma waitpoint schema while it’s fresh in my mind

    * Got Prisma working with testcontainers

    * Use beforeEach/afterEach

    * Simple Prisma and Redis test

    * Return Redis options instead of a client

    * Simplified things

    * A very simple FIFO pull-based queue to check the tests working properly

    * Use vitest extend

    * Separate redis, postgres and combined tests for faster testing

    * Some fixes and test improvements

    * Pass a logger into the queue

    * A queue processor that processes items from the given queue as fast as it can

    * Test for retrying an item that wasn’t processed

    * First draft of waitpoints in the Prisma schema

    * Remove the custom logger from the test

    * Added a completedAt to Waitpoint

    * Notes on the flow for an execution starting

    * Added redlock, moved some files around

    * Starting point for the TaskRunExecutionSnapshot table

    * Added relationships to TaskRunExecutionSnapshot

    * Change some tsconfig

    * Moved some things around

    * Added some packages

    * WIP on the RunQueue

    * Fix for some imports

    * Key producer with some tests

    * Removed the nv type from the keys… it’s not useful to do global queries

    * Passing unit tests for all the public key producer functions

    * Some basic tests passing for the RunQueue

    * Simple enqueue test working

    * Enqueue and dequeue for dev is working

    * Don’t log everything during the tests

    * Enqueuing/dequeuing from the shared queue is working

    * Tests for getting a shared queue

    * The key producer sharedQueue can now be named, to allow multiple separate queues

    * The key producer uses the name of the queue as the input

    * Extra info in the Prisma schema

    * Dequeuing a message gets the payload and sets the task concurrency all in one Lua script

    * Adding more keys so we can read the concurrency from the queue

    * Setting the concurrency with dequeue and enquque is working

    * Improved the tests and fixed some bugs

    * Acking is resetting the concurrencies

    * Check the key has been removed after acking

    * Nacking is working

    * Changed the package to CommonJS + Node10 so it works with Redlock

    * Moved the database, otel and emails packages to be in internal-packages

    * Moved some Prisma code to the database package

    * Started using the RunEngine for triggering

    * Progress on run engine triggering, first waitpoint code

    * Create a delay waitpoint

    * Moved ZodWorker to an internal package so it can be used in the run engine as well as the webapp

    * Web app now uses the zod worker package

    * Added parseNaturalLanguageDuration to core/apps

    * internal-packages/zod-worker in the lockfile

    * Pass in the master queue, remove old rebalance workers code

    * Add masterQueue to TaskRun

    * Fixed the tests

    * Moved waitpoint code into the run engine, also the zod worker

    * Completing waitpoints

    * An experiment to create a new test container with environment

    * More changes to triggering

    * Started testing triggering

    * Test for a run getting triggered and being enqueued

    * Removed dequeueMessageInEnv

    * Update dev queue tests to use the shared queue function

    * Schema changes for TaskRunExecutionSnapshot

    * First execution snapshot when the run is created. Dequeue run function added to the engine

    * Separate internal package for testcontainers so they can be used elsewhere

    * Remove the simple queue and testcontainers from the run-engine. They’re going to be separate

    * Fix for the wrong path to the Prisma schem,a

    * Added the testcontainers package to the run-engine

    * redis-worker package, just a copy of the simple queue for now

    * The queue now uses Lua to enqueue dequeue

    * The queue now has a catalog and an invisible period after dequeuing

    * Added a visibility timeout and acking, with tests

    * Added more Redis connection logging, deleted todos

    * Visibility timeouts are now defined on the catalog and can be overridden when enqueuing

    * Dequeue multiple items at once

    * Test for dequeuing multiple items

    * Export some types to be used elsewhere

    * Partial refactor of the processor

    * First stab at a worker with concurrency and NodeWorkers

    * Don’t have a default visibility timeout in the queue

    * Worker setup and processing items in a simple test

    * Process jobs in parallel with retrying

    * Get the attempt when dequeuing

    * Workers do exponential backoff

    * Moved todos

    * DLQ functionality

    * DLQ tests

    * Same cluster for all keys in the same queue

    * Added DLQ tests

    * Whitespace

    * Redis pubsub to redrive from the worker

    * Fixed database paths

    * Fix for path to zod-worker

    * Fixes for typecheck errors, mostly with TS versions and module resolution

    * Redlock required a patch

    * Moved the new DB migrations to the new database package folder

    * Remove the run-engine package

    * Remove the RunEngine prisma schema changes

    * Delete triggerTaskV2

    * Remove zodworker test script (no tests)

    * Update test-containers readme

    * Generate the client first

    * Use a specific version of the prisma package

    * Generate the prisma client before running the unit tests

commit fc60947
Author: Dan <[email protected]>
Date:   Tue Oct 8 14:36:03 2024 +0100

    Supabase database webhook example upgrade (#1386)

    * Added overview for guides and examples section and split them all out

    * New supabase guide wip

    * Updated images and improved docs

    * Trimmed the supabase prereqs

    * Supabase guide wip

    * more updates

    * Replaced old database webhook guide

    * Created one intro page and removed snippets

    * Updated guide sidebar titles

    * Code updates

    * More improvements

    * Updates and added images

    * Compressed image

    * Updated guides descriptions and edge function basic

    * Removed bold

    * Updated redirects

    * Fixed broken links

    * Updated intro

commit 07f82ea
Author: nicktrn <[email protected]>
Date:   Tue Oct 8 13:28:54 2024 +0100

    Release 3.0.11

commit 13ebfcc
Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Date:   Tue Oct 8 13:24:38 2024 +0100

    chore: Update version for release (#1381)

    Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

commit 2a04d17
Author: nicktrn <[email protected]>
Date:   Tue Oct 8 09:24:23 2024 +0100

    Simplify showLogs expression

commit 002ae4b
Author: nicktrn <[email protected]>
Date:   Tue Oct 8 09:22:29 2024 +0100

    Fix dotenv overrides for dev runs (#1388)

    * override dashboard dev env vars with local .env

    * add changeset

    * add simple task for testing env vars

commit 047cb00
Author: nicktrn <[email protected]>
Date:   Tue Oct 8 09:22:05 2024 +0100

    Disable schedules for deleted orgs on next tick (#1383)

    * disable schedules for deleted orgs

    * add debug logs

commit 2c014f7
Author: James Ritchie <[email protected]>
Date:   Sun Oct 6 13:02:00 2024 -0700

    Override log retention (#1385)

    * set full log retention as admin

    * If run.logsDeletedAt is set, don’t bother getting the trace

commit a69e04f
Author: nicktrn <[email protected]>
Date:   Sat Oct 5 14:18:58 2024 +0100

    Include push output in logs for self-hosted deploys (#1382)

    * include push output in logs

    * changeset

commit c5488df
Author: nicktrn <[email protected]>
Date:   Sat Oct 5 13:12:47 2024 +0100

    Fix CLI downgrade check (#1380)

    * fix downgrade detection

    * remove unused semver package from webapp

    * add changeset

commit 1caec27
Author: Eric Allam <[email protected]>
Date:   Fri Oct 4 15:33:35 2024 -0700

    docs: Max duration (#1379)

    * maxDuration docs

    * Update the init command to set the maxDuration and include a commented out maxDuration in the config file

commit e14c954
Author: Eric Allam <[email protected]>
Date:   Fri Oct 4 15:02:05 2024 -0700

    Release 3.0.10

commit 8e61f5d
Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Date:   Fri Oct 4 14:59:07 2024 -0700

    chore: Update version for release (#1378)

    Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

commit 08db565
Author: Eric Allam <[email protected]>
Date:   Thu Oct 3 12:38:25 2024 -0700

    improve the timed out description

commit 6d08842
Author: Eric Allam <[email protected]>
Date:   Thu Oct 3 12:43:26 2024 -0700

    feat: Add maxDuration to tasks (#1377)

    * WIP

    * Get max duration working on deployed runs

    * Actually set the timed out runs to status = TIMED_OUT

    * The client status for TIMED_OUT is now MAX_DURATION_EXCEEDED

    * New TimedOutIcon

    * Added new timedout icon

    * Add ability to opt-out of maxDuration with timeout.None

    * MAX_DURATION_EXCEEDED -> TIMED_OUT

    * changeset

    * Improved styling for the status tooltip content

    ---------

    Co-authored-by: James Ritchie <[email protected]>

commit 665ccf8
Author: nicktrn <[email protected]>
Date:   Thu Oct 3 12:33:18 2024 +0100

    Update github actions and self-hosting docs

commit 1ff7b86
Author: Eric Allam <[email protected]>
Date:   Wed Oct 2 18:26:36 2024 -0700

    Add max queue depth limits (#1376)

    * Add runs to an env queue, as well as the actual queue

    * Add queue size limit guard on triggering tasks

commit c531a9d
Author: Eric Allam <[email protected]>
Date:   Wed Oct 2 15:30:39 2024 -0700

    fix: cleanup ttl expire run graphile jobs (#1373)

    * fix: remove ttl expire run graphile jobs when a run is started or completed

    * Update expireEnqueuedRun.server.ts

commit 0bf500f
Author: Matt Aitken <[email protected]>
Date:   Wed Oct 2 15:30:16 2024 -0700

    Prioritize finishing waited runs (#1375)

    * If a tree node is missing, estimate the size as zero

    * Task to test prioritizing finishing existing runs after triggerAndWaits

    * When requeuing a run with a checkpoint, put it in the queue with the parent run time so it’s correctly prioritized

    * The same change but if there’s no checkpoint

Author: samejr