Age | Commit message (Collapse) | Author | |
---|---|---|---|
2021-10-26 | runner: remove another layer of child processes | Matthias Schiffer | |
With the restructuring around the poll loop and moving the job token handling to the task process, this has become unnecessary as well. | |||
2021-10-26 | runner: move jobserver token handling into task process | Matthias Schiffer | |
We can avoid holding onto a token while waiting on a lock this way. Abnormal exits that would make us lose tokens should not happen here; if they do, the runner PID1 will stop the whole build, so a lost token does not matter. | |||
2021-10-26 | runner: maintain list of task PIDs in context | Matthias Schiffer | |
2021-10-26 | runner: add RunnerContext struct | Matthias Schiffer | |
Bundle common variables passed to different functions. | |||
2021-10-26 | runner: move more code into the actual task process | Matthias Schiffer | |
Leave only the jobserver token handling in the task spawner process. | |||
2021-10-26 | runner: pass jobserver to task handler as reference | Matthias Schiffer | |
Some code simplification. | |||
2021-10-26 | runner: handle reaping and requests in the poll loop | Matthias Schiffer | |
2021-10-26 | runner: wait for signals using poll() | Matthias Schiffer | |
2021-10-26 | runner: use signal fd to wait for exiting child processes | Matthias Schiffer | |
Will be used to poll for new tasks and signals in the same process. | |||
2021-10-26 | executor: context: remove unneeded Serialize impl on TaskRef/OutputRef | Matthias Schiffer | |
2021-10-26 | Replace ipc-channel with UNIX sockets and manual FD passing | Matthias Schiffer | |
The new implementation is more flexible, as it allows mixed polling with other FD types, and it saves us a whole zoo of dependencies. | |||
2021-10-26 | Switch to 2021 edition | Matthias Schiffer | |
2021-10-26 | runner: hold lock on task state dir while task is running | Matthias Schiffer | |
Do not allow concurrent runs of the same task. | |||
2021-10-26 | runner: do not delete whole task dir, only layer and work dir | Matthias Schiffer | |
We will also store a lockfile in the taskdir that must not be deleted. | |||
2021-10-26 | runner: make lock() function more generic | Matthias Schiffer | |
2021-10-25 | Allow to set the maximum number of jobs on the command line | Matthias Schiffer | |
2021-10-25 | Add lockfile for build state directory | Matthias Schiffer | |
Use flock() to ensure that there are no concurrent instances using the same build state. More fine-gained locking may be implemented in the future. | |||
2021-10-25 | Update dependencies | Matthias Schiffer | |
2021-10-25 | Move main crate to subdirectory | Matthias Schiffer | |
2021-10-25 | Move runner into separate crate | Matthias Schiffer | |
2021-10-24 | Split defintions used by both runner and executor into separate crate | Matthias Schiffer | |
Also get rid of the Runner trait - different runner implementations do not make sense with our current design. | |||
2021-10-24 | Move util modules to container runner | Matthias Schiffer | |
The ipc module was unused and is removed completely. | |||
2021-10-24 | container: remove unneeded "pub" visibility from BUILD_UID/BUILD_GID | Matthias Schiffer | |
2021-10-24 | Move Hasher type aliases to runner | Matthias Schiffer | |
2021-10-24 | runner: merge TaskInput struct into Task | Matthias Schiffer | |
2021-10-24 | Move definition of Dependency into runner module | Matthias Schiffer | |
runner::TaskInput does not contain the dependency hashes anymore, this is handled internally in the runner module now. Unfortunately this also means that we can't use this struct directly for the InputHash anymore, but overall this reduces the complexity of the interface between runner and executor. | |||
2021-10-24 | jobserver: always make the reading pipe end nonblocking | Matthias Schiffer | |
Match the behaviour of GNU Make. | |||
2021-10-24 | jobserver: preserve token value | Matthias Schiffer | |
The Make manual recommends to preserve the token value when interacting with a jobserver pipe. While this is mostly meant for processes run by Make rather than processes running Make and with current Make, the token value is never changed, it is easy to implement this recommendation. | |||
2021-10-24 | container: pass jobserver pipe to tasks through MAKEFLAGS | Matthias Schiffer | |
Allow all tasks to share a common job token pool. | |||
2021-10-24 | examples: remove explicit parallelism handling | Matthias Schiffer | |
We are going to pass in a jobserver pipe through MAKEFLAGS. | |||
2021-10-24 | container: create additional child process for tasks | Matthias Schiffer | |
Ensure that we don't lose jobserver tokens even if a task exits abnormally. | |||
2021-10-24 | container: add reaper loop | Matthias Schiffer | |
Don't let the runner PID1 leave zombie processes unreaped. | |||
2021-10-24 | container: implement jobserver pipe | Matthias Schiffer | |
Limit the number of parallel tasks by the runner's CPU affinity. The jobserver is implemented as a pipe like in Make to allow passing it down to tasks. | |||
2021-10-24 | container: consistently use spawn-style fork | Matthias Schiffer | |
Also remove a few process::exit() that happen implicitly now. | |||
2021-10-24 | Update dependencies | Matthias Schiffer | |
2021-10-22 | container: add additional child process for task submission handling | Matthias Schiffer | |
The parent process will be used for process reaping and signal handling. | |||
2021-10-22 | container: refactor loop handling | Matthias Schiffer | |
2021-10-22 | container: create enum type for submission channel | Matthias Schiffer | |
2021-10-22 | container: drop task submission channel in child processes | Matthias Schiffer | |
2021-10-22 | executor: return error when reply channel is closed unexectedly | Matthias Schiffer | |
2021-10-21 | container: do not create /dev/mqueue | Matthias Schiffer | |
We do not mount /dev/mqueue, so there is no need to create the mount point. | |||
2021-10-21 | container: set process name | Matthias Schiffer | |
2021-10-21 | executor: run ready tasks in parallel | Matthias Schiffer | |
2021-10-21 | runner: expose reply channel | Matthias Schiffer | |
2021-10-21 | util: error: improve conversion of io::Error | Matthias Schiffer | |
Convert the whole source chain. | |||
2021-10-21 | container: make setup more similar to OCI runtime | Matthias Schiffer | |
- Make whole tree MS_PRIVATE - Create extra mount namespace after creating container mounts | |||
2021-10-21 | paths: update documentation | Matthias Schiffer | |
2021-10-20 | container: write task command to script file | Matthias Schiffer | |
2021-10-20 | container: do not call sync_all() on log file | Matthias Schiffer | |
Saves a few ms per task. | |||
2021-10-20 | container: do not use unshare command | Matthias Schiffer | |
We can create the user namespace ourselves. |