diff --git a/README.adoc b/README.adoc new file mode 100644 index 0000000..aa7f496 --- /dev/null +++ b/README.adoc @@ -0,0 +1,99 @@ += Stackable Agent + +== Purpose +The Stackable Agent is an alternative to the Kubernetes Kubelet that executes Pods not in containers but using systemd as its backend. +It is implemented in Rust as a https://github.com/deislabs/krustlet[Krustlet] provider. + +In principle this behavior is similar to that of the the https://github.com/virtual-kubelet/virtual-kubelet[virtual kubelet] project. +More specifically the functionality of this agent closely resembles that of https://github.com/virtual-kubelet/systemk[systemk]. +The reason why both projects currently exist is simply that systemk was not around when we started Stackable and so we had to go and build our own. + +WARNING: We are currently evaluating whether or not we should instead use systemk and contribute to that project instead of building our own agent under https://github.com/stackabletech/agent/issues/42[issue #42] + +The agent registers itself as a node with a kube-apiserver and will be considered by the Kubernetes scheduler for workloads (pods). +To avoid _normal_ Kubernetes pods being scheduled on the Stackable agent (it would not know what to do with these) the agent assigns the following taints to its node object: + +|=== +|Taint |Type|Value + +|kubernetes.io/arch +|NoSchedule +|stackable-linux + +|kubernetes.io/arch +|NoExecute +|stackable-linux +|=== + +These taints _suggest_ to the Kubernetes scheduler that only pods with matching tolerations should be scheduled on this node. + +== Installation +=== Build from source +Building from source is fairly straightforward if you have the rust toolchain installed, our CI chain tests against the latest stable version at the time the checks are run. +If you need to install this, generally the recommended way is to use https://rustup.rs/[rustup]. + +After rust is installed simply run + + cargo build + +To create the binary file in _target/debug_. + +=== Download binary +We do not at this time provide pre-compiled binaries, as we are still in the process of setting up the build jobs to create these. +This readme will be updated as soon as binary downloads are available! + +=== OS Packages +We do not at this time provide OS packages, that is due to change in the near future and this readme will be updated accordingly! + +== Configuration + +=== Command Line Parameters +The agent accepts the following command line parameters: + +include::documentation/commandline_args.adoc[] + +=== Config File +In addition to directly specifying them on the command line, the agent allows specifying a config file via the environment variable `CONFIG_FILE`. Values specified in the file will have to adhere to the format `--parameter=value`. + +This file can contain all command line parameters and will be parsed before the actual command line. +For parameters that are present in the file and on the command line, the command line will take precedence, unless it is a parameter that can be specified multiple times, in which case parameters from both, file and commandline, will be merged. + +.Example config file + --package-directory=/opt/stackable/agent/work/packages + --config-directory=/etc/stackable/agent + --server-cert-file=/etc/stackable/agent/secure/cert.crt + --server-key-file=/etc/stackable/agent/secure/key.key + +=== Kubernetes Config +The agent uses the default way of looking for a kube-apiserver, so if your system is already set up to connect to Kubernetes with kubectl you should be good to go right of the bat. + +The default location for the Kubernetes client config is `~/.kube/config`, if you want to change this location you can override this via the `KUBECONFIG` environment variable. + + export KUBECONFIG=/etc/stackable/agent/kubeconfig + + +=== Certificates +The agent requires a keypair and signed certificate to start a webserver which can be used to handle callbacks. +If these are not specified on the commandline, the agent will create a keypair, upload a certificate signing request to Kubernetes and wait for the certificate before continuing. +These steps require a certificate manager to be set up and running in your Kubernetes cluster, which may or may not be the case. + +You can also manually create these files and specify them on the command line. +The following example shows how to create these files using https://github.com/OpenVPN/easy-rsa[easy-rsa], but this can be done in any number of different ways as well. + + ./easyrsa init-pki + ./easyrsa build-ca + ./easyrsa gen-req krustlet1 + ./easyrsa import-req pki/reqs/krustlet1.req krustlet1-req + ./easyrsa sign-req serverClient krustlet1-req + # Convert key to pksc8 format + openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in pki/private/krustlet1.key -out pkcs8.key + +== Contributing +The agent is developed as an open source tool and we absolutely welcome any and all contributions! +Don't hesitate to drop us a line at info@stackable.de or reach out directly to any of our committers / contributors. + +We will run a bi-weekly dev call during which we will discuss current development, issues, our children and the general state of the world. +Please feel free to drop in if you have the time! + +Dial in info coming soon! + diff --git a/documentation/commandline_args.adoc b/documentation/commandline_args.adoc new file mode 100644 index 0000000..f3f6ec7 --- /dev/null +++ b/documentation/commandline_args.adoc @@ -0,0 +1,166 @@ + + + +=== tag + +*Default value*: No default value + +*Required*: false + +*Multiple values:* true + + +A "key=value" pair that should be assigned to this agent as tag. This can be specified multiple times to assign additional tags. + +Tags are the main way of identifying nodes to assign services to later on. + + +=== server-key-file + +*Default value*: No default value + +*Required*: false + +*Multiple values:* false + + +Private key file (in PKCS8 format) to use for the local webserver the Krustlet starts. + + +=== log-directory + +*Default value*: /opt/stackable/logs + +*Required*: false + +*Multiple values:* false + + +This directory will serve as starting point for all log files which this service creates. +Every service will get its own subdirectory created within this directory. +Anything that is then specified in the log4j config or similar files will be resolved relatively to this directory. + +The agent will need full access to this directory and tries to create it if it does not exist. + + +=== hostname + +*Default value*: No default value + +*Required*: false + +*Multiple values:* false + + +The hostname to register the node under in Kubernetes - defaults to system hostname. + + +=== server-cert-file + +*Default value*: No default value + +*Required*: false + +*Multiple values:* false + + +The certificate file for the local webserver which the Krustlet starts. + + +=== no-config + +*Default value*: No default value + +*Required*: false + +*Multiple values:* false + + +If this option is specified, any file referenced in AGENT_CONF environment variable will be ignored. + + +=== bootstrap-file + +*Default value*: /etc/kubernetes/bootstrap-kubelet.conf + +*Required*: false + +*Multiple values:* false + + +The bootstrap file to use in case Kubernetes bootstraping is used to add the agent. + + +=== server-bind-ip + +*Default value*: No default value + +*Required*: false + +*Multiple values:* false + + +The local IP to register as the node's ip with the apiserver. Will be automatically set to the first address of the first non-loopback interface if not specified. + + +=== data-directory + +*Default value*: /var/stackable/agent/data + +*Required*: false + +*Multiple values:* false + + +The directory where the stackable agent should keep its working data. + + +=== config-directory + +*Default value*: /opt/stackable/config + +*Required*: false + +*Multiple values:* false + + +This directory will serve as starting point for all log files which this service creates. + +Every service will get its own subdirectories created within this directory - for every service start a +new subdirectory will be created to show a full history of configuration that was used for this service. + +ConfigMaps which are specified in the pod that describes this service will be created relative to these run +directories - unless the mounts specify an absolute path, in which case it is allowed to break out of this directory. + +WARNING: This allows anybody who can specify pods more or less full access to the file system on the machine running the agent! + +The agent will need full access to this directory and tries to create it if it does not exist. + + +=== server-port + +*Default value*: 3000 + +*Required*: false + +*Multiple values:* false + + +Port to listen on for callbacks. + + +=== package-directory + +*Default value*: /opt/stackable/packages + +*Required*: false + +*Multiple values:* false + + +This directory will serve as starting point for packages that are needed by pods assigned to this node.\n Packages will be downloaded into the "_download" folder at the top level of this folder as archives and remain there for potential future use. + +Archives will the be extracted directly into this folder in subdirectories following the naming +scheme of "productname-productversion". + +The agent will need full access to this directory and tries to create it if it does not exist. \ No newline at end of file diff --git a/src/main.rs b/src/bin/agent.rs similarity index 97% rename from src/main.rs rename to src/bin/agent.rs index 2d3d0d1..74f8641 100644 --- a/src/main.rs +++ b/src/bin/agent.rs @@ -8,11 +8,8 @@ use kubelet::Kubelet; use log::{info, warn}; use stackable_config::ConfigBuilder; -use crate::agentconfig::AgentConfig; -use crate::provider::StackableProvider; - -mod agentconfig; -mod provider; +use stackable_agent::config::AgentConfig; +use stackable_agent::provider::StackableProvider; #[tokio::main(threaded_scheduler)] async fn main() -> anyhow::Result<()> { diff --git a/src/bin/generate_doc.rs b/src/bin/generate_doc.rs new file mode 100644 index 0000000..7396294 --- /dev/null +++ b/src/bin/generate_doc.rs @@ -0,0 +1,20 @@ +/// This is a helper binary which generates the file _documentation/commandline_args.adoc_ which +/// contains documentation of the available command line options for the agent binary. +/// +/// It gets the content by calling [`stackable_agent::config::AgentConfig::get_documentation()`] +/// +/// # Panics +/// This will panic if an error occurs when trying to write the file. + +fn main() { + use stackable_agent::config::AgentConfig; + use std::fs; + + let target_file = "documentation/commandline_args.adoc"; + fs::write(target_file, AgentConfig::get_documentation()).unwrap_or_else(|err| { + panic!( + "Could not write documentation to [{}]: {}", + target_file, err + ) + }); +} diff --git a/src/config_documentation/bootstrap_file.adoc b/src/config/config_documentation/bootstrap_file.adoc similarity index 100% rename from src/config_documentation/bootstrap_file.adoc rename to src/config/config_documentation/bootstrap_file.adoc diff --git a/src/config_documentation/config_directory.adoc b/src/config/config_documentation/config_directory.adoc similarity index 100% rename from src/config_documentation/config_directory.adoc rename to src/config/config_documentation/config_directory.adoc diff --git a/src/config_documentation/data_directory.adoc b/src/config/config_documentation/data_directory.adoc similarity index 100% rename from src/config_documentation/data_directory.adoc rename to src/config/config_documentation/data_directory.adoc diff --git a/src/config_documentation/hostname.adoc b/src/config/config_documentation/hostname.adoc similarity index 100% rename from src/config_documentation/hostname.adoc rename to src/config/config_documentation/hostname.adoc diff --git a/src/config_documentation/log_directory.adoc b/src/config/config_documentation/log_directory.adoc similarity index 100% rename from src/config_documentation/log_directory.adoc rename to src/config/config_documentation/log_directory.adoc diff --git a/src/config_documentation/no_config.adoc b/src/config/config_documentation/no_config.adoc similarity index 100% rename from src/config_documentation/no_config.adoc rename to src/config/config_documentation/no_config.adoc diff --git a/src/config_documentation/package_directory.adoc b/src/config/config_documentation/package_directory.adoc similarity index 100% rename from src/config_documentation/package_directory.adoc rename to src/config/config_documentation/package_directory.adoc diff --git a/src/config_documentation/plugin_directory.adoc b/src/config/config_documentation/plugin_directory.adoc similarity index 100% rename from src/config_documentation/plugin_directory.adoc rename to src/config/config_documentation/plugin_directory.adoc diff --git a/src/config_documentation/server_cert_file.adoc b/src/config/config_documentation/server_cert_file.adoc similarity index 100% rename from src/config_documentation/server_cert_file.adoc rename to src/config/config_documentation/server_cert_file.adoc diff --git a/src/config_documentation/server_ip_address.adoc b/src/config/config_documentation/server_ip_address.adoc similarity index 100% rename from src/config_documentation/server_ip_address.adoc rename to src/config/config_documentation/server_ip_address.adoc diff --git a/src/config_documentation/server_key_file.adoc b/src/config/config_documentation/server_key_file.adoc similarity index 100% rename from src/config_documentation/server_key_file.adoc rename to src/config/config_documentation/server_key_file.adoc diff --git a/src/config_documentation/server_port.adoc b/src/config/config_documentation/server_port.adoc similarity index 100% rename from src/config_documentation/server_port.adoc rename to src/config/config_documentation/server_port.adoc diff --git a/src/config_documentation/tags.adoc b/src/config/config_documentation/tags.adoc similarity index 100% rename from src/config_documentation/tags.adoc rename to src/config/config_documentation/tags.adoc diff --git a/src/agentconfig.rs b/src/config/mod.rs similarity index 94% rename from src/agentconfig.rs rename to src/config/mod.rs index 66aa119..ab7923b 100644 --- a/src/agentconfig.rs +++ b/src/config/mod.rs @@ -10,7 +10,7 @@ use pnet::datalink; use stackable_config::{ConfigOption, Configurable, Configuration}; use thiserror::Error; -use crate::agentconfig::AgentConfigError::{ArgumentParseError, WrongArgumentCount}; +use crate::config::AgentConfigError::{ArgumentParseError, WrongArgumentCount}; #[derive(Error, Debug)] pub enum AgentConfigError { @@ -278,6 +278,28 @@ impl AgentConfig { .into_string() .map_err(|_| anyhow::anyhow!("invalid utf-8 hostname string")) } + + pub fn get_documentation() -> String { + let mut doc_string = String::new(); + for option in AgentConfig::get_options() { + doc_string.push_str(&format!("\n\n\n=== {}\n\n", option.name)); + doc_string.push_str(&format!( + "*Default value*: `{}`\n\n", + option.default.unwrap_or("No default value") + )); + doc_string.push_str(&format!("*Required*: {}\n\n", option.required)); + doc_string.push_str(&format!("*Multiple values:* {}\n\n\n", option.list)); + + // We have not yet specified a documentation string for all options, as an interim + // solution we use the help string for the docs, if no proper doc has been written yet. + if option.documentation.is_empty() { + doc_string.push_str(&option.help); + } else { + doc_string.push_str(&option.documentation); + } + } + doc_string + } } impl Configurable for AgentConfig { diff --git a/src/lib.rs b/src/lib.rs new file mode 100644 index 0000000..84c6e89 --- /dev/null +++ b/src/lib.rs @@ -0,0 +1,2 @@ +pub mod config; +pub mod provider;