Tests for microvm configuration from command line parameter

Signed-off-by: Laura Loghin <[email protected]>

Author: lauralt

CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+### Added
+
+- New command-line parameter for both `firecracker` and `jailer`, named 
+  `vmm-config`, which represents a JSON that can be used for configuring and 
+  starting a microVM without sending any API requests.
+
 ## [0.18.0]
 
 ### Added

Cargo.lock

@@ -196,3 +196,9 @@ Possible mitigations are:
   kernel so as to use `vmalloc` instead of `kmalloc` for them.
 - Reduce memory pressure on the host.
 
+### How can I configure and start a microVM without sending API calls?
+
+Passing an optional command line parameter, `vmm-config`, to the Firecracker process 
+allows this type of configuration. This parameter must contain the entire JSON that 
+will be used for configuring all of the microVM's resources. One example of such JSON 
+can be found in `tests/framework/vm_config.json`.  

FAQ.md

@@ -242,6 +242,34 @@ curl --unix-socket /tmp/firecracker.socket -i  \
     }'
 ```
 
+#### Configuring the microVM without sending API requests
+
+If you'd like to boot up a guest machine without using the API socket, you can do that 
+by passing the parameter `vmm-config` to the Firecracker process. The command for starting 
+Firecracker with this option will look like this:
+
+```bash
+./firecracker --api-sock /tmp/firecracker.socket --vmm-config "JSON_BODY"
+```    
+
+"JSON_BODY" should contain the entire configuration for all of the microVM's resources. 
+You **must** provide in your JSON the configuration for the guest kernel and rootfs, as 
+these are mandatory, but all of the other resources are optional, so it's your choice if 
+you want to configure them or not. Because using this configuration method will also start 
+the microVM, you need to specify all desired pre-boot configurable resources in that JSON. 
+The names of the resources are the ones from the `firecracker.yaml` file and the names of 
+their fields are the same that are used in API requests. 
+The easiest way to send the configuration is by storing the JSON in a file and passing 
+the content of the file as command line parameter, for example: 
+
+```bash
+./firecracker --api-sock /tmp/firecracker.socket --vmm-config "$(cat /path_to_the_configuration_file)"
+``` 
+
+You can find an example of configuration file in `tests/framework/vm_config.json`.
+After the machine is booted, you can still use the socket to send API requests
+for post-boot operations.
+
 ## Building From Source
 
 The quickest way to build and test Firecracker is by using our development

docs/getting-started.md

@@ -14,6 +14,7 @@ jailer --id <id> \
        [--netns <netns>]
        [--daemonize]
        [--seccomp-level <level>]
+       [--vmm-config <vmm_config>]
 ```
 
 - `id` is the unique VM identification string, which may contain alphanumeric
@@ -38,6 +39,8 @@ jailer --id <id> \
     Firecracker.
   - 2 (default): advanced filtering. This adds further checks on some of the
     parameters of the allowed syscalls.
+- `vmm_config` represents the json that can be used for configuring and 
+  starting the microVM without using API requests.
 
 ## Jailer Operation
 
@@ -79,12 +82,14 @@ After starting, the Jailer goes through the following operations:
 - Drop privileges via setting the provided `uid` and `gid`.
 - Exec into `<exec_file_name> --id=<id> --api-sock=/api.socket
   --seccomp-level=<level> --start-time-us=<opaque>
-  --start-time-cpu-us=<opaque>`.
+  --start-time-cpu-us=<opaque> [--vmm-config <vmm_config>]`.
   Where:
   - `id`: (`string`) - The `id` argument provided to jailer.
   - `level`: (`number`) - the `--seccomp-level` argument provided to jailer.
   - `opaque`: (`number`) time calculated by the jailer that it spent doing
      its work.
+  - `vmm_config`: (`string`) - optional argument that can be used for 
+     configuring and starting a microVM without sending API requests.
 
 ## Example Run and Notes
 
@@ -191,6 +196,10 @@ We can now use the socket at
 `/srv/jailer/firecracker/551e7604-e35c-42b3-b825-416853441234/root/api.socket`
 to interact with the VM.
 
+In case we want to configure the VM without using the API socket, we can also 
+pass the `vmm-config` parameter, as mentioned in the previous section 
+(`Jailer Operation`).
+
 ### Observations
 
 - The user must create hard links for (or copy) any resources which will be

docs/jailer.md

@@ -113,10 +113,7 @@ impl Env {
             .parse::<u32>()
             .map_err(Error::SeccompLevel)?;
 
-        let config_json = match args.value_of("vmm-config") {
-            Some(s) => Some(String::from(s)),
-            None => None,
-        };
+        let config_json = args.value_of("vmm-config").map(String::from);
 
         Ok(Env {
             id: id.to_string(),
@@ -298,7 +295,7 @@ impl Env {
         }
 
         let mut command = Command::new(chroot_exec_file);
-        let init_command = command
+        command
             .arg(format!("--id={}", self.id))
             .arg(format!("--seccomp-level={}", self.seccomp_level))
             .arg(format!("--start-time-us={}", self.start_time_us))
@@ -311,9 +308,9 @@ impl Env {
             .gid(self.gid());
 
         if let Some(json) = self.config_json {
-            init_command.arg(format!("--vmm-config={}", json));
+            command.arg(format!("--vmm-config={}", json));
         }
-        Err(Error::Exec(init_command.exec()))
+        Err(Error::Exec(command.exec()))
     }
 }
 
@@ -379,6 +376,25 @@ mod tests {
         let gid = "1002";
         let chroot_base = "/";
         let netns = "zzzns";
+        let config_json = r#"{
+           "boot-source":{
+               "kernel_image_path": "vmlinux.bin",
+               "boot_args": "console=ttyS0 reboot=k panic=1 pci=off"
+           },
+           "drives": [
+               {
+                   "drive_id": "rootfs",
+                   "path_on_host": "xenial.rootfs.ext4",
+                   "is_root_device": true,
+                   "is_read_only": false
+               }
+           ],
+           "machine-config":{
+               "vcpu_count": 2,
+               "mem_size_mib": 1024,
+               "ht_enabled": false
+           }
+       }"#;
 
         // This should be fine.
         let good_env = Env::new(
@@ -391,7 +407,7 @@ mod tests {
                 chroot_base,
                 Some(netns),
                 true,
-                None,
+                Some(config_json),
             ),
             0,
             0,
@@ -408,6 +424,7 @@ mod tests {
         assert_eq!(format!("{}", good_env.uid()), uid);
         assert_eq!(good_env.netns, Some(netns.to_string()));
         assert!(good_env.daemonize);
+        assert_eq!(good_env.config_json, Some(config_json.to_string()));
 
         let another_good_env = Env::new(
             make_args(

jailer/src/env.rs

@@ -148,9 +148,7 @@ fn main() {
             .expect("'start-time-cpu_us' parameter expected to be of 'u64' type.")
     });
 
-    let vmm_config_json = cmd_arguments
-        .value_of("vmm-config")
-        .map(|s| String::from(s));
+    let vmm_config_json = cmd_arguments.value_of("vmm-config").map(String::from);
 
     let shared_info = Arc::new(RwLock::new(InstanceInfo {
         state: InstanceState::Uninitialized,

src/main.rs

@@ -62,7 +62,7 @@ def __del__(self):
         """Cleanup this jailer context."""
         self.cleanup()
 
-    def construct_param_list(self):
+    def construct_param_list(self, config_json):
         """Create the list of parameters we want the jailer to start with.
 
         We want to be able to vary any parameter even the required ones as we
@@ -94,6 +94,8 @@ def construct_param_list(self):
             jailer_param_list.extend(
                 ['--seccomp-level', str(self.seccomp_level)]
             )
+        if config_json is not None:
+            jailer_param_list.extend(['--vmm-config', str(config_json)])
         return jailer_param_list
 
     def chroot_base_with_id(self):
@@ -113,16 +115,17 @@ def chroot_path(self):
         """Return the MicroVM chroot path."""
         return os.path.join(self.chroot_base_with_id(), 'root')
 
-    def jailed_path(self, file_path, create=False):
+    def jailed_path(self, file_path, create=False, create_jail=False):
         """Create a hard link owned by uid:gid.
 
         Create a hard link to the specified file, changes the owner to
         uid:gid, and returns a path to the link which is valid within the jail.
         """
         file_name = os.path.basename(file_path)
         global_p = os.path.join(self.chroot_path(), file_name)
+        if create_jail:
+            os.makedirs(self.chroot_path(), exist_ok=True)
         jailed_p = os.path.join("/", file_name)
-
         if create:
             cmd = 'ln -f {} {}'.format(file_path, global_p)
             run(cmd, shell=True, check=True)

tests/framework/jailer.py

@@ -45,7 +45,8 @@ def __init__(
         microvm_id,
         build_feature='',
         monitor_memory=True,
-        bin_cloner_path=None
+        bin_cloner_path=None,
+        config_json=None
     ):
         """Set up microVM attributes, paths, and data structures."""
         # Unique identifier for this machine.
@@ -95,6 +96,9 @@ def __init__(
         self.machine_cfg = None
         self.vsock = None
 
+        # Optional json for configuring microvm from command line parameter.
+        self.config_json = config_json
+
         # The ssh config dictionary is populated with information about how
         # to connect to a microVM that has ssh capability. The path of the
         # private key is populated by microvms with ssh capabilities and the
@@ -205,9 +209,10 @@ def memory_events_queue(self, queue):
         """Set the memory usage events queue."""
         self._memory_events_queue = queue
 
-    def create_jailed_resource(self, path):
+    def create_jailed_resource(self, path, create_jail=False):
         """Create a hard link to some resource inside this microvm."""
-        return self.jailer.jailed_path(path, create=True)
+        return self.jailer.jailed_path(path, create=True,
+                                       create_jail=create_jail)
 
     def get_jailed_resource(self, path):
         """Get the jailed path to a resource."""
@@ -259,7 +264,7 @@ def spawn(self):
         self.network = Network(self._api_socket, self._api_session)
         self.vsock = Vsock(self._api_socket, self._api_session)
 
-        jailer_param_list = self._jailer.construct_param_list()
+        jailer_param_list = self._jailer.construct_param_list(self.config_json)
 
         # When the daemonize flag is on, we want to clone-exec into the
         # jailer rather than executing it via spawning a shell. Going

tests/framework/microvm.py

@@ -0,0 +1,23 @@
+{
+  "boot-source": {
+    "kernel_image_path": "vmlinux.bin",
+    "boot_args": "console=ttyS0 reboot=k panic=1 pci=off"
+  },
+  "logger": {
+    "log_fifo": "log_fifo",
+    "metrics_fifo": "metrics_fifo"
+  },
+  "drives": [
+    {
+      "drive_id": "rootfs",
+      "path_on_host": "xenial.rootfs.ext4",
+      "is_root_device": true,
+      "is_read_only": false
+    }
+  ],
+  "machine-config": {
+    "vcpu_count": 2,
+    "mem_size_mib": 1024,
+    "ht_enabled": false
+  }
+}

tests/framework/vm_config.json

@@ -0,0 +1,41 @@
+# Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+# SPDX-License-Identifier: Apache-2.0
+"""Tests microvm start with command line parameter."""
+
+import os
+
+import pytest
+
+import host_tools.logging as log_tools
+
+
+@pytest.mark.parametrize(
+    "vm_config_json",
+    ["framework/vm_config.json"]
+)
+def test_config_json_start(test_microvm_with_ssh, vm_config_json):
+    """Start a microvm using configuration sent as command line parameter.
+
+    Create resources needed for the configuration of the microvm, then
+    start a process which receives as command line parameter, one json
+    for that configuration.
+    """
+    test_microvm = test_microvm_with_ssh
+
+    test_microvm.create_jailed_resource(test_microvm.kernel_file,
+                                        create_jail=True)
+    test_microvm.create_jailed_resource(test_microvm.rootfs_file,
+                                        create_jail=True)
+
+    log_fifo_path = os.path.join(test_microvm.path, 'log_fifo')
+    metrics_fifo_path = os.path.join(test_microvm.path, 'metrics_fifo')
+    log_fifo = log_tools.Fifo(log_fifo_path)
+    metrics_fifo = log_tools.Fifo(metrics_fifo_path)
+    test_microvm.create_jailed_resource(log_fifo.path, create_jail=True)
+    test_microvm.create_jailed_resource(metrics_fifo.path, create_jail=True)
+
+    test_microvm.config_json = open(vm_config_json).read()
+    test_microvm.spawn()
+
+    response = test_microvm.machine_cfg.get()
+    assert test_microvm.api_session.is_status_ok(response.status_code)

tests/integration_tests/functional/test_cmd_line_start.py

@@ -9,7 +9,6 @@ kvm-ioctls = "0.2"
 libc = ">=0.2.39"
 epoll = "=4.0.1"
 futures = ">=0.1.18"
-hyper = { version = "=0.11.16", default-features = false }
 serde = ">=1.0.27"
 serde_derive = ">=1.0.27"
 serde_json = ">=1.0.9"