PR to 9fc93a4 (#2)

* lucet-idl: moved half of datatype validation & layout code to new side

* lucet-idl: datatype validation & layout calculated in new stuff

* lucet-idl: port in func validator

* lucet-idl: factor module names into their own validator

put all validators behind a single module name

* lucet-idl: port module datatype tests into new code

* lucet-idl: build out cursor, module validation for funcs

* lucet-idl: finish up module tests in env

* lucet-idl: add package-level validation and tests to new side

* lucet-idl: move new repr/cursor stuff to root of crate. codegen disabled!

* lucet-idl: a PackageRepr is a Package now

unboxing it fixed our lifetime problems

* lucet-idl: c generator turned on again

and it looks SO MUCH NICER dont you think??

* lucet-idl: build out idiomatic func defintion in C!

* lucet-idl: rust datatypes are generated

* lucet-idl: add rust guest abi import

* lucet-idl: rust backend ported to new cursors

* lucet-idl-test: fixup most issues

* lucet-idl: move atomtype/abitype parsing from lexer to validator

* lucet-idl: parse tree uses &'syntax str rather than String

one less clone is good

* lucet-idl: reject out-slice bindings

cant support these till we make an allocation protocol

* lucet-idl: refactor rust backend to be more reasonable

* lucet-idl-test: render harnesses for both rust guest and host

* bump cargo.lock

* lucet-idl-test: many fixes, very nearly works

* lucet-idl-test: fix testing! we can now round-trip test the example idl successfully!

* lucet-idl: get rid of `io` binding syntax, just support `inout`

* lucet-idl: only derive Eq and Ord on structs that do not contain floats

* lucet-idl: simplification in parser for func return value / whereclauses

* lucet-idl: explain c backend unreachable

* lucet-idl: make all of the cursor structs Copy

and eliminate a ton of clones as a result

* lucet-idl: use a while let

* lucet-idl: host func name is a rust-specific thing

* lucet-idl: validationerror derives Fail

* lucet-idl: prelude tests use PackageBuilder

which makes the thing being tested less obvious, but the invariant does
have to hold across these two modules. not a great design tbh

* lucet-idl: derive default unless contains enum

* lucet-idl: use MaybeUninit instead of null for struct member offset assertions

* lucet-idl: get rid of Deref impls of Datatype

rust should have a non-deref way to delegate method calls to a member

* lucet-idl: many code review fixes

* delete empty file

* lucet-idl: get rid of tons of unnecessary refs/clones of Location

* lucet-idl: fixup atom/abi relationships

allow i64 abi to represent smaller types

* lucet-idl: fix error messages in binding validation tests

and add an extra test case or two

* lucet-idl-test: fix stub

* lucet-idl-test: fixes

* lucet-idl: separate parse trees for package-level and syntax-level decls

rather than a recursive parse tree.

* lucet-idl: finish fixing parser split-up

* lucet-idl-test: split test plan into separate module; fixes

* lucet-idl: render rust cast to boolean as != 0

* lucet-idl-test: implement trivial test plan

* lucet-idl: func validator review comments

* lucet-idl: proptest found bug in parser :)

* bump cargo.lock

* Update the bincode crate to version 1.1 (bytecodealliance#274)

Using an old version prevents applications using the lucet crate
from also using bincode 1.1.

* delete faerie submodule

* upgrade to faerie 0.11, and latest cranelift, target-lexicon, wasmparser etc

* cargo install with debug to avoid full release build times

* Fixes bytecodealliance#288 - Remove cranelift-codegen from the dependencies of the lucet-runtime components such as lucet-module

* Fix a few typos

* lucet-idl: parse and validate `witx` format (bytecodealliance#275)

* lucet-idl: competent sexpr parser

* lucet-idl: start implementing grammar from https://github.com/WebAssembly/WASI/pull/74/files

* lucet-idl: add dan's wasi_unstable.wati as a test case

original commit, from https://github.com/sunfishcode/wasi:

commit 383385700a13acc2bc4ad2ac7e78a445d9053d21 (HEAD, refs/remotes/sunfishcode/moduletype)
Author: Dan Gohman <[email protected]>
Date:   Wed Aug 14 22:42:24 2019 -0700

    A wasi_unstable description based on module types.

    This sketches out a description of the current wasi_unstable API, using a syntax which anticipates the "module types" syntax [recently proposed to the CG](https://github.com/WebAssembly/meetings/blob/master/2019/CG-08-06.md#discuss-new-proposal-that-introduces-types-for-modules-and-instances-and-text-format-thereof-as-initially-discussed-in-design1289), though it does use a few extensions which can be easily lowered, and which mostly anticipate the [Interface Types](WebAssembly/interface-types#47 (comment)) proposal.

    This is derived from [the WatIDL proposal](WebAssembly/WASI#64), though it differs in a few ways:
     - It doesn't yet do anything special for capabilities, for now. File descriptors are just integers, for now. But the idea is that we could add OCAP concepts later.

     - It uses a new `(flags ...)` construct in place of structs of Bool fields, to describe flags values. This allows us to explicitly declare the underlying wasm type for flags values.

     - Types used by the API are declared outside of the moduletype, because we're using wat syntax as much as possible, and there, `(type ...)` inside a module declares an entry in the type section, which isn't what's intended here.

    The extensions to module types are:
     - It uses `string`, `array`, and `{s,u}{8,16,32,64}` types, which are expected to be provided in some form by the Interface Types proposal.  For now though, they can all be lowered in straightforward ways, by assuming that `string` is always just UTF-8.

     - It adds `typename` declarations, which can name any wasm type, and of the extra types mentioned in the previous item, or `struct`, `enum`, or `flags`.

     - It declares functions with multiple return values, which for now will need to be lowered to output (pointer) parameters.

* lucet-idl: enough infra to feed wasi_unstable.wati through parser

* lucet-idl: fix typo in wasi_unstable

lexer found it! cheers

* lucet-idl: better error reporting

* lucet-idl: parse a bunch more of wati

* lucet-idl: @interface is an Annot("interface")

* lucet-idl: switch from wasi_unstable.wati to multiple .witx

* lucet-idl: interface-types idl files are now called .witx

webassembly interface types has claimed the `.wit` file extension for
describing just module interfaces.

this idl is currently the `.wit` syntax, plus extensions. in lieu of a
better name, dan and I decided `.witx` will work for the time being.

* lucet-idl: renames of stuff in witx, plus use stmts

* lucet-idl: typenames.witx had an extra close paren

* lucet-idl: correct path to typenames in wasi_unstable.witx

* lucet-idl: change parser to own everything. start implementing use

* lucet-idl: witx use directive works!

* lucet-idl: better error messages in parser through lookahead

* lucet-idl: parser makes it through wasi_unstable

* lucet-idl: wasi_unstable.witx: fix two syntax errors in

* lucet-idl: witx use decls are now like #pragma once

so we dont have to care about cycles, and diamond dependencies
will be handled correctly

* lucet-idl: witx use resolution moved to toplevel.rs

* lucet-idl: make an AST for witx

* lucet-idl: rename interfacetypes to witx

* lucet-idl witx: file name goes in location, location throughout parse tree

* lucet-idl witx: validator is filling out

* lucet-idl witx: typenames.witx reorg so names are defined before use

this restriction makes it possible to perform validation linearly,
which is in line with validation elsewhere in wasm

* lucet-idl witx: validation of all datatypes complete

* lucet-idl witx: finish validation

* lucet-idl witx: fix tests

* lucet-idl: fix tests

* lucet-idl witx: syntax derives Eq. mock the filesystem to test use

more tests required

* lucet-idl witx: some fixups, plenty of toplevel tests

* lucet-idl witx: add docs, refactor error name, resolve warnings

* lucet-idl: add test case to load wasi_unstable.witx

* lucet-idl: rustfmt

* lucet-idl witx: review comments

* lucet-idl witx: really fix the path thing this time

* lucet-idl witx: simplify paths one more time

not my favorite corner of std!!

* split lucet-idl and witx-frontend into two separate crates

* Exit dev shell scripts on first command exiting with non-zero return (bytecodealliance#298)

* Exit dev shell scripts on first command exiting with non-zero return

Fixes issue bytecodealliance#297

* Create stages for Travis

* Remove set -e from devenv scripts

* add version info to modules

version information is comprised of both the current crate version and
the current git commit hash (if available).

the current git commit hash is only used in release builds to avoid
too much furstration in typical development workflows using tools like
"git commit --amend" or "git rebase", or just making non-conflicting
spot changes to only one of lucetc or lucet-runtime

* review comments

* Fixes bytecodealliance#299 - WasiCtx should open dev/null read write (bytecodealliance#302)

* need to go through the ELF view to read the tables slice too

we were going through the mapper to read each table, but incorrectly
assumed the pointer in the artifact to look at didn't need translation
too

* add a CallStackBuilder abstraction to tease apart Context::init

also adds documentation on what the guest stack looks like when
initialized

* upgrade to cranelift 0.43.1, wasmparser 0.39.1

* cranelift 0.44.0

* lucet-benchmarks: use module-level constant to explicitly pin OptLevel

* lucet-validate: initial commit

* lucet-validate: build up moduletype from parser

* lucet-validate: print out type sig of all imports, start func sig

* lucet-validate: better api to witx crate is in a branch

* upgrade to cranelift 0.43.1, wasmparser 0.39.1

* lucet-validate: calculate module type of InterfaceFunc

* lucet-validate: use release of wasmparser 0.39.1

* lucet-validate: parse module type of wasi sdk test program

* lucet-validate: use wasi crate to validate against spec

* lucet-validate: cranelift 0.44.0

* lucet-wasi-sdk: correct signature of `main` in test c code

* lucet-validate: use witx spec from WASI repo as a submodule

* lucet-validate: wip... starting to validate sigs against witx

* lucet-validate: all lucet-wasi test cases now validate

* move the stack guard page between the stack and heap

where it was protecting against stack underflow into globals, which are
extremely unlikely. Far more likely, are stack overflows into the
instance's heap, where we accidentally caught those via heap guard space
instead. If an instance's `Limits` had the number of heap guard pages as
0, which is permissible, an instance with full utilization would then
map a page that had been acting as a stack guard page as read/write and
remove the guard-ness of that page.

This commit moves a page to be an outside-of-heap dedicated stack guard
page.

* adjust signal handler to not always be so stack-hungry

Because the Lucet signal handler is run on any arbitrary thread that
recives a signal, if the signal handler is installed anywhere in the
process, it may be run on a signal stack we didn't set up. This means we
are still bound to be careful about how much stack is present, which
will often be SIGSTKSZ (observed to be ~8kb on Debian-likes, at least).

`handle_signal` is the entrypoint for any signal in the process, so its
call frame, locals, etc, are active for any path through the signal
handler. For other paths through the signal handler that may involve
subsequent calls, and in debug builds of lucet-runtime, the body of the
`SignalBehavior::Default` arm would include several `UContext` local
variables, as well as a `State` local which itself contains another
`UContext`. x86_64 `UContext` are large, resulting in this match arm
being responsible for ~5kb of stack usage in `handle_signal`, that is
only _actually_ used in this match arm.

So, by moving the arm to a closure and immediately calling it, we force
Rust to defer that stack usage in debug builds to only be used when the
closure is called. In release builds, the closure is likely inlined, but
all intermediate copies of UContext _probably_ go away and the entire
issue is _probably_ avoided.

A subsequent commit will introduce tests around signal handler stack use
for these various paths.

* switch sigprocmask to pthread_sigmask

`pthread_sigmask` is the safe choice for a multithreaded environment. Per `sigprocmask(2)`:

> The use of sigprocmask() is unspecified in a multithreaded process; see pthread_sigmask(3).

* switch to nightly-2019-09-25, the most recent with all components

* lucet-validate: witx_moduletype is now part of witx crate

* makefile: add lucet-validate to tests

* lucet-validate: latest witx branch, where module types renamed core types

* use goblin 0.0.24 everywhere

* [lucet-idl] parameterize C test with `WASI_SDK` env var

* use lucet-validate in lucetc

* lucetc: use lucet-validate Validator

* changes to lucetc threaded throughout repo

and lucet-wasi-sdk validates that imports work according to witx!

* delete witx-frontend: subsumed by wasi repo

* upgrade to witx 0.3.0

* delete lucet-validate's wasi-sdk test: that testis now inside lucet-wasi-sdk

* lucet-validate: test harness uses new api

* Make WASI `poll` test more deterministic, but disable pending debugging (bytecodealliance#320)

- Removes a sleep that wasn't relevant for testing poll
- Timeout step waits on an in-process pipe rather than stdin, which should make its behavior more consistent across environments
- Temporarily ignores the test while we debug its behavior in certain CI environments

* address review comments

* lucet-validate: add readme

* [lucet-validate] ignore non-c/wat files rather than failing test

Ephemeral files left around by editors were causing a panic.

* package lucet-validate, including wasi unstable witx

* rename `lucet-analyze->lucet-objdump`; add debs for it and `lucetc`

* skip Lucet build in the container step on travis

* 0.2.0

* make bump-global-version.sh gnu sed-friendly

-i.previous should be usable on either linux sed or macos sed, but the
space leads to linux sed interpreting it as `-i`, which is a valid flag
on its own, and `.previous`, as a sed command.

* lucetc: fix --opt-level {none, speed, speed_and_size} (bytecodealliance#328)

* cargo publish requires versions on dev dependencies to publish

* lucet-wasi/README.md doesn't exist any more, don't try to install it (bytecodealliance#330)

* dockerfile: use wask-sdk 7

* lucet-wasi-sdk: fix test to reflect behavior of clang-9

* the runtime's module version check was restrictive enough to be annoying

In the case that the runtime is a debug or from-cargo version, there's
no commit hash, so a build from any lucetc that can include a commit
hash would immediately fail the version check. Since lucetc may be
installed by `cargo install`, which will build release binaries, this is
more likely than initially expected.

* grammar

* rustfmt

* configure CI using Github Actions

there are two workflows:
* workflows/main.yml defines the Test and Lint jobs, which run in
  parallel on each push and PR.
    * Test takes about 19 minutes to complete.
    * Lint takes about 45 seconds to run `make indent-check`.
* workflows/fuzz.yml defines the Fuzz job, which runs nightly at
  midnight. It only runs on the master branch, so we need to merge this
  PR to test it. The fuzz smoke-test passes, so I expect that this will
  work. We will probably have to add some system to report failures it
  discovers. We may want to run it more often, as well.

there is one disabled workflow:
* workflows/mac-ci.yml.disabled will run the CI, except for fuzzing,
  under Mac OS. It is disabled because tests fail on Mac OS at the
  moment. It should be enabled as part of fixing Mac OS.

there is one Action:
* actions/test specifies (in action.yml) a job that builds the
  Dockerfile at the repo root and runs `make test` in the container.

The Test job uses the Dockerfile in the repo, but not the devenv scripts
to run it. Actions only runs Docker via the This means the devenv
scripts themselves have no CI under Actions.

Once we have Mac OS support enabled again, there is a lot less reason to
maintain the complex set of devenv scripts, which do a lot of fancy and
difficult-to-understand stuff to allow using `lucetc`, `lucet-wasi` and
other executables on the Mac (or windows, I guess).

We could instead document the one-liner invocations of Docker to
reproduce our standard developer environment locally.

* bump lucet crates to 0.2.1

* Improve lucetc error messages (bytecodealliance#269)

* move off debug repr for error reporting

this still prints the underlying error, like a wabt error in the case of invalid
 webassembly text, but does so in a more readable way

* Timeouts (bytecodealliance#150)

* add termination mechanism and use it to implement timeouts

The termination model is described in the `execution` module-level docs,
but to reiterate: when in guest code, terminate with a SIGALRM sent
immediately, and when in host code, terminate by setting a flag (setting
`execution_domain` to `Terminated`) and checking that flag when exiting
a hostcall.

Hostcalls currently have their safety, with respect to termination,
implemented by running the whole hostcall in `uninterruptable`, which
effectively makes all hostcalls critical sections. In the future, we may
want to expose the idea of interruptable hostcalls, both to avoid overhead
of synchronizing `KillState` when entering an exiting hostcalls, and to
avoid delayed termination.

In the extreme, even though this adds a tool for timeout-style logic,
it is at worst a request to stop execution, which requires hostcalls to
complete in a reasonable amount of time to guarantee that timeouts
are always witnessed promptly.

* [lucetc] add very basic JSON formatting for errors

This is meant to be a starting point for more semantic error reporting, but in order to get there
we'll need to do a broader refactor of our error types

* [lucet-runtime] relax unnecessary restrictions on some functions

- The `Send + Sync` restriction on the downcast methods was leftover from before the `State` refactor lifted those restrictions from `YieldedVal`s
- The mutable borrow on `Instance::get_embed_ctx_mut()` is checked at runtime by the `RefCell`s inside the underlying map. This patch gives us fewer compile-time guarantees, but holding a mutable borrow on the entire instance just to get at an embedding context is way too strict

* tests run again

* fix the number of magic globals that wasi-sdk clang creates

* fix lucet-wasi-sdk test for wask-sdk-7

* allow KillState's `exit_guest_region` to be name-mangled

* Bump versions to 0.3.0 (bytecodealliance#340)

* 0.3.0

* lucetc error messages include the underlying error from wabt
* lucet-runtime supports timeouts
* lucetc can emit errors as json
* lucet-runtime removes unnecessary restrictions on some functions

wabt update is necessary to avoid linker issues with multiple wabt in use at once

* [lucet-runtime] add `is_yielded`, etc methods for instances

Previously a client would have to keep around the `RunResult` to know which state an instance is in.

* [lucetc] json errors report as an array to support multiple errors

We don't have the capability to produce multiple errors yet, so this only returns an array of a
single element. Returning multiple errors will require refactoring of our code as well as upstream
dependencies.

* add (currently-failing) test for moving yielded instances

* fix the sending-yielded-instances bug in horrendous fashion

* move guest data from the guest stack to Instance

* run the benchmarks in debug mode for `make test`

* [lucet-runtime-internals] refactor guest-specific context data

This ended up changing around the interface to initializing and swapping contexts in some important
ways. Notably, a pointer to the parent context that `lucet_context_bootstrap` swaps into when the
entrypoint function returns is no longer written into the guest stack, but rather is a field on
`Context`. This field gets updated in the `to` context when calling `Context::swap()`, so that we
can swap `to` a context from different `from` contexts without trying to always swap back to the
original context.

This also gets us ready to abstract away instance/timeout-specific data from the `Context` struct by
making the use of a backstop callback function explicit but optional.

* [lucet-benchmarks] slightly simplify a setup iter

* bump crate versions to 0.3.1

Keeping this as a minor semver because API changes were restricted to `lucet-runtime-internals` functions that are not exported via `lucet-runtime`.

* [lucetc] restore output for the underlying cause of `lucetc::Error`s

* update cranelift to current master (0.46.1)

This is in preparation for adding an interface to customize machine-dependent features in
cranelift's codegen

* delete .travis.yml

* [lucetc] add CPU feature flags for code generation

This introduces a couple new command-line arguments and the associated programmatic APIs for
selecting optional CPU features at compile time.

Previously, we used `cranelift-native` to automatically detect and use the features available on the
host, but this can cause problems when compiling for a different CPU, even if the target triple is
constant.

Now, we can enable or disable individual features using `--target-feature=+sse3,+avx,-popcnt`, for
example. For convenience, we can also specify `--target-cpu=haswell`, for example, which enables a
fixed set of individual features.

The default behavior of auto-detection and use of features remains the same for now.

* [lucetc] add `native` and `sandybridge` CPU profiles

This required some reworking of how `CpuFeatures` is represented.

* fix rustfmt

* crank up the fuzz

* Include CPU features in module data (bytecodealliance#351)

* write cpu features into modules and verify compatibility when loading

* build spectests to ward away evil bit gremlins

* bump to 0.4.0

* Use app_from_crate!() instead of hardcoding clap base parameters

* Add include_str!("Cargo.toml") to update macros if Cargo.toml changes

* bump to 0.4.1 and refine some dependencies

There were some problems when trying to publish the crates due to parity-wasm versions differing in
our crates and Cranelift's. I went through and updated and/or fixed the versions of some crates that
look like they could've caused trouble.

* use crates.io version of wasmonkey, and fix parity-wasm version

* Replace lucet-wasi with wasi-common + lucet wrappers

* lucet-wasi: use pch/working branch of wasi-common

* lucet-wasi: export our own __wasi_exitcode_t alias, use master upstream

* lucet-wasi: re-export wasi-common's error

* lucet-wasi: export start symbol

* use nix 0.15 everywhere

wasi-common depends on nix 0.15, so upgrade everywhere using 0.13.1

* latest wasi-common master

* pin wasi-common to f4ac1299b2001b575cfc99f5544a4a96355a6bff in cargo.toml

* Apply suggestions from code review

Thank you Adam for a helpful and thorough code review!

Co-Authored-By: Adam C. Foltzer <[email protected]>

* heap_u32_mut: require a mutable heap (bytecodealliance#359)

* replace lucet_runtime ensure_linked in lucet-wasi

fixes bytecodealliance#360 where the runtime's C
API was being discarded by the linker.

* Make it a Bytecode Alliance project

* read function names from custom name section

* minor grammar fix in runtime docs

* specify signature size for module data

* pr review: use exported names

* pr review: use SecondaryMap for names

* delete lucet-idl. superceded by witx

* remove lucet-idl from bump-global-version

* add `#[lucet_hostcall]` attr macro; deprecate `lucet_hostcalls!`

The attribute macro is much more flexible syntactically, and in particular allows hostcall
implementations to be properly `rustfmt`ed rather than being stuck in the invocation of a macro.

* bump to 0.4.2

Deprecating an existing API and adding a new one are both minor semver changes

* redefine `lucet_hostcalls!` in terms of `#[lucet_hostcall]`

Prompted by @jedisct1's comment, this will hopefully stave off bitrot as we deprecate
`lucet_hostcalls!`.

Also tweaks a comment in response to @data-pup's review.

Thank you both!

* remove some unnecessary cloning in `#[lucet_hostcall]`

* use `lucet_runtime` rather than `$crate` in `lucet_hostcalls!`

* upgrade cranelift, wasi-common, witx deps, and friends

* pull in latest Cranelift release 0.51
* upgrade to wasi-common 0.7, which fixes a lot of the WasiCtxBuilder
  stuff that required many extra unwraps
* pull in latest WASI repo. This moves the preview0 stuff to another
  directory. Biggest change here is that the lucetc deb, which includes
  the wasi witx files, has a slightly different file layout now.
* transitive upgrades to target-lexicon, faerie

* lucet-validate: correct deb paths

* CI testing: add cargo-deb packaging

* CI: run `make package` on a parallel runner, so CI doesnt get longer

* Use actions/checkout@v1 to support submodules checkout

This commit fixes the CI by setting `actions/checkout` at version `v1`.
The current master of `actions/checkout` now obsoleted the
`with: submodules: true` input.
See [actions/checkout/releases/tag/v2-beta] for more info.

[actions/checkout/releases/tag/v2-beta]: https://github.com/actions/checkout/releases/tag/v2-beta

* show unsupported global import in error message

* misc. instance lints

* replace empty constructor w/ default impl

* we can derive default

* keep `new()` but defined in terms of default

* Make yield methods take `&mut Vmctx`; update semver

This prevents resources like embedder contexts or heap views from living across yield points, which
is important for safety since the host can modify the data underlying those resources while the
instance is suspended.

Since this is a semver breaking change, this patch also updates the package versions.

* Fixes bytecodealliance#276 - Allow use of aligned heaps

* add blank rustfmt config file

* into_iter() -> iter() for arrays (bytecodealliance#382)

This was previously accepted by the compiler but is being phased out;
it will become a hard error in a future rustc release.

See rust-lang/rust#66145
>

* switch rustc version back to stable; add note about callback safety

* add `make test-full` target; make `test` more focused

The benchmark and fuzz targets in particular take quite a long time to run, and are usually not the
focus of development. They're still worth running in CI, though, to catch bitrot.

* clean up symbol data code

* move bindings tests into file in tests/

* fix test assertion

* ModuleDecls cleanup (bytecodealliance#391)

* Addresses two lint warnings about redundant field names, using the shorthand notation instead.
* Moves `*_name_for` helper functions nested inside of `declare_funcs` out of a loop body. This is just a small readability fix for our loop.

* add `--target` option to lucetc

Currently, `lucetc` assumes that the machine that it's compiling
for (the "target") is the same as the machine on which `lucetc` is
running ("the host").  While this is a reasonable assumption to make,
many future uses of `lucetc` may require these machines to be distinct.
Towards that end, let's introduce a `--target` option for specifying the
machine `lucetc` is supposed to be generating code for, and thread that
information through to all the places that require it.

* [lucet-runtime] export the `KillSwitch` type from the public API

It was already possible to use via the `Instance::kill_switch()` method, but couldn't be named in a
type signature or a type declaration.

* Fixes bytecodealliance#357:  Migrate from using the failure to thiserror + anyhow. (bytecodealliance#374)

Co-authored-by: Pat Hickey <[email protected]>

* use exact dependencies for crates within the repo

Most of our thinking about semver for this project is concerned with the more "public-facing" crates
like `lucetc`, `lucet-runtime`, and `lucet-wasi`, as opposed to "implementation detail" crates like
`lucet-module`, `lucet-runtime-internals`, etc. As the project matures, we are paying more attention
to semver for all of the crates, but this change makes the released sets of packages more
consistent.

* Fix bad merge in cpu_features.rs. ISA lookup must be invoked on target (bytecodealliance#399)

Co-authored-by: Pat Hickey <[email protected]>
Co-authored-by: Frank Denis <[email protected]>
Co-authored-by: awortman-fastly <[email protected]>
Co-authored-by: Shravan Narayan <[email protected]>
Co-authored-by: Gabor Greif <[email protected]>
Co-authored-by: Eric Kilmer <[email protected]>
Co-authored-by: Adam C. Foltzer <[email protected]>
Co-authored-by: Tyler McMullen <[email protected]>
Co-authored-by: Till Schneidereit <[email protected]>
Co-authored-by: data-pup <[email protected]>
Co-authored-by: Jakub Konka <[email protected]>
Co-authored-by: Nathan Froyd <[email protected]>
Co-authored-by: iximeow <[email protected]>
Co-authored-by: Tanya L. Crenshaw <[email protected]>

Author: 15 people

.cargo/config

@@ -1,2 +1,2 @@
 [build]
-rustflags = ["-C", "link-args=-Wl,--export-dynamic"]
+rustflags = ["-C", "link-args=-rdynamic"]

.dockerignore

@@ -1 +1 @@
-target
+*

.editorconfig

@@ -0,0 +1,12 @@
+root = true
+
+[*]
+charset = utf-8
+end_of_line = lf
+indent_size = 4
+indent_style = space
+insert_final_newline = true
+trim_trailing_whitespace = true
+
+[Makefile]
+indent_style = tab

.gitattributes

@@ -0,0 +1,8 @@
+# Auto detect text and normalize to LF on commit regardless of "core.autocrlf".
+# See: https://help.github.com/en/articles/dealing-with-line-endings#example
+* text=auto eol=lf
+
+# Denote all files that are truly binary and should not be modified.
+*.png binary
+*.jpg binary
+*.wasm binary

.github/actions/test/action.yml

@@ -0,0 +1,18 @@
+name: 'Test Lucet'
+description: 'run tests using standardized lucet development environment'
+inputs:
+  target:
+    description: "Makefile target to execute"
+    required: true
+runs:
+  using: 'docker'
+  image: '../../../Dockerfile'
+  # The Dockerfile does not specify an entrypoint.
+  entrypoint: "/bin/sh"
+  args:
+      # Next arg is a command for sh to execute.
+    - '-c'
+      # rustup expects $HOME to be set to /root during `docker run` because thats what
+      # it was set to during container creation. Actions clears $HOME so we set it here.
+      # The test target of the Makefile is our standard CI.
+    - 'export HOME=/root; make ${{ inputs.target }}'

.github/workflows/fuzz.yml

@@ -0,0 +1,50 @@
+name: Fuzz
+on:
+  schedule:
+    - cron: "0 0 * * *"
+
+jobs:
+  fuzz:
+    name: Fuzz lucet-wasi
+
+    runs-on: ubuntu-16.04
+    steps:
+    - uses: actions/checkout@v1
+      with:
+        submodules: true
+
+    - name: Install Rust (rustup)
+      run: rustup update
+
+    - name: Install wasi-sdk (ubuntu)
+      run: |
+        curl -sS -L -O https://github.com/CraneStation/wasi-sdk/releases/download/wasi-sdk-7/wasi-sdk_7.0_amd64.deb
+        sudo dpkg -i wasi-sdk_7.0_amd64.deb
+
+    - name: Install native clang, csmith tools
+      run: |
+        sudo apt-get install -y --no-install-recommends \
+          software-properties-common \
+          clang-6.0 \
+          gcc-multilib \
+          csmith \
+          libcsmith-dev \
+          creduce
+        sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-6.0 100
+
+    - name: Build Binaryen tools
+      run: |
+        curl -sS -L "https://github.com/WebAssembly/binaryen/archive/version_${BINARYEN_VERSION}.tar.gz" | tar xzf -
+        mkdir -p binaryen-build
+        cd binaryen-build && cmake "../binaryen-version_${BINARYEN_VERSION}" && make wasm-opt wasm-reduce
+        echo "##[add-path]$PWD/binaryen-build/bin"
+      env:
+        BINARYEN_VERSION: 86
+
+    - name: Test lucet-wasi-fuzz with known seed
+      run: make test-fuzz
+
+    - name: Fuzz
+      env:
+          FUZZ_NUM_TESTS: 100000
+      run: make fuzz

.github/workflows/mac-ci.yml.disabled

@@ -0,0 +1,29 @@
+name: Mac OS CI
+on: [push, pull_request]
+
+jobs:
+  test:
+    name: Test
+    runs-on: macos-latest
+    steps:
+    - uses: actions/checkout@master
+      with:
+        submodules: true
+
+    - name: Install Rust (macos)
+      run: |
+        curl https://sh.rustup.rs | sh -s -- -y
+        rustup update
+        echo "##[add-path]$HOME/.cargo/bin"
+
+    - name: Install wasi-sdk (macos)
+      run: |
+        curl -sS -L -O https://github.com/CraneStation/wasi-sdk/releases/download/wasi-sdk-7/wasi-sdk-7.0-macos.tar.gz
+        tar xf wasi-sdk-7.0-macos.tar.gz
+        sudo mv wasi-sdk-7.0/opt /opt
+
+    - name: Test Lucet
+      run: make test-except-fuzz
+
+    - name: Ensure testing did not change sources
+      run: git diff --exit-code

.github/workflows/main.yml

@@ -0,0 +1,51 @@
+name: CI
+on: [push, pull_request]
+
+jobs:
+  test:
+    name: Test
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v1
+      with:
+        submodules: true
+
+    # Testing uses the development environment Docker container.
+    # This action builds the container and executes the test suite inside it.
+    - uses: ./.github/actions/test
+      with:
+        target: test-full
+
+    - name: Ensure testing did not change sources
+      run: git diff --exit-code
+
+  package:
+    name: Package
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v1
+      with:
+        submodules: true
+
+    # Testing uses the development environment Docker container.
+    # This action builds the container and executes the test suite inside it.
+    - uses: ./.github/actions/test
+      with:
+        target: package
+
+    - name: Ensure testing did not change sources
+      run: git diff --exit-code
+
+
+  rustfmt:
+    name: Rustfmt
+    runs-on: ubuntu-16.04
+    steps:
+    - uses: actions/checkout@v1
+      with:
+        submodules: true
+    - name: Install Rust (rustup)
+      run: |
+        rustup update
+        rustup component add rustfmt
+    - run: make indent-check

.gitignore

@@ -1,4 +1,8 @@
 target/
 *.rs.bk
 *.pyc
-host
+
+# devenv-installed directory
+/host
+
+core.*

.gitmodules

@@ -1,15 +1,12 @@
 [submodule "cranelift"]
 	path = cranelift
 	url = https://github.com/cranestation/cranelift
-[submodule "faerie"]
-	path = faerie
-	url = https://github.com/m4b/faerie
 [submodule "lucet-spectest/spec"]
 	path = lucet-spectest/spec
 	url = https://github.com/webassembly/spec
-[submodule "pwasm-validation"]
-	path = pwasm-validation
-	url = https://github.com/pchickey/pwasm-validator
 [submodule "sightglass"]
 	path = sightglass
 	url = https://github.com/fastly/sightglass
+[submodule "wasi"]
+	path = wasi
+	url = https://github.com/webassembly/wasi

.rustfmt.toml

@@ -0,0 +1 @@
+# This file tells tools we use rustfmt. We use the default settings.

.travis.yml

@@ -1,18 +1,14 @@
 # Contributor Covenant Code of Conduct
 
+*Note*: this Code of Conduct pertains to individuals' behavior. Please also see the [Organizational Code of Conduct][OCoC].
+
 ## Our Pledge
 
-In the interest of fostering an open and welcoming environment, we as
-contributors and maintainers pledge to making participation in our project and
-our community a harassment-free experience for everyone, regardless of age, body
-size, disability, ethnicity, sex characteristics, gender identity and expression,
-level of experience, education, socio-economic status, nationality, personal
-appearance, race, religion, or sexual identity and orientation.
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
 
 ## Our Standards
 
-Examples of behavior that contributes to creating a positive environment
-include:
+Examples of behavior that contributes to creating a positive environment include:
 
 * Using welcoming and inclusive language
 * Being respectful of differing viewpoints and experiences
@@ -22,56 +18,32 @@ include:
 
 Examples of unacceptable behavior by participants include:
 
-* The use of sexualized language or imagery and unwelcome sexual attention or
-  advances
+* The use of sexualized language or imagery and unwelcome sexual attention or advances
 * Trolling, insulting/derogatory comments, and personal or political attacks
 * Public or private harassment
-* Publishing others' private information, such as a physical or electronic
-  address, without explicit permission
-* Other conduct which could reasonably be considered inappropriate in a
-  professional setting
+* Publishing others' private information, such as a physical or electronic address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
 
 ## Our Responsibilities
 
-Project maintainers are responsible for clarifying the standards of acceptable
-behavior and are expected to take appropriate and fair corrective action in
-response to any instances of unacceptable behavior.
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
 
-Project maintainers have the right and responsibility to remove, edit, or
-reject comments, commits, code, wiki edits, issues, and other contributions
-that are not aligned to this Code of Conduct, or to ban temporarily or
-permanently any contributor for other behaviors that they deem inappropriate,
-threatening, offensive, or harmful.
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
 
 ## Scope
 
-This Code of Conduct applies both within project spaces and in public spaces
-when an individual is representing the project or its community. Examples of
-representing a project or community include using an official project e-mail
-address, posting via an official social media account, or acting as an appointed
-representative at an online or offline event. Representation of a project may be
-further defined and clarified by project maintainers.
+This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
 
 ## Enforcement
 
-Instances of abusive, harassing, or otherwise unacceptable behavior may be
-reported by contacting the project team at [email protected], or [email protected]
-if the message is sensitive. All complaints will be reviewed and investigated
-and will result in a response that is deemed necessary and appropriate to the
-circumstances. The project team is obligated to maintain confidentiality with
-regard to the reporter of an incident.  Further details of specific enforcement
-policies may be posted separately.
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the Bytecode Alliance CoC team at [[email protected]](mailto:[email protected]). The CoC team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The CoC team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
 
-Project maintainers who do not follow or enforce the Code of Conduct in good
-faith may face temporary or permanent repercussions as determined by other
-members of the project's leadership.
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the Bytecode Alliance's leadership.
 
 ## Attribution
 
-This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
-available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
 
+[OCoC]: ORG_CODE_OF_CONDUCT.md
 [homepage]: https://www.contributor-covenant.org
-
-For answers to common questions about this code of conduct, see
-https://www.contributor-covenant.org/faq
+[version]: https://www.contributor-covenant.org/version/1/4/