Skip to content

Test against ci precompiled binaries #595

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jneira opened this issue Nov 13, 2020 · 4 comments
Closed

Test against ci precompiled binaries #595

jneira opened this issue Nov 13, 2020 · 4 comments
Labels
CI Continuous integration type: enhancement New feature or request

Comments

@jneira
Copy link
Member

jneira commented Nov 13, 2020

  • With the lastest releases we can observe that several bugs affecting final users are related with the fact we are offering precompiled binaries and using them by default in the extension/ghcup
  • hls developers and ci are testing a version built from source in the same machine so the test suite is green for those bugs
  • imo we should run the test suite against the precompiled binaries before make a release, ideally in ci but, at least, locally
    //cc @bubba @alanz
@jneira jneira added CI Continuous integration type: testing labels Nov 13, 2020
@jneira
Copy link
Member Author

jneira commented Nov 13, 2020

The environment where run that test suite should be a clean one, different from the ci (for example putting ghc in a non standard location and so on)
Maybe circleci could be a good candidate for that environment.

@pepeiborra
Copy link
Collaborator

Big thumbs up, this is a must have

@jneira jneira added the type: enhancement New feature or request label Nov 13, 2020
@lukel97
Copy link
Collaborator

lukel97 commented Nov 19, 2020

I think I have a branch for this: master...bubba:static-binary-tests
Could be dusted off! It worked by downloading the build artefacts from the previous stage and then running the tests against that. The main change needed to the testsuite was the HLS_TEST_EXE env var so that we could point it to the binary. I might have time soon to look at it, but if anyone else wants to pick it up feel free to

@jneira
Copy link
Member Author

jneira commented Nov 19, 2021

From #2281 (comment) and #2281 (comment)

This could have helped to catch the bug about hs-boot files before releasing 1.5.0

In addition to @bubba suggestion another fuzzy test could be run hls in the cli for some real world projects. The obvious one is hls itlsef as it should be fast cause we have all dependencies in cache. We could choose some other projects known to be problematic.

//cc @pepeiborra

@jneira jneira self-assigned this Nov 19, 2021
@jneira jneira removed their assignment Feb 23, 2022
@jneira jneira closed this as completed Apr 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous integration type: enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants