Skip to content

Python builder can give a hint to build on linux system, whenever a dependency fails because of its C (native) dependency #208

Open
@mndeveci

Description

@mndeveci

Steps to reproduce the issue:
Whenever sam build command fails for certain dependencies which requires to be build on linux environment (because of C dependencies of that package), output logs can contain a hint for user to build their functions on a linux environment (probably using --use-container flag)

Steps to reproduce the issue:

  1. Init a project (ex: HelloWorld) which uses python3.8
  2. Update requirements.txt file inside the function, and add aws-xray-sdk as a dependency
  3. Try to build function on Mac or Windows by running sam build

Observed result:
Build fails with a dependency error;

C:\agent2_work\142\s\Assets\events\Post>sam build --template "C:\agent2_work\142\s\Assets\events\Post/Assets_Events_POST-build.yaml" --region us-west-2 --debug
Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
'build' command is called
No Parameters detected in the template
2 resources found in the template
Found Serverless function with name='AssetsEventsPOST' and CodeUri='.'
No Parameters detected in the template
Building function 'AssetsEventsPOST'
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'
Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Running workflow 'PythonPipBuilder'
Running PythonPipBuilder:ResolveDependencies
calling pip download -r C:\agent2_work\142\s\Assets\events\Post\requirements.txt --dest C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i
Full dependency closure: {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), wrapt==1.12.1(sdist), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), future==0.18.2(sdist), importlib-metadata==2.0.0(wheel)}
initial compatible: {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), importlib-metadata==2.0.0(wheel)}
initial incompatible: {future==0.18.2(sdist), wrapt==1.12.1(sdist)}
Downloading missing wheels: {future==0.18.2(sdist), wrapt==1.12.1(sdist)}
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i future==0.18.2
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i wrapt==1.12.1
compatible wheels after second download pass: {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), importlib-metadata==2.0.0(wheel)}
Build missing wheels from sdists (C compiling True): {future==0.18.2(sdist), wrapt==1.12.1(sdist)}
calling pip wheel --no-deps --wheel-dir C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i\future-0.18.2.tar.gz
calling pip wheel --no-deps --wheel-dir C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i\wrapt-1.12.1.tar.gz
compatible after building wheels (no C compiling): {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), future==0.18.2(wheel), importlib-metadata==2.0.0(wheel)}
Build missing wheels from sdists (C compiling False): {wrapt==1.12.1(sdist)}
calling pip wheel --no-deps --wheel-dir C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i\wrapt-1.12.1.tar.gz
compatible after building wheels (C compiling): {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), future==0.18.2(wheel), importlib-metadata==2.0.0(wheel)}
Final compatible: {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), future==0.18.2(wheel), importlib-metadata==2.0.0(wheel)}
Final incompatible: {wrapt==1.12.1(wheel)}
Final missing wheels: {wrapt==1.12.1(sdist)}
PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\actions.py", line 42, in execute
requirements_path=self.manifest_path,
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\packager.py", line 137, in build_dependencies
self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\packager.py", line 201, in build_site_packages
raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {wrapt==1.12.1(sdist)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflow.py", line 269, in run
action.execute()
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\actions.py", line 45, in execute
raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: {wrapt==1.12.1(sdist)}

Build Failed
Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': 'us-west-2', 'commandName': 'sam build', 'duration': 20257, 'exitReason': 'WorkflowFailedError', 'exitCode': 1, 'requestId': 'c5551ae9-caea-4b93-9c48-d07fa2cbc8e3', 'installationId': 'b7024037-b875-42d8-8e62-2abac9896f08', 'sessionId': 'd38fda56-1b47-4cf4-8840-65a6966432b4', 'executionEnvironment': 'CLI', 'pyversion': '3.7.6', 'samcliVersion': '0.53.0'}}]}
Telemetry response: 200
Error: PythonPipBuilder:ResolveDependencies - {wrapt==1.12.1(sdist)}

Expected result:
If the build failure is because of C dependency, it may contain more information for user which will suggest them to run build on a linux environment, probably using --use-container

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
This only happens on non linux environments (Mac and Windows).

Related issues

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions