Installation of Conan fails on macOS Big Sur (seems like a dependency failure) on an Intel based iMac

[Fallen-Zen]

I have a clean install of macOS Big Sur (11.0.1) and I cannot install Conan which is really required for my work. Is there any workaround to have it working?
What I have:

fallen@Fallens-iMac ~ % pip3 --version
pip 20.2.4 from /Library/Python/3.8/site-packages/pip (python 3.8)
fallen@Fallens-iMac ~ % python3 --version
Python 3.8.2
fallen@Fallens-iMac ~ % pip3 install conan
Defaulting to user installation because normal site-packages is not writeable
Collecting conan
  Using cached conan-1.31.2.tar.gz (631 kB)
Collecting PyJWT<2.0.0,>=1.4.0
  Using cached PyJWT-1.7.1-py2.py3-none-any.whl (18 kB)
Collecting requests<3.0.0,>=2.8.1
  Using cached requests-2.25.0-py2.py3-none-any.whl (61 kB)
Collecting urllib3!=1.25.4,!=1.25.5,<1.26
  Using cached urllib3-1.25.11-py2.py3-none-any.whl (127 kB)
Collecting colorama<0.5.0,>=0.3.3
  Using cached colorama-0.4.4-py2.py3-none-any.whl (16 kB)
Collecting PyYAML<6.0,>=3.11
  Using cached PyYAML-5.3.1.tar.gz (269 kB)
Collecting patch-ng<1.18,>=1.17.4
  Using cached patch-ng-1.17.4.tar.gz (17 kB)
Collecting fasteners>=0.14.1
  Using cached fasteners-0.15-py2.py3-none-any.whl (23 kB)
Requirement already satisfied: six<=1.15.0,>=1.10.0 in /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages (from conan) (1.15.0)
Collecting node-semver==0.6.1
  Using cached node_semver-0.6.1-py3-none-any.whl (10 kB)
Collecting distro<=1.5.0,>=1.0.2
  Using cached distro-1.5.0-py2.py3-none-any.whl (18 kB)
Collecting future<0.19.0,>=0.16.0
  Using cached future-0.18.2.tar.gz (829 kB)
Collecting pygments<3.0,>=2.0
  Using cached Pygments-2.7.2-py3-none-any.whl (948 kB)
Collecting deprecation<2.1,>=2.0
  Using cached deprecation-2.0.7-py2.py3-none-any.whl (11 kB)
Collecting tqdm<5,>=4.28.1
  Using cached tqdm-4.51.0-py2.py3-none-any.whl (70 kB)
Collecting Jinja2<3,>=2.9
  Using cached Jinja2-2.11.2-py2.py3-none-any.whl (125 kB)
Collecting python-dateutil<3,>=2.7.0
  Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting bottle<0.13,>=0.12.8
  Using cached bottle-0.12.19-py3-none-any.whl (89 kB)
Collecting pluginbase>=0.5
  Using cached pluginbase-1.0.0.tar.gz (41 kB)
Collecting idna==2.6
  Using cached idna-2.6-py2.py3-none-any.whl (56 kB)
Collecting cryptography<4,>=3.2
  Using cached cryptography-3.2.1.tar.gz (540 kB)
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: /Applications/Xcode.app/Contents/Developer/usr/bin/python3 /Library/Python/3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-build-env-5n4984tx/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.8,!=1.11.3; platform_python_implementation != '"'"'PyPy'"'"''
       cwd: None
  Complete output (106 lines):
  Collecting setuptools>=40.6.0
    Using cached setuptools-50.3.2-py3-none-any.whl (785 kB)
  Collecting wheel
    Using cached wheel-0.35.1-py2.py3-none-any.whl (33 kB)
  Collecting cffi!=1.11.3,>=1.8
    Using cached cffi-1.14.3.tar.gz (470 kB)
  Collecting pycparser
    Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
  Building wheels for collected packages: cffi
    Building wheel for cffi (setup.py): started
    Building wheel for cffi (setup.py): finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /Applications/Xcode.app/Contents/Developer/usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-install-nmrt27bi/cffi/setup.py'"'"'; __file__='"'"'/private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-install-nmrt27bi/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-wheel-ujtq2ccj
         cwd: /private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-install-nmrt27bi/cffi/
    Complete output (39 lines):
    running bdist_wheel
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.14.6-x86_64-3.8
    creating build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/backend_ctypes.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/error.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/setuptools_ext.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/__init__.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/cffi_opcode.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/vengine_gen.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/pkgconfig.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/model.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/ffiplatform.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/api.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/vengine_cpy.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/commontypes.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/lock.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/recompiler.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/cparser.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/verifier.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/_cffi_include.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/parse_c_type.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/_embedding.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/_cffi_errors.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    running build_ext
    building '_cffi_backend' extension
    creating build/temp.macosx-10.14.6-x86_64-3.8
    creating build/temp.macosx-10.14.6-x86_64-3.8/c
    clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/ffi -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c c/_cffi_backend.c -o build/temp.macosx-10.14.6-x86_64-3.8/c/_cffi_backend.o
    c/_cffi_backend.c:5854:2: error: Apple Arm64 ABI requires ffi_prep_cif_var
    #error Apple Arm64 ABI requires ffi_prep_cif_var
     ^
    c/_cffi_backend.c:6304:9: error: implicit declaration of function 'ffi_prep_closure' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        if (ffi_prep_closure(closure, &cif_descr->cif,
            ^
    2 errors generated.
    error: command 'clang' failed with exit status 1
    ----------------------------------------
    ERROR: Failed building wheel for cffi
    Running setup.py clean for cffi
  Failed to build cffi
  Installing collected packages: setuptools, wheel, pycparser, cffi
      Running setup.py install for cffi: started
      Running setup.py install for cffi: finished with status 'error'
      ERROR: Command errored out with exit status 1:
       command: /Applications/Xcode.app/Contents/Developer/usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-install-nmrt27bi/cffi/setup.py'"'"'; __file__='"'"'/private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-install-nmrt27bi/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-record-to1z5lfb/install-record.txt --single-version-externally-managed --prefix /private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-build-env-5n4984tx/overlay --compile --install-headers /private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-build-env-5n4984tx/overlay/include/python3.8/cffi
           cwd: /private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-install-nmrt27bi/cffi/
      Complete output (39 lines):
      running install
      running build
      running build_py
      creating build
      creating build/lib.macosx-10.14.6-x86_64-3.8
      creating build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/backend_ctypes.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/error.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/setuptools_ext.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/__init__.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/cffi_opcode.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/vengine_gen.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/pkgconfig.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/model.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/ffiplatform.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/api.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/vengine_cpy.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/commontypes.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/lock.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/recompiler.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/cparser.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/verifier.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/_cffi_include.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/parse_c_type.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/_embedding.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      copying cffi/_cffi_errors.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
      running build_ext
      building '_cffi_backend' extension
      creating build/temp.macosx-10.14.6-x86_64-3.8
      creating build/temp.macosx-10.14.6-x86_64-3.8/c
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/ffi -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c c/_cffi_backend.c -o build/temp.macosx-10.14.6-x86_64-3.8/c/_cffi_backend.o
      c/_cffi_backend.c:5854:2: error: Apple Arm64 ABI requires ffi_prep_cif_var
      #error Apple Arm64 ABI requires ffi_prep_cif_var
       ^
      c/_cffi_backend.c:6304:9: error: implicit declaration of function 'ffi_prep_closure' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
          if (ffi_prep_closure(closure, &cif_descr->cif,
              ^
      2 errors generated.
      error: command 'clang' failed with exit status 1
      ----------------------------------------
  ERROR: Command errored out with exit status 1: /Applications/Xcode.app/Contents/Developer/usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-install-nmrt27bi/cffi/setup.py'"'"'; __file__='"'"'/private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-install-nmrt27bi/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-record-to1z5lfb/install-record.txt --single-version-externally-managed --prefix /private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-build-env-5n4984tx/overlay --compile --install-headers /private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-build-env-5n4984tx/overlay/include/python3.8/cffi Check the logs for full command output.
  ----------------------------------------
ERROR: Command errored out with exit status 1: /Applications/Xcode.app/Contents/Developer/usr/bin/python3 /Library/Python/3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/c8/r7_f7w7x2psg2vdmv437fl6r0000gn/T/pip-build-env-5n4984tx/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.8,!=1.11.3; platform_python_implementation != '"'"'PyPy'"'"'' Check the logs for full command output.

[memsharded]

I see you have filed pyca/cryptography#5568 already, and they are deferring to cffi project.

I don't know how this could be avoided, but having the cffi fixed. Have you tried removing the "cryptograpy" dependency? It was added in Conan requirements.txt, because the https requests were failing otherwise. maybe there is a remote chance that it will work without it depending on your machine and your company servers security configuration.

[memsharded]

Could please someone with a Mac (@jgsogo @SSE4 @czoido ) play a little bit, remove the cryptography dependency and see what happens?

[czoido]

Hi @Fallen-Zen,
I have been doing some tests and removing cryptography seems to work for most of the normal use of Conan, so maybe it's an option for you to try that. Also, it seems like there are some possible workarounds: pypa/pip#9138 pyca/cryptography#5568 (comment)
Anyway I'll continue to make some tests to check if it's safe to remove that dependency.

[czoido]

Doing some research about cryptography, pyopenssl and idna and why are we adding them, I think that maybe we could consider removing those dependencies:

• idna was introduced because installing via requests was installing an incorrect version but looks like the issue is not happening any more. I think this one can be removed without risk and the installed one will be the one required by requests.
• pyopenssl and cryptography were introduced just for OSX here: memsharded@10d69ff because of Downloads from certain SSL domains fails in OSX #309

It seems like cryptography and pyopenssl will only be needed for TLS with SNI-support for Python 2. From requests versions > 2.24.0 loading this module will only be done if the Python ssl module is not present or if it does not have SNI support. For previous versions it will not raise if those libraries are not installed and just pass the exception.

Probably, removing cryptography and pyopenssl will only fail in case of an old Python 2 installation (the entire Python 3.4 ssl module was ported to 2.7.9) or ancient ssl versions installed.

[memsharded]

We are going to remove those dependencies in next 1.32, this has been merged already in #8075. This might solve the issue, but please report @Fallen-Zen otherwise. Thanks!