WIP: Copy the contents of requirements_all_ds.txt into requirements.txt

[justinclift]

As per discussion to try and get our build to include all data sources:

#59 (comment)

Note that I have no idea if this will actually pass our CI as is, so this is more experimental than anything for now.

Also update httplib2 to version 0.15.0, as 0.14.0 is apparently too old (according to my local pip install).

When trying this out from the command line on my local Ubuntu 20.04 system, pip installs most of them but throws these errors:

ERROR: boto3 1.14.63 has requirement botocore<1.18.0,>=1.17.63, but you'll have botocore 1.13.50 which is incompatible.
ERROR: dql 0.6.2 has requirement botocore>=1.17.55, but you'll have botocore 1.13.50 which is incompatible.
ERROR: dql 0.6.2 has requirement pyparsing==2.1.4, but you'll have pyparsing 2.3.0 which is incompatible.
ERROR: snowflake-connector-python 3.0.2 has requirement charset-normalizer<3,>=2, but you'll have charset-normalizer 3.1.0 which is incompatible.
ERROR: snowflake-connector-python 3.0.2 has requirement cryptography<41.0.0,>=3.1.0, but you'll have cryptography 2.8 which is incompatible.
ERROR: firebolt-sdk 0.16.2 has requirement cryptography>=3.4.0, but you'll have cryptography 2.8 which is incompatible.
ERROR: firebolt-sdk 0.16.2 has requirement python-dateutil>=2.8.2, but you'll have python-dateutil 2.8.0 which is incompatible.
ERROR: firebolt-sdk 0.16.2 has requirement sqlparse>=0.4.2, but you'll have sqlparse 0.3.0 which is incompatible.
ERROR: pinotdb 0.5.0 has requirement httpx<0.24.0,>=0.23.0, but you'll have httpx 0.24.0 which is incompatible.

We may need to drop those data sources in the short term to get things to a fully working state, then come back to them at some later point.

[justinclift]

Heh Heh Heh. It threw the error on the dql libray. I'll update this PR with the downgraded dql library version, so we can see what happens. 😄

[justinclift]

@Avey777 It looks like downgrading dql isn't a complete fix for the full build (with all data sources), as there's one test that seems to be persistently failing anyway.

We might need to do some "bisecting" type of thing, and try with (for example) half of the libraries excluded (repeated a few times) to try and narrow down which library or libraries are causing the problem.

I'll take a better look at this later on.

[Avey777]

The user requested cryptography==2.8
    pyopenssl 19.0.0 depends on cryptography>=2.3
    pysaml2 6.5.2 depends on cryptography>=1.4
    authlib 0.15.5 depends on cryptography
    snowflake-connector-python 3.0.2 depends on cryptography<41.0.0 and >=3.1.0

Maybe we can disable snowflake-connector-python 3.0.2 and then give it aty.

[Avey777]

I has been try it
cryptography from 2.8 to 3.1.0,<41.0.0
This won't work.

[justinclift]

Maybe we can disable snowflake-connector-python 3.0.2 and then give it aty.

Cool. That one was removed in the last commit that I pushed. I still haven't managed to get any sleep since yesterday, so I'm not really awake enough at the moment to do any serious thinking. So, I'll probably pick this up again tomorrow. 😄

Feel free to push changes to this branch to experiment with things if you want.