Page MenuHomePhabricator

Document how to install Python modules in a tool's home directory/virtual environment
Closed, ResolvedPublicFeature

Description

Intention:
I'm building a script to upload into Internet Archive pdf files that need some previous painful manipulation (converting into tiffs, then cropped, splitted and sometines rotated); a good set of metadata (those needed for a good Book template) could be uploaded too.

Reproducible: Didn't try


Version: unspecified
Severity: enhancement

Details

Reference
bz61824

Event Timeline

bzimport raised the priority of this task from to Lowest.Nov 22 2014, 3:05 AM
bzimport added a project: Toolforge.
bzimport set Reference to bz61824.

I forgot to report the main information! Python package that allows Internet Archive management can be found here:

https://pypi.python.org/pypi/internetarchive

After migrating into eqiad, local installation of internetarchive gives an ImportError and doesn't run any more.

The new message into eqiad is:

from requests.auth import AuthBase
ImportError: cannot import name AuthBase

pip doesn't run in eqiad so we can't even try to fix the issue; careful global installation is needed.

Have you tried installing the Python package in your home directory? Is that possible?

I tried, but pip seems lacking into eqiad; internetarchive installation is not simple, i.e. I got often a (random!) ImportError message when running my test python script with jsub, while I never got it running the same by itsource prompt.

So, a global, careful installation of internetarchive module is a good idea; I think that a better idea could be, to integrate it into pywikibot, given the interest of IA for wikisource projects and possible automatic alignment of IA-Common-Wikidata-Wikisource metadata.

I see that the bug is now assigned to marc Pelletier: thanks for attention! :-)

virtualenv --no-site-packages env
echo "source $HOME/env/bin/activate" >> .bashrc
source $HOME/env/bin/activate
pip install internetarchive


This will work, if you're interested in using virtualenv.

I furiusly browsed python doc to see what virtualenv is; it doesn't seems something so alarming, I'll try. Thanks MZMcBride.

@MZMcBride: Great: env, pip, internetarchive, OpalLib.py running! Thanks again.

Re-prioritizing since this only needs documentation.

I don't know enough about python to be certain how reusable that bit is, but there are instructions on setting up pywikibot in a virtualenv at https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Developing#Setup_pywikibot_on_Labs_.28locally.29 .

How general is that information?

coren removed coren as the assignee of this task.Mar 25 2015, 8:12 PM

Leaving for a volunteer virtualenv guru.

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

There are some instructions on setting up a virtualenv related to a webservice at:

These could be combined and cleaned up with some additional information such as links to good external documentation on using virtualenv generally and a note the the www/python/venv is only magical because the uwsgi webservice script will use the directory if it is found.

srodlund subscribed.

Reviewing as part of initiative to improve Toolforge / Cloud Services Technical Documentation

srodlund raised the priority of this task from Lowest to Low.Aug 29 2018, 10:32 PM
Aklapper changed the subtype of this task from "Task" to "Feature Request".Feb 4 2022, 11:13 AM
dcaro claimed this task.
dcaro subscribed.

I have double checked the current tutorials and updated them with the latest instructions, I think that would be enough for now. Feel free to reopen with more details of if you want to do a whole round of checking/reworking them, but the original concern I think is addressed :) (10 years later!)