aboutsummaryrefslogtreecommitdiff
path: root/programming/python/python_modules_primer.md
diff options
context:
space:
mode:
Diffstat (limited to 'programming/python/python_modules_primer.md')
-rw-r--r--programming/python/python_modules_primer.md269
1 files changed, 269 insertions, 0 deletions
diff --git a/programming/python/python_modules_primer.md b/programming/python/python_modules_primer.md
new file mode 100644
index 00000000..8932c19f
--- /dev/null
+++ b/programming/python/python_modules_primer.md
@@ -0,0 +1,269 @@
+# Python Modules Primer
+
+## Prerequisites
+
+These instructions assume a Linux environment.
+A macOS environment is similar, but not identical.
+A Windows environment is more different.
+
+## Previous knowledge
+
+### A refresher on the `PATH` variable
+
+If you execute the following command in your terminal:
+
+```
+$ echo hello
+```
+
+, the shell searches for the `echo` command in the directories listed in your `PATH` environment variable.
+You can display your `PATH` variable by running:
+
+```
+$ echo $PATH
+/home/user/.local/bin:/home/user/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin
+```
+
+The contents of the `PATH` variable depend on your particular environment.
+
+If you run the following command:
+
+```
+$ which echo
+/usr/bin/echo
+```
+
+The `which` command prints where the shell locates the `echo` command.
+
+### A refresher on shell scripts
+
+If you create a file named `foo.sh` with the following contents:
+
+```
+#!/bin/sh
+
+echo hello
+```
+
+You define a "shell script".
+The first line indicates that this shell script is executed by using the `/bin/sh` command.
+The rest of the file are commands to be executed by the shell command.
+These commands behave as if you typed them into your terminal, so if you execute this script, the command `echo hello` will be executed, printing `hello`.
+
+If you try to run `foo.sh` like you run the `echo` command, by typing its name, it does not work:
+
+```
+$ foo.sh
+bash: foo.sh: command not found...
+```
+
+, because the shell looks for the `foo.sh` in the directories listed in the `PATH` variable.
+Unless you created the `foo.sh` file in a directory like `/usr/bin`, the shell will not find the `foo.sh` command.
+
+A solution to this problem is to specify the path to the `foo.sh` file, instead of relying on the `PATH` variable.
+However, if you do this, you face a second problem.
+
+```
+$ ./foo.sh
+bash: ./foo.sh: Permission denied
+```
+
+This happens because only files with the executable permission can be executed in this way.
+To solve this, add the executable permission; then it works:
+
+```
+$ chmod +x foo.sh
+$ ./foo.sh
+hello
+```
+
+## The `import` statement in Python
+
+### Importing from the Python standard library
+
+Run the following commands by using the Python REPL:
+
+```
+$ python3
+Python 3.9.17 (main, Aug 9 2023, 00:00:00)
+[GCC 11.4.1 20230605 (Red Hat 11.4.1-2)] on linux
+Type "help", "copyright", "credits" or "license" for more information.
+>>> import datetime
+>>> datetime.datetime.now()
+datetime.datetime(2023, 9, 11, 21, 53, 16, 331236)
+```
+
+`import` works in a similar way to running a command in the shell.
+Python searches a number of directories looking for the `datetime` module.
+
+To see which directories are searched, run:
+
+```
+$ python3
+>>> import sys
+>>> sys.path
+['', '/usr/lib64/python39.zip', '/usr/lib64/python3.9', '/usr/lib64/python3.9/lib-dynload', '/home/alex/.local/lib/python3.9/site-packages', '/usr/lib64/python3.9/site-packages', '/usr/lib/python3.9/site-packages']
+```
+
+`sys.path` is a list of the directories that the `import` command searches.
+The contents of `sys.path` depend on your operating system and Python installation method.
+
+In my system, the `/usr/lib64/python3.9` directory contains the `datetime.py` module.
+
+```
+$ head /usr/lib64/python3.9/datetime.py
+"""Concrete date/time and related types.
+
+See http://www.iana.org/time-zones/repository/tz-link.html for
+time zone and DST data sources.
+"""
+
+__all__ = ("date", "datetime", "time", "timedelta", "timezone", "tzinfo",
+ "MINYEAR", "MAXYEAR")
+...
+```
+
+`/usr/lib64/python3.9` contains the modules in [the Python standard library](https://docs.python.org/3/library/).
+
+### Importing your Python files
+
+If you create a file with the `a.py` name:
+
+```
+def f():
+ return 2
+```
+
+, and another with the `b.py` name:
+
+```
+import a
+
+print(a.f())
+```
+
+, then:
+
+```
+$ python b.py
+2
+```
+
+
+This works, because `sys.path` contains `''`, which means "the current directory".
+
+(`sys.path` is very similar to the `PATH` variable. However, `sys.path` contains the current directory by default, whereas `PATH` does not.)
+
+When `import a` is executed, then Python searches the directories in `sys.path` for an `a.py` file; it is found when checking the `''` path.
+When `import datetime` is executed, Python searches in the current directory (because `''` comes first in the path), doesn't find it, but then finds it in the following `/usr/lib64/python3.9` directory.
+Python iterates over the `sys.path` directories, and loads the *first* matching file.
+
+## Installing libraries
+
+When writing Python software, sometimes it is enough with the modules included in the standard library.
+However, frequently you want to use other libraries.
+To use Python libraries, you must install them using the `pip` program.
+
+The `pip` program is not part of the `python3` package in some Linux distributions, and comes from the `python3-pip` package.
+
+The `pip` program can download libraries from https://pypi.org/ , the Python package index, and install them.
+`pip` installs libraries to a "Python environment".
+
+Old versions of `pip` defaulted to installing libraries to the "system" Python environment.
+In a Linux system, the system Python environment is located in a directory such as `/usr/lib64/python3.9`.
+By default, normal Linux users cannot write to `/usr`, so installing a package would fail.
+
+Modern versions of `pip` detect that they cannot write to the "system" Python environment, and then redirect the install to the "user" Python environment.
+The "user" Python environment is in a directory such as `~/.local/lib/python3.9`.
+
+You could use a command such as `sudo pip install` to grant `pip` the privileges required to write to `/usr`.
+However, this can make a Linux system unusable.
+Most Linux systems use software that uses the "system" Python environment.
+Altering the "system" Python environment can break such software.
+Do not run `sudo pip install` with root privileges unless you know why you need this.
+
+If you use a modern `pip` (or use the `--user` option), you can install libraries to the "user" Python environment.
+However, this is problematic because a Python environment can only contain a single version of a Python library.
+If you have two different Python programs that different versions of the same library, then these two programs cannot coexist in the "user" Python environment.
+
+In general, Python virtual environments are used to address this problem.
+
+## Creating Python virtual environments
+
+If you run:
+
+```
+$ python3 -m venv <some path>
+```
+
+This will create a directory with the path you specify, with the following contents:
+
+```
+<some path>
+├── bin
+│   ├── activate
+│   ├── pip
+│   ├── python
+├── include
+├── lib
+│   └── python3.9
+```
+
+The `python` and `pip` commands are copies of the same commands from the "system" Python environment.
+
+But these commands work differently from the "system" Python environment commands:
+
+```
+$ <some path>/bin/python
+>>> import sys
+>>> sys.path
+['', '/usr/lib64/python39.zip', '/usr/lib64/python3.9', '/usr/lib64/python3.9/lib-dynload', '<some path>/lib64/python3.9/site-packages', '<some path>/lib/python3.9/site-packages']
+```
+
+`sys.path` uses the `lib` directories in the virtual environment.
+
+When you use the `pip` program from the virtual environment, it installs the libraries to the virtual environment.
+
+You can create as many virtual environments as you need, and you can install different versions of libraries to each virtual environment.
+
+## Activating Python environments
+
+You can run the `python` and `pip` commands by specifying the full path, like we did when executing the `foo.sh` command earlier.
+
+By default, if you run `python`, the shell will invoke the `python` command from the "system" Python environment because it is in a directory included in the `PATH` variable.
+If you specify the full path, you override this.
+
+To save typing, the `bin` directory of a virtual environment contains an `activate` file.
+The `activate` file is a "special" shell script that must be invoked in one of the following two ways:
+
+```
+$ source <some path>/bin/activate
+```
+
+```
+$ . <some path>/bin/activate
+```
+
+`source` and `.` are synonyms.
+They are special shell commands that are needed for the `activate` command to work correctly.
+
+`activate` alters your path, so that the `bin` directory in your virtual environment comes first in your path.
+
+```
+$ echo $PATH
+/home/user/.local/bin:/home/user/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin
+$ . <some path>/bin/activate
+(some path) $ echo $PATH
+<some path>/bin:/home/user/.local/bin:/home/user/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin
+```
+
+, and thus if you run `python`, `<some path>/bin/python` will be executed instead of `/usr/bin/python`.
+
+Besides changing your prompt to indicate the virtual environment is activated, `activate` only alters your `PATH`.
+You can never use `activate` if you always specify the path to the virtual environment commands.
+
+## Further reading
+
+* [Some brief notes about Python dependency management](dependency_handling.md) continues this explanation, introducing the need for packaging tools.
+* [Installing Python Modules](https://docs.python.org/3/installing/index.html), from the official Python documentation, describes the `pip` program in more depth.
+* [`venv` - Creation of virtual environments](https://docs.python.org/3/library/venv.html), from the official Python documentation, describes virtual environments in more depth.