Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 11 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,22 @@ concurrency:
permissions: {}

jobs:
pre-commit:
name: pre-commit
prek:
name: prek
# External pull requests should be checked, but not our own internal pull
# requests again, as these are already checked by the push on the branch.
# Without this if condition, the checks would be performed twice, as
# internal pull requests correspond to both the push and pull_request
# events.
if:
github.event_name == 'push' ||
github.event.pull_request.head.repo.full_name != github.repository
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.cache/pre-commit
key: pre-commit|${{ env.pythonLocation }}|${{ hashFiles('.pre-commit-config.yaml') }}
- uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1
- uses: j178/prek-action@e98a699c41eb69ab013a45817a0406469a748f8d # v2.0.5

docs:
name: Build docs and check links
Expand Down
16 changes: 8 additions & 8 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ ci:

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: 3e8a8703264a2f4a69428a0aa4dcb512790b2c8c # v6.0.0
rev: v6.0.0 # v6.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
Expand All @@ -15,38 +15,38 @@ repos:
- id: check-added-large-files
args: ['--maxkb=1024']
- repo: https://github.com/tox-dev/pyproject-fmt
rev: cf78bb920f358be6dea099e6b1fd6f67bccb0955 # v2.25.2
rev: v2.26.0 # v2.25.2
hooks:
- id: pyproject-fmt
- repo: https://github.com/abravalheri/validate-pyproject
rev: 4b2e70d08cb2ccd26d1fba73588de41c7a5d50b7 # v0.25
rev: v0.25 # v0.25
hooks:
- id: validate-pyproject
- repo: https://github.com/sphinx-contrib/sphinx-lint
rev: c883505f64b59c3c5c9375191e4ad9f98e727ccd # v1.0.2
rev: v1.0.2 # v1.0.2
hooks:
- id: sphinx-lint
types: [rst]
- repo: https://github.com/pycqa/isort
rev: 7f321d375f31a2f03833bc9a548e8783f2b5c420 # 9.0.0b1
rev: 9.0.0b1 # 9.0.0b1
hooks:
- id: isort
additional_dependencies: ["toml"]
entry: isort --profile=black
name: isort (python)
- repo: https://github.com/psf/black-pre-commit-mirror
rev: 4160603246a6b365d4a2af661c6d71b0a0f50478 # 26.5.1
rev: 26.5.1 # 26.5.1
hooks:
- id: black
- repo: https://github.com/adamchainz/blacken-docs
rev: fda77690955e9b63c6687d8806bafd56a526e45f # 1.20.0
rev: 1.20.0 # 1.20.0
hooks:
- id: blacken-docs
args: [--line-length=79]
additional_dependencies:
- black
- repo: https://github.com/codespell-project/codespell
rev: 2ccb47ff45ad361a21071a7eedda4c37e6ae8c5a # v2.4.2
rev: v2.4.3 # v2.4.2
hooks:
- id: codespell
- repo: local
Expand Down
13 changes: 12 additions & 1 deletion CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,18 @@ emergencies when we need to start branches for older versions.
`Unreleased <https://github.com/veit/python-basics-tutorial-de/compare/25.1.0...HEAD>`_
---------------------------------------------------------------------------------------

Added
~~~~~

* 📝 Add pytest-leak-finder

Changed
~~~~~~~

* 👷🔧📝 Switch to prek

* Remove pre-commit

`25.1.0 <https://github.com/veit/python-basics-tutorial/compare/24.3.0...25.1.0>`_
----------------------------------------------------------------------------------

Expand All @@ -35,7 +47,6 @@ Added
Changed
~~~~~~~


* 📝 Update cookiecutter templates

* Add badges
Expand Down
73 changes: 50 additions & 23 deletions docs/document/shot-scraper.rst
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
shot-scraper
============

`shot-scraper <https://simonwillison.net/2022/Mar/10/shot-scraper/>`_ is a tool
to automate the process of updating screenshots.
`shot-scraper <https://shot-scraper.datasette.io/en/stable/>`_ is a tool to
automate the process of updating screenshots.

Installation
------------
Expand All @@ -15,50 +15,77 @@ Installation
.. note::
The second line installs the required browser.

.. seealso::
`shot-scraper Installation
<https://shot-scraper.datasette.io/en/stable/installation.html>`_

Use
---

shot-scraper can be used in two ways
shot-scraper can be used in several ways

#. …for single screenshots on the command line:

.. code-block:: console

$ shot-scraper https://jupyter-tutorial.readthedocs.io/de/latest/clean-prep/index.html -o ~/Downloads/clean-prep.png
$ uv run shot-scraper https://jupyter-tutorial.readthedocs.io/de/latest/clean-prep/index.html -o ~/Downloads/clean-prep.png

…or with additional options, e.g. for JavaScript and CSS selectors:
…or with additional options, for example for JavaScript and CSS selectors:

.. code-block::
.. code-block:: console

$ shot-scraper https://jupyter-tutorial.readthedocs.io/de/latest/clean-prep/index.html -s '#overview' -o ~/Downloads/clean-prep.png
$ uv run shot-scraper https://jupyter-tutorial.readthedocs.io/de/latest/clean-prep/index.html -s '#overview' -o ~/Downloads/clean-prep.png

#. …for a set of screenshots configured in a YAML file:

.. code-block:: yaml

- url: https://jupyter-tutorial.readthedocs.io/de/latest/clean-prep/index.html
output: ~/Downloads/clean-prep.png
- url: https://www.example.org/
width: 736
quality: 40
output: example.jpg
- url: https://jupyter-tutorial.readthedocs.io/de/latest/clean-prep/index.html
output: ~/Downloads/clean-prep.png
- url: https://www.example.org/
width: 736
quality: 40
output: example.jpg

Afterwards ``shot-scraper multi`` can be used, for example:

.. code-block:: console

$ shot-scraper multi shots.yaml
Screenshot of 'https://jupyter-tutorial.readthedocs.io/de/latest/clean-prep/index.html' written to '~(Downloads/clean-prep.png'
Screenshot of 'https://www.example.org/' written to 'example.jpg'
$ shot-scraper multi shots.yaml
Screenshot of 'https://jupyter-tutorial.readthedocs.io/de/latest/clean-prep/index.html' written to '~(Downloads/clean-prep.png'
Screenshot of 'https://www.example.org/' written to 'example.jpg'

.. seealso::
* `Taking multiple screenshots
<https://shot-scraper.datasette.io/en/stable/multi.html>`_

#. …for videos:

The ``shot-scraper video`` command captures a WebM video based on a
:doc:`Python4DataScience:data-processing/serialisation-formats/yaml/index`
storyboard. Storyboards describe the video as a sequence of scenes. Each
scene can open a page, wait for content, perform actions and pause between
steps, for example:

.. literalinclude:: storyboard.yml
:caption: storyboard.yml
:language: yaml

Then run the following command:

.. code-block:: console

$ uv run shot-scraper video storyboard.yml

This opens ``url``, records the scenes and saves the video as
:file:`demo.webm`.

As long as `FFmpeg <https://www.ffmpeg.org/>`_ is installed, you can use the
``--mp4`` option to convert the recorded WebM video to MP4 as well.

.. seealso::
* In the `README.md
<https://github.com/simonw/shot-scraper/blob/main/README.md>`_ file you
will find a complete overview of the possible options.
* In the shot-scraper-demo repository you will find a much more
comprehensive `shots.yaml
<https://github.com/simonw/shot-scraper-demo/blob/main/.github/workflows/shots.yml>`_
file.
* `Recording videos
<https://shot-scraper.datasette.io/en/stable/video.html>`_

GitHub Actions
--------------
Expand Down
16 changes: 8 additions & 8 deletions docs/document/sphinx/test.rst
Original file line number Diff line number Diff line change
Expand Up @@ -144,8 +144,8 @@ reST formatting
Whether the :doc:`Sphinx <start>` documentation is written in valid :doc:`rest`
format can be checked with `sphinx-lint
<https://pypi.org/project/sphinx-lint/>`_. We usually include this in our
:doc:`pre-commit
<Python4DataScience:productive/git/advanced/hooks/pre-commit>` configuration:
:doc:`prek <Python4DataScience:productive/git/advanced/hooks/prek>`
configuration:

.. code-block:: yaml
:caption: .pre-commit-config.yaml
Expand Down Expand Up @@ -194,10 +194,10 @@ Code formatting
The formatting of code blocks can be checked with `blacken-docs
<https://github.com/adamchainz/blacken-docs>`_, which uses
:doc:`Python4DataScience:productive/qa/black`. We usually integrate the library
via the :doc:`pre-commit
<Python4DataScience:productive/git/advanced/hooks/pre-commit>` framework:
via :doc:`prek <Python4DataScience:productive/git/advanced/hooks/prek>`:

.. code-block:: yaml
:caption: .pre-commit-config.yaml

- repo: https://github.com/adamchainz/blacken-docs
rev: 7ae9389351f4090e3993de28015a05a18ca6b8a7 # v1.12.1
Expand Down Expand Up @@ -310,8 +310,8 @@ You should then update your :ref:`.gitignore <gitignore>` file if necessary:

styles/*

You can configure Vale for the :doc:`pre-commit
<Python4DataScience:productive/git/advanced/hooks/pre-commit>` framework with:
You can configure Vale for :doc:`prek
<Python4DataScience:productive/git/advanced/hooks/prek>` with:

.. code-block:: yaml
:caption: .pre-commit-config.yaml
Expand Down Expand Up @@ -381,8 +381,8 @@ example with
commands =
interrogate --quiet --fail-under 95 src tests

You can also use ``interrogate`` with :doc:`pre-commit
<Python4DataScience:productive/git/advanced/hooks/pre-commit>`:
You can also use ``interrogate`` with :doc:`prek
<Python4DataScience:productive/git/advanced/hooks/prek>`:

.. code-block:: yaml
:caption: .pre-commit-config.yaml
Expand Down
34 changes: 34 additions & 0 deletions docs/document/storyboard.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
output: demo.webm
url: https://python-basics-tutorial.readthedocs.io/en/latest/

viewport:
width: 1280
height: 720

cursor: true
wait_for: "text=Python für Data Science"

scenes:
- name: Home
do:
- pause: 1

- name: Introduction
do:
- click: ".sidebar-tree a[href='intro.html']"
- wait_for: 'h1:has-text("Introduction")'
- screenshot: intro.png
- pause: 1

- name: Search
do:
- click: "input.sidebar-search"
- type:
into: "input.sidebar-search"
text: "shot-scraper"
delay_ms: 25
- press:
selector: "input.sidebar-search"
key: Enter
- wait_for: "text=Search Results"
- pause: 2
22 changes: 22 additions & 0 deletions docs/libs/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,28 @@ or

(.venv) $ python -m pip install "pandas>=2"

However, ``pip`` does not usually carry out any checks to protect against
tampering and allows arbitrary code from the distributions to be executed. You
can, however, use ``pip`` in a way that ensures more secure installation
mechanisms:

``--require-hashes``
enables hash verification mode.

This ensures that hashes have been provided in the requirements file. It is
also a convenient way to create a list of hashes yourself, as it displays
the hashes of the downloaded packages. However, only the preferred archive
is downloaded for each package, so you may still need to add hashes for
alternative archives using :samp:`pip hash {PACKAGE_NAME}` – for example, if
different operating system variants are available.

``--only-binary :all:``
prevents source code from being provided.

.. seealso::
* `Secure installs <https://pip.pypa.io/en/stable/topics/secure-installs/>`_
* `pip hash <https://pip.pypa.io/en/stable/cli/pip_hash/>`_

Proxy server
~~~~~~~~~~~~

Expand Down
6 changes: 3 additions & 3 deletions docs/packs/apps.rst
Original file line number Diff line number Diff line change
Expand Up @@ -149,9 +149,8 @@ specific version.
$ uv sync --exclude-newer-package "{PACKAGE}=P0D"

.. tip::
You can also use the
:doc:`Python4DataScience:productive/git/advanced/hooks/pre-commit` to
regularly update your :file:`uv.lock` file:
You can also use :doc:`Python4DataScience:productive/git/advanced/hooks/prek`
to regularly update your :file:`uv.lock` file:

.. code-block:: yaml
:caption: .pre-commit-config.yaml
Expand All @@ -160,6 +159,7 @@ specific version.
rev: 6a280ba12b7901e47757c868c8c13c6a624c9ecb # 0.11.7
hooks:
- id: uv-lock
args: ["--exclude-newer = 'P3D'", "--quiet"]

Restrict platform and Python versions
-------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion docs/packs/dataprep/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ License-File = [ "LICENSE" ]

[dependency-groups]
dev = [
"pre-commit",
"prek",
{ include-group = "docs" },
{ include-group = "tests" },
]
Expand Down
2 changes: 1 addition & 1 deletion docs/packs/distribution.rst
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,7 @@ Dependency groups

Recursive dependency groups are also possible. For example, for ``dev`` you can
take over all dependencies from ``docs`` and ``test`` in addition to
``pre-commit``:
``prek``:

.. literalinclude:: dataprep/pyproject.toml
:language: toml
Expand Down
Loading
Loading