Modernisation release (2.0.0): bug fixes, faster parsing, Python 3.10+ packaging

.env.example

@@ -0,0 +1,15 @@
+# Copy to .env for the live test suite. .env is gitignored - never commit credentials.
+#
+#   set -a && . ./.env && set +a
+#   APPLIANCE="$SNS_URL" PASSWORD="$SNS_PASSWORD" pytest -m live
+
+SNS_URL="10.0.0.254"
+SNS_USER="admin"
+SNS_PASSWORD="changeme"
+
+# optional, unlock the remaining connection modes
+#SERIAL="VMSNSX00000000A"
+#FQDN="firewall.example.com"
+#CABUNDLE="/path/to/ca.pem"
+#CERT="/path/to/user-cert-and-key.pem"
+#PROXY="socks5://user:pass@proxy:1080"

.gitignore

@@ -2,7 +2,19 @@
 dist
 build
 .eggs
-stormshield.sns.sslclient.egg-info
+*.egg-info
 __pycache__
 *.pyc
 .tox
+.pytest_cache
+.mypy_cache
+.ruff_cache
+.coverage
+htmlcov
+
+# local credentials - never commit
+.env
+.env.*
+!.env.example
+.venv
+venv

CHANGELOG.md

@@ -0,0 +1,152 @@
+# Changelog
+
+All notable changes to this project are documented here.
+This project adheres to [Semantic Versioning](https://semver.org/).
+
+## [2.0.0] - 2026-08-03
+
+Modernisation release. See [MIGRATION.md](MIGRATION.md) for the upgrade path.
+
+### Fixed
+
+- **`snscli -t` set a TOTP instead of a timeout.** `-t` was declared twice, for
+  `--timeout` and for `--totp`; argparse's `conflict_handler="resolve"` silently
+  gave it to `--totp`. `-t` now means `--totp` only, and the parser rejects
+  duplicated options instead of swallowing them.
+- **`upload()` poisoned the whole session.** It mutated `self.headers` in place,
+  so every request issued after an upload carried the multipart `Content-Type`
+  of that upload. The headers are copied now.
+- **`upload()` leaked a file descriptor** when the POST raised.
+- **`download()` left corrupted files on disk.** The payload was written in full,
+  *then* its size and CRC were checked. Downloads now land in a `.part` file that
+  is only moved into place after verification, so a failed transfer never
+  destroys a previously good file.
+- **`download()` returned an answer whose XML was truncated.** The `<nws>`
+  element it synthesised was never closed, so `ElementTree.fromstring(
+  response.xml)` raised `ParseError` and `snscli -o xml` died with an
+  `ExpatError` after every *successful* download. Present in 1.x.
+- **A malformed answer raised `IndexError`, `AttributeError` or `TypeError`**
+  instead of `ServerError`. The `if serverd is not None` guard was a no-op: an
+  `Element` is never `None`. The download header (its `section`/`key`, `size`
+  and `crc` nodes), the serverd session nodes (`sessionid`, `protocol`,
+  `sessionlevel`) and a `<serverd>` node with no `ret` now all report
+  `ServerError`.
+- **The library hijacked the root logger** (`logging.getLogger()`), capturing the
+  host application's logging configuration and emitting full API responses into
+  it at DEBUG level. It now logs under `stormshield.sns.sslclient`.
+- **No timeout by default**, so an appliance that stopped answering hung the
+  caller forever. `SSLClient.DEFAULT_TIMEOUT` is 30 s; pass `timeout=None` to
+  restore the old behaviour. `snscli --timeout 0` — or the 1.x `-1` — still
+  waits forever.
+- **`crc.compute_crc32()` raised `TypeError`** when handed a `str`, from a
+  leftover Python 2 branch calling `bytearray(data)` without an encoding. Both
+  it and `update_crc32()` now read a `str` as UTF-8.
+- **`ConfigParser` crashed on an empty answer** (`IndexError` on `lines[0]`,
+  `AttributeError` when `output` was `None`).
+- **`Response.__repr__` crashed** when `output` was `None`, and otherwise dumped
+  the entire payload.
+- A `raw` payload lost its trailing newline, a side effect of rendering the
+  answer to text and parsing it back.
+- **Paged answers looked complete.** The appliance reports row counts and
+  truncation flags as attributes of the `<serverd>` node (`total`,
+  `too_many_data`, `not_enough_space`, `data_changed`); the client dropped them
+  entirely. `CONFIG OBJECT LIST type=host start=0` returns at most 100 rows
+  while announcing `total=134`, so iterating `response.data` silently processed
+  a quarter of the objects with no way to notice. They are exposed as
+  `Response.meta` / `.total` / `.count` / `.truncated`, and `send_command()`
+  logs a warning when rows were held back.
+
+- **`sslverifyhost=False` silently widened the set of trusted authorities.**
+  That option (and `ip=`) mounts an adapter that builds its own SSL context.
+  The context came from `ssl.create_default_context()` with no `cafile`, which
+  activates the system trust store; urllib3 then loaded `cabundle` on top, so
+  the caller got *their* CA **plus every publicly trusted authority* — while
+  host name checking was off. Any certificate signed by any public CA, for any
+  name, was accepted. The context is now built with `cafile=cabundle`, which
+  keeps `create_default_context` from reaching for the system store.
+  Present in 1.x as soon as urllib3 2.x was installed. This is the one fix that
+  can break a *working* connection — an appliance whose certificate is signed by
+  a public CA was accepted before and is refused now. Pass the authorities you
+  trust as `cabundle`; see [MIGRATION.md](MIGRATION.md).
+
+### Changed
+
+- **Answers are decoded straight from their XML tree.** 1.x parsed the XML,
+  re-serialised it to ini text, then parsed that text back with regexes and
+  `shlex`. `Response.data` is now built from the tree, and `Response.output` is
+  rendered lazily on first access. Raw XML answer to fully decoded `data`,
+  measured on the captured answers in `tests/fixtures`:
+
+  | answer | 1.x | 2.0 |
+  |---|---|---|
+  | `USER LIST` (88 KB) | 27.7 ms | **3.9 ms** (×7.2) |
+  | `CONFIG OBJECT LIST` (23.5 KB) | 6.4 ms | **1.3 ms** (×5.1) |
+
+- **`crc.py` uses `zlib`** instead of a CRC table written in Python. The SNS CRC
+  is the non-finalised IEEE CRC-32, i.e. `zlib.crc32(data) ^ 0xFFFFFFFF`.
+  **157× faster** (113 ms/MB → 0.72 ms/MB), and the hand-written 256-entry
+  table (66 lines) removed.
+- `format_output()` builds its result with a list join instead of repeated
+  string concatenation, which was quadratic in the answer size.
+- `section_line` parsing uses one compiled regex instead of instantiating a
+  `shlex` lexer per line with a hand-maintained `wordchars` allow-list.
+- Download chunk size raised from 10 KB to 64 KB.
+- Connection failures are retried twice with backoff. Only connection
+  establishment is retried — a command that reached the appliance is never
+  replayed, as API commands are not idempotent.
+- `pygments`, `colorlog` and `pyreadline3` moved to the `cli` extra: importing
+  the library no longer pulls in terminal colouring dependencies.
+
+### Added
+
+- `SSLClient` is a context manager: `with SSLClient(...) as client:`.
+- `disconnect()` is idempotent and survives an already-dead connection.
+- `SNSError`, a common base class for every exception the library raises.
+- `Response.from_xml()` / `Response.from_tree()`, and `Response.get()` as a
+  shortcut for `response.parser.get()`.
+- `Response.serverd_code`, the code of the first serverd node (the transfer
+  state), distinct from `Response.code` which reports the final status.
+- `Response.meta`, `.total`, `.count`, `.offset` and `.truncated` for paged
+  answers. `truncated` compares `offset + count` against `total`, not `count`
+  alone: past the last page the appliance reports the full `total` alongside
+  zero rows, so the naive comparison never terminates.
+- `Response.to_dict()` and `Response.json()`. `data` uses `CaseInsensitiveDict`,
+  which `json.dumps` refuses; these return plain containers and a JSON string.
+  Note that every value is a string — the appliance sends XML attributes, so no
+  type information ever reaches the client.
+- Type annotations throughout, plus a `py.typed` marker.
+- `SNSCLI_PASSWORD` environment variable, so the password no longer has to
+  appear in `ps` output via `-p`.
+- `--retries` option on `snscli`.
+- An offline test suite (156 tests) built on XML answers captured from a real
+  appliance, and a locale-independent live suite behind the `live` marker.
+- `tox` environments for Python 3.10–3.13, plus a `live` environment and a
+  `lint` environment running ruff and mypy.
+
+### Removed
+
+- **Python 2 leftovers**: `from __future__ import unicode_literals`, the
+  `unicode` branch in `quote()`, the `raw_input`/`FileNotFoundError` shims, and
+  the `sys.version_info[0] < 3` branch in the parser.
+- **urllib3 1.x support** (end of life). `urllib3>=2.0` is now required, which
+  collapsed six near-identical `if URLLIB3V2` branches in the adapters.
+- `HostNameAdapter` and `DNSResolverHTTPSAdapter`, merged into the single
+  `SNSHTTPSAdapter`.
+- `setup.py`, replaced by `pyproject.toml` (PEP 517/621).
+- Python 3.7–3.9 support; the minimum is now 3.10.
+- `SSLClient.SRV_RET_MSG` and `SSLClient.AUTH_FAILED`, and
+  `ConfigParser.TOKEN_VALUE_RE`: dead since at least 1.0, referenced nowhere.
+  The `SRV_RET_*` and `SSL_SERVERD_*` code constants themselves are kept, and
+  `SSL_SERVERD_MSG` is still used to build error messages.
+- `MANIFEST.in`: setuptools builds an identical sdist without it now that
+  packaging is declared in `pyproject.toml`.
+
+## [1.1.2]
+
+- Disable compression to avoid an issue with later versions of urllib3 (#23)
+- Update readline lib for Windows
+
+## [1.1.1]
+
+- Ignore bad xml response from serverd (#22)
+- Update for SNS v5 (#17)

MIGRATION.md

@@ -0,0 +1,123 @@
+# Migrating from 1.x to 2.0
+
+Most code needs **no change**: `SSLClient(...)`, `send_command()`, `response.data`,
+`response.output`, `response.xml`, `response.ret` and `response.parser.get()` all
+behave as before.
+
+The points below are the ones that can actually break a caller.
+
+## Requirements
+
+| | 1.x | 2.0 |
+|---|---|---|
+| Python | 3.7+ | **3.10+** |
+| urllib3 | 1.x or 2.x | **2.0+** |
+
+Install the CLI dependencies explicitly if you use `snscli`:
+
+```console
+$ pip install 'stormshield.sns.sslclient[cli]'
+```
+
+## Behaviour changes
+
+### A default timeout is applied
+
+1.x waited forever. 2.0 defaults to 30 seconds:
+
+```python
+client = SSLClient(host="fw", password="pass", timeout=None)  # restore 1.x behaviour
+```
+
+### `repr(response)` no longer returns the payload
+
+`__repr__` returned `self.output`, and raised `TypeError` when it was `None`.
+It is now a short summary. Use `str(response)` or `response.output` for the text:
+
+```python
+print(response)          # unchanged: __str__ still returns output
+print(repr(response))    # <Response ret=100 code=00a00100 msg='Ok'>
+```
+
+### `sslverifyhost=False` no longer trusts the system authorities
+
+This is the one change that breaks *working* connections. An appliance reached
+with `sslverifyhost=False` **and** peer verification left on connected in 1.x
+whenever its certificate was signed by a publicly trusted CA — a common setup
+when the appliance is fronted by a corporate or commercial certificate. That
+worked by accident: the adapter's SSL context pulled in the system trust store
+on top of `cabundle` (see the CHANGELOG). 2.0 trusts `cabundle` and nothing else,
+so the same call now raises `SSLError`.
+
+Name the authorities you actually trust:
+
+```python
+import certifi
+
+# appliance certificate signed by a public CA
+client = SSLClient(host="fw", password="pass", sslverifyhost=False,
+                   cabundle=certifi.where())
+
+# appliance certificate signed by your own CA
+client = SSLClient(host="fw", password="pass", sslverifyhost=False,
+                   cabundle="/etc/ssl/private/company-ca.pem")
+```
+
+`snscli` takes the same file with `-C/--cabundle`. Callers already passing
+`sslverifypeer=False` (`snscli -k`) are unaffected, and so is the default
+factory-certificate setup, which the shipped bundle covers.
+
+### `raw` answers keep their trailing newline
+
+1.x dropped the final `\n` of a `format="raw"` payload. If you compared
+`response.data` against a literal, add the newline back or use `.rstrip()`.
+
+### Logging moved out of the root logger
+
+1.x called `logging.getLogger()`. If you relied on that to see the library's
+DEBUG output, target its namespace explicitly:
+
+```python
+logging.getLogger("stormshield.sns.sslclient").setLevel(logging.DEBUG)
+```
+
+### `snscli -t`
+
+`-t` was ambiguous and resolved to `--totp`. It now means `--totp` only, and
+`--timeout` has no short form. Scripts passing `-t` for a timeout were already
+setting a TOTP, so they were already broken — they now need `--timeout`.
+
+## Removed API
+
+| Removed | Replacement |
+|---|---|
+| `HostNameAdapter` | `SNSHTTPSAdapter(assert_hostname=False, cafile=bundle)` |
+| `DNSResolverHTTPSAdapter(cn, host)` | `SNSHTTPSAdapter(cn, cafile=bundle)` |
+| `URLLIB3V2` | urllib3 2.x is always assumed |
+| `setup.py install` | `pip install .` |
+
+These were internal plumbing; the public entry point has always been `SSLClient`.
+
+Always pass `cafile` — the certificate authority bundle you mounted the session
+with, or `None` only when peer verification is disabled. The adapter builds its
+own SSL context, and leaving `cafile` out makes it fall back to the system trust
+store: your bundle would no longer be the only authority trusted, which is the
+widening 2.0 fixed.
+
+## New things worth adopting
+
+```python
+from stormshield.sns.sslclient import SSLClient, SNSError
+
+# context manager: disconnects even when a command raises
+with SSLClient(host="10.0.0.254", user="admin", password="pass",
+               sslverifyhost=False) as client:
+    response = client.send_command("SYSTEM PROPERTY")
+    print(response.get("Result", "Version"))       # shortcut for parser.get()
+
+# one except clause for the whole library
+try:
+    ...
+except SNSError as exc:
+    ...
+```