Update chat/matrix-synapse to 1.53.0

Synapse 1.53.0 (2022-02-22)
===========================

No significant changes.


Synapse 1.53.0rc1 (2022-02-15)
==============================

Features
--------

- Add experimental support for sending to-device messages to application services, as specified by [MSC2409](https://github.com/matrix-org/matrix-doc/pull/2409). ([\#11215](https://github.com/matrix-org/synapse/issues/11215), [\#11966](https://github.com/matrix-org/synapse/issues/11966))
- Remove account data (including client config, push rules and ignored users) upon user deactivation. ([\#11655](https://github.com/matrix-org/synapse/issues/11655))
- Experimental support for [MSC3666](https://github.com/matrix-org/matrix-doc/pull/3666): including bundled aggregations in server side search results. ([\#11837](https://github.com/matrix-org/synapse/issues/11837))
- Enable cache time-based expiry by default. The `expiry_time` config flag has been superseded by `expire_caches` and `cache_entry_ttl`. ([\#11849](https://github.com/matrix-org/synapse/issues/11849))
- Add a callback to allow modules to allow or forbid a 3PID (email address, phone number) from being associated to a local account. ([\#11854](https://github.com/matrix-org/synapse/issues/11854))
- Stabilize support and remove unstable endpoints for [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231). Clients must switch to the stable identifier and endpoint. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#stablisation-of-msc3231) for more information. ([\#11867](https://github.com/matrix-org/synapse/issues/11867))
- Allow modules to retrieve the current instance's server name and worker name. ([\#11868](https://github.com/matrix-org/synapse/issues/11868))
- Use a dedicated configurable rate limiter for 3PID invites. ([\#11892](https://github.com/matrix-org/synapse/issues/11892))
- Support the stable API endpoint for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283): new settings in `/capabilities` endpoint. ([\#11933](https://github.com/matrix-org/synapse/issues/11933), [\#11989](https://github.com/matrix-org/synapse/issues/11989))
- Support the `dir` parameter on the `/relations` endpoint, per [MSC3715](https://github.com/matrix-org/matrix-doc/pull/3715). ([\#11941](https://github.com/matrix-org/synapse/issues/11941))
- Experimental implementation of [MSC3706](https://github.com/matrix-org/matrix-doc/pull/3706): extensions to `/send_join` to support reduced response size. ([\#11967](https://github.com/matrix-org/synapse/issues/11967))


Bugfixes
--------

- Fix [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) historical messages backfilling in random order on remote homeservers. ([\#11114](https://github.com/matrix-org/synapse/issues/11114))
- Fix a bug introduced in Synapse 1.51.0 where incoming federation transactions containing at least one EDU would be dropped if debug logging was enabled for `synapse.8631_debug`. ([\#11890](https://github.com/matrix-org/synapse/issues/11890))
- Fix a long-standing bug where some unknown endpoints would return HTML error pages instead of JSON `M_UNRECOGNIZED` errors. ([\#11930](https://github.com/matrix-org/synapse/issues/11930))
- Implement an allow list of content types for which we will attempt to preview a URL. This prevents Synapse from making useless longer-lived connections to streaming media servers. ([\#11936](https://github.com/matrix-org/synapse/issues/11936))
- Fix a long-standing bug where pagination tokens from `/sync` and `/messages` could not be provided to the `/relations` API. ([\#11952](https://github.com/matrix-org/synapse/issues/11952))
- Require that modules register their callbacks using keyword arguments. ([\#11975](https://github.com/matrix-org/synapse/issues/11975))
- Fix a long-standing bug where `M_WRONG_ROOM_KEYS_VERSION` errors would not include the specced `current_version` field. ([\#11988](https://github.com/matrix-org/synapse/issues/11988))


Improved Documentation
----------------------

- Fix typo in User Admin API: unpind -> unbind. ([\#11859](https://github.com/matrix-org/synapse/issues/11859))
- Document images returned by the User List Media Admin API can include those generated by URL previews. ([\#11862](https://github.com/matrix-org/synapse/issues/11862))
- Remove outdated MSC1711 FAQ document. ([\#11907](https://github.com/matrix-org/synapse/issues/11907))
- Correct the structured logging configuration example. Contributed by Brad Jones. ([\#11946](https://github.com/matrix-org/synapse/issues/11946))
- Add information on the Synapse release cycle. ([\#11954](https://github.com/matrix-org/synapse/issues/11954))
- Fix broken link in the README to the admin API for password reset. ([\#11955](https://github.com/matrix-org/synapse/issues/11955))


Deprecations and Removals
-------------------------

- Drop support for `webclient` listeners and configuring `web_client_location` to a non-HTTP(S) URL. Deprecated configurations are a configuration error. ([\#11895](https://github.com/matrix-org/synapse/issues/11895))
- Remove deprecated `user_may_create_room_with_invites` spam checker callback. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#removal-of-user_may_create_room_with_invites) for more information. ([\#11950](https://github.com/matrix-org/synapse/issues/11950))
- No longer build `.deb` packages for Ubuntu 21.04 Hirsute Hippo, which has now EOLed. ([\#11961](https://github.com/matrix-org/synapse/issues/11961))


Internal Changes
----------------

- Enhance user registration test helpers to make them more useful for tests involving application services and devices. ([\#11615](https://github.com/matrix-org/synapse/issues/11615), [\#11616](https://github.com/matrix-org/synapse/issues/11616))
- Improve performance when fetching bundled aggregations for multiple events. ([\#11660](https://github.com/matrix-org/synapse/issues/11660), [\#11752](https://github.com/matrix-org/synapse/issues/11752))
- Fix type errors introduced by new annotations in the Prometheus Client library. ([\#11832](https://github.com/matrix-org/synapse/issues/11832))
- Add missing type hints to replication code. ([\#11856](https://github.com/matrix-org/synapse/issues/11856), [\#11938](https://github.com/matrix-org/synapse/issues/11938))
- Ensure that `opentracing` scopes are activated and closed at the right time. ([\#11869](https://github.com/matrix-org/synapse/issues/11869))
- Improve opentracing for incoming federation requests. ([\#11870](https://github.com/matrix-org/synapse/issues/11870))
- Improve internal docstrings in `synapse.util.caches`. ([\#11876](https://github.com/matrix-org/synapse/issues/11876))
- Do not needlessly clear the `get_users_in_room` and `get_users_in_room_with_profiles` caches when any room state changes. ([\#11878](https://github.com/matrix-org/synapse/issues/11878))
- Convert `ApplicationServiceTestCase` to use `simple_async_mock`. ([\#11880](https://github.com/matrix-org/synapse/issues/11880))
- Remove experimental changes to the default push rules which were introduced in Synapse 1.19.0 but never enabled. ([\#11884](https://github.com/matrix-org/synapse/issues/11884))
- Disable coverage calculation for olddeps build. ([\#11888](https://github.com/matrix-org/synapse/issues/11888))
- Preparation to support sending device list updates to application services. ([\#11905](https://github.com/matrix-org/synapse/issues/11905))
- Add a test that checks users receive their own device list updates down `/sync`. ([\#11909](https://github.com/matrix-org/synapse/issues/11909))
- Run Complement tests sequentially. ([\#11910](https://github.com/matrix-org/synapse/issues/11910))
- Various refactors to the application service notifier code. ([\#11911](https://github.com/matrix-org/synapse/issues/11911), [\#11912](https://github.com/matrix-org/synapse/issues/11912))
- Tests: replace mocked `Authenticator` with the real thing. ([\#11913](https://github.com/matrix-org/synapse/issues/11913))
- Various refactors to the typing notifications code. ([\#11914](https://github.com/matrix-org/synapse/issues/11914))
- Use the proper type for the `Content-Length` header in the `UploadResource`. ([\#11927](https://github.com/matrix-org/synapse/issues/11927))
- Remove an unnecessary ignoring of type hints due to fixes in upstream packages. ([\#11939](https://github.com/matrix-org/synapse/issues/11939))
- Add missing type hints. ([\#11953](https://github.com/matrix-org/synapse/issues/11953))
- Fix an import cycle in `synapse.event_auth`. ([\#11965](https://github.com/matrix-org/synapse/issues/11965))
- Unpin `frozendict` but exclude the known bad version 2.1.2. ([\#11969](https://github.com/matrix-org/synapse/issues/11969))
- Prepare for rename of default Complement branch. ([\#11971](https://github.com/matrix-org/synapse/issues/11971))
- Fetch Synapse's version using a helper from `matrix-common`. ([\#11979](https://github.com/matrix-org/synapse/issues/11979))


Synapse 1.52.0 (2022-02-08)
===========================

No significant changes since 1.52.0rc1.

Note that [Twisted 22.1.0](https://github.com/twisted/twisted/releases/tag/twisted-22.1.0)
has recently been released, which fixes a [security issue](https://github.com/twisted/twisted/security/advisories/GHSA-92x2-jw7w-xvvx)
within the Twisted library. We do not believe Synapse is affected by this vulnerability,
though we advise server administrators who installed Synapse via pip to upgrade Twisted
with `pip install --upgrade Twisted` as a matter of good practice. The Docker image
`matrixdotorg/synapse` and the Debian packages from `packages.matrix.org` are using the
updated library.


Synapse 1.52.0rc1 (2022-02-01)
==============================

Features
--------

- Remove account data (including client config, push rules and ignored users) upon user deactivation. ([\#11621](https://github.com/matrix-org/synapse/issues/11621), [\#11788](https://github.com/matrix-org/synapse/issues/11788), [\#11789](https://github.com/matrix-org/synapse/issues/11789))
- Add an admin API to reset connection timeouts for remote server. ([\#11639](https://github.com/matrix-org/synapse/issues/11639))
- Add an admin API to get a list of rooms that federate with a given remote homeserver. ([\#11658](https://github.com/matrix-org/synapse/issues/11658))
- Add a config flag to inhibit `M_USER_IN_USE` during registration. ([\#11743](https://github.com/matrix-org/synapse/issues/11743))
- Add a module callback to set username at registration. ([\#11790](https://github.com/matrix-org/synapse/issues/11790))
- Allow configuring a maximum file size as well as a list of allowed content types for avatars. ([\#11846](https://github.com/matrix-org/synapse/issues/11846))


Bugfixes
--------

- Include the bundled aggregations in the `/sync` response, per [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675). ([\#11612](https://github.com/matrix-org/synapse/issues/11612))
- Fix a long-standing bug when previewing Reddit URLs which do not contain an image. ([\#11767](https://github.com/matrix-org/synapse/issues/11767))
- Fix a long-standing bug that media streams could cause long-lived connections when generating URL previews. ([\#11784](https://github.com/matrix-org/synapse/issues/11784))
- Include a `prev_content` field in state events sent to Application Services. Contributed by @totallynotvaishnav. ([\#11798](https://github.com/matrix-org/synapse/issues/11798))
- Fix a bug introduced in Synapse 0.33.3 causing requests to sometimes log strings such as `HTTPStatus.OK` instead of integer status codes. ([\#11827](https://github.com/matrix-org/synapse/issues/11827))


Improved Documentation
----------------------

- Update pypi installation docs to indicate that we now support Python 3.10. ([\#11820](https://github.com/matrix-org/synapse/issues/11820))
- Add missing steps to the contribution submission process in the documentation.  Contributed by @sequentialread. ([\#11821](https://github.com/matrix-org/synapse/issues/11821))
- Remove not needed old table of contents in documentation. ([\#11860](https://github.com/matrix-org/synapse/issues/11860))
- Consolidate the `access_token` information at the top of each relevant page in the Admin API documentation. ([\#11861](https://github.com/matrix-org/synapse/issues/11861))


Deprecations and Removals
-------------------------

- Drop support for Python 3.6, which is EOL. ([\#11683](https://github.com/matrix-org/synapse/issues/11683))
- Remove the `experimental_msc1849_support_enabled` flag as the features are now stable. ([\#11843](https://github.com/matrix-org/synapse/issues/11843))


Internal Changes
----------------

- Preparation for database schema simplifications: add `state_key` and `rejection_reason` columns to `events` table. ([\#11792](https://github.com/matrix-org/synapse/issues/11792))
- Add `FrozenEvent.get_state_key` and use it in a couple of places. ([\#11793](https://github.com/matrix-org/synapse/issues/11793))
- Preparation for database schema simplifications: stop reading from `event_reference_hashes`. ([\#11794](https://github.com/matrix-org/synapse/issues/11794))
- Drop unused table `public_room_list_stream`. ([\#11795](https://github.com/matrix-org/synapse/issues/11795))
- Preparation for reducing Postgres serialization errors: allow setting transaction isolation level. Contributed by Nick @ Beeper. ([\#11799](https://github.com/matrix-org/synapse/issues/11799), [\#11847](https://github.com/matrix-org/synapse/issues/11847))
- Docker: skip the initial amd64-only build and go straight to multiarch. ([\#11810](https://github.com/matrix-org/synapse/issues/11810))
- Run Complement on the Github Actions VM and not inside a Docker container. ([\#11811](https://github.com/matrix-org/synapse/issues/11811))
- Log module names at startup. ([\#11813](https://github.com/matrix-org/synapse/issues/11813))
- Improve type safety of bundled aggregations code. ([\#11815](https://github.com/matrix-org/synapse/issues/11815))
- Correct a type annotation in the event validation logic. ([\#11817](https://github.com/matrix-org/synapse/issues/11817), [\#11830](https://github.com/matrix-org/synapse/issues/11830))
- Minor updates and documentation for database schema delta files. ([\#11823](https://github.com/matrix-org/synapse/issues/11823))
- Workaround a type annotation problem in `prometheus_client` 0.13.0. ([\#11834](https://github.com/matrix-org/synapse/issues/11834))
- Minor performance improvement in room state lookup. ([\#11836](https://github.com/matrix-org/synapse/issues/11836))
- Fix some indentation inconsistencies in the sample config. ([\#11838](https://github.com/matrix-org/synapse/issues/11838))
- Add type hints to `tests/rest/admin`. ([\#11851](https://github.com/matrix-org/synapse/issues/11851))

4 files changed, 16 insertions, 30 deletions

diff --git a/chat/matrix-synapse/Makefile b/chat/matrix-synapse/Makefile
index 944b3ecc829..67366d5a7ae 100644
--- a/chat/matrix-synapse/Makefile
+++ b/chat/matrix-synapse/Makefile
@@ -1,7 +1,6 @@
-# $NetBSD: Makefile,v 1.42 2022/01/30 11:17:10 wiz Exp $
+# $NetBSD: Makefile,v 1.43 2022/02/25 16:18:36 js Exp $
 
-DISTNAME=	matrix-synapse-1.51.0
-PKGREVISION=	1
+DISTNAME=	matrix-synapse-1.53.0
 CATEGORIES=	chat
 MASTER_SITES=	${MASTER_SITE_GITHUB:=matrix-org/}
 EGG_NAME=	matrix_synapse-${PKGVERSION_NOREV}
diff --git a/chat/matrix-synapse/PLIST b/chat/matrix-synapse/PLIST
index 21c0cfd5a72..a085528ffba 100644
--- a/chat/matrix-synapse/PLIST
+++ b/chat/matrix-synapse/PLIST
@@ -1,4 +1,4 @@
-@comment $NetBSD: PLIST,v 1.21 2022/01/30 10:14:16 js Exp $
+@comment $NetBSD: PLIST,v 1.22 2022/02/25 16:18:36 js Exp $
 bin/export_signing_key
 bin/generate_config
 bin/generate_log_config
@@ -1674,6 +1674,10 @@ ${PYSITELIB}/synapse/storage/schema/main/delta/65/07_arbitrary_relations.sql
 ${PYSITELIB}/synapse/storage/schema/main/delta/65/08_device_inbox_background_updates.sql
 ${PYSITELIB}/synapse/storage/schema/main/delta/65/10_expirable_refresh_tokens.sql
 ${PYSITELIB}/synapse/storage/schema/main/delta/65/11_devices_auth_provider_session.sql
+${PYSITELIB}/synapse/storage/schema/main/delta/67/01drop_public_room_list_stream.sql
+${PYSITELIB}/synapse/storage/schema/main/delta/68/01event_columns.sql
+${PYSITELIB}/synapse/storage/schema/main/delta/68/02_msc2409_add_device_id_appservice_stream_type.sql
+${PYSITELIB}/synapse/storage/schema/main/delta/68/03_delete_account_data_for_deactivated_accounts.sql
 ${PYSITELIB}/synapse/storage/schema/main/full_schemas/16/application_services.sql
 ${PYSITELIB}/synapse/storage/schema/main/full_schemas/16/event_edges.sql
 ${PYSITELIB}/synapse/storage/schema/main/full_schemas/16/event_signatures.sql
@@ -1842,9 +1846,6 @@ ${PYSITELIB}/synapse/util/templates.pyo
 ${PYSITELIB}/synapse/util/threepids.py
 ${PYSITELIB}/synapse/util/threepids.pyc
 ${PYSITELIB}/synapse/util/threepids.pyo
-${PYSITELIB}/synapse/util/versionstring.py
-${PYSITELIB}/synapse/util/versionstring.pyc
-${PYSITELIB}/synapse/util/versionstring.pyo
 ${PYSITELIB}/synapse/util/wheel_timer.py
 ${PYSITELIB}/synapse/util/wheel_timer.pyc
 ${PYSITELIB}/synapse/util/wheel_timer.pyo
diff --git a/chat/matrix-synapse/distinfo b/chat/matrix-synapse/distinfo
index abf62f58b80..1f7c83bafb1 100644
--- a/chat/matrix-synapse/distinfo
+++ b/chat/matrix-synapse/distinfo
@@ -1,7 +1,7 @@
-$NetBSD: distinfo,v 1.31 2022/01/30 11:17:10 wiz Exp $
+$NetBSD: distinfo,v 1.32 2022/02/25 16:18:36 js Exp $
 
-BLAKE2s (matrix-synapse-1.51.0.tar.gz) = d29dfd91e95584096ceff17fea2095b1f1aaa7cb9bc88e0a71c5d4ff5cca3689
-SHA512 (matrix-synapse-1.51.0.tar.gz) = 52b96c4ee74894934fdaf683c8a3f0e9f4d6eb227ad916d94aaefd878c70cdc4fc0edbf44082dcbd5de5930fe279501207614a2bd6c28e2b2c64425ce74f137a
-Size (matrix-synapse-1.51.0.tar.gz) = 7665238 bytes
+BLAKE2s (matrix-synapse-1.53.0.tar.gz) = 9c6942a9de01469c46629e96a77443ddbec20bda43281397b22631afa8274cdc
+SHA512 (matrix-synapse-1.53.0.tar.gz) = a6ab6219f8ab3da37968e7480e59677fe640b4b2741bdfc75e407fd11deeed47a36efa7bb3c04ae1022fb07ad401763b7d747860d127cfb785a79433dfbcc4b7
+Size (matrix-synapse-1.53.0.tar.gz) = 7696244 bytes
 SHA1 (patch-synapse_handlers_room.py) = f9a62add7171898ec0ea76360f0a4c9969609537
-SHA1 (patch-synapse_python_dependencies.py) = ed198e743148c50c2066f2e20310ca6f5e29b19a
+SHA1 (patch-synapse_python_dependencies.py) = f6f12aa43f8739d83a9b31e169eb97b47f4a2cc5
diff --git a/chat/matrix-synapse/patches/patch-synapse_python_dependencies.py b/chat/matrix-synapse/patches/patch-synapse_python_dependencies.py
index c0c777fc528..4fd3dfef093 100644
--- a/chat/matrix-synapse/patches/patch-synapse_python_dependencies.py
+++ b/chat/matrix-synapse/patches/patch-synapse_python_dependencies.py
@@ -1,23 +1,12 @@
-$NetBSD: patch-synapse_python_dependencies.py,v 1.5 2022/01/30 11:17:10 wiz Exp $
+$NetBSD: patch-synapse_python_dependencies.py,v 1.6 2022/02/25 16:18:36 js Exp $
 
 Patch out cryptography check: It doesn't need a version this high and it was
 added by upstream as an ugly hack to force a patched OpenSSL, in case it is
 linked statically.
 
-Relax other restrictions since pkgsrc has/will have newer versions.
-
---- synapse/python_dependencies.py.orig	2022-01-25 11:42:32.000000000 +0000
+--- synapse/python_dependencies.py.orig	2022-02-22 11:32:11.000000000 +0000
 +++ synapse/python_dependencies.py
-@@ -51,7 +51,7 @@ REQUIREMENTS = [
-     # we use the TYPE_CHECKER.redefine method added in jsonschema 3.0.0
-     "jsonschema>=3.0.0",
-     # frozendict 2.1.2 is broken on Debian 10: https://github.com/Marco-Sulla/python-frozendict/issues/41
--    "frozendict>=1,<2.1.2",
-+    "frozendict>=1",
-     "unpaddedbase64>=1.1.0",
-     "canonicaljson>=1.4.0",
-     # we use the type definitions added in signedjson 1.1.
-@@ -84,11 +84,8 @@ REQUIREMENTS = [
+@@ -84,9 +84,6 @@ REQUIREMENTS = [
      "Jinja2>=2.9",
      "bleach>=1.4.3",
      "typing-extensions>=3.7.4",
@@ -25,8 +14,5 @@ Relax other restrictions since pkgsrc has/will have newer versions.
 -    # with the latest security patches.
 -    "cryptography>=3.4.7",
      "ijson>=3.1",
--    "matrix-common==1.0.0",
-+    "matrix-common",
+     "matrix-common~=1.1.0",
  ]
- 
- CONDITIONAL_REQUIREMENTS = {