Synapse is an open-source Matrix homeserver written and maintained by the Matrix.org Foundation. A malicious user on a Synapse homeserver X with permission to create certain state events can disable outbound federation from X to an arbitrary homeserver Y. Synapse instances with federation disabled are not affected. In versions of Synapse up to and including 1.73, Synapse did not limit the size of `invite_room_state`, meaning that it was possible to create an arbitrarily large invite event. Synapse 1.74 refuses to create oversized `invite_room_state` fields. Server operators should upgrade to Synapse 1.74 or newer urgently.


cpe:2.3:a:matrix:synapse:*:*:*:*:*:*:*:*
D7B262AE-3361-41B7-8BF8-D893316A98C4
1.74.0
OR
### Description

federation between my homeserver and matrix.org was broken. My homeserver was able to send messages but messages sent from matrix.org were never received. I was not the only server affected by this issue and another homeserver admin found a solution in [#synapse-admins](https://matrix.to/#/#synapse:matrix.org):

```
--- synapse/app/_base.py.2022118        2022-11-09 10:52:11.000000000 +0100
+++ synapse/app/_base.py        2022-11-18 00:38:43.054743699 +0100
@@ -650,7 +650,9 @@
     #
     # in short, we somewhat arbitrarily limit requests to 200 * 64K (about 12.5M)
     #
-    max_request_size = 200 * MAX_PDU_SIZE
+    #max_request_size = 200 * MAX_PDU_SIZE
+    max_request_size = 2000 * MAX_PDU_SIZE
 
     # if we have a media repo enabled, we may need to allow larger uploads than that
     if config.media.can_load_media_repo:
```

This patch worked for the original author and I confirm it is working for me too. I also made sure the parameter `client_max_body_size` in my nginx reverse-proxy was high enough. Increasing the value in nginx was not enough to fix the issue, the patch above was needed. I was affected by this bug since last weekend and I did a lot of tests by changing many values in nginx by increasing timeouts and size but nothing worked. I also tried changing synapse cache values and configs but that didn't do the trick either.



### Steps to reproduce

I do not know how to reproduce the initial conditions that caused the bug but the patch fixed the federation between my server and matrix.org.

### Homeserver

matrix.org was the server unable to send messages

### Synapse Version

1.72.0rc1 for matrix and I run 1.71.0

### Installation Method

Docker (matrixdotorg/synapse)

### Platform

Ubuntu LTS and I use the Ansible playbook from https://github.com/spantaleev/matrix-docker-ansible-deploy

### Relevant log output

```shell
here is an example of logs that I was seeing on my side:

Nov 17 21:46:19 localhost matrix-nginx-proxy[2836800]: 2022/11/18 02:46:19 [error] 30#30: *13315398 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 172.18.0.19, server: matrix-nginx-proxy, request: "PUT /_matrix/federation/v1/send/1668680027792 HTTP/1.0", upstream: "http://172.18.0.21:18111/_matrix/federation/v1/send/1668680027792", host: "matrix.anontier.nl"


here is a log that was submitted to me by a matrix developer:

2022-11-16 08:19:04,015 - synapse.http.matrixfederationclient - 672 - INFO - federation_transaction_transmission_loop-1891708-- - {PUT-O-1387436} [anontier.nl] Request failed: PUT matrix://anontier.nl/_matrix/federation/v1/send/1668471702441: HttpResponseException('502: Bad Gateway')
2022-11-16 08:20:01,940 - synapse.http.matrixfederationclient - 629 - INFO - federation_transaction_transmission_loop-1891708-- - {PUT-O-1387436} [anontier.nl] Got response headers: 502 Bad Gateway
```


### Anything else that would be useful to know?

_No response_
Federation was broken between my homeserver and matrix.org (and I fixed it with this patch)
This is a GitHub Issue
repo:synapse
owner:matrix-org
Title : Federation was broken between my homeserver and matrix.org (and I fixed it with this patch)
Issue date:

--- start body ---
### Description

federation between my homeserver and matrix.org was broken. My homeserver was able to send messages but messages sent from matrix.org were never received. I was not the only server affected by this issue and another homeserver admin found a solution in [#synapse-admins](https://matrix.to/#/#synapse:matrix.org):

```
--- synapse/app/_base.py.2022118        2022-11-09 10:52:11.000000000 +0100
+++ synapse/app/_base.py        2022-11-18 00:38:43.054743699 +0100
@@ -650,7 +650,9 @@
     #
     # in short, we somewhat arbitrarily limit requests to 200 * 64K (about 12.5M)
     #
-    max_request_size = 200 * MAX_PDU_SIZE
+    #max_request_size = 200 * MAX_PDU_SIZE
+    max_request_size = 2000 * MAX_PDU_SIZE
 
     # if we have a media repo enabled, we may need to allow larger uploads than that
     if config.media.can_load_media_repo:
```

This patch worked for the original author and I confirm it is working for me too. I also made sure the parameter `client_max_body_size` in my nginx reverse-proxy was high enough. Increasing the value in nginx was not enough to fix the issue, the patch above was needed. I was affected by this bug since last weekend and I did a lot of tests by changing many values in nginx by increasing timeouts and size but nothing worked. I also tried changing synapse cache values and configs but that didn't do the trick either.



### Steps to reproduce

I do not know how to reproduce the initial conditions that caused the bug but the patch fixed the federation between my server and matrix.org.

### Homeserver

matrix.org was the server unable to send messages

### Synapse Version

1.72.0rc1 for matrix and I run 1.71.0

### Installation Method

Docker (matrixdotorg/synapse)

### Platform

Ubuntu LTS and I use the Ansible playbook from https://github.com/spantaleev/matrix-docker-ansible-deploy

### Relevant log output

```shell
here is an example of logs that I was seeing on my side:

Nov 17 21:46:19 localhost matrix-nginx-proxy[2836800]: 2022/11/18 02:46:19 [error] 30#30: *13315398 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 172.18.0.19, server: matrix-nginx-proxy, request: "PUT /_matrix/federation/v1/send/1668680027792 HTTP/1.0", upstream: "http://172.18.0.21:18111/_matrix/federation/v1/send/1668680027792", host: "matrix.anontier.nl"


here is a log that was submitted to me by a matrix developer:

2022-11-16 08:19:04,015 - synapse.http.matrixfederationclient - 672 - INFO - federation_transaction_transmission_loop-1891708-- - {PUT-O-1387436} [anontier.nl] Request failed: PUT matrix://anontier.nl/_matrix/federation/v1/send/1668471702441: HttpResponseException('502: Bad Gateway')
2022-11-16 08:20:01,940 - synapse.http.matrixfederationclient - 629 - INFO - federation_transaction_transmission_loop-1891708-- - {PUT-O-1387436} [anontier.nl] Got response headers: 502 Bad Gateway
```


### Anything else that would be useful to know?

_No response_
--- end body ---

comments :
--- start comments ---

--- end comments ---