Use reverse proxy mode to do real-time replay

1

Suppose we have a real server A and a shadow server B, both of which are behind a mitmproxy instance that works in reverse mode. All the service requests are destined for server A through HTTPS originally.

Is it possible that, for each request destined for server A, we duplicate and replay that request to server B through an HTTPS connection simultaneously? Then the response from server A is redirected by mitmproxy to the original client; but the response from B will be dropped automatically. Here I want to do real-time replay which is different from other posts I have seen that record a log and then replay. Also, the mitmproxy needs to establish two https connections to A and B concurrently.

Please let me know if mitmproxy could do that using its scripting interface or how to extend it to achieve that.

2

Hi @ppsun,

You are likely looking for something similar to https://github.com/mitmproxy/mitmproxy/blob/master/examples/complex/dup_and_replay.py :slight_smile:
Does that help?

3

Thanks. Actually I have noticed this script. However, it is replaying HTTP request. What if the proxy also talks to the shadow server B using HTTPS? Then the mitmproxy would also need to negotiate the TLS crypto parameters with B through the TLS handshake process. Will mitmproxy do TLS handshake with B automatically?

4

Yes. Replay is fairly isolated from the actual proxy mechanism actually: https://github.com/mitmproxy/mitmproxy/blob/master/mitmproxy/proxy/protocol/http_replay.py

5

Got it. I’ll check the source code and do a real-time replay test. Thanks for the help!

6

I have tested the dup_and_replay script for mitmproxy version 2.0.0. However, I encountered a couple of problems.

Problem#1: This line of script “ctx.master.view.add(f)” leads to an error message:

“AttributeError: ‘DumpMaster’ object has no attributed ‘view’”

Problem#2: then I go ahead and comment out the line mentioned in Problem#1, but the main mitmproxy thread got stuck when setting “block” to be True in the ‘replay_request’ function. But if I set ‘block’ to false, then I can see that the request is constantly replayed and response is received.

Do you know what is actually happening here?

7

Yes - that access mitmproxy-specific functionality I believe. I’d be happy to accept a PR that fixes that.

From the replay docs:

        block: If True, this function will wait for the replay to finish.

            This causes a deadlock if activated in the main thread.

You can only block if you are in a separate thread, e.g. with @concurrent.

The constant refresh thing sounds like you call .replay() for the replayed flow again. If you replay a request, it goes through the event hooks like a normal request.

8

Got it. Thanks. I see why it keeps replaying the request.

1 Like