Hello everyone!
I am trying to set up a text environment where I am getting data from an external web service from multiple clients on my network. These clients are test code that are configured in a specific way and uses the incoming data to calculate some output I want to analyze. I need the exact same data to be distributed to the clients.
Sadly the web service is really not liking it when I gear up the amount of clients requesting data from it, and blocks my API-KEY almost immediately after starting up more than 2. I need around 100 at least.
So I have an idea of setting up a proxy that cached all the server responses from 1 of the clients requests and serves that up to the others.
The clients can be configured to direct the traffic directly to mitmproxy, so I dont need a transparent proxy. I am thinking a reverse proxy is the way to go here, but maybe a forward proxy will work just as well, if not better (easier)?
Some caveats:
- The server requires HTTPS calls
- The server responses have all the no-cache rules set
Im pretty sure mitmproxy can handle this for me. Am I right about that?
A few questions:
- Can I let mitmproxy handle the https part, and just let the client communicate to the proxy with http?
- How do I make mitmproxy ignore all the no-cache directives in the response and control when it should refresh the response with new data?
I have read the docs, but cant seem to find much about cache …in fact. Is there any more documentation than what is served at https://docs.mitmproxy.org… ? It doesnt seem to cover all the options mitmproxy system gives?
Thanks for any responses!
/Tomas