Hi there!
First post, complex question(s)! ^^
I’m working on a heavily manipulating proxy to improve the usability of a flash based browser game - since I can’t use a simple userscript there, my plan (and working attempt so far) is a basic userscript in the browser and a mitmproxy/ mitmdump inline script as backend.
According to my current draft, I need up to 4 concurrent flows:
-
X1: XHR long polling for the proxy to send information to the userscript
-
X2: normal XHR request to send commands / auxiliary info to the proxy
-
S1: game’s requests, which I have to modify before sending or tamper with the response before delivering it back to game
-
S2: proxy’s game server requests, which have to be sent & received while S1 is being intercepted/ on hold.
So far only X1 & S1 are implemented, by using the concurrent decorator for request() and forking for game url and magic url.
Now for the first questions/ problems:
-
I tried to use a tiny flask app for handling the script-proxy communication; that failed completely - the blocking thread for long polling blocks the whole proxy. Is this intended, or has just nobody noticed this so far? (According to source, the server hosting the app(s) is started threaded when registering the app in mitm)
-
when falling back to handle both X1 & S1 by proxy, I first tried a ‘magic path’ on the game server’s host to save one proxy rule in the browser.
The game runs on https, so my XHR requests have to be https too.
But that turned out as problem, about every 10th game request got stuck, wasn’t replied by the server. I don’t know anything about SSL connections in detail, but my feeling says that when mitm connects to the game server for faking my magic path, the original game request aborts.
By using a completely differnet magic url, it doesn’t happen - thousands of game requests without any trouble.
Is this a problem with mitm, is it a restriction of SSL, or could it be a restriction from the game server?
Wouldn’t it be possible to reuse an existing connection, at least when it’s only used for a magic url, and no request has to be made…?
– Depending on this answer, my whole project might be for the bin; but thinking positive and hoping for some solution, I’ll keep the questions coming: -
what would be a preferable way of sending those extra requests S2?
I thought of duplicating one original flow S1 (before request), storing it & duplicate this one again for every tailoring a S2 request. But then I’d have to filter out my tailored requests in the response() method;
So an indepent request/reply mechanism would be preferable - this is not a mitm realted question, but since I’m quite new to Python and all its possibilities (in fact, I started to dig in for mitm & this project) maybe someone more experienced could give some hints. -
for holding a request (for now only my XHR), I currently use a simple
while True:
if context.my_flag is True:
break
time.sleep(0.1)
flow.reply(...)
My first try was to use flow.intercept(context._master) [and later call accept_intercept from another thread], but since exiting the request / response methods automatically releases the interception, I had no luck with that 
4.1 Are there some plans to implement a real hold interception? (I can remember vaguely some other guy asking this questions quite some while ago, in the newsgroup, I think…)
4.2 If not, what would be a better way than the one I use so far? Seems to me that this sleeping makes the whole proxy awfully lame - could the threading package improve perfomance? I thought of trying threading.Condition, but as stated above, I’m quite new to Python and that threading stuff seems a bit complicated to me… so I better ask first, if you say that this would be the wrong approach, I can skip that ^^
Thanks in advance for any advise, and a big thanks to you guys behind mitmproxy!
p3trus