Earlier this year I wrote code that works with mitmproxy 0.15 or 0.16 (start mitmproxy, find right request, print it and replace its body from file).
import os
from mitmproxy import controller, proxy
from mitmproxy.proxy.server import ProxyServer
class MyProxy(controller):
with open("ad.xml", "r") as ad:
ad_data = ad.read()
print ("open file: ", ad_data)
def __init__(self, server):
controller.Master.__init__(self, server)
def run(self):
try:
return controller.Master.run(self)
except KeyboardInterrupt:
self.shutdown()
def handle_request(self, flow):
if 'ad.mail.ru/vast' in flow.request.url:
print ("request: ", flow.request.host)
print (flow.request.url, flow.request.path_components)
flow.reply()
def handle_response(self, flow):
if 'ad.mail.ru/vast' in flow.request.url:
print ("response: ", self.ad_data)
flow.response.content = (self.ad_data)
flow.reply()
config = proxy.ProxyConfig(port=8080)
server = ProxyServer(config)
ProxyInstance = MyProxy(server)
ProxyInstance.run()
But now it doesn’t work and I can’t write something similar to work with flow.
Right now I work with code from example, but can’t understand how to get access to methods like flow.url etc: github.com/mitmproxy/mitmproxy/blob/0.18.x/examples/flowbasic
from mitmproxy import flow, controller, options
from mitmproxy.proxy import ProxyServer, ProxyConfig
class MyMaster(flow.FlowMaster):
def run(self):
try:
flow.FlowMaster.run(self)
except KeyboardInterrupt:
self.shutdown()
@controller.handler
def request(self, f):
print("request", f)
@controller.handler
def response(self, f):
print("response", f)
opts = options.Options(app_port=8080)
config = ProxyConfig(opts)
state = flow.State()
server = ProxyServer(config)
m = MyMaster(opts, server, state)
m.run()
Right now I can start proxy in background with this code, but don’t understand how to manipulate with it (using api?):
from mitmproxy import master, controller, options
from mitmproxy.proxy import ProxyServer, ProxyConfig
import time
import threading
def background(f):
'''
a threading decorator
use @background above the function you want to run in the background
'''
def bg_f(*a, **kw):
threading.Thread(target=f, args=a, kwargs=kw).start()
return bg_f
class MProxy(master.Master):
@background
def run(self):
try:
master.Master.run(self)
except KeyboardInterrupt:
self.shutdown()
def shutdown(self):
self.shutdown()
@controller.handler
def request(self, f):
print("request", f)
@controller.handler
def response(self, f):
print("response", f)
opts = options.Options(cadir="~/.mitmproxy/")
config = ProxyConfig(opts)
server = ProxyServer(config)
m = MProxy(opts, server)
m.run()
time.sleep(60) # there is should be another code
m.shutdown()
You can execute mitmproxy in headless mode using mitmdump -s script.py. If you are writing tests, you could for example sys.exit(exitcode) from your script to signal test success/failure.
Thanks for reply. I have some questions for inline scripts then:
Is there some way to start script within PyCharm for easier debugging?
Can it be combined with testing framework like py.test?
Right now i see it this way (but it seems very complicated):
I have a file with hundred of tests belongs to some functionality (e.g. GA counters), it uses Selenium for navigation and Proxy for verification (e.g. open main page, check presence of GA counters).
Unfortunately, I need to check request\response, modify response (status code, headers, body), drop connection, simulate lags etc. Looks like I can’t do this through Mitmweb’s API.
I mean a little different thing (may be I still don’t understand correctly how it all works):
I can access Mitmproxy API only when load my code as inline script (mitmdump -s script.py)?
But this way I can’t load it through py.test (pytest sript.py).
Each file with python scripts will contain a hundred of tests like these:
def Test1:
Mitmproxy: change response body if request satisfies condition
Selenium: open page
Selenium: make action
Mitmproxy: check for correct request1 presence
def Test2:
Mitmproxy: change response body if request satisfies condition
Selenium: open page
Selenium: make action
Mitmproxy: check for correct request2 presence
I used a little helper file (mitmstarter.py) to run mitmdump + scripts from within PyCharm:
if __name__ == "__main__":
import sys
from mitmproxy.main import mitmdump
sys.argv = ['mitmdump', '-s', 'my_script.py']
mitmdump()
But that was with Py 2.x & mitmproxy 0.16 or so… didn’t have enough time since then to fiddle / update to current versions.
If it doesn’t work anymore, it still might be the right approach to run / debug from within PyCharm or any other IDE.
Sorry, I should have been more clear. To debug mitmdump (or mitmproxy/mitmweb) in PyCharm, you have to pick venv/bin/mitmdump-script.py (venv/Scripts/mitmdump-script.py on Windows) as the file to run/debug.
Mitmproxy’s functionality is a bit limited when it comes to simulating lagging (you have to use the concurrent decorator for that, and that’s more of a workaround), but other than that all those should be possible.
! 
