MITMWeb - Programmatic clear of session


#1

TL;DR - How can I clear the session in MITMWeb without user interaction?

Hi all,

I’m attempting to programatically clear sessions in MITMWeb to manage running memory usage, either through a script/add-on (which don’t sound like are working in MITMWeb yet), a shell command, or an API call to the web interface. I need to do this so MITMWeb can capture traffic and display it in the web interface, but clear the screen when it needs to free up memory.

In the UI, this is accomplished by clicking “mitmproxy --> new” and clicking OK on the popup, but I’m unable to find a way for something other than the user to trigger this (in my case, the Docker container will sense it needs to clear memory with a health check and trigger the command to clear screen).

So far, I’ve been able to inspect the API POST for the “clear” command, but it seems to be checking that the request originated from the web (http://:8081/clear?_xsrf=2|53adbe93|38e2d80bf2518464244af0d73afd0442|1528152407). Making this API Call from curl generates the following log:

WARNING:tornado.general:403 POST /clear?_xsrf=2|53adbe93|38e2d80bf2518464244af0d73afd0442|1528152407 (192.168.13.234): XSRF cookie does not match POST argument
WARNING:tornado.access:403 POST /clear?_xsrf=2|53adbe93|38e2d80bf2518464244af0d73afd0442|1528152407 (192.168.13.234) 0.96ms

Is there something I’m missing here? Some sort of way to format the request to get it accepted? The last-resort option is to kill and restart MITMWeb, but we’ll lose packets while it starts up again even with the “append to file” option.


#2

I’m here as the Github issue https://github.com/mitmproxy/mitmproxy/issues/237 doesn’t point to anything specific that I can use programatically.


#3

I was actually meddling with this earlier, and came up with:

$ export CSRF_COOKIE=$(http --print h :5556/ | perl -lane'print $1 if /Cookie: (.*);/')
$ http -v POST ":5556/clear?$CSRF_COOKIE" "Cookie:$CSRF_COOKIE"

(using httpie client, but I’m sure it could be relatively easily converted to curl format)

Basic principle is to make any request first to get the csrf cookie, then call the /clear and pass the csrf token as both a cookie and query param.

Feel free to improve into a script for generic resetting, I’d find it useful.


#4

:tada: :tada: :tada:
That’s exactly what I needed! For posterity, I’ll post what it looks like on my end with a Web UI IP of http://192.168.13.234:8081/:

# Install HTTPie and Perl on Debian/Ubuntu
sudo apt-get install -y httpie perl

# Make the API call
export WEB_UI_URL="http://192.168.13.234:8081/" 
export CSRF_COOKIE=$(http --print h ${WEB_UI_URL} | perl -lane'print $1 if /Cookie: (.*);/')
http -v POST "h${WEB_UI_URL}clear?$CSRF_COOKIE" "Cookie:$CSRF_COOKIE"

#5
view = ctx.master.addons.get("view")
if view.store_count() >= MAX_STORE_COUNT:
    view.clear()