Extending the UI

I want to integrate HTTPolice into mitmproxy.

Currently I have a simple script that can be used with mitmdump in a sort of batch mode: it produces one big report with all the collected flows on exit.

But it would be much better if I could show a report for every flow as the user views it. HTTPolice can produce both brief plain-text reports, which could be printed in the console UI, and extended HTML reports, which could be shown from mitmweb.

Is there any way for a script to tap into the mitmproxy and/or mitmweb UI? I have found pull request #700 which sounds promising, but I can’t find any mentions of “plugins” in the current code or docs.

1 Like

As a simpler approach, I tried logging to mitmproxy.ctx.log, so that my reports could be seen in the output of mitmdump. But messages logged in response are printed to the console before the request/response summary, like this:

E 1000 Malformed ETag header GET http://httpbin.org/response-headers?ETag=foo-bar
              << 200 OK 90b

which is impossible to parse visually. I guess some sort of after_response event would help here.

Better, how about a method like this:

flow.add_annotation('my text')

such that my text would then be shown in the “Details” pane in mitmproxy and mitmweb? And annotated flows could be marked with an icon in the flow list.

Do the developers feel that this would be a good feature?

1 Like

Hey @vfaronov,

Sorry for the slow reply - I wanted to take some time to form a longer reply, but then of course did not find that time and it slipped through. :see_no_evil: Sorry and thanks for bumping this! :slight_smile:

Long story short: Yes, UI integration is something we’d love to have, but we haven’t really figured out how to do it best. We’d be super interested in working with addon developers and see what is useful for them.

mitmdump output is realized as an addon as well, so in other words, we’d need to make sure that your addon can run after the terminal logging one. More control over addon order seems to be a reasonable feature request.

We kind of added something like this, namely flow.metadata = {}. We have not commited to making this a public/reliable/documented API yet, but if you add stuff there it appears in the details tab. Of course being able to add icons to the flow list would be very useful as well. What would work best for your here? We could potentially add flow.icons, which would just be a (non-persistent) string of UTF-8 icons modifiable by scripts.

FWIW; most of our dev discussions are happening on Slack at the moment - you are more than welcome to join us there for hopefully faster turnaround times. :slight_smile:


Thank you @mhils for your reply.

Great! Definite +1 from me for adding this to the public API.

It’s a bit strange that it prints repr(), not str() — I wonder what use cases you had in mind? But of course it’s trivial to wrap my string in a custom class that returns self from __repr__, so not a problem.

Just to be clear, the string that I want to put there is a few lines worth of text, maybe a few short paragraphs.

mitmweb doesn’t show this metadata though, but that must be easy to fix.

All I need is a way to distinguish flows that have (my) metadata from flows that don’t.

flow.icons sounds great — I would then mark some flows with the letter ‘E’, others with the letter ‘c’, and the rest with no icon at all.

By analogy with flow.metadata, I think flow.icons could be a set of (single-character) strings, rather than just one string. That would make it easier to toggle the icons independently, and potentially each icon could be drawn in its own column (across all flows). But of course I can manage a single string as well.

I see. Well, if I had metadata and icons, logging would be a secondary feature at best, so managing the order is not important for me then. Anyway I can add my own indications of which flows I’m referring to.

Thank you, I’ll definitely pop in at some point. mitmproxy is a great tool and I’d love to contribute. The above features (metadata in mitmweb, icons) sound like they could be a good fit for me. But probably not sooner than a couple months, so if you wish to do it yourself, by all means have a go :smile:

Yes, that’s stupid. Feel free to send a PR that fixes it!

Works for us as well I’d say - order may be useful though, so maybe list.

I’m super busy in March myself, but super excited for this! :slight_smile: