Help with --ignore regex

help
1

Steps to reproduce the problem:

  1. Use Upstream mode (not sure if it only happens there)
  2. Make a regex to ignore http://sandbox.itunes.apple.com/bag.xml?ix=6&os=9&locale=en_US
  3. Connect to said address.

I have tried many regex permutations, like apple.com, .*apple.com:80, sandbox.itunes.com:\d and many many many others.

Same thing for http://app-adforce.jp/ad//p/tmck?_app=1503&_bundle_id=jp.co.bandainamcogames.NBGI0169&_bv=1.2.2&_model=iPhone6%2C1&_os_ver=9.3.3&_sdk_ver=v2.11.1g.
My latest tries for that were .*app-adforce\.jp[^\s]* and http:\/\/app-adforce\.jp/ad.+ and various combinations.

I’m not sure if my assumptions about the checked string are wrong or my regexes are incorrect…

I looked at the code and it seems to check against base_url:port? Like in the last example, it’d be app-adforce.jp:80? Making a regex with this assumption doesn’t seem to work, so maybe the fact that it’s Upstream mode has something to do with it (although it’s more likely that I’m just wrong).

Full command (although probably irrelevant)

"../mitmproxy/mitmweb.exe" -v -U http://localhost:8888 --ignore .*app-adforce\.jp[^\s]* --ignore .*\.?apple\.com:?[^\s]* --ignore .*s3-us-west-1\.amazonaws\.com:443.* --ignore ocsp.apple.com.* --ignore .*apple.com.* --ignore http://app-adforce\.jp:80

I think some of those work, but I haven’t tested it. Just --ignore .* definitely works (blocks everything).

Edit: I’d also like to escape http://ocsp.apple.com/ocsp03-wwdr01/ME4wTKADAgEAMEUwQzBBMAkGBSsOAwIaBQAEFADrDMz0cWy6RiOj1S%2BY1D32MKkdBBSIJxcJqbYYYIvs67r2R1nFUlSjtwIISCB2VPs6tfI%3D and the above doesn’t work for that either.

System information

Mitmproxy version: 2.0.2 (release version) Precompiled Binary
Python version: 3.5.3
Platform: Windows-10-10.0.14393
SSL version: OpenSSL 1.0.2j  26 Sep 2016
Windows version: 10 10.0.14393  Multiprocessor Free
2

Hi,

Have you looked at the limitations listed in http://docs.mitmproxy.org/en/stable/features/passthrough.html#limitations? In regular (and upstream) mode, explicit HTTP requests are never ignored.

3

Oh, well it just says regular there, how could have I known it was for upstream as well?

If so, then how did .* work? :thinking: IIRC I tried it and I saw no requests appear in the web interface.

4

@mhils Hello again. In that Ignore Domains page it mentions a “Limit” option, but I couldn’t find anything on it?
It’s also on the bottom of the pages but there’s no docs linked.

I’m assuming that feature doesn’t exist yet, so I’m currently trying to hack mitmproxy’s source to allow for ignore domains in Upstream mode, do you have a suggested way to go about it? Could that even work?

My goal is: I have a phone in which I set my local IP as a proxy and I want to direct all HTTP and HTTPS traffic to a Flask server I’m running locally, except some specific domains.

Thank you for your help.

5

I’m quite confused…

I put some logging lines here: https://github.com/mitmproxy/mitmproxy/blob/master/mitmproxy/proxy/config.py#L27
and the return False line below, and every request seems to go to False (which reflects what I’m seeing), but host which the regex checks against is always localhost:8888 (my flask server which I’ve set as upstream SPEC) . Is that how it’s supposed to be? Is that why --ignore-hosts can’t work with upstream mode? Why is it checking against the target host and not each incoming address/URL?

Sorry for inundating you with questions, I appreciate any help.

You obviously know about all this, so if there’s a very different way to achieve what I want, I’m eager to hear it as well.