Hey all, new to mitmdump/proxy but I am finding this tool to be fantastic.
I have a requirement to capture ALL web traffic from the proxy (especially sourced in javascript), and parse it (ideally from command line) looking for specific things. This tool is SO CLOSE to answering all my questions but I can not see all the traffic (especially responses) when using mitmdump to export a dump to plain text file.
The odd thing is, I see FULL responses/requests when viewing the same dump file via mitmproxy or mitmweb (using mitmproxy -n -r ). When viewing via mitmdump I see some of the responses that contain this:
To create my dump file here is the process I am performing:
- mitmdump --flow-detail 3 -w test.dump
- Browse to website that I want to capture traffic from
- stop mitmdump with a cntl-c
- redirect dump capture to a txt file: mitmdump -n --flow-detail 3 -r test.dump > test.txt
- test.txt will “cut off” some of my traffic responses
- Verify by loading original test.dmp file in mitmproxy, and sure enough I see all traffic
Can anyone assist and let know where I am going wrong with this method?
Running on ubuntu. Version info below:
Mitmproxy: 4.0.4
Python: 3.6.8
OpenSSL: OpenSSL 1.1.0i 14 Aug 2018
Platform: Linux-5.0.0-32-generic-x86_64-with-Ubuntu-18.04-bionic